Building GCP networks from space

Read the conference paper
Have a look at the patent

Canonical MIN diagram

How do satellite image providers know where their images are looking on the earth?

Every satellite image has an implicit location on the earth embedded in its metadata. In theory, if you pick a point on the image and you know what the point’s elevation is, you can use the image metadata to estimate a 3-D earth coordinate for it. The accuracy of that estimated coordinate is an entirely different question!

Satellite imaging systems with high pointing accuracy (such as DigitalGlobe’s WorldView and Geoeye satellites) utilize star-trackers and other high-end satellite geopositioning systems on the satellite bodies themselves, but these systems are extremely expensive, and would break smallsat budgets. As a result, the geolocation error in the raw imagery can be very great.

Smallsat providers usually use ground control points (GCPs) derived by survey, or from other image sources, to better position their satellite images on the earth. GCPs are objects that are both recognizable in satellite imagery, and have a known coordinate on the earth. Smallsat providers try to minimize the difference between a GCP’s true coordinate and the location given by the image metadata, by adjusting the positioning information associated with the images until the errors are minimized.

This means that smallsat geolocation accuracies are dependent on the quality of the GCPs that are available in the location being imaged. In the United States and Europe, very high quality GCPs are freely available; but in South America, Africa, and Asia, this is not the case. As a result, there can be a lot of uncertainty in the geolocations of objects visible in the imagery, and orthorectified satellite imagery mosaics are often plagued by seam lines.

A Metric Information Network (MIN) is a network of correlated GCPs that can be used as a framework for correcting earth imagery. The value of such a system is that smallsat and other image providers can use this dense network of GCPs as a consistent skeleton on which their images can be draped, improving the images’ geolocation accuracy and smoothing seamlines between images in mosaics. Furthermore, since a MIN is built using satellite imagery, this method can be used to produce sets of GCPs in remote and denied areas — wherever there are stable features to be matched on. If the massive quantity of satellite images of the earth can be thought of as the ‘skin of the earth’, then a MIN can be thought of as the skeleton on which the skin lies.

In 2007, John Dolloff and Michele Iiyama described a prototype method for generating a MIN that could be extended, using sequential bundle adjustment methods, to cover continent-sized areas. The GCPs in this case were extracted as tie points from overlapping images taken from satellites (as shown in the diagram at the top of the page), and then sequentially updated with other observations until their geolocation error was reduced. The resulting network can be as dense as the application requires. Using DigitalGlobe’s WorldView-1 imagery, the resulting CE90 (the 90th percentile of horizontal error) of the ground control point network was shown to be between 1 and 2 meters. At the time, the NRO required satellite imagery to be produced and delivered with lower limits on resolution of .5 meters; this restriction has since been lifted, and GCPs can now be produced with CE90s under a meter using only high-resolution remote sensing imagery from the WorldView class satellites.

My team at DigitalGlobe developed this prototype into an implementation that ran on a high performance cluster (HPC), and that was limited in scale only by the number of nodes in the HPC. We also broke the steps of MIN development up into two phases. In the first phase, tie points are extracted from a large collection of imagery (for example, the DigitalGlobe WorldView archive itself), clustered to find sets of tie points that are stable over time, and are then stored in a database. In the second phase of processing, the tie points are used to drive an image adjustment process that corrects estimation errors in the position of the object being observed.

On Febuary 2, 2016, the patent (US9251419) was issued to DigitalGlobe for the HPC implementation of the Metric Information Network.