Copyright © Immersive Video Solutions
ALL RIGHTS RESERVED.
Using everyday terminology, Structure-from-Motion (SfM) is the name of an image (photo) processing technique for the construction of three-dimensional point cloud data accompanied by a) rendered realistic 3D scenes [models] and b) ortho-photographs. These deliverable products are constructed from digital photographs (images) taken at points along a path prescribed to produce the most cost-effective, visually desirable results. For some applications, Structure-from-Motion products can rival those produced using LIDAR in terms of accuracy, cost, timeliness, and visual appeal. The term FODAR has been introduced into the marketplace to extend SfM technology's competitive posture relative to LIDAR.
Minimising systematic error surfaces in digital elevation models using oblique convergent imagery
Rene Wackrow and Jim H. Chandler
First published: 16 March 2011
There are increasing opportunities to use consumer‐grade digital cameras, particularly if accurate spatial data can be captured. Research recently conducted at Loughborough University identified residual systematic error surfaces or domes discernible in digital elevation models (DEMs). These systematic effects are often associated with such cameras and are caused by slightly inaccurate estimated lens distortion parameters. A methodology that minimises the systematic error surfaces was therefore developed, using a mildly convergent image configuration in a vertical perspective. This methodology was tested through simulation and a series of practical tests. This paper investigates the potential of the convergent configuration to minimise the error surfaces, even if the geometrically more complex oblique perspective is used. Initially, simulated data was used to demonstrate that an oblique convergent image configuration can minimise remaining systematic error surfaces using various imaging angles. Additionally, practical tests using a laboratory testfield were conducted to verify results of the simulation. The need to develop a system to measure the topographic surface of a flooding river provided the opportunity to verify the findings of the simulation and laboratory test using real data. Results of the simulation process, the laboratory test and the practical test are reported in this paper and demonstrate that an oblique convergent image configuration eradicates the systematic error surfaces which result from inaccurate lens distortion parameters. This approach is significant because by removing the need for an accurate lens model it effectively improves the accuracies of digital surface representations derived using consumer‐grade digital cameras. Carefully selected image configurations could therefore provide new opportunities for improving the quality of photogrammetrically acquired data.
Wikipedia: Structure from motion (SfM) is a photogrammetric range imaging technique for estimating three-dimensional structures from two-dimensional image sequences that may be coupled with local motion signals. It is studied in the fields of computer vision and visual perception. In biological vision, SfM refers to the phenomenon by which humans (and other living creatures) can recover 3D structure from the projected 2D (retinal) motion field of a moving object or scene.
BEaWARE of the (SfM) systematic doming (elevation) error!
Structure from Motion (SfM) Photogrammetry ISSN2047-0371
Natan Micheletti1 , Jim H Chandler2 , Stuart N Lane1 1
Institute of Earth Surface Dynamics,
University of Lausanne (email@example.com) 2
School of Civil and Building Engineering, Loughborough University
SfM involves a process that automatically finds and matches a limited number of common features between images which are then used to establish both interior and exterior orientation parameters. A subsequent procedure then extracts a high resolution and colour-coded point cloud to represent the object. For this reason, the acquisition of imagery with the right characteristics is critical.
A range of cameras can be used but a digital SLR camera equipped with fixed focus lens will generate the most accurate data as widely varying zoom settings can cause difficulties (Shortis et al., 2006, Sanz-Ablanedo et al., 2012). Images do not need to be acquired from the same distance or have the same scale (see Figure 1). On the contrary, it is advisable to acquire multi-scale image sets which initially capture the whole site with a few frames before obtaining closer range images to capture the desired detail at the required precision. This is particularly important when capturing areas of detail which are physically obscured by other features (i.e. occlusions). The whole set of images is used for feature extraction, so it is fundamental to ensure that the scene is static and that exposures capture the detail required. Flash photography frequently creates inconsistent image textures which can confuse the feature-matching process (Micheletti et al., 2014).
The spatial relationship between images is more flexible than traditional photogrammetric image acquisition using stereo-pairs (Chandler, 1999, Remondino et al., 2014). However, it is critical to acquire imagery from as many different spatial positions as possible. The wide range of image directions then creates a dataset with a strong geometry, important to recover both internal camera models and precise, and hence accurate, object coordinates.
The exact number of photographs required is dictated on a case-by-case basis and is a function of occlusion, shape complexity and scale. A range between 10 and 100 should be a good starting point for most applications at close (cm to 10s of m) and intermediate (< Structure from Motion 4 British Society for Geomorphology Geomorphological Techniques, Chap. 2, Sec. 2.2 (2015) 1km) scales. Micheletti et al. (2014) demonstrated that increasing the number of images produces denser meshes and improves model accuracy.
More significantly, this investigation showed how larger datasets help to remove outliers when the number of images is already sufficient for a good representation of the surface of interest. Hence, very large datasets are not always necessary as even small image sets are able to provide outputs of very satisfying quality, provided image geometry remains strong throughout the area of interest. An important practical constraint is computer memory and the associated time users are willing to wait for results.
Guidelines and tips for imagery acquisition are often provided with tools (e.g. by Autodesk at www.123dapp.com). A summary of key points has been provided by Micheletti et al., 2014) and includes:
- Plan camera survey and registration or scaling method in advance.
- Capture the whole subject first, and then the detail, ensuring that occlusions are captured adequately (see item 3).
- Ensure appropriate coverage. Basic principle: every point on the subject must appear on at least three images acquired from spatially different locations.
- Static scene.
- Consistent light.
- Avoid overexposed and underexposed images.
- Avoid blurred images – normally arising from slow shutter speed and/or camera movement.
- Avoid transparent, reflective or homogeneous surfaces.
As for sensors, SfM applications allow a wide range of surveying platforms options for camera deployment. Again, the best choice varies on a case-by-case basis, depending on object of interest and scale. Usually, handheld devices and tripod-based terrestrial imagery are employed for small landforms. Larger scenes are nowadays mostly surveyed using small-scale UAVs (including multi-copters and fixed-wing drones, e.g. Ryan et al., 2015). These platforms are becoming more popular amongst academics and industrial surveyors due to their increasing affordability. Their clear advantage is the possibility of placing the sensor in locations that would otherwise be difficult to capture with hand-held sensors. Nevertheless, the use of such platforms can create weak image geometry, poor camera models and hence low accuracy data (see below).
Post-processing and possible error sources
As in traditional photogrammetric methods, every stage of a 3-D reconstruction using SfM photogrammetry can create significant errors that propagate through to the final product. The reliance upon a “black box” calibration routine to model camera geometry is particularly problematic. Weak image geometry will generate an imprecise, but more importantly, an inaccurate set of parameters to model camera geometry. A conventional block of vertical aerial imagery is geometrically weak and both a calibrated metric camera and abundant ground control points were traditionally required to maintain mapping accuracies as well as defining a coordinate system. Calibrating a camera “in- Structure from Motion 6 British Society for Geomorphology Geomorphological Techniques, Chap. 2, Sec. 2.2 (2015) situ” using a conventional block of vertical imagery acquired using a UAV is likely to generate inaccurate data. This typically manifests itself in the form of a systematic error surface or “dome” caused by an inaccurate lens model (Wackrow and Chandler, 2008; James and Robson, 2014) which can often be overlooked (e.g. Ouédraogo et al., 2014). One simple recommendation is to strengthen image geometry by obtaining oblique imagery in addition to the vertical dataset acquired for object coverage. This requires particularly careful attention to be given to the design of UAV surveys.
Intermediate scale applications (to 100s of m), the quality of derived data is clearly related to image quality, scale and geometry. If image geometry is weak in any area then inaccurate data can easily be generated, particularly if black box calibration routines are used to determine camera geometry.
If traditional (linear / parallel) flight lines are being used;
then why not use curved non-traditional (non-linear / non-parallel) flight lines?
Note: The SfM doming error is particularly noticeable when traditional (linear/parallel) flight lines are used for longer/narrower (corridor) type projects. Please read both of the following respected references to gain a realistic perspective regarding ways to mitigate the systematic SfM doming (elevation) error.
IGS is providing a couple well known technical references (below) in the field of SfM technology (otherwise introduced here as FODAR) to let our customers know why IGS is committed to the use of non-traditional (non-linear / non-parallel) flight lines for collecting digital images for the development of 3D point clouds, orthophotographs, and accurate digital elevation models that are geometrically and geographically correct. Today, SfM processing software inherently introduces a systematic doming (elevation) error when traditional (linear/parallel) flight lines are used to capture the required photographs (images). Unlike the majority of SfM (drone photography) service firms who have never heard of (or deny the existence of) the systematic SfM doming error, IGS has done its homework and offers a solution that delivers direct cost benefits (along with mitigation of the doming error) on every project flown.