Where Things Go Wrong When Combining Different Data Sources

A drone can see a rooftop that a tripod-mounted scanner never will, and a terrestrial scanner can resolve a facade at a precision no aerial platform can match. That is one reason people combine sensor sources. The other reason is just as important: LiDAR and imagery do not capture the same information about a surface in the first place. LiDAR gives you precise geometry, imagery gives you texture and color, and combining them is meant to get you both at once. Vendors and service providers have built a lot of marketing around this idea, and the workflow genuinely works. What that marketing tends to skip over is how much has to go right before “combine the sources” is a finished step instead of the hardest part of the project.

What “combining sources” usually means in practice

Reality mapping projects that mix data sources typically draw from some combination of platforms (aerial, UAV, and terrestrial) and sensor types (LiDAR and imagery). Aerial and UAV platforms cover large areas quickly but lose detail on vertical surfaces and anything occluded from above. Terrestrial platforms, whether a stationary scanner or a mobile system, deliver very high precision at close range but are slow to cover large sites and cannot see rooftops. LiDAR delivers precise geometry regardless of lighting, while imagery delivers the color and texture that LiDAR alone cannot provide.

The appeal across dimensions is the same: use each source for what it does well, and avoid relying on it where it is weak. Some hardware has been built specifically around this idea rather than leaving it to post-processing. Leica’s CityMapper-3, for example, captures imagery and LiDAR from the same airborne platform in a single flight, tightly aligning the two rather than requiring a separate combination step afterward. That kind of hybrid sensor is a good illustration of how much industry effort has gone into making this pairing work, and it is worth looking at if you want to see the combined-capture approach taken to its logical conclusion.

Why combining sources is harder than it looks

Each of the following challenges shows up repeatedly in real projects, and they compound rather than occurring in isolation. None of them make combining sources a bad idea, they just mean the fusion step deserves the same planning attention as the capture itself.

How much the datasets actually overlap

Merging two datasets requires a shared reference between them, either through surveyed control points or through overlapping geometry that a registration algorithm can match. An aerial dataset and a terrestrial dataset often only meet at a thin sliver, such as a roofline or a single facade. That is not enough shared structure for reliable registration unless you planned for it. A deliberate overlap area, or a tie scan that links the two datasets, turns registration from a best-effort exercise into a controlled one.

Coverage, resolution, and update frequency rarely match

Aerial data typically covers a large area at lower resolution and gets refreshed infrequently. UAV and terrestrial data are more local, higher resolution, and easier to recapture often. Combining sources means combining all three of these mismatches at once: different coverage extents, different resolutions, and different capture intervals. The practical question this raises is which dataset becomes the base reference that the others get tied to. Most often it ends up being the large-area, lower-resolution aerial dataset, simply because it is the one thing that spans the whole site.

A dataset can look correct on its own and still be offset

Every dataset can have excellent relative accuracy, meaning its own points are consistent with each other, while still sitting in the wrong absolute position once you bring in a second dataset. This is easy to miss because nothing about the dataset looks wrong in isolation. Coordinate system or datum mismatches are one common cause, but not the only one, and the shift often only becomes visible once the two sources are placed side by side.

Simultaneous capture is possible, but it is not the default

Hybrid sensors like the CityMapper, dual-hatch aircraft carrying multiple sensors, and bathymetric systems that collect several LiDAR wavelengths and imagery at once all eliminate the time gap between sources entirely. Most projects do not have that setup, though. A terrestrial scan and a drone flight are often days or weeks apart, and it is just as common to pair fresh capture with something pulled out of an archive. On a site that changes over time, that gap can look exactly like a registration error even though it is really a difference in what was actually there when each dataset was captured.

Different sensors have different blind spots

Imagery struggles in shadow, while LiDAR keeps collecting regardless of light. A UAV can fly underneath a bridge deck where an airborne platform cannot see at all. Different LiDAR wavelengths penetrate water to different depths. None of these are flaws in a single sensor, they are just the physical limits of what each one can observe, and combining sources is partly an attempt to cover one sensor’s blind spot with another sensor’s strength. That only works if you know in advance where each blind spot actually falls.

Practical tips for combining sources well

• Plan your overlap and control before you are in the field. Decide where datasets need to meet, and make sure that area has enough shared geometry or surveyed control points to register against.
• Decide which dataset will serve as your base reference before you start, based on which one spans the area you need tied together, rather than defaulting to whichever one happens to be the most precise.
• Where the site is likely to change over time, keep capture windows close together, or use a simultaneous multi-sensor platform if that is available to you. When you are working with archived data, record capture dates clearly so real change is not mistaken for registration error.
• When possible, perform a combined alignment across all datasets during processing rather than adjusting them individually at a later stage. Reprocessing from the raw data, such as adjusting LiDAR strip parameters or reworking image orientation, corrects the actual source of misalignment, while shifting or rotating a finished point cloud only masks it and can introduce its own distortion.
• Take advantage of the overlapping area to assess how well the datasets actually match, rather than checking each dataset only on its own. Doing this in multiple areas can help indicate which dataset needs adjustment, rather than just confirming that a mismatch exists.

Deciding whether combining sources is worth it

The decision to combine sensor sources often comes down to whether the added value is worth the effort. Software vendors continue to work on reducing that effort, and as fusion tools get better at handling the reconciliation work automatically, that will open the door to applications that are not practical to attempt today.

Effort is not the only cost, though. If every source you want is already sitting on the shelf, the calculation is fairly simple. If getting the added source means an extra flight or an extra scan, that acquisition cost needs to be justified by what the combined result actually adds, not just by the fact that adding another sensor sounds like a natural next step. Fusion is not a guaranteed win, and how much value it adds depends heavily on what the application actually needs.

From a processing point of view, aligning the geometry is often only the first step when merging data. Even correctly registered datasets often still need further adjustment before they look like one coherent product. That adjustment cuts both ways. Tuning a dataset to look good, such as pushing contrast on aerial imagery, can work against the radiometric information the data was captured to preserve, particularly in shadowed areas or bright hot spots. This tension between a visually pleasing output and a radiometrically correct one is a long-standing discussion in its own right, and it applies just as much when combining sources as it does within a single dataset.

Added as an afterthought, sensor fusion often just adds cost for a result that quietly underwhelms. Combined with intention though, it can genuinely raise the quality of a reality mapping product and increase the possibilities for further applications.