If you fly a consumer drone such as a DJI Mini 4 Pro around a rural property and feed the MP4 clips into a modern end-to-end neural 3D reconstruction tool, the first render often looks impressive from a distance—until you slice through a roof or compare two separate flights. Suddenly, roof planes split into ghosted double layers, terrain tilts away across a hillside, and sharing the result with an architect, contractor, or collaborator means asking them to download a 200 MB .ply file and install desktop software.
This article documents how I consolidated multi-flight consumer drone video and stills over a stone rural farmhouse and wooded hillside in southern France into crisp, metric, single-surface 3D point clouds—dropping roof-plane RMSE from 21.37 cm down to 6.23 cm—and built a progressive, zero-install Three.js WebGL2 viewer backed by a custom gzipped binary format (.pcloud) that streams smoothly in any desktop or mobile browser.
The complete generalized pipeline, binary exporter, and WebGL2 viewer are open-sourced under the MIT License at github.com/Madderose/drone-pointcloud-web3d.
1. Interactive 3D Live Demo
Explore the consolidated metric 3D point clouds directly below—no browser extensions, desktop installs, or account logins required:
.pcloud streaming2. Why Uncalibrated Neural SLAM Drifts on Multi-Flight Drone Video
Consumer drones write rich per-frame telemetry into .SRT sidecar files—GPS latitude/longitude, barometer-fused relative altitude, and focal length—yet neural feed-forward SLAM pipelines commonly fail when merging separate flights over real-world terrain:
- Per-Flight Scale & Horizon Drift: Two clips flown minutes apart (a wide perimeter loop over the wooded hillside plus a low 300° architectural orbit) accumulate independent scale drift and tilt. Merging them without global bundle adjustment creates ghosted double roofs and doubled masonry walls.
- Uncalibrated Wide-Angle Distortion: Assuming fixed pinhole intrinsics bends straight roof ridges toward frame edges by tens of centimeters.
- Sky & Haze Artifacts: Without conservative sky masking and multi-view visibility checks, dense depth estimators project clouds and distant haze as floating confetti above roofs and meadows.
- Friction-Heavy Sharing: Raw
.plyor Rerun.rrdfiles routinely exceed 100–500 MB and require specialized desktop viewers, locking out stakeholders on phones or standard browsers.
3. Quantitative 5-Method Benchmark & Full Hillside Expansion
To isolate geometry quality from pose drift, I built a unified 191-frame multi-source dataset around the farmhouse core—combining Flight 1 perimeter departure/return frames, Flight 2 close 300° orbit frames, and three oblique stills—and solved a single global COLMAP 4.2.1 sparse OPENCV model: 191/191 images registered (100%), 142,485 3D points, 0.5813 px mean reprojection error, 7.802 mean track length, and 0.784 m GPS ENU RMSE via a single global rigid Sim(3) transform.
Five dense reconstruction strategies were evaluated quantitatively on the exact same farmhouse region of interest:
| Reconstruction Method | House Coverage | Roof Plane RMSE | Roof P90 Error | Open 0.5 m Cell Z-Span | Open Cells > 0.5 m |
|---|---|---|---|---|---|
| MASt3R-SLAM (Old Baseline) | 59.6% | 21.37 cm | 36.63 cm | 0.97 m | 90.71% |
| Refined MASt3R (COLMAP Poses) | 100.0% | 21.19 cm | 34.78 cm | 0.73 m | 68.29% |
| Depth Anything 3 (Pose-Conditioned) | 100.0% | 32.08 cm | 55.73 cm | 0.80 m | 80.51% |
| COLMAP CUDA PatchMatch MVS | 100.0% | 9.50 cm | 13.80 cm | 0.17 m | 13.97% |
| OpenMVS + Visibility Carving | 100.0% | 6.23 cm | 8.90 cm | 0.18 m | 11.05% |
Even when given clean COLMAP poses, feed-forward depth models still leave 21–32 cm of roof-plane thickness and 68–80% double-layered open cells. By contrast, OpenMVS multi-view PatchMatch followed by Delaunay visibility carving drops roof-plane RMSE by 3.4× to 6.23 cm (8.90 cm p90) and cuts noisy open cells (`> 0.5 m` vertical span) from 90.71% down to 11.05%. Architectural measurements extracted directly from the consolidated 5 cm cloud (1,674,668 points) match site dimensions: 17.54 m East–West ridge length, 7.29 m × 5.59 m west wing, 6.81 m east wing width, and 4.83 m courtyard clearance to the north outbuilding.
Expanding Across the Full Wooded Hillside
To cover the entire surrounding forest and terraced southern slope without introducing a single centimeter of offset against the 5 cm farmhouse core, we extracted intermediate loop frames from Flight 1 (312 total frames across all 6 flight/still segments) and locked the expanded SfM reconstruction to the exact 191 reference camera centers of the farmhouse model using RANSAC + Huber IRLS rigid Sim(3):
- Full-Estate SfM:
312/312images registered (100.0%),226,164sparse 3D points (0.5407 pxmean reprojection error), shared 191-camera alignment RMSE = 0.0318 m (3.18 cm), GPS ENU RMSE = 1.153 m. - Footprint & Density (15 cm voxel): 2,875,063 points covering 6.776 ha (
67,762 m²), compared to4.274 ha(529,490points) in the legacy uncalibrated cloud (+443% points, +58.5% area). - Zero-Seam Core Lock & Terrain Crispness: Median nearest-neighbor distance to the 5 cm farmhouse cloud in the shared zone is 0.0699 m (down from
1.6935 min the legacy baseline), while open-terrain 1 m vertical IQR tightened from1.1475 mto 0.2605 m (−77.3%).
4. Pipeline Architecture & Open-Source Repository
The open-source pipeline (Madderose/drone-pointcloud-web3d) is organized into four modular CLI stages plus the WebGL2 exporter:
- Telemetry & Conservative Sky Masking (
pipeline/01_prepare_dataset.py): Parses DJI.SRTsubtitles and JPG EXIF GPS into a local East-North-Up (ENU) frame, resizes frames to2048×1152(Lanczos4), and masks only low-gradient (Sobel < 32.0) desaturated/blue sky regions in the top 45% connected to rowy = 0, eroded by3×3to protect horizon treetops. Generatespairs.txtcombining temporal video neighbors (±12frames) with each frame’s nearest spatial ENU GPS neighbors. - Metric SfM & Rigid
Sim(3)ENU Alignment (pipeline/02_run_sfm.py): Extracts GPU SIFT features (8192keypoints/frame,OPENCVcamera model), runs GPU brute-force guided matching overpairs.txt, evaluates bothglobal_mapperand incrementalmapper, filters tracks (min_track_len=3,max_reproj_error=2.0 px), refines viabundle_adjuster, and applies one global RANSAC + Huber IRLS rigidSim(3)to GPS ENU priors (or to--reference-sparsewhen locking wide hillside scans onto an architectural core). - OpenMVS Dense MVS & Visibility Carving (
pipeline/03_run_openmvs.py): Converts the undistorted PINHOLE workspace withInterfaceCOLMAP, runsDensifyPointCloud+ReconstructMesh(Delaunay visibility carving), prunes floating noise farther than0.20–0.45 mfrom the carved mesh using Open3D’sRaycastingScene, applies SOR/ROR outlier filtering, and voxel-downsamples to 5 cm or 15 cm. - Quantitative Audit & Progressive Export (
pipeline/04_audit_cloud.py&web_exporter/export_pcloud.py): Audits 0.5 m / 1.0 m vertical cell spread and roof-plane RMSE, then exports shuffled gzipped.pcloudpayloads andmanifest.json.
Copy-Paste Quickstart
git clone https://github.com/Madderose/drone-pointcloud-web3d.git
cd drone-pointcloud-web3d
python -m venv .venv
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
# Linux/macOS: source .venv/bin/activate
pip install -r requirements.txt
# Option A: End-to-end YAML orchestrator
cp config.example.yaml my_site.yaml
python pipeline/run_all.py --config my_site.yaml
# Option B: Step-by-step CLI
python pipeline/01_prepare_dataset.py \
--videos "./media/orbit.mp4:0:-1:10:orbit" "./media/loop.mp4:0:660:15:loop_a" \
--stills ./media/still_01.jpg ./media/still_02.jpg \
--out ./work/dataset
python pipeline/02_run_sfm.py --dataset ./work/dataset --out ./work/sfm --colmap-bin colmap
python pipeline/03_run_openmvs.py --undistorted ./work/sfm/undistorted --out ./work/mvs --openmvs-bin /path/to/openmvs/bin
python pipeline/04_audit_cloud.py --ply ./work/mvs/consolidated_cloud.ply --out ./work/eval
python web_exporter/export_pcloud.py \
--site "My Site — Drone 3D" \
--layer "preview:Quick Preview:./work/mvs/consolidated_cloud.ply:preview:220000:0.25" \
--layer "hd:Main Cloud (HD):./work/mvs/consolidated_cloud.ply:primary:1400000:0.04" \
--trajectory-srt "orbit:Orbit Flight:./media/orbit.srt:#F59E0B:10" \
--out ./viewer/data
python -m http.server 8080 --directory ./viewer
5. Real-World Engineering Pitfalls Found Along the Way
Three practical failures surfaced during production runs—each straightforward to avoid once diagnosed:
WinError 1114):COLMAP 4.2.1 supports learned
ALIKED_N16ROT + ALIKEDLIGHTGLUE via ONNX Runtime, which on Windows systems with mixed CUDA 12/13 DLLs crashes in onnxruntime_providers_cuda.dll with WinError 1114. Falling back to native CUDA SIFT (--FeatureExtraction.type SIFT --FeatureExtraction.use_gpu 1) + SIFT_BRUTEFORCE with guided_matching=1 is fast, rock-solid, and yielded 0.581 px mean reprojection error.-w Relative Paths & Silent Windows Logs:On Windows, OpenMVS executables resolve undistorted images relative to
-w <UNDIST_DIR> and often write failure diagnostics into DensifyPointCloud*.log in the working directory rather than stderr. Always pass an absolute -w path, set cwd to a dedicated MVS output directory, prepend the OpenMVS binary folder to PATH, and inspect the newest *.log on non-zero exit codes..gz Double-Decompression Trap:Serving gzipped point clouds named
*.bin.gz over shared Apache hosting causes Apache to send Content-Encoding: gzip. The browser transparently decompresses the response inside fetch(), so feeding those bytes into DecompressionStream('gzip') or fflate crashes with a corrupt-header error. Renaming the payload to .pcloud (with AddType application/octet-stream .pcloud in .htaccess) ensures raw gzipped bytes reach JavaScript intact.6. WebGL2 Progressive Viewer & .pcloud (CHPC v1) Spec
Instead of building a multi-level octree server, the viewer relies on a 48-byte Little-Endian binary header followed by quantized coordinates and colors, compressed with gzip level 9:
| Byte Offset | Type (LE) | Field | Description |
|---|---|---|---|
0..3 |
char[4] |
magic |
"CHPC" |
4..7 |
uint32 |
version |
1 |
8..11 |
uint32 |
count |
Number of points N |
12..15 |
uint32 |
flags |
0 (reserved) |
16..47 |
float32[8] |
bbox |
minX, minY, minZ, maxX, maxY, maxZ, zP02, zP98 (ENU meters) |
48..48+6N-1 |
uint16[3N] |
xyz_u16 |
Quantized (X, Y, Z) mapped onto [0, 65535] per axis |
48+6N..48+9N-1 |
uint8[3N] |
rgb_u8 |
Point colors (R, G, B) |
Two implementation details keep the WebGL2 viewer snappy across phones and laptops:
- Shuffled Buffers = Zero-Cost GPU Quality Scaling: Points are uniformly shuffled (
np.random.default_rng(42)) prior to quantization. Switching between Eco (25%), Standard (65%), and Ultra (100%) simply updatesgeometry.setDrawRange(0, Math.floor(count * ratio))with zero buffer reallocation or spatial bias. - Progressive Hydration & Circular Splats: The viewer loads a ~220k-point preview (
~1.95 MB) for near-instant first paint, then streams the 1.4M-point HD core (~12.45 MB) and hillside layer in the background. Inside the custom GLSL shader, ENU(X, Y, Z)maps to Three.js(X, Z, -Y), with antialiased circular splats, live clip-box uniforms, and Turbo DSM elevation coloring.
7. Credits, Open-Source Licensing & Disclaimers
Built by Jonas Hopp (@Madderose • jonas.hopp.fr). Source code is published at github.com/Madderose/drone-pointcloud-web3d.
| Component | Role | License | Compliance Notes |
|---|---|---|---|
| drone-pointcloud-web3d | Pipeline scripts, .pcloud exporter, WebGL2 viewer |
MIT | Copyright © 2026 Jonas Hopp (@Madderose) |
| COLMAP / PyCOLMAP | GPU SIFT SfM, Bundle Adjustment, Sim(3), PatchMatch MVS |
BSD 3-Clause | Schönberger & Frahm (CVPR 2016) |
| OpenMVS | Dense MVS & Delaunay visibility carving | GNU AGPL-3.0 | Invoked via external user-installed CLI binaries only (not bundled or linked). Hosting OpenMVS as a network/SaaS service triggers AGPL-3.0 copyleft; serving static .pcloud files via the MIT Three.js viewer contains zero OpenMVS code. |
| Open3D, Three.js, fflate, PyYAML | Mesh proximity filtering, WebGL2 viewer, gzip fallback, config | MIT | — |
| OpenCV | Video frame extraction, Lanczos4 resize, sky masks | Apache 2.0 | — |
| NumPy, SciPy, Matplotlib, Pillow | Numerics, cKDTree, publication plots, EXIF |
BSD / HPND | — |
| MASt3R / MASt3R-SLAM (benchmarked) | Evaluated neural SLAM baseline | CC BY-NC-SA 4.0 | Non-commercial license; benchmark comparison only |
| Depth Anything 3 (benchmarked) | Evaluated pose-conditioned depth baseline | Apache 2.0 | Code Apache 2.0 / model license; benchmark comparison only |
Always operate UAVs in compliance with local aviation rules (EASA / DGAC / FAA), maintain visual line-of-sight, and obtain consent before capturing or publishing imagery over private property. Note that consumer non-RTK GPS telemetry exhibits roughly 0.5–1.5 m absolute ENU positional offset even when local multi-view geometry achieves 5–10 cm relative precision—making these point clouds visualization and planning aids rather than legal cadastral or licensed land-surveyor deliverables.
Commentaires récents