3D Scans for VR, by Maxime Lhuillier

Email: Maxime { dot } Lhuillier { at } free { dot } fr
Address: Institut Pascal, UMR 6602 CNRS/UCA,
Campus Universitaire des Cézeaux (4 av. Blaise Pascal),
63178 Aubière Cedex (TSA 60026, CS 60026) - France.
Tel: +33 (0)4 73 40 75 93
Fax: +33 (0)4 73 40 72 62

Last update on December, 10th 2021.

These scans are reconstructed from input videos taken by helmet-held 360 cameras. They can be explored by moving like a pedestrian (seated VR). They are results of methods that i design, implement and experiment, without manual clean up.

Table of Scans

Name Acquisition 360 camera Triangles Shots Standalone Quest Oculus & WMR OpenVR Sketchfab
Snowy Town walk 670m Garmin Virb 697k shots apk (169Mo) zip (148Mo) zip (143Mo) reduced
Forest walk 800m Garmin Virb 2.1M shots apk (337Mo) zip (319Mo) zip (314Mo) reduced
Semi-medieval town walk 1km Gopro Max 2.3M shots apk (408Mo) zip (381Mo) zip (376Mo) reduced
Campus bike 5.2km Garmin Virb 2.8M shots apk (721Mo) zip (666Mo) zip (661Mo) reduced
Verdon's canyon kayak 2.5km Gopro Max 1.8M shots apk (199Mo) zip (196Mo) zip (191Mo) reduced
Basalt canyon walk 800m Gopro Max 2.9M shots apk (786Mo) zip (764Mo) zip (759Mo) reduced
Collonges-la-Rouge walk 1.6km Gopro Max 3.4M shots apk (777Mo) zip (746Mo) zip (741Mo) reduced

For each environment, the table provides the name, informations about the acquisition (camera and approximate length of trajectory), the number of triangles, a video and screenshots, the downloadable applications of visualisation in three cases of VR headsets:
(1) standalone Oculus Quest (including Go and Quest 2),
(2) headsets compatible with Windows Mixed Reality (WMR) and Oculus headsets linked to a PC,
(3) headsets compatible with OpenVR (in SteamVR).
Two controllers are needed in all cases (except the Go). The applications are generated by Unity 2019.4 with preview packages. In theory, an application works for all headsets in its case, but in practice i cannot check every headset.

The table also provides links to explore reduced versions of the scans without headset. Only segments of the environments are textured in the reduced versions (with lower resolution than VR) due to a space limitation. Remainder of reduced version has colors encoding surface normals.

Send me an email if you have a VR headset that does not work in one of the three cases above.

Installation and control instructions

In case (1), the application is an APK file, which can be installed with the ADB command-line tool. I checked it for both Quest and Go. More details are here.
In case (2), the application is in a zipped directory. I checked it with two headsets (Oculus Quest and Lenovo Explorer) linked to a PC running Windows 10. Run the .exe file once the headset and the controllers are recognized and initialized (by the Oculus application or the WMR application).
In case (3), the application is also in a zipped directory. I checked it with one headset (Lenovo Explorer) linked to a PC running Windows 10. Run the .exe file once the headset and the controllers are recognized and initialized (eg by the WMR app), then SteamVR (which includes OpenVR) is automatically launched.

In all cases, the control instructions of the VR applications are those described by the shot links in the table above.
In cases (2) and (3), the user can quit a VR application by pressing simultaneously the two triggers. Furthermore, joysticks can be replaced by touchpads.

Alternative installation methods: use SideQuest for case (1) or SteamVR Workshop for case (3).

Photogrammetry Methods

Papers about methods and algorithms are available here.

In short: the structure-from-motion step includes a keyframe selection of the multi-camera video (that is good for matching and reconstruction) and a calibration refinement (from an initial rough guess), only features (interest points and edge curves) are reconstructed, then the surface is computed using topological constraints (manifoldness and lowered genus) and visibility constraints in a 3D Delaunay triangulation of the points (including sampled curves), last the texturing is done from the keyframes and the 3D coordinates are changed for VR.

Here are papers and their computation steps: calibration refinement (3DV'15, CVIU'17), structure-from-motion (BMVC'07, IVC'09), surface reconstruction (CVIU'18 improved by IC3D'19), curves are matched (PAMI'02) and integrated in the surface, texturing (IC3D'20), coordinate change (IC3D'21). The methods are sometimes old since my main goal is to show what can be done using what i published. (It is also easier to maintain software and methods that are well known for me.)

Other details

The table below provides md5 sums, if needed.

Name md5sum for Standalone Quest md5sum for Oculus & WMR md5sum for OpenVR
Snowy Town 199150e9ead1283826c12df1c87cd525 6665cac805b59f0f64e06f3f291e7094 831bb86b2821272e19bdca4d3052305c
Forest a560d8cec85ddf40a0b8a4a6e67e5fae b6427e7239738bbf5cb270e67bc98018 f77f3b8cb672d6e0117a317ac7820d30
Semi-medieval town ff1ed3c39df7fc4bc08667a6c784e26c dd7219649b0a6c1152ea1b241c8459b4 7f12dc39db9cc536c65721553d114546
Campus 146018af613f24760546f8df83b90845 8dd0573a500832a1af79d667d8f11a1a 35ecf97f7d4acb86b7c5d0d2f70eeb18
Verdon's canyon 9c9525ba953921d245ddba4f20da31c4 4653b020e62f1de6607d5c5ed014e4d9 a1972a8712c00621d714e6975088af57
Basalt canyon b2672804739b8aefda31d3f04ad15b36 b058e87f0777ab6b14d74455316c285d cf7c6686af032d0cdb9ec6593d0edfdf
Collonges-la-Rouge ff2632eac68dbaed57e25b7d9a11d0b1 37ba7416587c60c8a6941a53c563b81a fa59979e0df8b3db05caf618910d0a48


Privacy policies: every application in this page does not process data of the VR user.