Photogrammetric Scans for VR

Email: Maxime { dot } Lhuillier { at } free { dot } fr
Address: Institut Pascal, UMR 6602 CNRS/UCA,
Campus Universitaire des Cézeaux (4 av. Blaise Pascal),
63178 Aubière Cedex (TSA 60026, CS 60026) - France.
Tel: +33 (0)4 73 40 75 93
Fax: +33 (0)4 73 40 72 62

Last update on July, 3th 2022.

These scans are reconstructed from input videos taken by helmet-held 360 cameras. They can be explored by moving like a pedestrian (seated VR). They are results of methods that i design, implement and experiment, without manual clean up.

Table of Scans

Name Acquisition 360 camera Triangles Shots Standalone Quest Oculus & WMR OpenVR Sketchfab
Snowy Town walk 670m Garmin Virb 697k shots apk (169Mo) zip (148Mo) zip (143Mo) reduced
Forest walk 800m Garmin Virb 2.1M shots apk (337Mo) zip (319Mo) zip (314Mo) reduced
Semi-medieval town walk 1km Gopro Max 2.3M shots apk (408Mo) zip (381Mo) zip (376Mo) reduced
Campus bike 5.2km Garmin Virb 2.8M shots apk (740Mo) zip (686Mo) zip (681Mo) reduced
Verdon's canyon kayak 2.5km Gopro Max 1.8M shots apk (199Mo) zip (197Mo) zip (192Mo) reduced
Basalt canyon walk 800m Gopro Max 2.9M shots apk (786Mo) zip (764Mo) zip (759Mo) reduced
Collonges-la-Rouge walk 1.6km Gopro Max 3.4M shots apk (801Mo) zip (770Mo) zip (765Mo) reduced
Ruin in a forest walk 400m Gopro Max 1.7M shots apk (391Mo) zip (375Mo) zip (370Mo) reduced

For each environment, the table provides the name, informations about the acquisition (camera and approximate trajectory length), number of triangles, video and screenshots, downloadable VR applications in three cases of headsets:
(1) standalone Oculus Quest (including Go and Quest 2),
(2) headsets compatible with Windows Mixed Reality (WMR) and linked to a PC, including Oculus headsets,
(3) headsets compatible with OpenVR (in SteamVR) and linked to a PC.
Two controllers with joysticks or touchpads are needed in all cases (except the Go). The applications are generated by Unity 2019.4 with preview packages. In theory, an application works for all headsets in its case, but in practice i cannot check every headset. Actually i experiment Oculus Quest and Go, Lenovo Explorer and HP Reverb G2, with a PC running Windows 10.

The table also provides links to explore reduced variants of the scans without headset. Only segments of the environments are textured in the reduced variants (with lower resolution than VR) due to a space limitation. Remainder of reduced variants has colors encoding surface normals.

Send me an email if you have a VR headset that does not work in one of the three cases above.

What's new in june 2022 ?

Now there are discontinuous motions (ie "teleportations") by using a virtual laser pointer to reduce the risk of VR-sickness. They can be used in conjunction with the continuous (translation) motions.
Joysticks and touchpads can be used interchangeably.
The location of the user in the environment is drawn in a map: a top view of the surface vertices with gray levels encoding vertex density. The map and the information pannel are hand-held and switchable.
The vertical offset of the user is adjustable to make easier various VR setting: sitting/standing/walking in the tracking area. This holds for both constrained and unconstrained VR motions. The former is better for continuous motions above the ground and the latter is better for discontinuous motions.
The Oculus Go has neither discontinuous motion nor adjustable vertical offset.
There is a new scan: Ruin in a forest.

Installation and control instructions

In case (1), a VR application is an APK file, which can be installed with the ADB command-line tool. More details are here.
In case (2), a VR application is in a zipped directory. First the PC must recognize the headset and the controllers, eg if you are in the default environment of WMR, or if the link cable is activated for the Quest. Then download and uncompress the zip file chosen in the table, run the .exe file. Adjust the vertical offset at the beginning if needed. Press simultaneously the two index triggers to quit the VR application.
In case (3), first install Steam and SteamVR on your PC, then install and run the VR application as in case (2). Note that SteamVR is automatically launched by running the .exe file. Press the Windows button or the Oculus button of the controllers to quit simultaneously the VR application and SteamVR.

In all cases, the control instructions of a VR application are those described by its shot link in the table above.

Alternative installation methods: use SideQuest for case (1) or SteamVR Workshop for case (3).

Photogrammetry Methods

Papers about methods and algorithms are available here.

In short: the structure-from-motion step includes a keyframe selection of the multi-camera video (that is good for matching and reconstruction) and a calibration refinement (from an initial rough guess), only features (interest points and edge curves) are reconstructed, then the surface is computed using topological constraints (manifoldness and lowered genus) and visibility constraints in a 3D Delaunay triangulation of the points (including sampled curves), last the texturing is done from the keyframes and the 3D coordinates are changed for VR.

Here are short names of the papers and their computation steps: calibration refinement (3DV'15, CVIU'17), structure-from-motion (BMVC'07, IVC'09), surface reconstruction (CVIU'18 improved by IC3D'19), curves are matched (PAMI'02) and integrated in the surface, texturing (IC3D'20), coordinate change for VR (IC3D'21). The methods are sometimes old since my main goal is to show what can be done using what i published. It is also easier to maintain software and methods that are well known for me.

Other details

The table below provides md5 sums, if needed.

Name md5sum for Standalone Quest md5sum for Oculus & WMR md5sum for OpenVR
Snowy Town 10dc817654a7b5b641f14329727d34c6 16ce76edeb1091869047c1988588fe59 39b77352a876477b3e4343b30cddcac6
Forest 41f28161ff6ef98dbaa296ec9066b8f1 0953ec8a03dab65eabeee1940564e717 f134e4b5f59f387e5cebe0ce01e41f32
Semi-medieval town aa8f0e33f1579f2b14e9b67ceb0064f1 77e71ebc80d2d3514ea124bb450026fc e0b69bb0daf33196b3fbee226e2d34e0
Campus 5360354907b46ec9a1c7fc6137d8f186 699a86e841d0b6ec6bd5e3947d26466a a3dadbe848c32b543b2e3a40056e6703
Verdon's canyon 5bd2e1ba9e2826af9432615cd1bdb06e 6ddcfb9dfc07d38b6e493b88b4af7e48 dcca08918a0f4aa8024900c4ed89330e
Basalt canyon 45ca8d10c3f1ebb6a8522d0197ba0101 313554a4b3099f7122d09cf3f0c159a6 782f39247c475c8a8e6b5f1057c7e11e
Collonges-la-Rouge 1d9f6b15b5d48323c3406c9214846edb 351aeb59a1f30ea7e2977e6c34b66950 e45efe94c0af83eaca209373d84330fd
Ruin in a forest 191ba75fe6be73d9e4b8ce8d2c207617 d8605b06ce72e8df37aa0f5e18a798e4 8153b6a4c525b5f9731671bcb1924f0e


Privacy policies: the applications in this page do not process data of the VR user.