Masking out static objects
Ideally, the configured reconstruction volume only contains the subject to be filmed, and no other objects. In practice, this is not always feasible. For example, filming a speaker while recording high-quality audio may require putting a microphone close to the speaker’s mouth.
For such cases, it is possible to mask out static objects such that they will not appear in reconstructions.
Note
Currently, masks can only be applied when processing an existing recording on disk, not to live snapshots or directly reconstructed videos.
For best results, make sure that:
The object to be masked out does not move, allowing to keep the mask small.
The object to be masked out does not significantly occlude the subject to be filmed, allowing for good camera views on the subject.
For preparing the mask, first take a snapshot of the scene with the final recording setup. The created mask will be applicable to all recordings created with this recording setup; since changing the setup (either by changing individual configurations such as the floor calibration, or by going through the guided recording setup again) may change the resulting coordinate system, this could necessitate updating the mask.
Locate the snapshot file on disk and open it in a 3D modeling program, for example in Blender. Add a new 3D object which envelops the 3D region that you would like to mask out. Leave some space around the object to account for possible measurement noise, but not a large amount, since the mask object will act like an occluder between the cameras and the subject.
When done modeling the mask object(s), make sure of the following:
If you created multiple objects, merge all of their triangles into a single object and delete all other objects (such as the original snapshot mesh).
The object must have a manifold surface without self-intersections. If you have intersecting objects, this may be resolved with the Boolean Union operation.
Apply any potential object transformation to its vertices, resulting in an identity transformation on the object.
Then, export the object in GLB format.
Locate the recording(s) on disk which you would like to apply the mask to.
In each recording’s directory, save the mask mesh with the name mask_mesh.glb
.
Any reconstruction created of these recordings will then take the mask into account.