Orbbec Femto Bolt
ScannedReality Studio supports the Femto Bolt camera that was developed by Orbbec together with Microsoft and is the de-facto successor to the Microsoft Azure Kinect.
This documentation page covers Femto Bolt-specific topics and is structured as follows:
The first two sections give an overview of the requirements and constraints to consider when planning to buy Femto Bolt cameras:
The next sections discuss camera synchronization and cable extension with the respective cables required for each task, followed by a parts list summary:
Then, assuming that you obtained one or more Femto Bolt cameras and the necessary cables, the setup steps to prepare for video recording are described. They are split into first-time and repeat setup steps:
The documentation page ends with a reference of the camera settings that are available for Femto Bolt cameras in ScannedReality Studio, as well as potential troubleshooting steps in case there are issues with the cameras:
PC requirements
Operating system: Femto Bolt cameras currently require using Windows for reliable operation.
Connection: The Femto Bolt has a USB-C port for its data connection to the PC. As of March 2024, the cameras ship with a USB-C to USB-A cable that is ca. 1.5 meters long. Unless you plan to record only very small scenes, this length is too short for practical use for volumetric video recording. See the section on cable extension below for notes on extending the cable length.
The USB connections between the cameras and the PC must allow for sufficient bandwidth to stream video from all cameras at the same time. Bottlenecks can happen in multiple places:
Separate USB ports on a PC may use the same USB host controller internally, which may limit the number of cameras that work on these ports at the same time. This is not always clear from the specifications and may have to be tried out.
If a PC does not offer a sufficient number of suitable USB ports, it may be possible to extend it with additional ports via PCIe expansion cards. We verified that this four-port card, which comes with a separate USB controller for each port, allows for connecting four Femto Bolt cameras.
The camera’s video data must be transferred to the CPU for processing it. Some USB ports and PCIe slots are connected to the CPU directly, while others share a common connection to the CPU via the motherboard’s chipset. This common connection may theoretically become a bottleneck as well. In case this happens, cameras should be moved to other USB ports that have a direct connection to the CPU, if possible.
Processing: The Femto Bolt requires the GPU of the host computer to process the depth images as they are being recorded. Any modern mid- to high-end GPU should be sufficient for this.
The CPU speed must be sufficient for handling the high USB bandwidth in order to prevent frame drops. For using 10 cameras, we recommend a high-end 16-core CPU.
Disk write speed: ScannedReality Studio uses the Femto Bolt’s native MJPEG color stream, which means that the video bandwidth varies depending on the image content.
To give a rough idea of the write bandwidth that is required in practice, we tested several typical operation modes and report the maximum bandwidths that we observed below. Please be aware that these are rough estimates which are meant to be used with a safety margin.
Depth: Narrow FOV unbinned, Color: 3840 x 2160 pixels MJPEG, 30 fps: 107 MB / second
Depth: Narrow FOV unbinned, Color: 2560 x 1440 pixels MJPEG, 30 fps: 45 MB / second
Depth: Narrow FOV unbinned, Color: 1920 x 1080 pixels MJPEG, 30 fps: 37 MB / second
Other requirements
Power connection: While it is possible to power the camera by using a USB-C to USB-C cable for both power and data, we do not recommend this because according to the camera’s datasheet, this limits the depth camera to at most 640 x 576 pixels and the color camera to at most 1920 x 1080 pixels.
To be able to use all camera resolutions, or if using a USB-C to USB-A cable for data, a separate power connection must be used. As of March 2024, the cameras ship with a power cable that is ca. 1.5 meters long. Just like the data cable, this power cable is in general too short for volumetric video recording. Standard power extension cables may be used to extend this.
Temperature range: Keep in mind that the camera must only be operated in temperatures between 10 and 25 degrees Celsius.
Sunlight: Sunlight may affect the camera’s active infrared sensing.
Interference: Cameras whose fields-of-view overlap must be synchronized to avoid interference. With typical camera operation modes and timings, at most 10 cameras can be synchronized tightly while avoiding interference. See the next section on synchronization below.
Synchronization
All Femto Bolt cameras whose fields-of-view overlap must be synchronized to avoid interference of their active sensing. Up to 10 cameras with overlapping field-of-view can be tightly synchronized while avoiding interference.
To synchronize the devices, they must be connected to Orbbec’s Sync Hub Pro. A single Sync Hub Pro supports synchronization of up to nine cameras. For ten or more cameras, two Sync Hub Pro must be used which need to be connected with a CAT5e or better LAN cable via their “Hub to Hub” ports. Two hubs allow connecting up to 17 cameras, however, only up to 10 work without taking special precautions to avoid interference.
For each camera, a sync adapter is needed which is connected to the camera (each Sync Hub Pro already includes two adapters). The Sync Adapter then must be connected with a CAT5e or better LAN cable to the Sync Hub. One arbitrarily chosen camera is connected to a Hub’s “Primary In” port, while all other cameras are connected to the Hubs’ secondary ports.
All LAN cables should be less than 10 meters in length.
The Sync Hub Pro support an optional USB-C power supply that is not included. If such a power supply is used, the synchronization voltage may be switched from the default 1.8V to 3.3V or 5V. According to Orbbec support, this is only required if external devices are used within the synchronization network that require these voltages as synchronization trigger, and it does not affect the possible cable lengths for synchronization, or the signal reliability.
Cable extension
Both the data cable and the power cable included with the camera (as of March 2024) are ca. 1.5 meters long and are thus in general too short for volumetric video recording.
For extending the USB data cable, notice that active USB extension cables are required for reliable connections over longer distances. We have successfully tested StarTech’s 10 meter active extension cable. In our tests, using the optional power supplies that come with these cables was not required.
For extending the power cable, standard power extension cables can be used.
Note
In contrast to the Azure Kinect, the Femto Bolt’s power supplies (as shipped in March 2024) do not have USB ports, thus an extension via USB cables is not possible for them.
Parts list
This section summarizes the parts that are needed for using Femto Bolt cameras in a volumetric recording system based on the detailed description in the previous sections.
Per camera, the following parts are needed:
Power extension cable,
Active USB extension cable (verified product),
LAN cable for connection to the Sync Hub Pro (CAT 5e or better, less than 10 meters long).
The following is needed in addition:
For 2 to 9 cameras: 1x Sync Hub Pro.
For 10 to 17 cameras: 2x Sync Hub Pro and a LAN cable for the Hub to Hub connection (CAT 5e or better, less than 10 meters long).
First-time setup
This section describes first-time setup steps for using Femto Bolt cameras.
Updating firmware
We strongly recommend to keep the camera firmware updated to the latest version. See Orbbec’s instructions for how to update the firmware on Femto Bolt devices.
Windows-specific setup
For image metadata support on Windows, each camera must be used at least once with Orbbec Viewer. If you use Orbbec Viewer to update the firmware on all cameras, then this will happen automatically and no further actions are necessary.
However, if you use the cameras on a new PC without updating their firmware on this PC, then please use each camera once with Orbbec Viewer on this PC to complete this setup step before using them with ScannedReality Studio.
ScannedReality Studio will warn you if metadata support is not enabled for all connected cameras.
Linux-specific setup
Important
These instructions are given for future reference. As of now, Femto Bolt cameras require using Windows for reliable operation.
To use Femto Bolt cameras on Linux without root rights, udev rules must be added. Orbbec’s instructions for this may for example be followed by running these commands in a terminal:
curl -sSL https://github.com/orbbec/OrbbecSDK/raw/main/misc/scripts/99-obsensor-libusb.rules > /tmp/99-obsensor-libusb.rules
sudo mv /tmp/99-obsensor-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload && sudo udevadm trigger
Furthermore, if you want to use more than a single camera on one PC at the same time, then you most likely need to increase the configured kernel USB memory. Please see Orbbec’s instructions for how to do this.
Recording setup
This section describes how to plug in and configure your Femto Bolt cameras to prepare for video recording, assuming that you obtained the necessary hardware according to the sections above, and followed the first-time setup steps.
The image below shows the cable connection scheme for up to nine cameras:
Cabling scheme for up to nine synchronized Femto Bolt cameras.
For ten or more cameras, two Sync Hub Pro must be used, with the two hubs connected by a LAN cable, and the secondary cameras arbitrarily distributed over the two hubs:
Cabling scheme for ten or more synchronized Femto Bolt cameras.
Each camera will have the following connections:
USB data connection to the PC,
Power connection to a power outlet,
Sync port connection to the Primary or a Secondary sync port on a Sync Hub Pro. This connection is made via a sync adapter and a CAT5e or better LAN cable. One arbitrary camera is connected to the “Primary In” port of one hub, and the other cameras are connected to any of the available secondary ports.
With these connections in place, you may start the guided recording setup.
In contrast to the Azure Kinect, Femto Bolt cameras cannot sense whether they are connected to a primary or secondary sync port. Thus, in the device configuration setup step, the camera sync mode must be manually configured:
If you use a single camera only, set it to “standalone”.
If you use synchronized cameras, set each of them to “primary” or “secondary” as described below.
To identify the primary camera, please look up the serial number of the camera that is connected to the “Primary In” sync port. The serial number can be found here on the bottom side of the camera:
Location of the serial number on the bottom side of a Femto Bolt camera. The serial number can also be obtained by scanning the QR code there.
Configure the device with this serial number as primary, and all other devices as secondary, for example like this:
Example of sync mode configuration for 10 synchronized Femto Bolt cameras.
The program remembers this configuration, thus it only needs to be repeated if changing the sync cabling.
After configuring the sync mode, the sensors can be started.
Settings reference
This section describes the settings that are available for Femto Bolt cameras within ScannedReality Studio.
Settings regarding the cameras’ operating modes can be configured both in the settings tab while the sensors are not active, and in the device configuration screen as part of the guided recording setup.
All other settings can be configured in the sensor configuration screen while the cameras are running, either as part of the guided recording setup, or by going there from the recording screen.
Device configuration
The color and depth cameras of the Femto Bolt can be operated in different modes having different image resolutions, fields-of-view, frame rates, and ranges.
For detailed specifications, please see the Femto Bolt hardware specifications.
We generally recommend to use the unbinned narrow field-of-view mode for depth, and the highest possible color resolution that works with your hardware setup.
As default, ScannedReality Studio uses the unbinned narrow field-of-view depth mode with the maximum color resolution and fps (3840 x 2160 pixels at 30 fps).
The subordinate delay stepping determines the trigger delay between successive synchronized cameras. There should usually be no need to change this.
For a description of depth bias compensation, see reconstruction settings.
Sensor configuration
In this list of settings, the upper settings correspond to camera driver settings, and additional software color correction settings are available below.
The software settings are applied by ScannedReality Studio on top of the raw images received from the camera driver. This happens at the time when the images are being processed and thus does not affect how they are written into recordings. For existing recordings, these settings can thus still be edited later in the recording details screen.
However, using the software settings will reduce the color precision. If possible, it is therefore preferable to achieve the desired effects by improving the lighting during recording.
In our experience with the Femto Bolt, with yellow-ish indoor lighting it may happen that no camera white balance setting produces a good image. In this case, the white balance factors in the software color correction settings must be used to achieve good white balance.
The powerline frequency setting should be set to the line frequency used in your area to avoid flickering under electrical lighting. This can be looked up online.
Troubleshooting
If you experience generic camera issues, see Orbbec’s Femto Bolt troubleshooting page.
If the cameras generally work but you experience a significant number of dropped frames, consider the following possibilities:
Do several USB ports that are used by the cameras share bandwidth internally? If possible, try it with other USB ports that do not share bandwidth.
Does the GPU on the host PC not keep up with processing the depth images of all cameras?
Does the CPU on the host PC not keep up with receiving data from USB?