Setting up a Camera

Setting up an Intel® RealSense™ Camera

Warning

To use the D435i or D455 depth camera, most x64 systems will require building and activating kernel modules to complete the setup of the Intel RealSense SDK 2.0 features required by Slamcore software.

The D435i and D455 depth cameras also require USB 3.0 connection.

To start using our software with live camera data you will need to do the following:

  1. Install Intel® RealSense™ Kernel Modules

  2. Verify RealSense Setup

  3. Install the Camera Firmware

  4. Register Your Camera

Install Intel® RealSense™ Kernel Modules

Check your machine’s kernel version by running uname -r in a terminal window, and follow the steps below to set up the camera depending on your system:

First check whether Intel RealSense’s DKMS kernel drivers package supports your system’s Ubuntu kernel version on their package installation site. For supported kernel versions, install the pre-built packages following Intel RealSense’s instructions.

For older kernels not supported by Intel RealSense DKMS package, build and install the librealsense2 module package from source.

To download the sources without cloning the repository, use the command:

$ wget https://github.com/IntelRealSense/librealsense/archive/master.zip && unzip master.zip

Warning

Ensure that your Intel RealSense camera is unplugged when running the scripts. Reboot your system when the patch is completed.

We open source a DKMS package to provide full support for Intel RealSense cameras on the all NVIDIA Jetson platforms running L4T between r32.4.4 and r32.6.1.

Note

Installation of this DKMS package is not required to use an Intel RealSense D435i camera with NVIDIA Jetson platforms.

Install the pre-built packages following the details on Slamcore’s DKMS package installation site.

Verify RealSense Setup

After following the setup instructions it is important to make sure that everything has been set up correctly. Firstly, you should confirm the camera firmware version matches the one required.

  1. Install RealSense’s SDK if you have not done so.

  2. Start up the realsense-viewer program.

  3. Connect your RealSense camera.

  4. Find the firmware version of the camera:

    • Go to Info (in the connected cameras side panel) > Firmware Version

    • Make sure it corresponds to one of the required versions: 5.12.3.0 - 5.12.15.50. This may be displayed in the form 05.12.12.100

Now confirm that all of the data streams are being received correctly from the camera.

  1. Start all three of the streams: Stereo Module, RGB Camera and Motion Module and ensure the 2D view is selected.

  2. On each panel, click the “Show frame metadata” icon.

  3. Check that none of the panel display the message “Per-frame metadata is not enabled at the OS level! Please follow the installation guide for details.”.

    This message on any of the panels indicates that the kernel module patches are not correctly installed or activated. If this happens, check that the patches were installed correctly according to the instructions for your kernel version.

Note

If the message still appears, contact Slamcore support at support@slamcore.com providing the following details:

  • The machine you are using,

  • Your system’s Ubuntu kernel version,

  • The camera firmware version,

  • The link to the kernel patching instructions used,

  • The patch version used from Intel RealSense’s repository (if applicable).

Install the Camera Firmware

Follow the instructions provided by Intel RealSense to install one of the camera firmware versions 5.12.3.0 - 5.12.15.50.

Register Your Camera

Finally, register your camera to start using it with the Slamcore software. You will need the serial number which can be found printed on the camera or the box. Alternatively you can connect your camera and run:

$ rs-enumerate-devices | grep Serial

Two serial numbers will be displayed, take the Serial Number.

Login to your Slamcore account at the Slamcore Portal.

  1. Click on the Manage Cameras option.

  2. Click on the Activate New Camera link.

  3. Select your camera type (RealSense D435i, RealSense D455, OAK-D S2 Fixed-Focus).

  4. Enter your camera serial number.

  5. Click Add Camera.

RealSense Camera Calibration (Optional)

While the RealSense cameras can work out-of-the-box without a custom camera calibration, we recommend performing the RealSense Camera Calibration to see potential improvements in accuracy and robustness of SLAM, especially before you do a thorough accuracy evaluation of our system.

Set up an OAK-D S2 Camera

Note

Our support for the Luxonis OAK-D S2 is experimental and only on demand. Please contact us directly at support@slamcore.com if you would like to use the camera.

We currently only support the camera in Fixed-Focus mode and only the stereo pair. We do not support its color or depth streams.

To start using our software with live camera data from the OAK-D cameras you will need to do the following:

Set Udev Rules on your Machine

When first attempting to access the OAK-D camera, you may get permission issues from Luxonis’ DepthAI:

Insufficient permissions to communicate with X_LINK_UNBOOTED device with name
"1.1". Make sure udev rules are set

Resolve this by setting the udev rules according to Luxonis’ DepthAI documentation, then unplug the camera and plug it back into USB afterwards:

echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
sudo udevadm control --reload-rules && sudo udevadm trigger

Register the Camera

As the OAK-D S2 Fixed-Focus camera is currently supported only on demand, you must contact us at support@slamcore.com to register the camera. Please email us with the following details:

  • Slamcore account email address

  • Your OAK-D camera serial number (typically 18-digit long)

OAK-D Camera Calibration

We will reply to your email with the next steps to calibrate your camera before using it with our SLAM.