RealSense Camera Calibration

This tutorial provides a guide to capture sequences that can be used to enhance the calibration of your Intel RealSense D435i or D455. During the calibration, the following parameters are estimated from the visual-inertial datasets:

  • Camera intrinsics and distortion parameters

  • Camera to IMU (Sensor) extrinsics

  • Time offset between the camera and IMU measurements

  • Noise densities for the accelerometer and gyro IMU biases

  • Accelerometer and gyro IMU biases priors

Once the calibration is applied to your account, you may see improvements in the accuracy and robustness of SLAM when working with the Slamcore SDK.

Requirements

This tutorial assumes that you have the following:

  • An Intel RealSense D435i or D455 plugged into a portable local machine via USB

  • The latest version of our Slamcore SDK installed on the local machine

  • A RealSense box (or substitute object) to use as a visual reference point in the scene

Recording Environment

To perform the VI calibration customised to your camera, we require two sequences recorded from that specific camera, with visual and inertial data. A guideline for the ideal recording environment:

  • An office-sized recording space,

  • Has walls with abundant visual textures and clutter, and

  • Is well-illuminated.

Note

If blank/texture-less walls, ceiling or floor are present, avoid pointing the camera at them.

If only a large warehouse-sized space is available, limit your movements to one corner that contains visual textures and clutter in the scene, and avoid pointing the camera far into the distance.

The goal is to record sequences that are easy for visual-inertial SLAM to perform well in.

_images/camera_calibration_record_env.png

Fig. 60 Recording Environment for Calibration Sequences

Capturing a Sequence

Record 2 sequences of about 1-2 minutes duration each, with the camera performing slow and steady motion around the RealSense box (or substitute object).

Before beginning the capture, you may want to read through the full instructions below and take a look at our demonstration of a calibration sequence capture:

  1. Ensure your camera and laptop setup is portable. You may find it easier to hold the camera with its tripod attached.

  2. Place the camera and the RealSense box on a convenient surface, 1-2 m apart.

  3. Mark this “starting position” of the camera with duct tape, or as accurately as possible. You will have to return the camera to the spot at the end of the recording.

  4. Launch the Dataset Recorder GUI or CLI tool, with the depth stream turned off.

    $ slamcore_dataset_recorder --no-depth

    (More details at Slamcore Dataset Recorder)

    $ slamcore_dataset_recorder_cli --no-depth -o <output-directory>

    (More details at Dataset Recorder CLI)

    $ source /opt/ros/foxy/setup.bash

$ ros2 launch slamcore_slam run_dataset_recorder.launch.py \
>   override_realsense_depth:=true \
>   realsense_depth_override_value:=false \
>   output_dir:=<output_directory>

    (More details at Usage - Dataset Recording)

  5. Imagine a spherical surface in the recording space, with the RealSense box in the centre of the sphere.

    _images/camera_calibration_record_distance.png

    Fig. 61 Recording Setup

  6. Begin recording. Pick up the camera from the start marker, slowly and smoothly move it along the spherical surface with the camera facing the RealSense box at all times. Ensure that the trajectory does not contain sudden jerks or rotations.

  7. Perform the following arcs multiple times around the RealSense box, arching as far as possible while avoiding blank walls in the camera’s field of view at all times:

    _images/camera_calibration_record_movement.png

  8. The sequence should be around 1-2 minutes long. You may perform more arcs from a different spot to fill up the time.

  9. Return the camera to the start position marked previously, and stop the recording.

  10. Repeat the steps above to record a second dataset. You may use a different starting position or room that meets the requirements.

When you’re done, you should have two datasets in the save directory, which is /home/<user>/Documents by default.

Send Files to Slamcore for Calibration

Dataset Folder Structure

Change the dataset folder names to end up with the following dataset structure:

VI_calib0/
├── capture_info.json
├── imu0/
├── ir0/
└── ir1/
VI_calib1/

Dataset Validation

Before sending the datasets to Slamcore, ensure that the sequences recorded are suitable for calibration by running SLAM on them.

Launch the Slamcore Visualiser:

$ slamcore_visualiser dataset -u <path/to/dataset>

For all the datasets, ensure that the trajectories look reasonable with no large jumps and that loop closures are present (if possible).

Compress the files

Compress each individual sequence into a zip file each for ease of file sharing. For example, run in your terminal window:

# rename the dataset directory
$ mv <path/to/dataset> <path/to/VI_calib0>

# enter the parent directory of the dataset e.g. /home/<user>/Documents
$ cd <path/to/parent/directory>

# compress the dataset
$ tar -czvf VI_calib0.tar.gz VI_calib0/

You will have two compressed datasets:

VI_calib0.tar.gz
VI_calib1.tar.gz

Upload the sequences

Upload the compressed files to Slamcore’s MASV portal below or at https://slamcore.portal.massive.app in one single package, filling in the following details:

  • Your email

  • Package name: <company_name>_<D435i/D455>_<serial_number>

  • Message: any details or description of the data provided

Finally, click SEND. A page indicating “Transfer Complete” will be shown if the upload is successful. You can email us at support@slamcore.com about the data upload.

_images/camera_calibration_upload_to_masv.png

Fig. 62 The process to upload to Slamcore’s MASV portal

The MASV portal has been embedded below and can be used to share upload and share the files.

_images/slamcore_logo_primary1.png

Calibration Confirmation

Once received, we may take up to 1-2 working days to generate the calibration and results. We will email you for a confirmation and will apply the calibration permanently to your camera on your Slamcore account via https://portal.slamcore.com. As all changes are applied in the back end, you will not be able to see the calibration.

The calibration data will be effective the next time you run Slamcore.

Note

Please email us if you wish to revert the calibration data to the default due to poor results from the new calibration data.