Panoptic Segmentation API & Person Detection Plugin
The panoptic segmentation API allows for panoptic segmentation to be run as part of the position SLAM system, providing semantic segmentation, 3D bounding boxes and the position of object instances relative to the camera in real time. In this release, as part of the API, we provide a person detection plugin that can segment and detect people in the environment. Custom plugins will be supported in the future. For more information, see Panoptic Segmentation API.
ROS 2 Humble and Ubuntu 22.04
This release adds the
ros-humble-slamcore-rospackage which provides the Slamcore ROS 2 Wrapper for ROS 2 Humble on x86_64 and arm64 systems.
Packages are also available for the Slamcore C++ API for Ubuntu 22.04 on both architectures and the Slamcore Tools for x86_64.
NVIDIA Jetson Orin
This release adds support for all NVIDIA Jetson Orin platforms, including the Orin AGX, Orin NX and Orin Nano running JetPack 5.
A new default SLAM configuration, along with algorithmic and performance improvements in this release contribute to improved tracking by integrating more observations without compromising performance.
UX and UI Improvements
This release brings various improvements to the online Slamcore Portal user interface, allowing users to add a custom tag to their cameras, as well as displaying the camera calibration being used. Improvements have also been made to our CLI tools, providing greater visibility of the system by displaying the camera serial number and presets being used when running our software.
Please be aware of the following changes and incompatibilities with previous versions.
Coordinate Frame Convention Change
This affects trajectory generation and may affect existing static transforms used when integrating Slamcore into your system. For more information, see Migrating from Earlier API Versions.
Rebuilt for latest ROS 2 Foxy dependencies
ros-foxy-slamcore-rospackages have been rebuilt for compatibility with the most recent versions of ROS 2 Foxy dependencies.
Unlabelled metadata in API
Some items in the
metadatastream were published without corresponding labels. These have been removed in all the upgraded packages.
Slamcore’s new look
The GUI tools have been updated to reflect Slamcore’s new visual branding.
Simultaneous SLAM and recording
The new version of the software allows users to capture a dataset while running SLAM. This allows much more rapid development, evaluation and debugging.
We are pleased to be able to offer a Python API alongside the existing C++ API for faster prototyping and integration. Please see the Python API documentation for details.
NVIDIA JetPack 5 and ROS 2 Galactic
slamcore-devpackages are now available for JetPack 5. The new
ros-galactic-slamcore-rospackage provides integration with ROS 2 Galactic on x86_64 and arm64 platforms.
SAFE_V2 Feature Extractor
This version introduces the
SAFE_V2feature extractor which is much faster and more power-efficient especially on low-power platforms while maintaining comparable accuracy. Please see the frontend parameters section for how to enable it in your integration.
Please be aware of the following changes and incompatibilities with previous versions.
Session files from earlier versions are no longer compatible. New session files will need to be generated from mapping datasets where available.
JSON configurations from earlier versions may be incompatible. The
slamcore-toolspackage contains a utility to migrate these for use with the current release. For more information, see Migrating from Earlier API Versions.
ros-galactic-slamcore-rospackages have a new dependency on the
slamcore-devpackage. Please ensure you install both the ROS package and
slamcore-devfrom the same release.
The ROS 2
world_frameparameter/launch argument has been renamed to
The “Tracking Status” no longer reports Relocalisation or Loop Closure events. These are now part of a “SLAM Events” stream in the C++ API or ROS topic. See Tracking Information for more details.
Extended functionality in the C++ API
Users can now configure, access and use height mapping, wheel odometry support and local point cloud functionality directly from the C++ API. Also, with
TrajectorySubsystemusers can now export trajectories directly from the C++ API.
Additional system configuration and presets
This release provides additional settings and presets that allows users more control of the SLAM configuration as well as RealSense camera settings. This includes the ability to set feature extraction ROI, camera auto-exposure settings and RealSense depth presets.
Support for OpenLORIS and TUM VI datasets
Slamcore Visualiser and Slamcore Dataset Processor can now be run on the OpenLORIS and TUM-VI datasets.
Dataset compression tool
To reduce the size of datasets, a new tool for compressing datasets has been introduced.
Slamcore Tools improvements
There have been several improvement to the Slamcore Visualiser, including the ability to export trajectories, display current frame rate, processing time as well as camera orientation in the Euler angles. Users can also visualise the depth point cloud and enable camera “Chase mode”.
Slamcore Dataset Processor provides more detailed feedback.
Support for ground robots
A new, more robust method for estimating the main plane of motion for mobile ground robots has been introduced.
Performance in multisession mode is improved by enabling loop closure support.
Publish local point cloud in ROS 2.
Access to RealSense camera streams
Users can now enable and access all the RealSense camera streams, including color.
This minor release improves the implementation of multisession for better performance and improves the likelihood of relocalisation.
Updated Intel RealSense Camera Support
Added support for RealSense cameras that do not offer a 250Hz accelerometer stream.
ROS Dataset Recorder Killed Before Completion
In previous versions of the Slamcore ROS Wrapper when the dataset recorder was stopped using a
SIGINT, it was possible that the process of finalising dataset writing would exceed the shutdown time limit of the monitoring
roslaunchprocess. This causes
roslaunchto escalate to a
SIGTERMwhich will terminate the dataset recorder process and results in a corrupt dataset.
The finalisation step has been removed in this version for all dataset recording tools.
Ubuntu 20.04 packages
slamcore-devpackages are now available for Ubuntu 20.04 (Focal Fossa) on x86_64.
Users can now download the
slamcore-dev-jetsonpackages which contain headers, a shared library and example code to build applications using Slamcore software.
The API supports the RealSense cameras and operates in SLAM, visual odometry or Multisession modes.
Please see the C++ API for more details.
Wider NVIDIA Jetson support
Slamcore packages can now be used on NVIDIA Jetson Xavier AGX as well as the previously supported NVIDIA Jetson Xavier NX.
Memory usage and performance
Memory usage and speed in session save have both been improved in this release, especially on large sessions.
The Slamcore Visualiser has been enhanced to show multiple image streams with the 3D position and map view. Additionally, a first-person “AR” view is also available.
In the Slamcore ROS Wrapper, all topics and service names are now snake_case.
In previous versions the
--fpscommand-line flag was not usable. The issue has been resolved in this release.
The Slamcore Visualiser and Slamcore ROS Wrapper both now support generating height maps when using RealSense Depth cameras. These height maps can be converted to occupancy maps and tuned by hand using the new Slamcore Session Explorer tool.
When running in Multisession Relocalisation mode, the Slamcore Visualiser and Slamcore ROS Wrapper will display the map saved in the session file and the sensor’s pose relative to it.
Intel RealSense D455 Support
All the Slamcore software now supports the Intel RealSense D455 on x86 platforms.
The Slamcore ROS Wrapper has been improved in a number of ways including now publishing a
/slamcore/odomtopic which provides pose, angular velocity and twist of the sensor without discontinuous jumps due to loop closure and relocalisation.
Additionally, the Slamcore ROS Wrapper now allows users to use the RGB stream. Please contact Slamcore to enable this feature.
In previous versions of the Slamcore ROS Wrapper it was not possible to enable the depth stream when running on the NVIDIA Jetson Xavier NX. The issue has been resolved in this release.
The Slamcore ROS Wrapper now allows users to use wheel odometry. To do so, users will need to record a number of datasets for calibration using the procedure in the wheel odometry tutorial and provide these to Slamcore to perform the calibration.
With the calibration file from Slamcore, users will be able to process datasets containing wheel odometry using all the current tools:
Slamcore Dataset Processor, and
Slamcore ROS Wrapper.
Users will also be able to run live SLAM with wheel odometry in the Slamcore ROS Wrapper.
slamcore-position-jetsonnxDebian packages now include configurations to tune SLAM for several common use-cases. These are usable with the Slamcore Visualiser Slamcore Dataset Processor and Slamcore ROS Wrapper.
The Slamcore Dataset Processor and Slamcore Dataset Writer CLI tools have been added to the main .deb.
Save and Load a session in the Slamcore ROS Wrapper
The Slamcore ROS Wrapper now allows users to save and load a session file, similar to save and load functionality in Slamcore Visualiser.
Map optimisation performance during session save has been significantly improved.
SLAM performance has also been improved.
The interface to the ROS node has been redesigned for better integration and robotics deployment.
The node will now start as soon as it is launched and will not publish pose information when it is not successfully tracking.
Stream switches have also been removed in favour of directly detecting subscribers.
The RQT window will no longer be launched by default in visualisations as the node starts automatically. RQT can be launched manually if required.
Session saving file format
Sessions created with this release can not be used with earlier releases. This is due to essential changes format required to support future features. Sessions created with older versions of the software are still supported.
D435i timestamp wrap-around error
The range of the hardware timestamps provided by
librealsensecan overflow within reasonable SLAM session time. This issue is resolved in this release.
Minor changes for consistency and usability
slamcore_positiontool is now provided as
Logging verbosity has been reduced.
ROS Diagnostic Topic
A new diagnostic topic has been added in ROS to allow querying the status of SLAM.
Checking for Correct Installation of RealSense SDK 2.0
Checks have been added to verify that the Intel RealSense SDK 2.0 has been correctly installed. An error message is generated if this is detected. For the Visualiser a popup window is displayed. For ROS a message is displayed and execution is stopped.
Session saving file format
Changes have been made to the format for saved sessions. Sessions created with this release can not be used with earlier releases of our software. Sessions created with older versions of the software are still supported.
Improvements have been made to the relocalisation mode, when positioning within an existing map.
Additional telemetry is being collected about the state of the SLAM system to better diagnose issues and improve the software in the future.
Restarting SLAM in Visualiser
Previously stopping a SLAM session and then starting again without resetting could lead to unexpected and poor position estimations. This behaviour is corrected.
Multiple map(session) Saving
During mapping runs it is now possible to save maps and then continue building the sparse map.
Save and Load a session in the Slamcore Position Visualiser
The Slamcore Position Visualiser now allows users to save a session file after running SLAM. The file is an optimised representation of the explored environment and contains the accumulated sparse map.
Loading the session file puts the Slamcore Position Visualiser into
MultiSession Localisationmode, where the system will localise against the loaded map and report its pose in the map’s frame.
This mode visually distinguishes between currently recognised landmarks loaded from the session file, other loaded landmarks, and temporary landmarks to visualise the operation of
Saving and loading sessions is supported for the Mynteye2 and D435i, datasets recorded with either camera using the Slamcore Dataset Recorder, as well as sequences from the EuRoC MAV dataset.
Several algorithmic improvements are included to improve detection and landmark merging.
The Slamcore Position Visualiser, Slamcore Dataset Recorder and Slamcore ROS wrapper now all support the depth stream from the Intel RealSense Depth Camera D435i. Depth is enabled by default in all the applications.
Depth requires additional bandwidth and processing time to collect and display. Also, under some conditions, it can degrade the quality of SLAM. For these reasons it is simple to disable in all the applications. Please refer to their corresponding sections for how to do so.
Additional topics in ROS
The Slamcore ROS wrapper now publishes additional topics to aid integration:
/slamcore/accel- accelerometer data;
/slamcore/gyro- gyroscope data;
/slamcore/<various>/camera_info- camera intrinsics; and,
/tf_static- transformations between various coordinate frames in the system.
Slamcore Dataset Recorder performance improvement
The performance of the Slamcore Dataset Recorder has been significantly improved. This will be most visible on less powerful systems.
D435i dataset metadata corruption
When recording a dataset using the Slamcore Dataset Recorder from a D435i, it was possible to corrupt the received metadata, resulting in either a fatal error or corrupted dataset. This issue has been resolved in the current release.
The behaviour of the Slamcore Dataset Recorder is now to record datasets in the directory where it is launched. For example, to record into
/home/slamcore/datasets, run:$ cd /home/slamcore/datasets $ slamcore_dataset_recorder
SLAM from user datasets
The Slamcore Position Visualiser and the Slamcore ROS Wrapper now support running SLAM from a dataset captured using the Slamcore Dataset Recorder.
Experimental support for Intel RealSense Depth Camera D435i
The Slamcore Position Visualiser, Slamcore Dataset Recorder and Slamcore ROS wrapper now all support the Intel RealSense Depth Camera D435i.
Configurable algorithm parameters for Slamcore position
The Slamcore Position Visualiser and Slamcore ROS Wrapper now allow users to specify:
The positioning mode: SLAM or odometry.
The maximum number of features per frame.
Separate packages for SLAM and Visualisation in ROS Melodic
A standalone package for the Slamcore position ROS Melodic Wrapper allows integration into existing ROS installations with minimal dependencies.
An additional visualisations package provides visual tools for Slamcore position using ROS’s
Intel processor of the Haswell or later architecture (with AVX2 extensions)
8GB of RAM
USB 3 port for the camera
ROS Melodic for the use of the API features
Internet connection at the start of each session.
Supported Camera Sensors
Pre-calibrated Mynteye2 sensors provided by Slamcore.
Visualiser. This stand-alone application shows: trajectory, camera view, tracking status, distance travelled, map features.
Dataset Recorder. This tool allows the capturing of detailed datasets which can be sent to Slamcore for analysis if the user requires it.
ROS Melodic Wrapper. This allows integration of the Slamcore position tracking into an existing ROS installation.
Six degrees-of-freedom tracking using visual-inertial SLAM.
Loop closure and relocalisation.
Six degrees-of-freedom pose.
Angular and linear velocities of the sensor.
Number of features tracked.