Multi-Agent Localisation

Warning

Multi-Agent Localisation is experimental.

The tutorial for Localisation Mode takes you through the step to create a master point-cloud map, save it and then calculate the live position of a robot in the reference frame of this map. Because this point-cloud map is created using the naturally occurring features of the environment, it is unique to the space and not the machine. This means that any machine with compatible hardware will be able to localise itself using the same pre-built point-cloud. Because the positioning software only uses the passive stereo cameras and IMU from the RealSense, there is technically no limit to the number of machines that can operate simultaneously in the same environment.

The process for multi-agent localisation is the same as described in the tutorial for Localisation Mode. The only difference is that once the session file for the point-cloud map is saved, you can now share with any machine with compatible hardware with the Slamcore software installed.

Step 1 - Create a Master Configuration File

You can change some of Slamcore’s default parameters using a configuration file. Download multi_agent_master.json to a convenient directory, e.g. ~/slamcore/custom/multi_agent_master.json.

The ProfileName parameter can be set to any name that you wish to be displayed on the Slamcore Visualiser when the configuration file is being used.

The positioning mode is controlled by the PositioningMode parameter and can be set to SLAM or ODOMETRY_ONLY. For this tutorial we will set it to SLAM.

You can also refer to Configuration Overview to set other parameters which will affect the performance of the system. For example, in this tutorial we set the number of features detected on each frame which are used to triangulate the position of the sensor. The number of features can be changed with the NumKeypoints parameter. The higher the number the more the system is able to track but computational resource will also increase.

As this step does not need to operate in real-time, you can choose a higher number. We recommend >300.

Step 2 - Create the Offline Master Point-cloud

Step 2.1 - Launch Slamcore Visualiser

Launch the tool using the master dataset you have captured of the test environment:

$ slamcore_visualiser dataset -u <path/to/dataset> -c ~/slamcore/custom/multi_agent_master.json

This will open the Slamcore Visualiser tool:

_images/visualiser_multi_agent_master.png

Fig. 54 Slamcore Visualiser Opening Screen

Step 2.2 - Generate the Offline Master Point-cloud

Click the Start button to start processing the dataset.

The tool will now process the dataset to create a point-cloud of the space.

Fig. 55 Example point cloud

When the dataset has been fully processed, you will see the following message:

Fig. 56 Message when dataset finished processing

Step 2.3 - Save the Point-cloud

Now save the point-cloud by clicking the Save button:

You will be asked where you want to store this file.

_images/visualiser_progressbar_cropped.png

Fig. 57 Message displayed whilst saving dataset

The point-cloud will be saved as a single .session file which may take a few minutes to an hour depending on how large the environment and how many points have been mapped.

Step 3 - Copy the Session Map File to any Compatible Machine

The point cloud is stored as a small, single session file. This can be found in the location you specified in the previous step. You can copy this file to any compatible robot that meets our Software Requirements, with Slamcore software installed and set up (see Getting Started and Setting up a Camera).

Step 4 - Run Localisation using a Pre-built Point-cloud

You can test this system using pre-recorded datasets but to demonstrate this mode we will use a live sensor feed.

Step 4.1 - Launch the Positioning Tool on any Compatible Machine

You will need to decide how many features you wish to track per frame during the evaluation, and change the configuration file accordingly. This does not need to be the same as the settings you used to create the original point-cloud. Here real-time operation may be important so we would recommend setting NumKeypoints to 150.

Download the file to each machine you will be running the software on: multi_agent_deploy.json

With your sensor connected to the machine(s), run in the terminal window:

$ slamcore_visualiser -c ~/slamcore/custom/multi_agent_deploy.json

Note

You may skip Step 4.2 if you load the point-cloud in the command line with:

$ slamcore_visualiser -l <path/to/session/file> -c ~/slamcore/custom/multi_agent_deploy.json

This will launch the Slamcore Visualiser tool.

_images/visualiser_multi_agent_deploy.png

Fig. 58 Slamcore Visualiser Opening Screen

Step 4.2 - Load the Point-cloud to Use for Live Localisation

Before clicking start, you will need to load the saved point-cloud that you copied to this new machine. Click the Load button and select the session file you wish to use.

Step 4.3 - Run the System Live

Click the Start button to start processing the live sensor feed:

As before, it may take a few seconds for the system to recognise its location. When this happens you will see the 6DoF pose estimation marker update in real-time. If the system is not able to localise, the 6DoF axis will remain stationary even when you move the sensor. This can normally be fixed by moving the sensor in a figure of eight.

_images/visualiser_multisession_pointcloud.png

Fig. 59 View when system is localising in the saved point-cloud

Note

The system is very tolerant to both lighting and structural change. If there is an extreme difference in the conditions from when the original dataset was captured, then this mode may struggle to estimate a position. To resolve this you will need to capture a new master dataset of the full environment, generate a point-cloud from this dataset and save this new point-cloud.