LIDAR Data Setup

Overview

It is the process of preparing and configuring Dataloop system to effectively utilize LIDAR data. Setting up LIDAR data involves several key steps:

• Files Setup
• PCD Processing
• Mapping JSON Format
• Upload Framed Annotations to the LiDAR Studio

Supported Files

The setup process prepared the files for work in a point-cloud annotation/segmentation task. Supported files are:

• PCD Files: Point Cloud Data files are used to generate the 3D scenes. Both single and multiple PCD files are supported.
• JPEG/PNG Images: Usually showing the angle and viewport of the cameras that captured the main scene. These images will have real-time projections of 3D objects annotated in the main point-cloud scene.

Files Setup

Setup or stitching of PCD files with camera images is performed by arranging files in designated folders in the root of the dataset.

1. Since every PCD is a frame, you must organize or prepare the files first. File names must be numbers starting from 0, it defines the order of the frames in the 3D scene.

2. lidar folder: All PCD files are placed in a folder named lidar.

3. frames folder: This folder contains subfolders, each with JPEG or PNG images per frame.
   1. The folder contains subfolders with names corresponding directly to the PCD file names; hence, subfolder names must match the 0-N numerical order.
   2. The order of the images will dictate the order in which they will be displayed in the PCD Studio.

4. mapping.json file: This file must contain all the calibrations' data for the sensor and the cameras.
   1. The .json file is mandatory and enables sensor fusion.
   2. You can use any names for the JSON, but for best practice, use mapping.json.

An example of arranging the files in a selected folder inside the dataset

The same structure as in the example of arranging the files in the root of the dataset should be kept inside the selected folder.

PCD Processing

Once all files are ready, contact Dataloop to execute the PCD processing, a step in which the following takes place:

1. Ground detection: On each provided .pcd file.
2. PCD stitching: All PCD files are stitched together to create a LiDAR video file, where each frame contains the following information:
   1. PCD file: For further information about how a PCD file must look, refer to Why a New File Format?.
   2. JPEG/PNG files: The images of the available cameras for the given frame.
   3. JSON file: The JSON data is taken from mapping.json and includes all the metadata needed for the given frame.

Mapping JSON Format

{
    "frames": {
        "0": {
            "path": <>, // for frame 0: "lidar/0.pcd" (Relative path from the mapping.json file)
            "timestamp": <>
            "position": { // lidar sensor location (used as the center of the world)
                "x": <>,
                "y": <>,
                "z": <>
            },
            "heading": { // lidar sensor rotation (Quaternion)
                "x": <>,
                "y": <>,
                "z": <>,
                "w": <>
            },
            "images": { // if no images are provided, add an empty dict
                "0": {
                    "image_path": <>, // for frame 0 image 0: "frames/0/0.jpg" (Relative path from the mapping.json file)
                    "timestamp": <>,
                    "intrinsics": { // camera intrinsic
                        "fx": <>, // Focal length in pixels.
                        "fy": <>,
                        "cx": <>, // Optical center (the principal point), in pixels.
                        "cy": <>,
                    },
                    "extrinsics": { // camera extrinsic
                        "translation": { // camera location in world coordinates (in relation to the lidar sensor)
                            "x": <>,
                            "y": <>,
                            "z": <>
                        },
                        "rotation": { // rotation of the camera (Quaternion)
                            "w": <>,
                            "x": <>,
                            "y": <>,
                            "z": <>
                        }
                    },
                    "distortion" : { // distortion parameters
                        "k1": <>,
                        "k2": <>,
                        "p1": <>,
                        "p2": <>,
                        "k3": <>,
                        "k4": <>
                    }
                }
            }
        }
    }
}

Upload Framed Annotations to the LiDAR Studio

Dataloop allows you to upload framed annotations to the LiDAR Studio via SDK.
Make sure that the frames.json file includes the fps key in its metadata, set to a default value of 1 if it is not already present.

{
    "system":   {
        "originalname": "frames.json",
        "size": 32712,
        "encoding": "7bit",
        "shebang": {
            "dltype": "PCDFrames"
        },
        "taskStatusLog": [],
        "mimetype": "application/json",
        "isBinary": false,
        "refs": []
    },
    "fps": 1
}