LiDAR 2.0

Overview

Dataloop offers an advanced LiDAR (Light Detection and Ranging) annotation and detection tool. By loading supported LiDAR files into the Dataloop platform, annotators can label and segment out objects captured by pulsed lasers, and provide precise three-dimensional information for data scientists and industry professionals to build AI models for cutting-edge applications.

The main sections of an LiDAR Studio is described below:

• Section 1: Label Picker and Annotation Tools
• Section 2: Annotations List, Attributes, and Item Details
• Section 3: Frames Navigator
• Section 4: The Canvas
• Section 5: 2D Editor
• Section 6: 3D Editor
• Section 7: Settings

Section 1: Label Picker and Annotation Tools

This section allows the annotator to use labels and annotation tools to perform the annotation process. The available labels and tools are determined in the Recipe.

Label Picker

The Label Picker is a feature within the annotation studio that allows the annotator to select labels to assign to specific objects or elements in the data being annotated.

LiDAR Annotation Tools

The following annotation tools are available for the LiDAR studio:

• Cube 3D
• Instance Semantic
• 3D Polyline

Height Hide from Bottom or Top

Utilizing these filters enables you to refine the display of point clouds on the canvas according to their height, simplifying the process of identifying and annotating objects. This hide height feature is essential while annotating with Instance Semantic Segmentation. Read more about it on the instance segmentation tool.

Section 2: Annotations List, Attributes, and Item Details

This section allows you controlling and managing annotations involves utilizing the annotations list and attribute controls, particularly when attributes are configured in the Recipe. Click on the following sections to view more details:

• Annotation List
• Attributes
• Item Details
  
  

Annotation Controller

The annotation controller directs the annotator to the specific annotation being viewed on the canvas. The annotation controller will display on the canvas by selecting the annotation from the list.

Hover over the annotation name to perform the following actions with the Annotation Controller:

• Set end time: You can set end time for the annotation based on the frames.
• Set object ID: You can add a unique ID for the annotation.
• Delete Annotation: Click on the trash icon to delete the annotation.

The distance below the annotation controller presents the distance of the annotation from the PCD file center.

Navigate Annotations Around the Canvas

Lidar annotation studio allows you to navigate annotations around the canvas. You can perform the following actions on the LiDAR studio:

• Move UP and Down: Click on the purple arrow to adjust the height of the annotation.
• Move Sideways: Click on the green and red arrows to move the annotation sideways accordingly.
• Drag Annotations: Click in the center of the annotation, and drag it to move it to a new location.
• To pan the canvas, press Ctrl+drag.
• To activate anchor mode and change the point of view angle, press the Space bar + drag. The anchor mode will present a sphere in the center of the screen. When zooming in, the zoom-in will be done relative to the center of the screen.

To learn more actions, click on the Keyboard Shortcuts or refer to the Keyboard Shortcuts.

Section 3: Frames Navigator

This feature is designed to facilitate navigation and interaction with different frames of LiDAR data. If the LiDAR file comprises a sequence of frames, you can switch between PCD frames. The selected frame's scene will be showcased on the canvas, and the 2D images of that scene will be synchronized accordingly.

Ego Vehicle Position

The Ego Vehicle position refers to the position of the ego vehicle within the PCD file, and is precisely centered within the PCD file. The ego vehicle is typically the vehicle on which the LiDAR sensor is mounted. It serves as the reference point for interpreting and analyzing the surrounding environment based on the LiDAR point cloud.

Show or Hide Ground

When this feature is enabled, the ground is displayed. By default, it is enabled, and the ground is visible. Clicking it causes the ground to disappear.

Show or Hide Attributes

When this button is activated, the attributes of the annotation are shown over the item, next to each annotation. By default, it is deactivated. Clicking it will reveal the attributes.

Timeline

The timeline enables users to visually observe annotations and their corresponding frames. By default, the timeline feature is in a collapsed position. Clicking Expand Timeline will reveal the timeline, displaying a maximum of 5 annotations at a time, with scroll functionality.

Linear Interpolation

Linear Interpolation automatically calculates the linear change of size and position between two keyframes, allowing users to speed up work by eliminating the need to make small changes on every frame. By default, Linear interpolation is active in the LiDAR studio. If the dropdown value is “None”, then the interpolation tool is inactive, and the created annotation won’t be interpolated.

• Cuboid: Cuboid annotation tool supports interpolation. If the interpolation is enabled, the annotator can annotate and the interpolation will be done automatically. Between each 2 keyframes there will be interpolated frames.

• Polyline: The polyline annotation tool supports interpolation. When activated, annotators can annotate, and automatic interpolation occurs between consecutive keyframes. The polyline points can be adjusted at each frame, with any modification treated as a keyframe (including attribute, label, and location changes).

• Instance Semantic: Instance semantic annotation tool does not support interpolation. For annotating with Instance Semantic on various frames, the user should annotate manually at each frame. Each manual annotated frame will be counted as keyframe.

Use the controls on the left to set a keyframe at the current position.

• Set Start Frame: It removes the previous Start Frame.
• Set End Frame: The annotation will end on this frame.
• Set occlusion on/off: Set ON when the annotation is hidden and off when it's back. Changes are visualized over frames with different opacities.

Section 4: The Canvas

The canvas renders the Point Cloud data based on the details contained in the PCD file. After annotations are created, they will appear on the canvas, allowing users to make edits directly within the canvas interface.

How to Create 3D Cuboid Annotation?

To create 3d Cuboid Annotations, refer to the 3D Cuboid page.

How to Edit Cuboid Annotation?

Use one of the following three options to edit cuboid annotation:

• Canvas Editing
• 2D Editor
• 3D Editor

Canvas Editing

Use the following options to adjust the Cuboid:

• Translate: The translation option allows the annotator to reposition the cuboid within the PCD file along a specific axis using directional arrows or through unrestricted manual adjustments.
• Scale: The Scale option allows users to scale the cuboid
• Rotate: The rotation option enables the annotator to pivot the cuboid around the X, Y, and Z axes.

Instance Semantic Annotation

Instance segmentation annotation involves color-coding the pixels within an image or, in the case of LiDAR data, the point clouds. As the name suggests, instance segmentation focuses on annotating individual instances, enabling the differentiation of objects that might share the same label.

To learn more, refer to the Instance Semantic 3D page.

3D Polyline Annotation

3D polyline annotation involves marking and delineating a series of connected line segments within a three-dimensional space.

To learn more, refer to the 3D Polyline page.

Section 5: 2D Editor

The 2D Editor displays Cuboid and Polyline annotations projected on the image, while the annotation location on the PCD file corresponds to their position on the image. It is the visual assistant for corresponding images that showcase both the angle and the viewport of the cameras captured by the main scene.

To open the 2D editor, hover over the desired image on the right-side panel and click Open 2D Editor.


Upon opening the 2D editor, you will have access to a range of capabilities, including:

• Selecting annotations within the Image view.
• Editing annotations, which includes modifying the annotation ID.
• Deleting annotations.
• Adjusting the scale, rotation, and translation of the annotation, effectively alters the geometry of the 3D cube.
• Setting the end time for the annotation.

In order to exit the 2D Editor, click ESC or Exit 2D Editor on the right-side panel.

Projection

The projection signifies the angle at which the camera captures data within the PCD file.
Activate the projection by clicking the button on the 2D Editor panel.

Section 6: 3D Editor

Selecting a Cuboid annotation from the annotations list opens the 3D editor, displaying the cuboid in three dimensions and enabling users to easily edit it.

• To hide the Point Cloud Background points, utilize the Hide Background button.
• Expand the 3D editor to obtain a larger view of the annotation.

Zoom to Mouse Cursor in LiDAR Side Panel

Annotators can now zoom directly to the mouse cursor in the 3D Camera View. This feature allows for more precise navigation and focus on specific areas within the LiDAR data, enhancing the efficiency and accuracy of the annotation process.

Section 7: Settings

The setting configuration is available on top of the LiDAR studio.

• Annotation Opacity: Users can adjust the opacity of annotation to have a better sight of the PCD scene.
• Point Size: Users can adjust the point cloud size. Resizing the point cloud will help annotators to have a clearer view of the Point Clouds.
• Color Point by Intensity: Intensity serves as a valuable tool for visualizing the strength of laser returns within a LiDAR point cloud.
• Color Point by Height: Coloring points in a LIDAR point cloud by height is a common visualization technique that helps convey information about the elevation of the terrain or objects. This feature assigns colors to points based on their height values, where lower points are colored with green color and higher points are colored with red color.

• Bind to Annotation Center: Upon enabling this configuration, it will impact the distance-based hiding behavior. When activated, the concealed area will be relative to the center of the chosen annotation. Conversely, with this configuration turned off, the hidden area will be relative to the PCD center.
• Hide by Distance: By adjusting this scale (in meters), the canvas will filter out points that are not within the distance range.

Actions Available on the LiDAR Studio

How to Clone a LiDAR File in the Data Browser?

When you clone a LiDAR file (frames.json) from the Dataset Browser using the right-click -> File Actions -> Clone option, it does not properly clone the file completely. This method may result in incomplete cloning, leading to potential issues with data integrity and usability.
Ensure you use the following method to clone the LiDAR files accurately:

1. In the Dataset Browser, download the frames.json file.
2. Re-upload the Frames.json file to the dataset.
3. In the dataset browser, select the original file again.
4. Go to the Metadata tab in the right-side panel.
5. Click on the Pencil icon.
6. Copy the shebang data.

7. Navigate back and select the newly cloned item.
8. Go to the Metadata tab in the right-side panel.
9. Click on the Pencil icon.
10. Paste the shebang data under the system section.
11. Click on the Save icon.

LiDAR Studio Keyboard Shortcuts

General Shortcuts

Annotation Tools Shortcuts