Annotation Studio Basics

Overview

As a broad and highly customizable platform, Dataloop has a built-in annotation studio for image, video, audio, text, documents, and Lidar. The Dataloop team is always working on introducing new ones, and is made it possible for customers to build their own. Dataloop follows a certain template for the annotation-studio, to make it easy to switch between them and quickly get familiar with and become highly productive.

The main sections of the Annotation Studio are explained below.

Section 1: Label Picker and Annotation Tools (left-side panel)

This section allows you to use labels and annotation tools to perform the annotation process. The available labels and tools are determined in the Recipe.

Label Picker

A Label Picker is a feature within the annotation studio that allows you to pick or select labels to assign to specific objects or elements in the data being annotated. You can perform the following activities on the Label Picker section.

• Scroll and click a label to activate it.
• Use the search bar to easily find labels.
• Resize the label list to better fit your number of labels by clicking and dragging the separator line.

Annotation Tools

Annotation tools are designed to facilitate the process of data annotation. Data annotation involves adding metadata, labels, or tags to raw data, making it understandable for machines.

By default, the first tool in the list is selected. To activate a different tool, click on the available tools in the list.

Annotation Tools for Different Annotation Studios

Explore the Annotation Tools in Annotation Studio, each carefully designed to cater to specific annotation needs. The following table provides a concise overview of these tools, offering a versatile toolkit for insightful and collaborative annotation within the platform.

Section 2: Annotation Lists, Attributes, Modalities, and Item Details (right-side panel)

This section allows you controlling and managing annotations involves utilizing the annotations list and attribute controls, particularly when attributes are configured in the Recipe.

Annotations Lists: Annotation Selection

The annotations list displays all annotations added to the item and incorporates numerous features, facilitating efficient work even with dozens or hundreds of annotations.

Selecting an annotation from the list enables you to apply operations, reflecting the activation of its controller on the item.

The list enables two selection modes:

• Single Selection: Click anywhere on an annotation to have it selected, replacing any previous selection you made.
• Multiple Selection: Use the checkbox to add annotations to your selection.

Annotations Lists: Action Buttons

Hover over annotations to view related action icons, including:

• Edit annotation label and attributes.
• Hide or Show Selected (Keyboard shortcut key - H) annotations: Click on the Eye icon or press the H keyboard shortcut key to hide or show selected annotations.
• Hide or Show All (Keyboard shortcut key - J): Click on the Keyboard shortcut key J to hide or show all annotations, regardless of the selections.
• Labeling QA task controls:
  • Open Issue: Create an issue if there is any correction required in the annotation.
  • For review: Flag an issue for review after correcting it.

From the top-side of the annotation list, you can use the following actions:

• Clone: Click on the clone icon to clone one or more annotations from the list. You cannot clone semantic annotations.
• Change Label & Attributes: Click on it to change the annotation labels, attributes, and speaker names (for Audio Annotations).
• Open Issue: Create an issue if there is any correction required in the annotation.
• For review: Flag an issue for review after correcting it.
• Approve: It allows you to approve the item after verifying the annotations.
• Hide or Show Annotations: (Shortcut key - H) annotation visibility by clicking on the Eye icon or press H shortcut key. Once any annotation is hidden, a banner indicates that "One or more hidden annotations. Show All". By clicking on the Show All, all the hidden annotations will display again.
• Delete: Click on the Trash icon to delete an annotation.

How to Clone Annotations?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select one or more annotations from the list to clone.
3. Click on the Clone icon from the top-side of the annotation list. The cloned annotation will be created and added to the list.

How to Change Label and Attributes of a Single Annotation?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotation from the list.
3. Click on the Change Labels & Attributes icon from the top-side of the annotation list, or hover over the annotation and click on the Edit icon. The Annotation Definition popup window is displayed.
4. To change the Label, select the Label from the Labels tab, or
5. To change the Attributes, select the Attribute from the Attributes tab.
6. Click Save Changes.

How to Change Label and Attributes of Multiple Annotations?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotation from the list.
3. Click on the Change Labels & Attributes icon from the top-side of the annotation list. The Annotation Definition popup window is displayed.
4. To change the Label, select the Label from the Labels tab, or
5. To change the Attributes, select the Attribute from the Attributes tab.
6. Click Save Changes.

How to Hide or Reveal a Single Annotation?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotation from the list.
3. Click on the Hide or Reveal Annotations (Eye) icon from the top-side of the annotation list, or hover over the annotation and click on the Hide or Reveal Annotations (Eye) icon.

How to Hide or Reveal Multiple Annotations?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotations from the list.
3. Click on the Hide or Reveal Annotations (Eye) icon from the top-side of the annotation list.

How to Delete Annotations?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotation(s) from the list.
3. Click on the Trash icon from the top-side of the annotation list.
4. Click Yes to confirm the deletion.

Annotations Lists: Grouping Modes

By default, the annotation list displays all annotations as they are created, listed sequentially.

• After reaching 15 annotations, the list automatically switches to group-by-label mode to optimize loading.
• Users can click the Annotation Grouping button to toggle between the list view and group-by-label or group-by-creator modes.

How to Group Annotations By Label or Creator?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Click the Down-arrow next to the Annotation Grouping icon.
3. Select the Label or Creator option from the list as required.

How to Remove Grouping of the Annotation List?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Click on the Annotation Grouping icon to remove the grouping of annotations.

Annotations Lists: Annotations Filter

The filter in the annotation studio includes the following sections:

1. Annotation type: It displays annotations based on the annotation tool types, such as box, point, polygon, note, etc. This filter type is not available for Audio items.
2. Annotation status: It displays annotations based on the annotation status, such as no status, issue, for review, etc.
3. Annotation properties: It displays annotations based on the annotation properties, such as object ID, description, etc.

• By default, all the filter are unchecked. When you select a filter, it displays only the annotations according to the selection you made.
• Each filter shows the total number of annotations.
• When you select multiple filters, only annotations that satisfy all filters will be displayed.

How to Filter Annotations?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Click on the Annotation Filter icon next to the Annotation Search field.
3. Select the filter criteria as required and click on the same filter icon or click outside to apply the changes.

How to Add a Note to the Annotation?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotations from the list.
3. The selected annotation will be listed at the bottom of the annotation list, click on the Note icon next to the Information icon.
4. Enter the note as required and click Save.

How to View Annotation Information?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Select the annotations from the list.
3. The selected annotation will be listed at the bottom of the annotation list, click on the Information icon. The following details are displayed:
   1. Annotation creator's and updated by email ID.
   2. Annotation ID.

Annotations Lists: Sorting

Click on the arrows icon to sort your annotations on either Ascending or Descending by:

• Creation Time (default)
• Update Date/time
• Label
• Type
• Confidence

How to Sort the Annotation List?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Click on the Annotation Sort icon next to the All Annotations list.
3. Select the sort by options from the list. The available options are:
   1. Creation Time: Sort the annotations by their creation time.
   2. Update Date or Time: Sort the annotations by their update time.
   3. Label: Sort the annotations by their label name.
   4. Type: Sort the annotations by the annotation type.
   5. Confidence: It is a numerical representation that indicates how certain the model or the annotator is about a specific annotation or prediction. This value is usually expressed as a probability between 0 and 1, where: 0 represents no confidence at all, and 1 represents complete confidence.

Annotations Lists: Advanced Search Box

Type in the search box to have the annotations list that displays only annotations that match the search criteria. You can search in every aspect of the annotations:

• Tool: Enter the tool name, like BOX or POINT.
• Label: Enter any label from the recipe.
• Creator: Enter the name of the annotations' creator to view only related annotations.

Annotations Lists: Focus Mode

The annotation focus mode allows you to filter out annotations that are out of view from the annotation list when you zoom or pan. The annotation list will then display only those annotations that are available within the zoomed area.

How to Focus an Annotation?

1. In the Annotation Studio, identify the annotation list on the right-side panel.
2. Click on the Annotation Focus icon to focus only the selected annotations, when you zoom or pan.

Attributes

When attributes are defined in the recipe and annotation is selected (annotation is also selected immediately after it’s created) the attributes section will be visible, allowing the user to set them.

• Select the attributes relevant to the object.
• Use the Search field to search for a specific attribute value.
• Skipping a ‘mandatory’ attribute will show a yellow flag, to highlight it for the user.

Attributes' Value Search

You can quickly and efficiently search for attribute values to streamline the annotation process. Use the Search field to find specific attribute values.

Attributes: Reviewing Attributes

Annotations with attributes selected will be marked with an 'A' sign, hover over it to see the attributes. For mandatory attributes, the 'A' will be highlighted in yellow.
Click on the 'A' button on the bottom left to toggle the display of all the annotation's attributes on the canvas.

Item Details

The Item tab displays information according to the type of the selected item, including:

• Created Date: The creation date of the selected item.
• Description: The text description, which can be viewed and edited from this panel, allows you to add item descriptions (including support for emojis) during file uploads. This feature provides an additional method to search for items using specific text or descriptions. Click on the Pencil icon to edit details.
• Item ID: Unique identification for the item. Click on the Copy icon to copy the item ID.
• Dataset ID: Unique identification for the item's Dataset. Click on the Copy icon to copy the Dataset ID.
• File Path: A URL link to the item on the Dataloop platform. Click on the Copy icon to copy the file path.
• Item path: A URL link to the item on the Dataloop platform. Click on the Copy icon to copy the item path.
• Item Metadata: Item metadata refers to the descriptive information and attributes associated with individual items within a dataset. Click on the Pencil icon to edit details.
• Functions: It displays the functions performed with this item, providing information such as Application Name, Function, Status, Date, Log details, and options to view execution details, among others.
• Status Log: It allows you to track and monitor your team's actions on data items. You can easily follow task status changes, see who applied each status, such as Completed, Discard, Status Completed Removed, or any custom status), and when it was done. The way status logs are shown in the annotation studio depends on the user's role, either as an Annotator" or an Annotation Manager and above:
  • Annotators - Task-Specific View: Annotators only see status logs for the tasks they are assigned to.
  • Annotation Managers and Above - Full View: Managers and higher roles can see all status logs across different tasks. This gives them a complete overview of what's happening in the project, even for tasks they’re not directly managing.
• Import Annotations: Click on the Import icon to import annotations in a JSON file or paste annotation source info.
• Export Annotations: Click on the Export icon to export annotations and masks.
• Recipe: Click on the Recipe icon to open the Recipe page of the selected item.

How to View Item Details?

1. In the Annotation Studio, click on the Item tab on the right-side panel.
2. You can view the details available in the Item Details section.

How to View Recipe?

1. Go to the Annotation Studio.
2. In the Label Picker section of the left-side panel, click on the Go to Recipe to view the recipe that is linked to the current dataset.
   Or
3. Go to the Annotation Studio.
4. Select the Item tab in the right-side panel.
5. Click on the Recipe icon available on top of the right-side of the Item tab.

How to Import Annotations into an Item?

1. Go to the Annotation Studio.
2. Select the Item tab in the right-side panel.
3. Click on the Import Annotations icon available on top of the right-side of the Item tab.
4. In the Import Annotations popup, click on the following:
   1. Import JSON file: Upload a JSON file.
   2. Paste: You can directly copy and paste the JSON data representing the annotations.
5. Click Import.

How to Export Annotations from an Item?

1. Go to the Annotation Studio.
2. Select the Item tab in the right-side panel.
3. Click on the Export Annotations icon available on top of the right-side of the Item tab.
4. Select the Export Annotation option from the list. A JSON file will be downloaded.

How to Export Mask from an Item?

In image annotation, a mask typically represents a specific area or object within the image, often in the form of a binary or grayscale image where the annotated regions are marked

1. Go to the Annotation Studio.
2. Select the Item tab in the right-side panel.
3. Click on the Export Annotations icon available on top of the right-side of the Item tab.
4. Select the Export Mask option from the list. An image file (PNG) will be downloaded.

How to View Applications and Functions Available on the Item?

1. Go to the Annotation Studio.
2. Select the Item tab in the right-side panel.
3. Click on the Functions section at the bottom of the right-side panel -> Item tab. The available functions are listed.

Section 3: Item Info & Controls (top-side panel)

Item Info & Controls are available depends on the type of annotation studio. The following features are mainly available on the annotation studios.

Undo-Redo

Applicable for all actions performed directly over Canvas when creating or updating annotations.

How to Undo or Redo Actions?

1. Go to the Annotation Studio.
2. In the top-side control:
   1. Select the Undo icon to cancel the change you made.
   2. Select the Redo icon to reverse the change you had previously undone.

Item description

The item description feature allows you to add a description for the items available in the dataset.

How to Edit Item's Description?

1. Go to the Annotation Studio.
2. In the top-side control, select the Item Description icon next to the Redo icon.
3. Enter the description and click Save. The newly added description will be available under the Item tab.

Item info

Details and enables copying item-related information and context such as File-name, item path, dataset ID, and more.

How to View Item Information?

1. Go to the Annotation Studio.
2. In the top-side control, select the Item Info icon. The following details are displayed:
   1. File Name
   2. File Path
   3. Item ID
   4. Dataset ID
   5. Dataset Name.

Keyboard shortcuts

Keyboard shortcuts are also accessible through the top panel of each Annotation Studio, with each studio featuring both general and studio-specific keyboard shortcuts.

How to View Keyboard Shortcuts?

The available shortcut keys for each annotation studio are listed below:

• [Image Studio Shortcuts]
• [Audio Studio Shortcuts]
• [Video Studio Shortcuts]
• [NLP or Text Studio Shortcuts]
• [RLHF or Prompt Studio Shortcuts]
• [Segmentation Studio Shortcuts]
• [Comparison Studio Shortcuts]
• [LiDAR Studio Shortcuts]

Adjustments

The adjustments setting allows you to configure the following features according to your requirement:

1. Opacity: It controls the transparency level of an annotation or element within the image.
2. Brightness: It controls how light or dark the image appears.
3. Contrast: It adjusts the difference between the light and dark areas of an image. It enhances the distinction between light and dark areas, making annotations more pronounced.
4. Border thickness: It controls the width of the border surrounding an annotation.
:::(Warning) ()
Opacity, brightness, and contrast are only supported for images up to 8192 px (either width or height).
:::

How to Adjust the Opacity of Items in Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Adjustments icon next to the Keyboard Shortcut icon.
3. Adjust the Opacity progress bar as required.

How to Adjust the Brightness of Items in Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Adjustments icon next to the Keyboard Shortcut icon.
3. Adjust the Brightness progress bar or enter a value as required.

How to Adjust the Contrast of Items in Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Adjustments icon next to the Keyboard Shortcut icon.
3. Adjust the Contrast progress bar or enter a value as required.

How to Adjust Items' Border Thickness?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Adjustments icon next to the Keyboard Shortcut icon.
3. Adjust the Border Thickness progress bar or enter a value as required.

Work Settings

Various options for annotators to toggle and tune their work, such as:
1. Keep zoom and position: When moving between items, especially frame snapshots out of video files.
2. Pixel measurement: This shows the pixel length value for Box, Polygon, and Polyline tools.
3. Pixel smoothing: Smoothen the display of pixels when viewing low-resolution images at high zoom.

4. Cross Grid Measurement - provides a cross-grid for the mouse cursor, a tool that may assist in achieving higher annotation accuracy.
5. Coordinate - displays the mouse coordinates over the item when hovering it.
7. PDF - Open/close the PDF instructions document, when it is set from the Recipe page.

How to View or Hide the Pixel Measurement in the Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Settings icon.
3. Enable the Pixel Measurement option to view the pixel value on the annotation.

How to Enable to Disable the Keep Zoom and Position Feature in the Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Settings icon.
3. Enable the Keep Zoom and Position option. It allows you to keep the Zoom and Position settings of an item when moving between items, especially frame snapshots out of video files.

How to Enable to Disable the Pixel Smoothing in the Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Settings icon.
3. Enable the Pixel Smoothing option, by default it is enabled. It smoothens the display of pixels when viewing low-resolution images at high zoom.

How to Enable or Disable the Cross Grid Measurement Feature in the Annotation Studio?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Settings icon.
3. Enable the Cross Grid Measurement option, by default it is enabled. It displays a cross-grid for the mouse cursor, a tool that may assist in achieving higher annotation accuracy.

How to Enable or Disable the Coordinates Feature in the Annotation Studio

1. Go to the Annotation Studio.
2. In the top-side control, click on the Settings icon.
3. Enable the Coordinates option. It displays the mouse coordinates over the item when hovering it.

How to Copy Annotations from the Previous Item?

1. Go to the Annotation Studio.
2. In the top-side control, click on the Copy annotation from previous item icon.
3. Click Yes to confirm. A confirmation message is displayed.

Section 4: Item's View Controls (bottom-side panel)

The controls on the bottom-side panel display based on the annotation contexts and work controls.

Assignment Context

Assignment controls, including moving between items, displaying the item gallery, and the status buttons (Complete / Discard). It displays only while working on an annotation or QA task.

1. Browse between the assignment items using the Left and Right arrows
2. Open the Thumbnails' gallery viewer, and click a thumbnail to open that item
3. Save button - Clicking the button when it is enabled triggers saving changes to the Dataloop platform, before the auto-saving feature takes care of that.
4. Status buttons - Complete and Discard.

Work Controls

From left to right, work controls at the bottom panel deliver the following functionality:

1. Zoom control - Small steps using (+) and (-) buttons.
2. Fit to screen.
3. Preview Attributes: Enable/disable the attribute details visibility over the item, next to each annotation.