Data Browser

Overview

The dataset browser allows you to explore and navigate through dataset items. It typically provides a user-friendly interface for searching, visualizing, and accessing data within a dataset. You can filter, sort, and view the dataset items, including images, text, audio, video, LiDAR, etc. making it easier to analyze and work with large volumes of data.

The Dataset Browser page features are explained in the following sections (marked on the screenshot):

Section 1: Access the Dataset Browser

Access the dataset browser using one of the following options:

• In the Dataloop left-side menu, select Data.
  • For existing datasets, double-click on the name datasets to open the dataset browser,
  • For a new dataset, see the Create a dataset article.
• In the Project Overview > Data Management widget, click on your Dataset name.

The Classic Browser is displayed by default. To use the new version of the Dataset Browser, click Switch to Browser 2.1 (section 1, that is marked on the above screenshot).

Section 2: Search and Filter Items

The dataset browser filter enables you to refine datasets by filtering them based on:

• The item features, such as filename, media type, etc.
• Annotation features, such as filter by label, type, etc.
• Task ID and Task Names
• CLIP Based - Free Text Search

For more information about the Search Filter, see the Schema Based Search article.

Section 3: Viewer

The Dataset browser offers two viewing options, Thumbnail and Details. By default, the thumbnail view is shown, and you can use the respective View icons to switch between these options.

Thumbnail View

• Initially set to Thumbnail view, showcasing smaller versions of items in a dataset includes images, documents, and video files arranged in a preview grid.
• Audio, Lidar, and RLHF files are represented by a thumbnail indicating the data type, without an actual preview.
• Items are organized by their latest upload date.
• Allows for quick browsing and selection of specific content without needing to open each item.

In the Thumbnail view, you can:

• Adjust the thumbnail size using the slider control at the bottom left of the page.
• Use smaller thumbnails to view more items on a page.
• By default, the Show File Name option in the Settings is enabled to show the file names. You can toggle it to hide the file names.
• Use the Sort By option to sort the items in the dataset. By default, the items are displayed according to the Creation Date.

Details View

• In this view, users can see a list of items with their associated details including File Name, File Created Date, Media Type, Item's annotation status, etc.
• This makes it easier to access and review specific information about each item in a structured and organized manner.

In the Details view, you can:

• Click on the checkbox next to the file name to select all the items in the current page.
• Click on the Manage Columns to hide columns.
• Sort items based on the Columns.

Common Features in the Thumbnail and Details Views

• The number of selected items is highlighted.
• Click on the Select All option to select all the items in the current page.
• Total number of dataset items, and breadcrumbs navigation to give a clear path back to higher levels, such as sub-folder and folder.
• Settings > Show Hidden Files: By default, the Show Hidden Files option is disabled. You can toggle it to show the hidden files.
• Items per page: By default, 100 items are displayed per page. You can select 2, 25, 50, 100, 250, 500, and 1000 items per page.
• Use the page navigation options to view next and previous pages, or enter a specific page number and click Go to view the page.

Difference Between Annotated, Completed, Approved, and Not Annotated Items

An item earns the status annotated if it receives any form of annotation, such as a classification, a note, or any other tool-generated annotation, regardless of whether this occurs in the dataset browser or during a task. However, being annotated doesn't necessarily mean an item is completed.

• This situation may arise if the item is annotated via the dataset browser, where annotations can be saved without assigning a status, or if it's annotated during a task without the complete button being engaged, perhaps because the annotator plans to return to it later.
• Completed status is assigned to an item when an annotator finalizes their work on it by clicking the complete button during a task.
• An approved status is granted after an item undergoes a QA task and the QA tester decides to approve it by clicking the respective button.
• An item is deemed not annotated if it lacks any annotations. Interestingly, an item can be marked as completed and yet be considered not annotated if the complete button was clicked without any actual annotation work being done on it.

View Context

You can utilize one of the following view contexts to showcase items in the Dataset browser. When applying a filter, it is implemented within the scope chosen by the user, whether it's the entire dataset or a specific folder.

The dataset browser enables organizing file items in file-system-like:

• Items based: By default, this view is displayed and in this view it shows all items regardless of their folder structure, enabling the application of filters and displaying all items at the Root Folder (Dataloop). When you select a folder, it shows Items Only, and it does Not show any sub-folders if available.
• Folders based: It shows items based on the folders or subfolders you selected. When you select the Root Folder (Dataloop), it shows items and folders if available in the Root folder.

You can perform the following actions:

• Click Folder based to view items and sub-folders in a folder.
• Create folders: Select the root folder and click Add Folder or the new folder icon when you hover Root Folder (Dataloop).

• Create Sub-Folders: Select the folder and click Add Folder or the new folder icon when you hover the selected folder.

• Move items between folders:

  1. Select one or more items from the current page,
  2. Right-click and select File Actions > Move to Folder.
  3. Select the folder from the list.
  4. Click Move.

• Edit the folder name, hover over the folder, and click on the Pencil icon.

• Select a folder and right-click to:
  • Rename: Rename the selected folder.
  • Move: Moved items in the selected folder to another folder.
  • Copy item path: It copies the complete item path.
  • Create Trigger: It allows creating a trigger function for the selected folder items.
  • Delete: It allows you to delete the selected folder and items in the folder.

Section 4: Settings, Actions, and Details

Data Browser allows users to perform various actions, manage settings, and access detailed information about specific items or features.

Dataset Details

The Dataset Details provides the following information related to the dataset. To view the Dataset Details, no items should be selected.

• Dataset ID: The ID of the dataset. Click on the Copy icon to copy the dataset ID.
• Recipe: The name and the link of the recipe that is configured to the dataset. Click on the link to open the recipe page.
• Dataset Analytics: Dataset analytics refers to the process of collecting, analyzing, and deriving insights from a dataset. Click on the link to view the Progress tab of the Analytics page.
• Project: The project name of the dataset. Click on the Copy icon to copy the project ID.
• Owning Organization: The name of the organization to which the dataset is affiliated. Click on the Copy icon to copy the Organization ID.

ML Data Split

ML Data Split Chart displays the distribution of items across machine learning subsets, including Test, Train, and Validation, showing both the total numbers and percentages.

• This chart is generated when an item is assigned to one of the ML subsets.
• When hovering over a subset, the chart specifically highlights the subset being hovered over.
• Clicking on a subset grays it out, removing it from the chart.
• Clicking on it again restores its visibility on the chart.

Upload Items

Clicking on the Upload (Sync for external cloud storage) icon allows you to upload files and folders into Dataloop's storage.

Item Tab

Choose an item to view the following details in the right-side panel:

• File Name: The name of the selected item. Click on the copy icon to copy the file name.
• Created at: The creation date of the selected item.
• Description: The text description of the item. Click on the pencil icon to add or edit descriptions. Also, item descriptions can be added during file uploads, serving as an additional way to search for items containing specific text or descriptions.
• File path: The folder path where the file is located. Click on the copy icon to copy the file path.
• Item ID: Unique identification for the item. Click on the copy icon to copy the item ID.
• Item path: A URL link to the item on the Dataloop platform. Click on the copy icon to copy the item path.
• Parent Item Link: If the item is a clone, it shows the parent item as a link. The link takes to the datasets where the original item is located, filtered to that item.
• Labeling Tasks: This section provides the number of annotations and classifications associated with the selected item.

Automation Tab

The Automation tab allows your selected item to run with FaaS, Pipeline, or Model Predictions. Click on the following options to create a function, pipeline, or model prediction execution:

• Run with FaaS: It lists the functions of the all the activated FaaS services. Select a function to execute with the selected items.
• Run with Pipeline: It allows you to select a pipeline to execute with the selected items.
• Run Model Predictions: It allows you to select a model to generate predictions for the selected items. Only trained and deployed models are available for selection.

When executions are available, you can search executions by function, application, or pipeline. Also, the following details are displayed:

• Pipeline: The name and link of the pipeline. Click on the link to view the pipeline.
• Application name: Name of the application.
• Function name: Name of the function.
• Execution Status: Success, Running, and Failed.
• Updated At: Date and time of the execution update.
• Rerun: If needed, click the Play icon to rerun the execution.
• Filter icon: Filter executions based on the status, such as Success, Failed, Running, and Pending.

Metadata Tab

Item metadata refers to the descriptive information and attributes associated with individual items within a dataset.

You can perform the following actions:

• Click on the copy icon to copy the metadata.
• To edit the metadata:
  1. Click the Edit icon to open the editor.
  2. Make changes as required.
  3. Click on the Save icon to save the changes.

Dataset Actions

Dataset Browser allows you to perform the following actions based on dataset and item level. The following actions are available to perform when you click on the Dataset Actions. A few actions are not applicable if you select more than one item.

Right-Click: You can also use the right-click to perform the following actions. The menu is opened only for the individual item on which the right-click is performed.

Once you click on either Dataset Actions or Right-Click on an Item, the following options are listed:

File Actions

You can perform the following action on a file within the Dataset. Please note that actions from the right-click menu cannot apply to multiple selected items simultaneously.

• Rename
• Export JSON
• Download files
• Clone
• Classification
• Move to Folder
• Open File in New Tab
• Open File In Studio
• Delete Annotations
• Delete Items

Labeling Tasks

• Create New Task
• Add to Existing Task

Models

• Predict
• Extract Embeddings
• Assign to Subset -> Test
• Assign to Subset -> Train
• Assign to Subset -> Validation
• Remove from Subsets

Deployment Slot

• COCO Converter
• YOLO Converter
• VOC Converter

Run with FaaS

Run with FaaS

Run with Pipeline

Run with Pipeline

Download Annotations

Download Annotations

To view more actions available on the dataset actions, refer to the Manage Datasets article.

Section 5: View Number of Items and Annotations

The Data Browser provides you the number of items and annotations available in the dataset. You can view the following information on the top-right side of the data browser page:

• All Dataset Items: It displays the number of items available in the dataset.
• Annotated Items: It displays the number of items that are annotated.
• Annotations: It displays the number of annotations available in the all annotated items.