Search Your Data

The Dataloop Dataset Browser is a powerful interface that enables users to efficiently search, filter, and refine datasets to identify required items with ease. It provides multiple levels of search functionality, ranging from simple searches to advanced queries, ensuring flexibility for diverse user needs.

Basic Search

The Basic Search feature enables quick filtering and discovery of items using predefined parameters. It helps you easily locate items by applying simple conditions such as labels, attributes, status, users, and more. Clicking on the + allows you to add more search types into the query.

Advanced Search

Filters are integral components of the Dataset Browser, providing users with the capability to refine and narrow down the displayed items based on specific criteria. These filters offer a powerful tool for managing and exploring large datasets efficiently.

Advanced Search uses a search schema that allows users to query items and annotations data with higher granularity:

• Search Items using structured schema fields.

• Explore diverse use cases of search schemas within the Dataset Browser.

• View available schemas directly in the interface for guidance.

Search Items by Item's Data

1. In the Data Browser, click on the Items field.

2. Select or enter the required search query.

3. Click Search to view the search result.

Search by Annotation Status

By Annotation Status -> Annotated or Not.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

annotated = true 

or

annotated = false 

3. Click Search to view the search result.

Search by Annotation's Count

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

annotationsCount = 5

3. Click Search to view the search result.

Search by Collection List

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

collection = 'Collection A'

3. Click Search to view the search result.

Search by Creation Date

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

createdAt = (03/12/2023)

3. Click Search to view the search result.

Search by Creator Email

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

creator = 'your@email-ID'

3. Click Search to view the search result.

Search by File Name

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

name = '63dbdd10be95cbe35df0a78b.jpg' 

3. Click Search to view the search result.

Search by Dataset ID

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

datasetId = '65950d1d5c356a5e51f6727e'

3. Click Search to view the search result.

Search by Description Status

Description Status is Available or Not

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

described = true

or

described = false  

3. Click Search to view the search result.

Search by Description Text

1. In the Data Browser, click on the Item field.

2. Select or enter the query as follows:

description = 'description-text'

3. Click Search to view the search result.

Search by Folder Directory

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

dir = '/sub-folder name' 

or

dir IN '/sub-folder name/sub-folder name'  

3. Click Search to view the search result.

Search by File Path

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

filePath = '/folder name/sub-folder name/fileName.jpg' 

3. Click Search to view the search result.

Search by Height

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

itemHeight = 234

Use the height value (for example, "height": 234) from the item metadata.

3. Click Search to view the search result.

Search by Width

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ItemWidth = 234

Use the width value (for example, "width": 234) from the item metadata.

3. Click Search to view the search result.

Search by Item ID

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ItemID = '65798f8f81b02fbafe34fcad'

3. Click Search to view the search result.

Search by Discarded Items

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ItemStatus = 'Discard' 

3. Click Search to view the search result.

Search by Completed Items

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ItemStatus = 'Complete' 

3. Click Search to view the search result.

Search by Approved Items

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ItemStatus = 'Approve' 

3. Click Search to view the search result.

Search by JSON (Media Type)

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'application/json' 

3. Click Search to view the search result. It displays the all the JSON files.

Search by PCD (Media Type) Files

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'application/pcd' 

3. Click Search to view the search result. It displays the all the PCD files.

Search by Audio Files

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'audio/*'

3. Click Search to view the search result. It displays the all the audio files.

Search by Image Files

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'image/*'

3. Click Search to view the search result. It displays the all the image files.

Search by Text Files

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'text/*'

3. Click Search to view the search result. It displays the all the text files.

Search by Video Files

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

MediaType = 'video/*'

3. Click Search to view the search result. It displays the all the video files.

Search by Frames Per Second

Search Video Files by Frames Per Second (Metadata - fps) in the Dataset.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.fps = 25

Use the fps value (for example, "fps": 25) from the item metadata.

3. Click Search to view the search result.

Search by System Metadata's Mimetype

Search Items by System Metadata's Mimetype in the Dataset.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.mimetype = 'image/jpeg'

Use the mimetype value (for example, "mimetype": "image/jpeg", "mimetype": "text/html", or "mimetype": "audio/mp3" ) from the item metadata.

3. Click Search to view the search result.

Search by File Size Using Metadata

Search Items by File Size from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.size = 'value' 

3. Click Search to view the search result.

Search by Frames Per Second Using Metadata

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.fps = 15

Use the fps value (for example, "fps": 15) from the item metadata system.

3. Click Search to view the search result.

Search by Duration Using Metadata

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.duration = 30.8

Use the duration value (for example, "duration": 30.8) from the item metadata system.

3. Click Search to view the search result.

Search by Width Using Metadata

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.width = 320

Use the Width value (for example, "width": 320) from the item metadata system.

3. Click Search to view the search result.

Search by Height Using Metadata

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.height = 240

Use the Height value (for example, "height": 240) from the item metadata system.

3. Click Search to view the search result.

Search by Dataset Tags Using Metadata

Search Items by the Dataset Tags (Train, Validation, or Test) from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.tags.train = true 

Use the Tags value from the item metadata system.
for example,

    "tags":
        "train": true

Or

   "tags":
       "validation": true

Or

    "tags":
        "test": true

Use the Tags value from the item metadata system.

3. Click Search to view the search result.

Search by Reference ID Using Metadata

Search Items by the Reference ID from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.refs.id = 649c421b9084a344b862289b

Use the refs ID value (for example, "id": "649c421b9084a344b862289b") from the item metadata system.

3. Click Search to view the search result.

Search by Reference Type Using Metadata

Search Items by the Reference Type from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.refs.type = 'assignment' 

Use the Type value (for example, "type": "assignment") from the item metadata system.

3. Click Search to view the search result.

Search by Encoding Using Metadata

Search Items by the Encoding from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.encoding = 7bit

Use the Encoding value (for example, "encoding": "7bit") from the item metadata system.

3. Click Search to view the search result.

Search by Original Name Using Metadata

Search Items by the Original Name from the Item Metadata.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

metadata.system.originalname = video-tutorial-v1.mp4

Use the originalname value (for example, `"originalname": "video-tutorial-v1.mp4") from the item metadata system.

3. Click Search to view the search result.

Search by Model Test Set Items

Search Items are Designated for a Model Test Set.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ModelTestSet = true

or,

ModelTestSet = false 

3. Click Search to view the search result.

Search by Model Train Set Items

Search Items are Designated for a Model Train Set.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ModelTrainSet = true 

or,

ModelTrainSet = false 

3. Click Search to view the search result.

Search by Model Validation Set Items

Search Items are Designated for a Model Validation Set.

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

ModelValidationSet = true 

or,

ModelValidationSet = true 

3. Click Search to view the search result.

Search by Updated Date

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

updatedAt = (10/04/2024)

3. Click Search to view the search result.

Search by User Who Updated

1. In the Data Browser, click on the Items field.

2. Select or enter the query as follows:

updatedBy = 'name@dataloop.ai'

3. Click Search to view the search result.

Search Items by Annotation's Data

1. In the Data Browser, click on the Annotation field.

2. Select or enter the required search query.

3. Click Search to view the search result.

Annotation ID

Search Annotations by Annotation ID.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

annotationId = '6638e5f4fb9b79bc00d04288'

Copy the annotationId value from opening a labeled item -> select an annotation from the right-side panel -> click on the 'i' (info) icon -> copy annotation ID.

3. Click Search to view the search result.

Confidence Value

Search Annotations by Label's Confidence Value.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

Confidence = '402.33'

3. Click Search to view the search result.

Creation Date

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

createdAt = (03/12/2023)

3. Click Search to view the search result.

Annotation Creator

Search Annotations by Annotation Creator.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

creator = 'creator's email ID' 

3. Click Search to view the search result.

Dataset ID

Search Annotations by Dataset ID.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

datasetId = '65950d1d5c356a5e51f6727e'

3. Click Search to view the search result.

Attribute’s Name

Search Annotations by Attribute’s Name.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

attributeName.<name of the attribute> = '<attribute value>'

For example,

• Attribute Name = How is it. By default, it displays all available attributes.

• Attribute Value = Good or Bad. By default, it displays all available attribute values.

then, the query will be attributeName.How_is_it = 'Good'

3. Click Search to view the search result.

Attribute ID

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

attributeName.<ID of the attribute> = '<attribute value>'

For example,

• Attribute ID = 1. By default, it displays all available attribute ID.

• Attribute Value = Good or Bad. By default, it displays all available attribute values.

then, the query will be attributeId.1 = 'Good'

You can copy the attributeId value from the Recipe -> Labels & Attributes -> Right-side section (Create Section)

3. Click Search to view the search result.

Item ID

Search Annotations by the Item ID.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

itemId = '65798f8f81b02fbafe34fcad'

3. Click Search to view the search result.

Label Name

Search Annotations by the Label Name.

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

label = 'Cat'

3. Click Search to view the search result.

Model Name

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

modelName = 'yolov3'

You can copy the model name value from the JSON file of the item.

3. Click Search to view the search result.

Parent Annotation (Parent ID)

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

parentId = '1234'

3. Click Search to view the search result.

Source of the Annotation

Search Annotations by the Source of the Annotation.

1. In the Data Browser, click on the Annotations field to find where the annotation is created.

2. Select or enter the query as follows:

type = 'ui' 

or

type = 'sdk' 

3. Click Search to view the search result.

Label's Attribute

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

metadata.system.attributes.1 = 'Yes' 

3. Click Search to view the search result.

Annotation Type

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

type = 'cube'

3. Click Search to view the search result.

Update Date

1. In the Data Browser, click on the Annotations field.

2. Select or enter the query as follows:

updatedAt = (03/12/2023)

3. Click Search to view the search result.

Search Items by Task's Data

1. In the Data Browser, add the Tasks filter.

2. In the Tasks' field, select task's ID or Name criteria from the list.

3. Enter the value and click Search to view the search result.

Task's ID

1. In the Data Browser, add the Tasks filter.

2. In the Tasks' field, select the TaskID from the list.

3. Enter the value as follows.

TaskID= 'task's ID'

4. Click Search to view the search result.

Task's Name

1. In the Data Browser, add the Tasks filter.

2. In the Tasks' field, select the TaskName from the list.

3. Select or enter query as follows:

TaskName = 'task's Name'

4. Click Search to view the search result.

Free Text - CLIP Based

1. In the Data Browser, click on the Add Filters.

2. Select the NLP (CLIP) from the list. A new field NLP (CLIP) is added.

3. Enter the required text in the NLP (CLIP) field. For example, "Image with red cars".

4. Click Search to view the search result.

Search Query Variables

DQL Search

The Dataloop Query Language (DQL) provides a powerful and flexible way to search, filter, and retrieve items within a dataset. Unlike Basic Search, which uses predefined filters, DQL enables users to build custom, fine-grained queries that can combine multiple conditions and operators.

DQL Syntax Highlights

DQL queries are written in a structured JSON-like format. Some of the common elements include:

• Field Operators:

  • =, != (equals / not equals)

  • >, <, >=, <= (range comparisons)

  • in, not in (match against lists)

• Logical Operators:

  • $and, $or, $not (combine multiple conditions)

• Wildcard & Regex Support:

  • Search partial text patterns or match flexible strings.

Learn more

Use the DQL Query Editor to Search

In addition to data search using the basic UI, it is possible to filter data using the Dataloop Query Language.

1. In the Dataset Browser, click on the DQL tab.

2. Click on the DQL Editor icon. The DQL Search window is displayed.

3. Edit the query as required, or select a saved query from the list.

4. Once edited, click Search to run the search query, or click Save As to save it.

Copy the DQL Filter Query

1. In the Dataset Browser, click on the DQL tab.

2. Click on the Copy Filter icon. A confirmation message is displayed, and the query will be copied in DQL format.

Filters Operands

The following operators are applied within and between filters, unless otherwise specified when manually modifying a DQL query.

Cross Filter Operand

A relationship between multiple filters in a single query is based on the AND operand.
For example, filtering by status Annotated AND the user john@doe.ai AND the annotation type: Box, will return items that are annotated, have Box annotation, and have john@doe.ai as an annotator in one or more of the annotations.

Inter-Filter Operand

The operand relationship between multiple values in a specific filter is based on the OR operator.
For example, filtering by labels with the values Person and Dog, the filter will return all items with annotations of either of these labels, not necessarily both at the same time.

Search Items by Free Text - CLIP Based

NLP (Natural Language Processing) with CLIP (Contrastive Language-Image Pre-Training) refers to the integration of language understanding capabilities with visual recognition. CLIP is a neural network model that leverages NLP techniques to process and understand text while simultaneously analyzing and comprehending images. This enables the model to interpret and align textual descriptions with corresponding visual content effectively.

You can input any descriptive text, and the model will retrieve images or data entries that best match the query, significantly enhancing the user experience by making the search process more intuitive and flexible. Additionally, you can use it alongside other search fields such as Annotations and Items to further refine and improve your search results.

1. In the Data Browser, click on the Add Filters.

2. Select the NLP (CLIP) from the list. A new field NLP (CLIP) is added.

3. Enter the required text in the NLP (CLIP) field. For example, "Image with red cars".

4. Click Search to view the search result.

Search-Filter Actions

The Dataloop platform allows you to customize and save search queries within data querying or search systems, designed to streamline and optimize user interactions with large datasets.

Save a Search Query

Dataloop platform offers the ability to save specific search filter criteria, allowing for efficient and consistent future searches.

1. In the Dataset Browser, create a search query.

2. Click Filter Actions.

3. Select Save. A dialogue window is displayed.

4. Enter a name for the new filter query.

5. Click Save. A confirmation message is displayed.

Use a Saved Search Query

Dataloop platform offers the ability to reuse specific search filter criteria.

1. In the Dataset Browser, click on the Filter Actions.

2. Select Saved Filters and choose the saved filter from the list. Clicking on the saved filter allows the search query to run and displays the result.

Delete a Saved Search Query

Dataloop platform offers the ability to reuse specific search filter criteria.

1. In the Dataset Browser, click on the Filter Actions.

2. Select the Save Filters.

3. Find the query to be deleted and click on the Delete icon when you hover over.

4. Click Delete Query. A confirmation message is displayed.

Searchable and Unsearchable Metadata

• Searchable metadata refers to key-value pairs stored under metadata.user that are indexed by Dataloop. Learn more.

• Unsearchable metadata in Dataloop refers to custom key-value pairs stored under alternative subpaths like metadata.user.unsearchable, which are not indexed by the platform. Learn more.