Data Filtering
  • Dark
    Light
  • PDF

Data Filtering

  • Dark
    Light
  • PDF

Filters are part of the Dataset and Task Browsers, enabling you to filter items based on every aspect of your files.

Simply, click on the filter you wish to apply from the list of filters on the left, select or enter the value you wish the filter to have, and click “Submit Filter”. Each filter field is explained below.

The following example demonstrates filtering the dataset for items uploaded between 1st Dec, 2021 and 31st Dec, 2021, of item status “Annotated,” and that include the labels “text” and “difficult.”

image.png

When multiple filters are used, the relationship between them will be the AND logical operator.

For example, filtering by status: "Annotated" AND contributor:”john@doe.ai” AND annotation type: “Box”, will return items that are annotated, have Box annotation and have john@doe.ai as an annotator in one or more of the annotations.

However, the relationship between multiple values in each filter will be the OR logical operator. For example, entering "dog" and "cat" in the Labels filter, will result in all items that have a "dog" label OR "cat" label.

Create a Filter

  1. Click on the pencil icon to select the filter fields you need and click "Apply".
  2. Click on the field tab and select your value.
  3. Click "Submit Filter" to filter the items.
  4. Click "DQL" to see the filter as a DQL query. You can also edit the filter directly by editing the query.
  5. Click the 3 dot icon to save, delete, or add a new filter.
  6. Your created filters will be easily accessible from the dropdown list on the left.

newFilterPage.gif

Filter Context

Whether you filter an entire dataset, specific folders, or specific tasks – you can see your filtering context and make quick adjustments.

filters context.gif

Filter Fields

Field Value
Directory

Select a directory to filter the items in that directory.

image.png

Checkboxes allow you to select multiple folders. Use the icons on the right to create a new subfolder or rename an existing one.

folder Management.gif

Tasks

Filters can be applied to datasets/folders or tasks, not both at once.

Click the "Tasks" icon:

Filter items by task names they are assigned to.

image.png

  • Use the checkboxes to select multiple tasks.
  • Select 'No task assignment' if you wish to filter unassigned items.

Item Status

Dataset Browser

image.png

  • Selecting “Annotated” will return only the annotated items in the Dataset.
  • Selecting “Not Annotated” will return only the unannotated items in the Dataset.

Task Browser

Using the Item Filter In the Task Browser allows you to select the following options to filter items by status in a specific task.

Annotations Tasks:
image.png

QA Tasks:

image.png

If different Actions are created when creating the task (e.g., through a pipeline), these Actions will replace the default Actions of Complete/Discard for annotation tasks and Approve/Discard for QA tasks.

Select dates to filter items by status that was created in that date range.

Untitled Project.gif

Labels

This filter will return the items based on the selected Annotations that are saved in the item.

For example, if you input “dog” and “cat” the filter will return all items that have “dog” Annotations and all the items that have “cat” Annotations.

Contributors

Filters will return all items annotated by the selected Contributor.

Item Uploaded Date

The “From…” field will return the items uploaded to the platform from that given date

The “To…” field will return the items uploaded to the platform before that given date

File Name

This filter will return items whose filenames include the text entered in the filter.

For example, submitting “jpg” and “dog” will result in all the items that have either one of the values in the filename.

File Extension

The filter will filtere items based on their file extension.

For example, “png” and "jpg" will return all files whose extension is either png or jpg.

Attributes

This filter will return the items based on the selected Annotations that are saved in the item.

For example, if you input “dog” and “cat” the filter will return all items that have “dog” attributes Annotations and all the items that have “cat” attributes Annotations.

To select all item with NO attributes (e.g. find items where annotators may have forgot to add attributes), edit the DQL query to include an empty array.
"attributes": {}

       "metadata": {
        "system": {
            "attributes": {},
            ......
                         }
                           }
Annotation Type

Selecting a tool from the dropdown list will return all items that contain the selected tool.

For example, if you input “Box” and “Point” the filter will return all items that have “Box” Annotations and all the items that have “Point” Annotations.

See Annotation Tools for reference.

Item Size

Avalable size units are KB, MB and GB.

The “Min…” field will return the items of larger number from the given size

The “Max…” field will return the items of smaller number from the given size

Item Metadata

Filter items based on the metadata of an item.

Text will be used for values that are text.

Number will be used for values that are numbers.

The following example shows how to find items that have the field "details": {"car-color": "blue"} in the user metadata.

The "user" metadata includes fields in the item metadata that are outside the "system" field.

For example, the field "details" is a part of the user metadata:

{
	"system": {
		"originalname": "name2.png",
		"size": 675834,
		"encoding": "7bit",
		"mimetype": "image/png",
		"annotationStatus": []
	},
	"details": {
		"car-color": "blue"
	}
}

image.png

Key: details.car-color
Value (case sensitive): blue

recipe

  • Select Item Metadata in filters
  • Toggle the switch to User (this tells the filter to look outside of the "system" field in the item's metadata)
  • In Key input details.car-color
  • In Value input blue

*Both Key and Value are case-sensitive

Annotation System Metadata

Filter items based on the metadata of annotations in items.

  • Text will be used for values that are texts
  • Number will be used for values that are numbers

In the annotation metadata are 2 relevant fields:

  • status is the annotation status, options: approved/issue/review
  • description is the annotation description, is is free text and case sensitive

Here is an example of how to find items that have annotations with "Approved" status

       "metadata": {
        "system": {
            "status": "approved",
            "startTime": 0,
            "endTime": 1,
            "frame": 0,
            "endFrame": 1,
            "snapshots_": [
            ......
            ],
            "automated": false,
            "isOpen": false,
            "system": false,
            "description": "Happy dog"
    }
}

recipe

  • Select Annotation Metadata in filters
  • Select Add Field and Text
  • In Key input metadata.system.status
  • In Value input approved

*Both Key and Value are case-sensitive

Item Media Type

Filter is based on the item media type. For example “image/png” will return all png images and “video/mp4” will return all mp4 videos

You can also write "image" or "video" to filter image and video items.


What's Next