Labeling Tasks
  • 21 May 2024
  • Dark
    Light
  • PDF

Labeling Tasks

  • Dark
    Light
  • PDF

Article summary

Overview

Labeling tasks page provides you the comprehensive list of annotation and QA tasks, assignments, and task's annotation issues for your project in three different tabs.

  • Tasks: The Tasks tab on the Labeling Tasks page displays the list of annotation and quality assurance tasks created using the Labeling Tasks page or Pipeline, along with their respective information.
  • Assignments: The Assignments tab on the Labeling Tasks page displays the list of task's assignments along with their respective information.
  • Issues: The Issues tab on the Labeling Tasks page displays the list of Items with Annotation issues created during the quality assurance task along with their respective information.

Upon selecting a task, assignment, or issue from the list, the right-side panel presents specific details, including general details, item's and task's status, data source and recipe, assignments, and metadata. Additionally, the labeling tasks page provides a summary of the total count of tasks (not in archive), assignments (not in Archive), issues, and reviews on the top right-hand side of the page.

The Labeling Tasks page can be broken down into the following sections (marked on the screenshot), as outlined below:


Section 1: Create Task and Refresh Buttons

  • Create Task button: To create an Annotation or a QA task, click on the Create Task button. For more information, see the Annotation and QA tasks article.
  • Refresh button: To refresh the list of tasks, assignments, or issues, click on the Refresh button.

Section 2: Search and Filters

The Labeling Tasks page enables you to search and filter the tasks, assignments, or issues. You have the flexibility to choose multiple filter criteria.

Search

The Labeling Tasks page enables you to enter the following search criteria to search tasks, assignments, and issues in the Search field:

  • Tasks: To search for a task, utilize the following criteria:
    • Task Name: This displays tasks with the matching task name.
    • Task ID: It lists the tasks based on their unique task ID. You can either use the Copy Task ID icon next to the Task Name or use the URL to get the Task ID.
    • Dataset name: This displays tasks with the matching dataset name.
  • Assignments: To search an assignment, utilize the following criteria:
    • Assignment Name: This displays the assignment with the matching name.
    • Task ID: This lists assignments based on the task ID of the assignment. You can either use the Copy Task ID icon next to the Task Name or use the URL to get the Task ID.
  • Issues: To search an issue, use the Annotation Name. This displays the issues based on the annotation name.

After entering the search criteria, you can begin the search by either pressing the Enter (or Return) key or by clicking outside the Search field.

Filter

The Labeling Tasks page enables you to use the following filter criteria to filter tasks, assignments, and issues:

Filter Criteria for Tasks

When you click on the Select filters on the Tasks tab, the following filter criteria will be listed:

Filter TypeCriteriaDescription
ChannelPipelineThis displays the tasks, those created using the Pipeline channel.
ChannelWorkflowThis displays the tasks, those created using the Workflow (or new UI - Labeling Tasks) channel.
Task TypeLabeling TaskThis displays the tasks those are Labeling-type tasks.
Task TypeQA TaskThis displays the tasks that are QA-type annotation tasks.
QualityConsensus TaskThis displays the tasks with a Quality task type of Consensus.
QualityQualification TaskThis displays the tasks with a Quality task type of Qualification.
QualityHoneypot TaskThis displays the tasks with a Quality task type of Honeypot.
PriorityHighThis displays the tasks with a high priority.
PriorityMediumThis displays the tasks with a medium priority.
PriorityLowThis displays the tasks with a low priority.
PriorityN/AThis displays the tasks with no priority.
Annotation StatusOpen IssuesThis displays the tasks with annotations that have open issues.
Annotation StatusFor ReviewThis displays the tasks with annotations that have issues corrected for review.

Filter Criteria for Assignments

When you click on the Select filters on the Assignments tab, the following filter criteria will be listed:

Filter TypeCriteriaDescription
Task TypeLabelingThis displays the assignments of those are Labeling-type tasks.
Task TypeQAThis displays the assignments that are QA-type tasks.
PriorityHighThis displays the assignments with a high priority.
PriorityMediumThis displays the assignments with a medium priority.
PriorityLowThis displays the assignments with a low priority.
PriorityN/AThis displays the assignments with no priority.
Annotation StatusOpen IssuesThis displays the assignments with annotations that have open issues.
Annotation StatusFor ReviewThis displays the assignments with annotations that have issues corrected for review.

Filter Criteria for Issues

When you click on the Select filters on the Issues tab, the following filter criteria will be listed:

CriteriaDescription
Open IssuesThis displays the list of items with annotations that have open issues.
For ReviewThis displays the list of items with annotations that have issues corrected for review.

Filter Using the Project Owner or Username

  • Select the Project Owner's name from the Select owner dropdown list to filter the tasks.
  • Select the Assignment's Username from the Select user dropdown list to filter the assignments and issues.

After you select the filter criteria, the system initiates the search function.

Clear Filters: To clear the search or filter criteria, click on the Clear Filters button.


Section 3: List of Tasks, Assignments, and Issues

List of Tasks

The Tasks tab displays a table containing a list of tasks, with their respective details presented in individual columns. Clicking on the Task Name will navigate you to the Item's page for that specific task.

Open Issue, Assignment Issue and For Review Signs
  • The presence of Open Issue and Assignment Issue signs typically indicates the existence of open issues within tasks or assignments. Click on the link to view more details.
  • The presence of For Review signs typically indicates that issue corrections are ready for review. Click on the link to view more details.
  • Task Name: This displays the name of the task. You can click on it to view the task's browser page. Also, you click on the Copy icon to Copy the Task ID.
  • Task Related*: This displays the name of the parent task.
  • Status*: This displays the status of the tasks.
  • Progress*: This displays the item's completion status.
  • Type*: This displays the task type, labeling or QA tasks.
  • Remaining*: This displays the number of remaining items to be completed in the labeling task.
  • Assignments*: This displays the number of assignments associated to the tasks.
  • Open Issues*: This displays the number of items with annotations that have open issues.
  • For Review*: This displays the number of items with annotations that have issues corrected for review.
  • Groups: This displays the group name of the task owner.
  • Task Owner: This displays the email ID and avatar of the task owner.
  • Priority: This displays the tasks with a High, Medium, or Low priority.
  • Channel: This displays the channel type used to create the tasks, Pipeline or Workflow.
  • Dataset: This displays the name of the dataset used to create the task.
  • Updated At: This displays the task update date, for example, Sep, 15 2023.
  • Quality: This displays the tasks with a quality task types, Consensus, Quality, and Honeypot.
  • Created At: This displays the task creation date, for example, Sep, 15 2023.
  • Due Date: This displays the due date for the task set by the task creator. It helps the annotation team to prioritize tasks and plan their schedule accordingly.
  • Allocation Method: This displays the task allocation method - Distribution or Pulling type.

Show/Hide Columns: You can customize the visibility of the column fields in the table view by clicking on Show/Hide Columns.

The default column fields are marked with "*".

List of Assignments

The Assignments tab provides the list of assignments in a table along with their details on each column. Clicking on the Assignment's Name takes you to the Item's page of the assignment.

  • Users: This displays the avatar of the assignment's users.
  • Assignments: This displays the name of the assignments. Use the Copy Assignment ID icon next to the assignment name to copy the Assignment ID.
  • Type: This displays the task type, labeling or QA tasks.
  • Status: This displays the status of the assignments.
  • Priority: This displays the assignments' with a High, Medium, or Low priority.
  • Progress: This displays the item's completion status.
  • Open Issues: This displays the number of items with annotations that have open issues.
  • For Review: This displays the number of items with annotations that have issues corrected for review.

You can customize the visibility of the column fields in the table view by clicking on Show/Hide Columns.

List of Issues

The Issues tab presents a table listing items with annotation issues, and it provides details for each item in the respective columns.

  • Item: This displays the thumbnail view of the item.
  • Annotation: This displays the label name used to annotate the item.
  • Assignment: This displays the name of the item's assignment.
  • Dataset: This displays the name of the dataset.
  • Status: This displays the current status of the issue.
  • Open Since: This displays the creation date of the issue.

Also,

  • Click on the More Details to view the details including, File Name, Item ID, Dataset ID, etc.
  • Clicking on the Annotation label, Assignment Name, and Dataset takes you to the respective page.
  • Double-clicking on the issue takes you to the respective type of annotation studio of the item.

Section 4: Task Actions and Assignment Actions

When you click on a task or an assignment, a Task Actions or Assignment Actions action button is available on the right-side panel of the labeling tasks page.
When you click on the Task Actions or Assignment Actions button, the following list of actions becomes available:

Task Actions

  • Edit Task: Edit the task allows you to change task details, such as the assignees, due date, and more. For more information, see the Edit an Annotation article.
  • Open Task Browser: This allows you to open and view the task browser page.
  • Open Task Analytics: This takes you to open and view the Analytics page of the task. For more information, see the Analytics article.
  • Delete Task: This allows you to delete the selected task.

Assignment Actions

  • Clear Batch | This allows you to clear the assignment batch of the tasks which are pulling tasks. |
  • Load Batch | This allows you to load the assignment batch of the tasks which are pulling tasks. |
  • Open Assignment Browser | This allows you to open and view the assignment browser page. |
  • Open Assignment Analytics | This allows you to open and view the Analytics page of the assignment. For more information, see the Analytics article. |
  • Reassign: This allows you to reassign the task to a new user.

Section 5: Task and Assignment Details

When you select a task or an assignment, the right-side panel of your Labeling Task page displays comprehensive information regarding the selected task or assignment, offering the following details for both tasks and assignments:

Open Issue, Assignment Issue, and For Review Alerts
  • The presence of Open Issue and Assignment Issue alerts typically indicates the existence of open issues within tasks or assignments. Click on the link to view more details.
  • The presence of For Review alerts typically indicates that issue corrections are ready for review. Click on the link to view more details.

Tasks - General Details

The General Details section of the task provides you the following general information:

  • Browser icon: This allows you to open and view the task browser page.
  • Analytics icon | This takes you to open and view the Analytics page of the task.
  • Task Owner: This displays the email ID and avatar of the task owner.
  • Priority: This displays the tasks with a High, Medium, or Low priority.
  • Channel: This displays the channel type used to create the tasks, Pipeline or Workflow.
  • Created At: This displays the task creation date, for example, Sep, 15 2023.
  • Updated At: This displays the task update date, for example, Sep, 15 2023.
  • Pipeline: This displays the pipeline name and link if the task is channel is pipeline.
  • Description: The description provided for the selected task. To add a description, click on the Edit icon. You can also use SDK to add or edit description.

Use the Copy Task ID icon next to the task name to copy the task ID.

Tasks - Status

The Status section of the tasks provides you the status of the item as follows:

Progress: This displays the item's completion status.
Open Issues: This displays the number of items with annotations that have open issues.
For Review: This displays the number of items with annotations that have issues corrected for review.

Tasks - Data Source, Instructions & Statuses

This section of the task provides you the following information, related to the dataset and recipe used to create the task:

  • Dataset: The name of the dataset used to create the selected task.
  • Recipe: The name of the recipe selected for this task.
  • Statuses: The annotation task status of the items in the task.

Tasks - Assignments

This section of the task provides you the following information, related to the task assignment:

  • Allocation Method: This displays the type of allocation method used in the task.
  • Assignments: This displays the number of assignments available for the task.
  • Groups: This displays the number of groups assigned.

Tasks - Metadata (Read Only)

This section of the task provides you the metadata information of the selected task. You have the option to copy the metadata, but editing it is not possible.

For example,

{
  "system": {
    "nextIndex": 3,
    "batchSize": 5,
    "maxBatchWorkload": 7,
    "allowedAssignees": [
      "name@dataloop.ai"
    ],
    "groups": [],
    "relatedTasks": [],
    "disableWebm": false,
    "groupsMembers": []
  }
}

Assignments - General Details

The General Details section of the assignment provides you the following general information:

  • Browser icon: This allows you to open and view the assignment browser page.
  • Analytics icon | This takes you to open and view the Analytics page of the assignment.
  • Task Name: This displays the assignment name. You can click on it to navigate to the assignment browser page.
  • User: This displays the avatar of the assignment's users.
  • Priority: This displays the assignments with a High, Medium, or Low priority.
  • Channel: This displays the channel type used to create the tasks, Pipeline or Workflow.
  • Created At: This displays the assignment creation date, for example, Sep, 15 2023.
  • Updated At: This displays the assignment update date, for example, Sep, 15 2023.
  • Description: The description provided for the selected assignment's task. To add a description, click on the Edit icon. You can also use SDK to add or edit description.

Use the Copy Assignment ID icon next to the assignment name to copy the Assignment ID.

Assignments - Status

The Status section of the assignment provides you the following information:

Progress: This displays the item's completion status.
Open Issues: This displays the number of items with annotations that have open issues.
For Review: This displays the number of items with annotations that have issues corrected for review.

Assignments - Metadata (Read Only)

This section of the assignment provides you the metadata information of the selected assignment. You have the option to copy the metadata, but editing it is not possible.

For example,

{
  "system": {
    "datasetId": "64a671e0d854daf23d6ed2b9",
    "taskId": "65091cc7d93bfc635d4f1878",
    "active": true
  },
  "datasetId": "64a671e0d854daf23d6ed2b9"
}

Section 6: View the Number of Tasks, Assignments, Issues, and Reviews

The Labeling Tasks page provides you the number of Tasks, Assignments, Issues, and Reviews available. You can view the following information on the top-right side of the Labeling Tasks page.