---
title: "Review (QA) Task"
slug: "review-qa-task"
updated: 2026-07-04T09:54:18Z
published: 2026-07-04T09:54:18Z
canonical: "docs.dataloop.ai/review-qa-task"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Review (QA) Task

## Overview

The quality assurance process ensures that annotations are compliant with certain quality standards or requirements, as well as identifying and addressing issues, as well as verifying that the desired level of quality has been achieved.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/QA%20Flow%281%29.png)

The QA process consists of the following steps:

1. Reviewers can use two tools to indicate quality issues during QA.
  1. Create an **Issue** with an annotation. It is used when there is a quality problem with an annotation, such as a wrong position, label, attribute, etc.
  2. Create a **Note** annotation. It is used to indicate that an annotation is missing. The note annotation is created with an issue on it.
2. Having an issue with an item in a QA task removes its status from the original task where the annotation was created.
  1. For note annotations, since they are created in the QA task, the issue is assigned by default to the last person who sets a status on the item. However, the note reporter can manually change the note assignee. [Learn more](https://docs.dataloop.ai/docs/note#create-an-qa-issue-note) article.
3. Annotators will see open issues on assignments, and the task owner or manager can see them on the task level.
4. Annotators can start correcting the annotations.
5. After correcting an annotation, the annotator flags it **For review** and sets the status to **Complete**. This will make the item appear again in the QA task.

Note

If necessary, annotators can create an issue in the QA task.

1. The reviewer can view how many **For review** annotations are pending on their assignments and open them for review.
2. If corrections are accepted, the reviewer can flag them as **Approved**. If all corrections are approved, the reviewer can set the item's status to **Approve**.

To gain a clearer understanding of the QA process, see the [QA Process Example](/docs/review-qa-task#qa-process-example).

Important

To ensure people learn from their mistakes and progress on the learning curve, Dataloop QA always favors the original annotation creator by default for any correction work.

---

## Review (QA) Process

#### QA User Roles

- **Create QA Tasks**: An Annotation Manager creates QA tasks, redistributes and reassigns these tasks to annotators, reviews their tasks, etc.
- **QA Tasks Assignees**: QA tasks can be assigned to annotators, annotation managers, developers, and project owners.
- **Report Issues**: Issues can be reported only by annotation manager role and above.

Info

If necessary, annotators can create an issue in the QA task.

---

#### QA Process Example

The following example gives you an idea of steps involved in the QA process.

**Step 1: Create a QA Task**

The annotation manager views the completed annotation task and creates a QA task (in this example, the annotation manager assigns the QA task to self and acts as a QA reviewer, but you can assign it to roles). Learn to [Create a review QA task](/docs/review-qa-task#create-a-review-qa-task).

**Step 2: Create Issues During the QA**

During the QA task, the QA tester (in this case, the annotation manager):

- Discovers issues with a couple of annotations and opens an issue with them.
- In addition, the QA tester discovers a few objects that are not annotated, and since there is no annotation, adds an issue for them.
- Creates a note (a ladybug icon) over the unannotated objects with instructions for the annotator. For more information about the Note, see the [QA Note Annotation](https://docs.dataloop.ai/docs/note#qa-note-annotation) article.

When an **issue** or **note** is created:

- The **Completed** status is removed from the item.
- The assignment becomes active again.
- An email notification will be received.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/qa_process_qa_create_issues.png)

Resolve the Issues

The item cannot be completed or approved until the issues have been resolved and approved by the QA tester.

On the Tasks page, the progress bar shows that one item is not completed, and there are issues remaining to be resolved.

**Step 3: View Issues Reported During the QA**

The annotation manager can view **tasks**, **open issues**, and **for review** items. The annotator can see that there are open issues with the tasks in the **Tasks** page.

- Hovering over the red exclamation displays the number of open issues.
- Double-clicking on the exclamation opens the item with issues.

**Step 4: Correct the Issues**

1. The annotator corrects the annotations and marks those annotations **For Review** in the **Annotations** tab by clicking on the hourglass icon.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28119%29.png)

Info

If necessary, annotators can create an issue in the QA task.

1. The annotator clicks **Complete**. On the **Tasks** page, the task is shown with items **For review**.

Similarly, the **Tasks** page on the QA tester’s platform shows that the annotator has submitted items for review.

**Step 5: Review QA Corrections and Approve**

The QA tester clicks the **For Review** to view the corrected annotations on the items.

If the issues have been corrected satisfactorily, the QA tester approves them by:

1. Selecting the annotations that are marked for review.
2. Clicking on the **Approve** checkmark icon.

At this point, the **For review** icons disappear, and the QA tester can click **Approve**.

**Both the annotation task and the QA task are now complete.**

---

## Create Review (QA) Tasks

You can create Review (QA) tasks based on the following two scenarios:

- **A Review (QA) Task from a Labeling task**: To validate annotations created by assignees.
- **A Standalone Review (QA) Task**: To validate annotations that are uploaded to the platform, for example, Annotations created by your model.

On the tasks page, Review (QA) Tasks are linked to their respective annotation tasks. Click the "+" icon next to an annotation task to see all Review (QA) tasks related to it.

To create a Review (QA) task, follow the instructions for each section:

1. Open the **Human Tasks** from the left-side menu.
2. Click **Create Task**. The task type section popup is displayed.
3. Select the **Review (QA)** *Single Task*.
4. Click **Continue**.

#### 1. General

Enter or select the required details in the **General** section:

1. **Task Name**: By default, **your task name - QA** is displayed. Modify, if needed.
2. **Owner**: By default, the current user's email ID is displayed. Click on it to select a different owner from the list.
3. **Priority**: Select a priority from the list. By default, **Medium** is selected.
4. **Completion Due Date** (Optional): Select a task due date from the calendar. If a date is chosen, you will have the option to lock the QA task once the due date has passed.
5. **Lock Task After Due Date**: When enabled, all assignments in this task will be locked for editing after the due date passes. Read [here](/1-3-8/docs/manage-your-tasks#lock-unlock-tasks) to unlock the task.
6. Click **Next: Data Source**.

#### 2. Data source

Enter or select the required details in the **Data Source** section.

1. **Select Dataset**: By default, the dataset used to create the task is displayed.
2. (Optional) **Filters**: Refine data selection by selecting specific folders, using DQL filters, or subsampling (randomly and equally distributed). The Folder or DQL field is **Active** only if you do not select any items in the Dataset.
  1. **Folders**: Select a folder from the dataset.
  2. **Selected Filters / Saved DQL Query**: Select a filter or saved DQL query from the list.
  3. **Data Sampling**: Enter the **Percentage** or **Number of Items** for the task. Data sampling does not give an exact number of items.
    1. **Percentage**: The option selects the items randomly. **For example**, if the percentage is 100% for four items, then 75% is for three items (It can be 1/4, 3/4, or 4/4) from the selected dataset.
    2. **Number of Items**: The allows you to select the items sequentially from the start of the dataset, not randomly.
  4. **Collections**: Choose a collection from the list to filter and display items within the selected collection.
3. Click **Next: Recipe and Instructions**.

#### 3. Recipe and Instructions

In the **Recipe and Instructions** section, enter or select a recipe according to your task and data type. The number of Labels and Attributes is displayed on the top-right side of the page:

1. **Recipe**: Select a recipe from the list. A preview of the recipe is displayed on the right-side panel.
  1. If there is no recipe, select the [Project Recipe](https://docs.dataloop.ai/docs/recipes-overview#project-recipes), or click **Set Recipe** to create an unique recipe for your task. **Annotation Managers** can select the **Project Recipe** and view its details through Quick Preview; however, they do not have permission to open or modify the Project Recipe.
  2. (Optional) Click on the recipe drop-down, and click on the **three dots** of a recipe to view more actions like [clone recipe](/1-3-8/docs/recipe-actions#clone-recipes), open the recipe in a new tab, etc.
  3. (Optional) Click on the **three dots** from the right-side **Recipe Preview** for more actions like [clone recipe](/1-3-8/docs/recipe-actions#clone-recipes), open the recipe in the editor, [export ontology](/1-3-8/docs/recipe-actions#export-ontology), etc.
2. **QA Instructions (.pdf)**: The QA Instruction document is displayed, if available. Go to the **Recipe** section to upload a PDF instruction.
3. Click **Next: Statuses**. The **Statuses** section is displayed.

#### 4. Statuses

Enter or select the required details in the **Statuses** section:

1. By default, the **Approved** status is selected. Click **Add New Status** to add a new status.
2. Click **Next: Assignments**.

#### 5. Assignments

Enter or select the required details in the **Assignments** section.

1. **Allocation Method**: Select one of the following allocation methods:
  1. **Pulling**: The pulling distribution method means that annotators only pull a batch of items at a time and the maximum number of items in an assignment. You can make changes in the following fields if required: **Pulling batch size (items)** and **Max items in an assignment**.
  2. **Distribution**: The distribution allocation method means that the items will be distributed in advance among users, equally or based on a custom percentage.
    1. The **Auto Distribution** option distributes the task equally among the users. By default, it is checked.
    2. The **Show only unassigned users to any labeling task** option allows existing users to complete their task.
2. **Available Users**: Search for or select users from the list, and click the **Forward arrow** icon to add them to the Assigned Users list.
3. **Assigned Users**:
  1. Search for or view the assigned users from the list. The allocation percentage is equally distributed if you select Auto Distribution.
  2. Select and click the **Backward** arrow icon to remove them from the Assigned Distribution list.
4. Click **Create Task**.

---

## Set a QA status

When working on a QA assignment, the assignee or QA tester is expected to review annotations and set the item's status to:

- **Approved**, if it is annotated satisfactorily.
- **Discarded**, if the item is unsuitable for the task.
- or raise issues if there are problems with the annotation of the item that the QA tester wants the annotator to fix.

Setting the status on an item will trigger the studio to move on to the next item.

---

## View QA status

You can view the QA status of a task on the following pages, depending on the user role and the type of information requested.

- **Task's page**: When you click a task within the task page, it displays assignments, especially if there are items with open issues or those pending review. Additionally, on this page, you have the options to **Browse Tasks** and **Browse Issues**.
- **Assignments' page**: Click a task to see all its assignments. Each assignment line shows an indication of whether there are open issues or items pending review.
- **Issues' page**: On the **Tasks** > **Click on a task** → **Issues** page, you can see the full list of the issues in a project. For more information about the issue page, see the [Issues](https://docs.dataloop.ai/docs/issues) article.