---
title: "Studio Overview"
slug: "gen-ai-eval-overview"
updated: 2026-04-08T08:19:30Z
published: 2026-04-08T08:19:30Z
canonical: "docs.dataloop.ai/gen-ai-eval-overview"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Studio Overview

GenAI Evaluation Studio is a powerful, user-friendly tool for evaluating GenAI responses, provide feedbacks, etc. It enables you to design **custom evaluation forms** using the **Multimodal Layout Builder** and run structured assessments directly within the DDOE platform.

> [!NOTE]
> **Recipe Type** - **GenAI / Multimodal Recipe**

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/genai_studio.png)

## Key Capabilities

This studio allows you to **evaluate** the GenAI output—structured as a `.json` item—within a custom-designed layout using the Multimodal Layout Builder.

- **These layouts represent structured evaluation forms that can include**:
  - Text inputs
  - Dropdowns, radio buttons, and checkboxes
  - Star ratings and sliders
  - Multimodal content such as conversations, URLs, images, audio, and video
  - Custom validations, JavaScript logic, and CSS styling
- **Flexible Assignment Modes**:
  - **Unified Layout** for all items
  - **Per-Item Layouts** for varied formats
- **Integrated Annotations**: Every evaluator action is stored as an annotation.
- **Validation Controls**: Built-in and custom rules to ensure completeness.
- **Multimodal Support**: Evaluate text, images, audio, video, and mixed formats in a single interface.

---

## Use Cases

- Model evaluation & benchmarking
- Human-in-the-loop (HITL) reviews
- Multi-modal model assessment
- Data quality scoring
- Feedback collection for fine-tuning

## Workflow

1. **Design the Layout:** Create a **Multimodal Layout Recipe** in the Layout Builder to define the evaluation form.
2. **Assign the Layout:** Apply the layout at **task level** (Unified Layout) or per **individual item** (Per-Item Layout).
3. **Upload Items:** Import evaluation items manually or via SDK.
4. **Run the Task:** Annotators evaluate items in the **GenAI Evaluation Studio** using your custom form.
5. **Export Results:** Export annotations for reporting, analysis, or integration into downstream workflows.

> [!NOTE]
> **Any modifications made within the Evaluation Studio are treated as annotations.**
> 
> This includes actions such as providing a rating, selecting options, entering comments, or any other form of input captured during the evaluation process.

---

## Use GenAI Evaluation Studio

The **GenAI Evaluation Studio** is a dedicated interface within DDOE’s platform designed for **reviewing, scoring, and validating outputs from GenAI models** using structured forms and logic defined in a **Multimodal Recipe**.

### Choosing the Right Layout Mode

1. [**Unified Layout for All Items**](/1-3-8/docs/gen-ai-eval-overview#using-a-unified-layout) **(Default Approach):** In this mode, **all items in the task share the same layout recipe**.
  1. **Use Case**: Ideal for structured evaluations where every item follows a common annotation format.
  2. **Recommendation**: Use this as your go-to method for most evaluation scenarios.
2. [**Different Layout Assignment per Items**](/1-3-8/docs/gen-ai-eval-overview#using-different-layouts)**:** In this approach, **different items within the same task or dataset can have distinct layout recipes**.
  1. **Use Case**: Suitable for **mixed evaluation scenarios**, such as comparing chats, images, and structured responses within the same dataset.
  2. **Important Note**: This **overrides** the layout connected to the task or dataset.
  3. **Reference**: A full example script can be found under **Layout Editor → Sample Data →** `</>` **icon (Python script generator)**.

### Using a Unified Layout

All items share the same structure and evaluation format.

**Example:** Each item has one prompt, two images, and the evaluator selects the best image.

1. Create a [Multimodal Recipe](/1-3-8/docs/genai-recipe#create-genai-multimodal-recipes) via **Layout Builder**.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/genai_create_example1.png)

1. [Create a dataset](https://docs.dataloop.ai/docs/create-your-datasets) and connect the recipe to it. If the dataset is already available, use the [Switch Recipe](/1-3-8/docs/manage-your-datasets#switch-recipes) option to change it to the newly created recipe.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/create_dataset.png)

1. Upload your GenAI items (JSON files) to the dataset. You can upload via manually or SDK. An example JSON structure is given below:

```json
{
  "image": "https://upload.wikimedia.org/wikipedia/en/b/b9/MagrittePipe.jpg",
  "prompt": [
    {
      "role": "",
      "content": "Create an image of a pipe"
    }
  ]
}
```

1. If your items are in a CSV file, make sure to convert the CSV into JSON files as required.
2. [Create a task](/1-3-8/docs/create-labeling-tasks#🔖-create-a-labeling-task) with the same recipe.
3. Users with an Annotator roles can open the task in **GenAI Evaluation Studio**.
4. Perform the feedback and ratings as per the layout structure.
5. Export annotations when completed.

---

### Using Different Layouts

Different items within the same dataset or task can have unique layouts.

**Example:** One prompt produces a text response, another produces an image.

1. Create a [Multimodal Recipe](/1-3-8/docs/genai-recipe#create-genai-multimodal-recipes) via **Layout Builder** for each unique layout. You can use the [Template Library](/1-3-8/docs/genai-recipe#templates-library) to duplicate and modify similar layouts.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/genai_create_example1.png)

1. [Create a dataset](https://docs.dataloop.ai/docs/create-your-datasets) for the items, if it is not available.
2. If your items are in a CSV file, make sure to convert the CSV into JSON files as required.
3. Upload your GenAI items (JSON files) **via SDK** and include layout mentioned in the metadata as given below:

```json
'system': {
            'shebang': {
                'dltype': 'evaluation-studio'
            },
            'evaluation': {
                'layoutName': '{INSERT RELEVANT LAYOUT RECIPE ID HERE}'
            }
        }
```

```json
{
  "image": "https://upload.wikimedia.org/wikipedia/en/b/b9/MagrittePipe.jpg",
  "prompt": [
    {
      "role": "",
      "content": "Create an image of a pipe"
    }
  ]
}
```
  1. To receive the Recipe ID, read [here](/1-3-8/docs/manage-your-recipes#copy-recipe-id).
  2. An example JSON structure is given below:
4. [Create a task](/1-3-8/docs/create-labeling-tasks#🔖-create-a-labeling-task) and you can choose any recipe. Even if a recipe is assigned at the task level, **item metadata overrides it**.
5. Users with an Annotator roles can open the task in **GenAI Evaluation Studio**.
6. Perform the feedback and ratings as per the layout structure.
7. Export annotations when completed.

### Open the JSON file

1. Open the **Dataset Browser** and select the **JSON** file.
2. Double-click on the **JSON** file. The GenAI Multimodal Studio is displayed.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/gen_ai_eva_studio.png)

1. Make the required evaluation steps as per the structure. The structure will vary according to the GenAI recipe structure.
2. Click **Save**.

### JSON Formats

You can export the [items](/1-3-8/docs/evaluation-studio-json-output#item-json-output) and [items with annotations](/1-3-8/docs/evaluation-studio-json-output#item-json-output-with-annotations) as JSON files.