---
title: "Create GenAI - Multimodal Recipes"
slug: "genai-recipe"
tags: ["Recipe Structure", "Layout Editor", "Multimodal Recipe"]
updated: 2026-04-08T08:19:27Z
published: 2026-04-08T08:19:27Z
canonical: "docs.dataloop.ai/genai-recipe"
---
> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Create GenAI - Multimodal Recipes
## Overview
A **GenAI** / **Multimodal Recipe** defines how you evaluate GenAI outputs in DDOE. It is a **JSON configuration** that controls the full evaluation experience, including:
- The **structure and layout** of the evaluation form
- The **fields and components** shown to evaluators
- The **user interaction flow**
- The **validation rules and logic** applied to submitted data
This recipe is best suited for **AI output evaluation and validation**, rather than traditional data annotation workflows.
Multimodal Recipes support multiple GenAI output types, such as **text**, **multimodal outputs**, and **conversational responses**. You can:
- Use **pre-built recipes** for common evaluation use cases
- Create **custom recipes** by editing the JSON directly
It is especially useful for **human-in-the-loop evaluation**, quality assurance, and model comparison, where subjective judgment and structured scoring are required.
**Common use cases include:**
- LLM response evaluation and ranking
- Image–text alignment and relevance checks
- Human feedback collection and scoring
- Structured GenAI benchmarking and model comparison
| **Layout** | **Layout Builder** |
| --- | --- |
| A **Layout** defines how the evaluation form appears to the user. It controls the **visual organization** of the form, including: - The order of sections and fields - Grouping of related questions - Arrangement of components (rows, columns, sections) - Placement of multimodal content (for example, text next to images) The layout focuses on **presentation and flow**. It does not define business logic, data storage, or validation rules. | It allows you to create and manage layouts using a visual interface. You use the Layout Builder to: - Design evaluation forms without writing JSON - Add, arrange, and configure form components - Organize sections and groups - Preview the evaluator experience in real time The Layout Builder automatically updates the **layout configuration** inside the Multimodal Recipe. |
## Types of GenAI / Multimodal Recipes
The GenAI / Multimodal Recipes feature lets you build evaluation recipes tailored to different annotation task types.
**Available recipe types:**
- **Prompt–Response RLHF:** Supports structured prompt engineering workflows, enabling annotators to evaluate and rank AI-generated responses in a questionnaire-style interface. You can use conditional logic to dynamically show or hide questions.
- **Standard GenAI Evaluation Recipe:** A customizable, template-based layout for multimodal GenAI evaluation, allowing annotators to provide structured feedback on AI-generated content. Ideal for quickly getting started using prebuilt templates.
- **Unique GenAI Evaluation:** A fully customizable, multimodal evaluation layout that lets annotators provide feedback on AI-generated content. Designed for building a studio layout entirely from scratch.
## Create Prompt-Response RLHF Type Recipes
Supports structured prompt engineering workflows, enabling annotators to evaluate and rank AI-generated responses in a questionnaire-style interface. You can use conditional logic to dynamically show or hide questions.
1. Open **Recipes** from the left-hand navigation panel.
2. Click **Create Recipe** to bring up the configuration dialog.
.png)
1. Choose the **GenAI / Multimodal**. An annotation tasks type selection window appears.
2. Choose the **Prompt-Response RLHF** option from the list.
1. Click **Create Recipe**. The Recipe page will be displayed, and you can start creating or configuring:
- Labels
- Attributes
- Annotation tools
- Instructions
- Advanced Settings
---
## Create Standard GenAI Evaluation Type Recipes
A customizable, template-based layout for multimodal GenAI evaluation, allowing annotators to provide structured feedback on AI-generated content. Ideal for quickly getting started using prebuilt templates.
#### 1. Open the Recipe menu
1. Open **Recipes** from the left-hand navigation panel.
2. Click **Create Recipe** to bring up the configuration dialog.
.png)
1. Choose the **GenAI / Multimodal**. An annotation tasks type selection window appears.
2. Choose the **Standard GenAI Evaluation (template)** option from the list.
1. Click **Create Recipe**. The GenAI / Multimodal Recipe page will be displayed and you can start configuring with a temple from the left-side panel:
- Templates (from the left-side panel)
- Layout & Data
- JavaScript
- CSS
- Sample Data
- Instructions
- Advanced Settings
#### 2. Design the Layout
1. The editor displays a sample format by default. Use your own scripts (including JavaScript and CSS), if required. Refer to the Components library to view various components and its properties can used to customize your recipe structure.
2. Otherwise, click on the **Library** tab to view and use the predefined Templates.
3. To use a template, follow steps below:
1. Select a template from the **Library**, and click **Use Layout.** 🚨 *Once you confirm, the current form and layout will get overwritten by the selected template layout*🚨. Users can also check examples from the templates and copy and paste them to they own template (after clicking "back to edit").
2. Click **Use Layout** to confirm it.
#### 3. Generate Sample Data & Upload the JSON
1. After finalizing the layout, open the **Sample Data** section and click the `</>` **icon**.
2. A **Python Usage Sample** dialog appears containing an auto-generated script.
- Click **Copy** to copy the code.
- Run the script in your environment to upload a JSON data file into the target **dataset** while tagging it for *evaluation-studio* processing.
3. Click on the **Copy** icon to copy the python script and run the script to upload the `json` file to your dataset. Sample Python script is shown below:
```python
import dtlpy as dl
import json
import io
# Get the dataset
dataset = dl.datasets.get(dataset_id='undefined')
# Create a io.BytesIO object from the data json
data = {
"chat_a": [
{
"role": "user",
"content": "Hello!"
},
{
"role": "assistant",
"content": "Hi there! How can I help you today?"
}
],
"chat_b": [
{
"role": "assistant",
"content": "Good morrow, gentle friend!"
},
{
"role": "user",
"content": "Hi!"
},
{
"role": "assistant",
"content": "Hail, good sir or madam! How may I be of service to thee?"
}
]
}
buffer = io.BytesIO(json.dumps(data).encode('utf-8'))
buffer.name = '68404c727b5a5a853549a9a8-data.json'
buffer.seek(0)
item = dataset.items.upload(
local_path=buffer,
item_metadata={
'system': {
'shebang': {
'dltype': 'evaluation-studio'
},
'evaluation': {
'layoutName': '68404c727b5a5a853549a9a8'
}
}
}
)
print("Item uploaded successfully:", item.id)
print("Open in platform:", item.platform_url)
```
#### 4. Save and Create the Recipe
Click **Save** 💾. The new recipe will be listed in the **Recipes** page.
#### 5. Run an Evaluation
1. Locate the uploaded JSON item in the dataset.
2. Open it with **Multimodal Layout Studio**.
3. The Studio loads your Multimodal Recipe, allowing reviewers to fill in responses, raise flags, and leave comments.
4. All inputs are validated automatically—basic **required-field** checks plus any JavaScript rules you added—blocking invalid submissions.
### Recipe Structure
A Multimodal Recipe (written in JSON format) describes:
- **Sections**: Logical groupings of form elements (similar to tabs or blocks), organizing the form into manageable parts.
- **Layout Direction**: Defines the arrangement of components within sections — either horizontal or vertical.
- **Components**: The actual interactive elements used in the form, such as text inputs, sliders, file uploads, ratings, conversations, media viewers (image, audio, video, markdown, external URLs), and more.
- **Logic**: Rules for conditional visibility, dynamic behavior, hierarchical dependencies, and validation logic.
- **Styling**: Custom CSS definitions that control the visual appearance of the form and its components.
- **Behavior**: Custom JavaScript modules that introduce advanced interactivity, dynamic validations, and real-time form manipulation.
**A recipe template has the following sections:**
- [Layout & Data](/1-3-8/docs/genai-recipe#layout-editor)
- [Layout Editor](/1-3-8/docs/genai-recipe#layout-editor)
- [Sample Data](/1-3-8/docs/genai-recipe#sample-data)
- [JavaScript](/1-3-8/docs/genai-recipe#javascript)
- [CSS](/1-3-8/docs/genai-recipe#css)
- [Studio Preview](/1-3-8/docs/genai-recipe#studio-preview)
#### **Layout Editor**
The **Layout Editor** is a critical tool in the recipe creation process. It lets you **design and control the structure** of the evaluation form used by annotators in the studio. You can:
- **Customize your own layout using JSON schema**.
- Choose from a variety of UI components such as:
- `select` – dropdowns
- `text` – text inputs or prompts
- `radio` – radio buttons for single-choice selection
- `checkbox`, `slider`, etc.
For example,
```json
[
{
"components": [
{
"type": "markdown",
"key": "intro"
},
{
"type": "text",
"key": "text",
"title": "Text input",
"tooltip": "Tooltip...",
"hidden": false,
"required": true,
"errorMessage": "Error message...",
"placeholder": "Placeholder..."
},
{
"type": "markdown",
"key": "Components-select"
},
{
"type": "radio",
"key": "radio",
"title": "Radio input",
"options": [
{
"label": "Option1",
"value": "Value1"
}
]
},
{
"type": "markdown",
"key": "Components-checkboxes"
},
{
"type": "checkbox",
"key": "checkbox",
"title": "Checkbox input",
"options": [
{
"label": "Option1",
"value": "Value1"
}
]
},
{
"type": "markdown",
"key": "Components-slider"
},
{
"type": "slider",
"key": "slider",
"title": "Slider input",
"min": 0,
"max": 100,
"step": 1,
"default": 50
},
{
"type": "markdown",
"key": "Components-video"
},
{
"type": "video",
"key": "video",
"title": "Video player"
},
{
"type": "image",
"key": "childField",
"hierarchy": {
"condition": {
"parentField": {
"$eq": "true"
}
}
}
},
{
"type": "markdown",
"key": "Components-more-conditions"
},
{
"type": "markdown",
"key": "js-css"
}
]
}
]
```
### Sample Data
In the **GenAI / Multimodal Recipe** of the **DDOE platform**, the **Sample Data** section plays an essential role in the design and testing process for annotation workflows—especially within the **GenAI** **Evaluation Studio**. It allows you to populate **placeholders** in your **Layout Editor** with sample inputs. It also displays a sample of the annotation output after interacting with the Studio preview. DDOE highly recommends exploring the preview yourself to better understand how the configuration behaves and how the outputs are generated. The **Sample Data** section is designed to **simulate or preview** how a layout will behave **before integrating real data or running actual annotations**. It allows developers, annotation leads, and evaluators to:
- Populate the layout with mock inputs.
- Validate the design, spacing, logic, and visibility conditions in the Studio Preview.
- Test dynamic elements like JavaScript validation or conditional formatting in the Studio Preview.
The following JSON snippet defines separate chat histories: . These are matched to UI components (e.g., two chat boxes) created in the **Layout Editor**.
```json
{
"intro": "# Quickstart Guide\nBackground\n\n
\n\nThe GenAI Layout Builder lets you create multimodal layouts for LLM evaluation tasks. Define the structure with JSON building blocks, and adjust styling and behavior with CSS and JavaScript. Explore our template library to get started quickly. Once ready, use the layout recipe in your GenAI Evaluation tasks.\n\n\nLayout Structure
\n\n
\n\nA layout is a JSON array of blocks, each defining a set of components and settings:\n- `title` (string): Display blocks title\n- `hidden` (boolean): Hide the entire block\n- `layout` (object): Control blocks layout\n - `direction` (string, required): Set to 'horizontal' for horizontal layout\n - `wrap` (boolean): Set to false to disable components wrapping and enable horizontal scrolling\n- `components` (object[], required): The list of component for this block\n\n```json\n[\n {\n 'title': 'Section Title',\n 'hidden': true,\n 'layout': {\n 'direction': 'horizontal',\n 'wrap': false\n },\n 'components': [\n {\n 'type': 'text',\n 'key': 'nameInput1',\n 'title': 'Full Name',\n 'placeholder': 'Enter your name...',\n 'tooltip': 'Please also include your middle name',\n 'required': true,\n 'hidden': false\n }\n ]\n }\n]\n```\nCommon Properties
\n\n
\n\nAll components support these base properties:\n- `key` (string, required): Unique identifier for the field\n- `type` (string, required): Component type identifier\n- `title` (string): Display title for the field\n- `tooltip` (string): Hover tooltip text\n- `hidden` (boolean): Whether the component should be hidden\n- `required` (boolean): Whether the field is required\n- `allowStatus` (boolean): Whether users can report an issue or mark a field for review\n- `hierarchy` (object): Conditional visibility rules\n - `condition` (string): DQL condition to evaluate\nText Input
\n\n
\n\n- Type: `text`\n- Properties:\n - All common properties\n - `placeholder` (string): Placeholder text\n - `disabled` (boolean): Whether the field is disabled\n - `errorMessage` (string): Error message to display\n - `numRows` (number): Optionally increase the number of rows displayed\n\nSingle Selection - Dropdown
\n\n
\n\n- Type: `select`\n- Properties:\n - All common properties\n - `options` (array, required): Array of options\n - `label` (string): Display text\n - `value` (any): Option value\n - `disabled` (boolean): Whether the field is disabled\n\nSingle Selection - Radio Buttons
\n\n
\n\n \n- Type: `radio`\n- Properties:\n - All common properties\n - `options` (array, required): Array of options\n - `label` (string): Display text\n - `value` (any): Option value\n - `disabled` (boolean): Whether the field is disabled\n\nMultiple Selection - Checkboxes
\n\n
\n\n\n- Type: `checkbox`\n- Properties:\n - All common properties\n - `options` (array, required): Array of options\n - `label` (string): Display text\n - `value` (any): Option value\n - `disabled` (boolean): Whether the field is disabled\n\nNumber Single Selection - Slider
\n\n
\n\n\n- Type: `slider`\n- Properties:\n - All common properties\n - `min` (number): Minimum value\n - `max` (number): Maximum value\n - `default` (number): Default value\n - `step` (number): Step increment (minimum 0.01)\n - `disabled` (boolean): Whether the field is disabled\n\nNumber Single Selection - Star Rating
\n\n
\n\n\n- Type: `rating`\n- Properties:\n - All common properties\n - `maxStars` (number): Maximum number of stars (default: 5; min 1, max 10)\n - `showValue` (boolean): Whether to show numeric value\n\nImage Viewer
\n\n
\n\n\n- Type: `image`\n- Properties:\n - All common properties\n\nAudio Viewer
\n\n
\n\n\n- Type: `audio`\n- Properties:\n - All common properties\n\nVideo Viewer
\n\n
\n\n\n- Type: `video`\n- Properties:\n - All common properties\n\nURL Viewer
\n\n
\n\n\n- Type: `url`\n- Properties:\n - All common properties\n - `initialWidth` (number): Initial iframe width\n - `initialHeight` (number): Initial iframe height\n\nMarkdown Viewer
\n\n
\n\n\n- Type: `markdown`\n- Properties:\n - All common properties\n- Insert the markdown text as the value of the markdown component key\n\n*This entire layout is an example*Conversation
\n\n
\n\n\n- Type: `conversation`\n- Properties:\n - All common properties\n- Insert the conversation data as the value of the `convesation` component key, in the form of a JSON object[], and the following fields:\n - `role` (string): Use the word 'user' for the right bubble, or any other string for the left bubble title.\n - `content` (string): The bubble text.\n\nMore Examples
\n\n
\n\n\n- Show if riddle answer contains 'raven':\n```json\n{\n 'condition': { 'riddle_answer': { '$ilike': '*raven*' } }\n}\n```\n- Use AND logic to show if tea party size is 42 and time is frozen:\n```json\n 'condition': {\n '$and': [\n {\n 'tea_party_size': {\n '$eq': '42'\n }\n },\n {\n 'time_frozen': true\n }\n ]\n }\n```\n- Use OR logic to show if tea party size is 42 or time is frozen:\n```json\n{\n 'condition': {\n '$or': [\n {\n 'tea_party_size': {\n '$eq': '0'\n }\n },\n {\n 'tea_party_size': {\n '$eq': 'infinity'\n }\n }\n ]\n }\n}\n```\n- Use IN logic to show if tea party size is 0, 42 or infinity:\n```json\n{\n 'condition': {\n 'tea_types': {\n '$in': ['earl_grey', 'darjeeling']\n }\n }\n}\n```\n- Use EXISTS logic to show if tea party size is not 0:\n```json\n{\n 'condition': {\n 'riddle_answer': {\n '$exists': true\n }\n }\n}\n```\n- Advance exmaple:\n```json\n{\n 'condition': {\n '$or': [\n {\n '$and': [\n {\n 'tea_party_size': {\n '$eq': '6'\n }\n },\n {\n 'time_frozen': true\n }\n ]\n },\n {\n '$and': [\n {\n 'tea_types': {\n '$in': ['chamomile']\n }\n },\n {\n 'riddle_answer': {\n '$exists': true\n }\n }\n ]\n }\n ]\n }\n}\n```\n3. **Dynamic hiding**: Use JavaScript to control visibility\n```javascript\nmodule.exports = {\n run: async function (formData, formLayout) {\n // Hide field based on some condition\n formLayout[0].components[0].hidden = someCondition;\n return {\n formData,\n formLayout,\n };\n },\n};\n```\nJavaScript Module
\n\n
\n\n\nAdd custom validation and dynamic behavior by providing a JavaScript module:\n```javascript\n// Define your 'run' function here\nfunction run (formData, formLayout) {\n // This function will be called when form values change\n // formData: the data that is set on the form\n // formLayout: the layout of the form, you can change the layout to show error messages or other ui elements\n // Set an error message on the component if the name is too long\n if (formData.text2.length > 5) {\n formLayout[0].components[30].errorMessage = `The name is too long, Max 5 characters. ${formData.text2.length}/5`;\n window.dl.sendEvent({\n name: 'app:toastMessage',\n payload: {\n 'type': 'info', // info, error, warning, success\n 'message': 'The name is too long...'\n },\n });\n }\n else {\n formLayout[0].components[30].errorMessage = ''\n }\n // Log the form data to the console\n console.log('Consoling from the JS code');\n\n // Return the form layout and data\n return { formLayout, formData };\n}\n\n// Export the run function\nmodule.exports = {run};\n```\nCustom CSS
\n\n
\n\n\nAdd custom styling by providing CSS:\n```css\n.form-section {\n /* Custom section styling */\n}\n\n.field-container {\n /* Custom field styling */\n}\n\n.field-text2 {\n width: 250px;\n}\n```\nTry It: