---
title: "Organize Your Items"
slug: "organize-your-data"
tags: ["Folders", "Collections", "Access Collections", "Create Collections", "Add Items to a Collection", "Find Collections", "Clone Collections", "Rename Collections", "Remove Items from a Collection", "Remove Collections from Items", "Delete Collections", "SDK Code to Manage Collections", "Metadata", "Add Custom Metadata", "ML Subsets", "ML Data Split Chart", "Split Items Into Subsets", "SDK Code to Manage ML Subsets", "Display Custom Metadata", "Searchable Metadata", "Unsearchable Metadata", "Remove Metadata Keys"]
updated: 2026-10-30T06:53:40Z
published: 2026-10-30T06:53:40Z
canonical: "docs.dataloop.ai/organize-your-data"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Organize Your Items

Structuring your items within a dataset is key to maintaining clarity and efficiency as your projects grow. Efficient data organization is critical for streamlining dataset management, enhancing searchability, and improving collaboration. DDOE’s Data Browser provides structured ways to manage and access data using Folders, Collections, Metadata, and ML Subsets.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image(687).png)

## Folders

In DDOE, folders offer a straightforward way to organize unstructured data by grouping related items together. This hierarchical structure makes it easier to navigate large datasets, manage different data types, and keep your workspace tidy.

### Access Folders

The **left panel** in the dataset browser provides quick access to your folder structure. When the panel is collapsed, a folder icon appears, allowing you to reopen the panel and continue browsing your dataset.

It shows items based on the folders or subfolders you selected. When you select the Root Folder, it shows items and folders if available in the Root folder.

### Create Folders

- **Create folders**: Select the root folder and click on the **plus** icon or the **New Folder** icon when you hover Root Folder (DDOE).
- **Create Subfolder**: Select the folder and click **New Folder icon** or the new folder icon when you hover the selected folder.

### Filter By Folders

Quickly filter you data using the Folder tree component:

1. Select a folder and all its **subfolders** by clicking the checkbox next to the folder.
2. Click on the folder name to select the folder itself only.

Selected folders are automatically applied in the browser filters.

### Move Items Between Folders

1. Select one or more items from the current page.
2. Right-click or Click **Actions.**
3. Select the **File Actions** > **Move to Folder.**
4. Select the folder from the list.
5. Click **Move**.

### Rename Folder

Hover over the folder, and click on the **folder edit** icon.

---

## Collections

The Collections feature in DDOE's Data Browser helps organize data by allowing you to group specific sets of items based on task needs (e.g., annotation, review, training) into a collection folder. You can create up to **10** collection folders.

**Key Features of Collections in Data Browser**

- **Selective Grouping**: Choose specific items from a dataset to move into a collection based on criteria like image type, labeling status, or annotation requirements.
- **Easy Access**:
  - Collections provide a convenient way to quickly access and manage items that are organized in collection folders for relevant to a particular task, without sifting through the entire dataset.
  - Collections allows you to identify the list of **Unassigned Items** (the items that are not yet part of any collections).
- **Enhanced Collaboration**: Collections can be shared with team members, allowing specific data subsets to be easily shared and collaboratively worked on without impacting the primary dataset.
- **Task-Specific Organization**: Create collections based on different stages of your workflow, like "Pending Annotation," "Quality Review," or "Model Training Set," which helps keep the data organized according to your project’s progress.
- **Filter Collection Items Using Smart Search**: Use the **Items** field in the smart search to apply a collection filter query to find specific items within a collection. For instance, using the query `metadata.system.collections.c0 = true` will filter items that are part of the first collection (`.c0` ID for the firstly created collection folder and `.c9` ID for the lastly created collection folder).

---

### Create Collections

Creating Collections can be customized to match the requirements of your specific task, such as grouping items by type, project phase, or other relevant attributes.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image(689).png)

**Limitations**: - You can create up to 10 collection folders. - Each item can be tagged in a maximum of 10 collections at once.

1. Open the **Data Browser**.
2. Click on the **Collections** button located below of the Dataset name.
3. Click on **Create Collection**.
4. Type your desired collection's name, and press the **Enter** key. The new collection will now be created and displayed in Collections.

### Access Collections

To access the Collections in the Data Browser:

1. **Open the Data Browser**: Log in to your DDOE account and navigate to the Data Browser.
2. Click on the **Collections** button situated below the Datasets name. A popup will display with all your existing collections, an option to choose **Items without collections**, and allowing you to create new ones or manage current collections as needed.

### Add Items to a Collection

You can create a new collection by selecting items from your dataset and adding them to a designated collection.

1. Open the **Data Browser**.
2. Select the items you want to add to a collection.
3. Right-click on the selected items, or Click **Actions.**
4. Select **Collections** and choose your desired collection from the list (if not available, create a collection). The selected items will now be added to the chosen collection.

### Find Collections Using Smart Search

1. Open the **Data Browser**.
2. Click the **Advanced** search.
3. In the **Items** search field, enter the query code as `metadata.system.collections.c0 = true` where c0 is collection ID.
4. Click **Search** icon. The available collections will be listed as a dropdown.

### Clone Collections

1. **Open the Data Browser**: Log in to your DDOE account and navigate to the Data Browser.
2. Click on the **Collections** button situated below the Datasets name.
3. Hover-over the collection you want to clone.
4. Click on the **three dots** and select **Clone** from the list.
5. Click Yes to confirm the cloning process. The cloned collection will be created and named as `original_name-clone-1`.

### Rename Collections

1. **Open the Data Browser**: Log in to your DDOE account and navigate to the Data Browser.
2. Click on the **Collections** button situated below the Datasets name.
3. Hover-over the collection that to be renamed.
4. Click on the **three dots** and select **Rename** from the list.
5. Make the changes and press **Enter** key.

### Remove Items from Collections

1. Open the **Data Browser**.
2. Click on the **Collections** button situated below the Datasets name.
3. Click on the collection containing the items you want to remove.
4. Select the items, then right-click on them.
5. Select **Collections** -> **Remove From Collection**s option from the list.
6. Select the specific collection from which you want to remove the items (if they belong to multiple collections).
7. Click **Remove**. A successful deletion message will be displayed.

### Remove Collection from Items

1. Open the **Data Browser**.
2. Select Item(s) from the browser.
3. Right-click and select **Collections** -> **Remove from Collections**.
4. Select the Collection(s) that are to be removed.
5. Click **Remove**. A confirmation message is displayed.

### Delete Collections

1. Open the **Data Browser**.
2. Click on the **Collections** button situated below the Datasets name.
3. Hover-over the collection you want to delete.
4. Click on the three dots and select **Delete** from the list.
5. Click **Yes** to confirm the deletion process.

### SDK Code to Manage Collections

Leverage the DDOE SDK to create, update, delete, and manage collections at both the dataset and item levels.

[Learn more](https://developers.dataloop.ai/tutorials/data_management/datasets_and_versioning/chapter#collections-your-datas-best-friend-)

---

## ML Subsets

The **ML Subsets** view in DDOE’s Data Browser is a dedicated feature that streamlines machine learning workflows by helping you organize and manage datasets more efficiently. It enables you to classify and filter items according to their ML subset assignments—such as **train**, **validation**, and **test**—which are essential stages in the ML lifecycle for model development and evaluation.

![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image(690).png)

When you click on **ML Subsets**, a popup menu appears. From here, you can select [**Split Into Subsets**](https://docs.dataloop.ai/docs/manage-your-data#split-items-into-ml-subsets) to assign items to the desired subsets.

**Filtering by ML Subset Assignment**:

Easily locate items in your dataset based on their subset classification:

- **Train**: Items designated for training the ML model.
- **Validation**: Items used to tune hyperparameters and evaluate the model during training.
- **Test**: Items reserved for final evaluation of the trained model's performance.
- **Unassigned**: Items that have not been allocated to any specific subset.

**Use Cases:**

- **Training Pipeline**: Quickly access the train subset to prepare data for model training.
- **Model Validation**: Focus on the validation subset to monitor the model’s performance during tuning.
- **Final Testing**: Access the test subset to evaluate the final accuracy, precision, or other metrics.

Select and right-click on any item from the folders and perform [actions](https://docs.dataloop.ai/docs/dataset-actions) available.

### ML Data Split Chart

ML Data Split Chart displays the distribution of items across machine learning subsets, including Test, Train, and Validation, showing both the total numbers and percentages.

- This chart is generated when an item is assigned to one of the ML subsets.
- When hovering over a subset, the chart specifically highlights the subset being hovered over.
- Clicking on a subset grays it out, removing it from the chart.
- Clicking on it again restores its visibility on the chart.

### Why Use the ML Subsets View?

- It simplifies dataset management by clearly segregating data for training, validation, and testing, reducing errors in ML workflows.
- Enhances collaboration within teams by providing a consistent structure for dataset organization.
- Saves time by offering intuitive filtering and search options for specific subsets.

### View Items by ML Subsets

1. Select the **Data** page from the left-side menu.
2. Open the Dataset
3. to the **Data Browser**.
4. Select the **ML Subsets** from the top-bar. The available items are displayed in folder wise.
5. Select a subset. For example, Train, etc. and click outside of the ML Subsets pop-up. The items are added to the train subset will be filtered and displayed.

A search query for the train subsets will be also added and visible on the Items search field.

**If there are no items added to ML Subsets**, click [Split Into Subsets](/docs/organize-your-data#split-items-into-subsets).

### Split Items Into Subsets

Split Data Into Subsets feature allows users to divide their dataset into multiple subsets, such as train, validation, and test, based on a specified distribution. This splitting is important for ensuring that the dataset is well-prepared for machine learning or data analysis tasks. Custom Distribution: By default, the items are divided as follows:

- **Train** set: 80% of the data, which is used to train the machine learning model.
- **Validation** set: 10% of the data, which is used during training to fine-tune model hyperparameters and prevent overfitting.
- **Test** set: 10% of the data, which is used to evaluate the final model performance after training.

1. In the Dataset Browser, select one or more items.
2. Click **Actions**.
3. Select **Models** -> **Split Into Subsets**. The ML Data Split pop-up is displayed.
4. Customize the distribution by moving the slider. By default, the items are divided as mentioned above.
5. Click **Split Data**. A confirmation message is displayed, and the selected items are divided into respective subsets.
6. Click on the **ML Data Split** section in the right-side panel to view the items' distribution.

---

### SDK Code to Manage ML Subsets

This SDK code demonstrates how to filter dataset items, split them into ML subsets, assign specific items to a subset, remove an item from a subset, and retrieve items missing an ML subset in DDOE.

[Learn more](https://developers.dataloop.ai/tutorials/data_management/datasets_and_versioning/chapter#collections-your-datas-best-friend-)

---

## Metadata

Item metadata consists of descriptive information and attributes associated with individual items in a dataset. DDOE enables users to organize and categorize data items using metadata tags, making it easier to filter and analyze datasets efficiently.

With metadata-based filtering, users can define Filter Queries to refine searches based on specific attributes, such as field names or annotation labels. For example, a query can be created to filter items based on a particular metadata field or assigned annotations.

You can update item metadata using either the [UI](/docs/organize-your-data#metadata) or [SDK](https://developers.dataloop.ai/tutorials/data_management/dataloop_query_language/introduction/chapter#updating-metadata) allowing for efficient querying and retrieval of relevant data when needed.

### Add Custom Metadata

Adding custom metadata involves attaching additional information or tags to various types of data items. Custom metadata can be user-defined and is not limited to the predefined categories or attributes provided by the DDOE platform.

To attach metadata to any entity, such as Datasets, you can utilize the SDK's 'Update' function. To learn how to upload items with metadata, [read here](https://docs.dataloop.ai/docs/annotation-json-format).

```json
// Example
dataset.metadata["MyBoolean"] = True
dataset.metadata["Mycontext"] = Blue
dataset.update()
```

---

### Display Custom Metadata

The datasets page provides a list of all the datasets present within the project. The table contains default columns, including dataset name, the count of items, the percentage of annotated items, and additional information.

To include and display columns with your custom context (metadata fields):

1. In the **Dashboard**, click on **Settings**.
2. Select **Project Configuration**.
3. Select the **Dataset Columns** from the left-side menu.
4. Click **Update Setting**.
5. Click **Add** column.
6. Enter the required information as follows.
  1. **Name**: A general name for this column (not visible outside the project-settings).
  2. **Label**: The column header displayed on the Datasets page.
  3. **Field**: The Metadata field to map to this column.
7. Configure the desired feature settings as needed:
  1. **Link**: If the field value is a URL and should open in a new tab, select this option.
  2. **Resizable**: Check this option if the column needs to be resizable, useful for displaying long values.
  3. **Sortable**: Enable this option to allow sorting the table by clicking the column header.
8. Click **Apply**. A successful message is displayed.

After completing the above steps, the Datasets table on the Datasets page will display the custom column and the data you've populated there.

- To ensure that any new data added via SDK is reflected, refresh the page.
- You can use the search box to search for datasets that match your search term, provided that the search term is included in any of the custom columns you've added to the table. This allows you to filter datasets based on the custom metadata you've defined.

---

### Searchable Metadata

**Searchable metadata** refers to key-value pairs stored under `metadata.user` that are **indexed by DDOE**, enabling:

- Filtering via **DQL (DDOE Query Language)**
- Searching via **schema-based search**

These indexed keys are automatically available for use in UI filters, API queries, and pipeline automation conditions.

**How It's Created**

- Automatically **indexed** when new metadata is added to items (manually or via upload/sync), up to the platform limit (**100** searchable keys per dataset).
- By default, all new keys are assumed searchable until the searchable **limit** is reached.

#### Limitations of Searchable Metadata

| Constraint | Description |
| --- | --- |
| **Max Keys** | **100 searchable keys per dataset** under `metadata.user` |
| **Indexing Scope** | Only metadata under `metadata.user` is counted; `metadata.system` is excluded |
| **First 100 Keys** | Only the first 100 *unique keys* across all items are indexed. |
| **Schema Overflow Error** | Adding a 101st searchable key blocks item upload with a backend error. |

---

### Unsearchable Metadata

**Unsearchable metadata** in DDOE refers to custom key-value pairs stored under alternative subpaths like `metadata.user.unsearchable`, which are **not indexed** by the platform.

#### Key Behaviors

- **Visible in the DDOE UI** including the dataset browser, item view, and Studio
- **Not searchable or filterable** using DQL, schema-based search, or automation conditions
- **Handling Identical Keys:** Identical keys can be added to the unsearchable path if one of them is nested. For example, `metadata.user.event_data.companyType` and `metadata.user.companyType`.
- **Inherited Unsearchable Paths**: When a parent path (like `metadata.user.*`) is marked as **unsearchable**, it automatically applies to all of its **child fields** (such as `metadata.user.event_data.notes`).
  - This means you don’t need to explicitly add every subfield under that parent — once the parent is marked unsearchable, **the entire branch of fields becomes unsearchable by inheritance**.
- **Ideal for contextual tagging** without consuming searchable schema slots
- Any key listed in `unsearchablePaths` will **not be indexed or searchable**
- These keys remain **fully visible at the item level**, accessible to annotators and developers
- `unsearchablePaths` are **prefix-based** all nested keys under the specified prefix are also unsearchable
- **Removing a path** from `unsearchablePaths` will **restore searchability** for existing data under that path
- **Maximum of 200 unsearchable keys per dataset**

#### When to Use Unsearchable Metadata

Unsearchable metadata is particularly useful for:

- **Supplementing annotation context** (e.g., hints or descriptions for annotators)
- **Storing descriptive or trace information** (e.g., user notes, collection metadata)
- **Avoiding schema overflow** caused by exceeding searchable metadata limits

**Practical Example**:

When annotating turtle images, storing the **location** (e.g., `"region": "Pacific Ocean"`) as **unsearchable metadata** helps annotators infer species without using up one of the 100 (use up to 200) searchable key slots.

This feature empowers users to manage schema complexity while maintaining metadata visibility and usability in non-query contexts.

#### **Before Adding Keys to Unsearchable Paths**

Ensure the following criteria are met before adding any keys:

1. **Language Compliance**
  - The metadata fields must be in English.
  - Fields containing non-English characters or text should be reviewed and standardized before inclusion.
2. **Character Restrictions**
  - Metadata must not include special or non-printable characters such as:
    - Curly quotes (“ ”) or other non-standard quotation marks
    - Single quotes (')
    - Escape characters such as `\t` (tab) or `\n` (newline)

---

### Make the Metadata Key Unsearchable

You will learn how to retrieve the dataset schema, mark specific keys as unsearchable, and revert them to searchable. It helps you to overcome the errors received on metadata while uploading items.

Important

When you add or remove an unsearchable path, it triggers re-indexing. During this process, schema changes and new metadata additions are temporarily disabled until indexing is complete.

#### Step 1: Connect to the DDOE Platform

Sets the environment to **production** (for example) and logs in using your configured credentials.

```python
import dtlpy as dl
dl.setenv("prod")
dl.login()
```

#### Step 2: Access the Dataset and Target Item

Retrieve the specific **dataset** and **item** by ID.

```python
dataset = dl.datasets.get(dataset_id='your_dataset_id')
item = dataset.items.get(item_id='your_item_id')
```

#### Step 3: Add Metadata Keys

```python
# Add professional, descriptive metadata keys under 'metadata.user'
item.metadata['user'] = dict()
item.metadata['user'] = {
    "review_notes": "Animal partially occluded, needs double check",
    "source_device_id": "trailcam_09"
}
# Save the changes to the item
item = item.update()
```

- Adds a metadata keys:
  - `review_notes` and with value `Animal partially occluded, needs double check` under `metadata.user`
  - `source_device_id` and with value `trailcam_09` under `metadata.user`
- Updates the item so the new metadata is saved.

#### Step 4: Access Project and Dataset Schema

Retrieve the **project** and then the dataset again through the project object.

```python
# Access the project using the project ID
project = dl.projects.get(project_id='your-project-id')
# Re-fetch the dataset through the project context
dataset_from = project.datasets.get(dataset_id=dataset.id)
```

#### Step 5: Make Keys Unsearchable

```python
# Add keys to unsearchablePaths to prevent them from being indexed
success = dataset.schema.unsearchable_paths.add(paths=[
    "metadata.user.review_notes",
    "metadata.user.source_device_id"
])
```

✅ **Result:**

- Prevents `review_notes` and `source_device_id` from being indexed
- Makes it **unsearchable** via DQL or filters, do **not count** against the 100 searchable key limit, and are **excluded from queries or filters**
- Keeps the metadata **visible** in the UI (dataset browser, item view, etc.)

---

### Remove Keys (path) from Unsearchable Paths

If you decide that certain metadata keys should be searchable again, you can remove them from the list of unsearchable paths.

```python
dataset = dl.datasets.get(dataset_id='datasetId')
success = dataset.schema.unsearchable_paths.remove(paths=['review_notes', 'source_device_id'])
```

This step uses the `remove()` method to delete `review_notes` and `source_device_id` from the list of unsearchable paths, allowing them to be searchable again. The method returns True if the paths are successfully removed.