Embeddings

Overview

The Embeddings tab in the Dataloop platform provides a powerful way to visualize and interact with feature sets derived from your datasets. These feature sets represent the data in a numerical form (embeddings) that models can process and learn from.

What Are Embeddings?

Embeddings are numerical representations of data, such as images, text, or other objects, generated by AI models. They capture essential features and relationships in a compact, high-dimensional vector space. These embeddings can be used for tasks such as:

• Similarity searches (e.g., finding similar images or texts)
• Clustering (e.g., grouping items by similarity)
• Visualization (e.g., projecting high-dimensional data into 2D or 3D space)
• Training downstream models (e.g., using embeddings as input features for predictive models)

Key Features of the Embeddings Tab

Feature Set Extraction

• Enables you to extract embeddings (feature sets) from one or more datasets.
• Supports various types of data, including images, text, and other structured/unstructured data formats.

View Feature Sets

• Displays embeddings (numerical representations) generated from your datasets.
• Helps understand patterns, clusters, or relationships within the data.

Model Training

• Use these feature sets directly to train machine learning models.
• Provides a head start by utilizing pre-extracted embeddings, saving computational resources.

Model Evaluation

• Allows you to evaluate how well a model performs on the feature sets.
• Supports analysis of misclassified instances or clusters needing refinement.

Detailed Overview

Lists all generated feature sets with comprehensive metadata:

• Feature Set Name: Displays the name assigned to the generated feature set.
• Associated Datasets: Identifies the datasets from which the embeddings were extracted.
• Model Used: Specifies the model (e.g., a neural network) used to generate the embeddings. Clicking on the model name will open the Model's details page.
• Dimensions: Indicates the dimensionality of the embeddings (e.g., 512-dimensional vectors).
• Created By: Identifies the user or process that generated the feature set.
• Creation Date: Records when the feature set was generated.
• Updated Date: Records when the last modification date of the feature set.
• Search by Feature Set Name: Enables you to search for specific feature sets using their names.
• Filter Feature Sets by Creator Name: Allows filtering of feature sets based on their creator’s name.

Centralized Management

• Provides a dedicated workspace for managing all embeddings generated within your platform.
• Allows for searching, sorting, and filtering embeddings by various attributes.

Integration and Usage

Seamlessly integrates with other tools within the platform for tasks like similarity search, training models, or data analysis.

Visualization and Analysis

• Some platforms may include built-in visualization tools to explore embeddings in 2D/3D spaces (e.g., using UMAP).
• Facilitates data analysis by revealing patterns, clusters, or anomalies.

Scalable and Customizable

Can handle large-scale datasets and multiple embedding models.
Supports custom models if you want to use your trained models for feature extraction.

How to Extract Embeddings?

Refer to the Extract Embeddings section.

How to Copy Feature Set ID?

1. In the Embeddings tab, identify the feature set from the list.
2. Click on the three-dots and select the Copy Feature Set ID. The ID will be copied.

How to Delete Feature Sets?

1. In the Embeddings tab, identify the feature set from the list.
2. Click on the three-dots and select the Delete Feature Set.
3. Type DELETE and click Delete. A confirmation message is displayed.