--- title: "Platform Release Notes" slug: "platform-release-notes" tags: ["Enhancements", "New Features", "What's new"] updated: 2026-07-28T05:09:42Z published: 2026-07-28T05:09:42Z canonical: "docs.dataloop.ai/platform-release-notes" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.dataloop.ai/llms.txt > Use this file to discover all available pages before exploring further. # Platform Release Notes ## July 27th, 2026 This release is about infrastructure freedom. We're expanding DDOE's hybrid capabilities and delivering the platform's first full on-premises deployment option. ### DDOE On-Premises & Expanded Hybrid Setup As a Dell product, DDOE is built for the reality that enterprise data isn't all in the cloud. Large-scale AI workflows often run on Dell appliances, on-premises object stores, and private compute environments — where data is protected, regulated, or simply too large to move. We're expanding on **two** fronts: - **Expanded hybrid setup**: Connect your own Kubernetes cluster to DDOE and run FaaS applications, pipelines, and preprocessing on your self-managed hardware. Protected data that only your cluster can access stays that way. - **DDOE On-Premises Installer (first release)**: For organizations that need the full AI data platform without the cloud: DDOE can now be deployed entirely in your data center. Your data never leaves. No cloud egress risk. No cloud cost. Full DDOE capability, on your infrastructure. To request access or learn more, contact the [DDOE Team](https://dataloop.ai/salesforce-form/). ### Data Management & CloudOps #### Connect Your Compute Cluster You can now connect a self-managed Kubernetes cluster to DDOE directly from the UI or SDK. Once connected, it becomes a fully operational infrastructure target: deploy applications, execute pipelines, run preprocessing, and process data across projects on your own hardware. Configure node pools, volumes, environment variables, and connected storage integrations to match your setup. [Learn more](https://docs.dataloop.ai/docs/create-datasets#connect-your-compute-cluster) #### Preprocesses Management Preprocesses run automatically when new items land in a dataset - transforming, enriching, or routing data before it reaches annotation or training workflows. Now you can see them, manage them, and control them. The new Preprocesses tab gives you a unified view of everything running in your project, system-level, organization-wide, or project-specific. Pause or resume any preprocess, and add new ones directly from the Marketplace as pipelines or applications. [Learn more](https://docs.dataloop.ai/docs/preprocesses-overview) #### Organization Bots We're introducing a new bot type: the Organization Bot. Created automatically when your organization is provisioned, it's added to every project in the org — giving it the access it needs to run org-scoped preprocesses without any manual configuration. To deploy org-scoped preprocesses, use the new System Project (see below). > [!NOTE] > Note > > Manual bot creation is no longer available. [Learn more](https://docs.dataloop.ai/docs/bot-users#organization-bot-org-bot1) #### System Projects Every organization now has a System Project, created automatically at org creation with the organization owner as a member. It's the control point for org-level functions, the place where you deploy applications as preprocesses across all projects in your organization. System project names follow the pattern: [Organization Name] System Project. [Learn more](https://docs.dataloop.ai/docs/my-projects-1#system-projects) #### New Dataset Creation Flow We've redesigned the dataset creation flow to reflect the full range of storage options now available, including on-premises storages and compute cluster-connected storages. The new flow starts with your data source type, then walks you through selecting or creating an integration, defining where your data lives within that storage, and setting your dataset details. DDOE remembers your last selection, so next time you only need to update the path - not start over. A storage driver is created automatically in the background, keeping your dataset ready to sync whenever your data changes. [Learn more](https://docs.dataloop.ai/docs/create-datasets) ### Recipe and Ontology #### Project Recipe The Project Recipe is the centralized configuration that defines labels, attributes, and annotation settings for an entire project, the single source of truth for how data is labeled and structured across all workflows. Set it once at the project level. It becomes the default for every dataset you create. Developers can clone a subset of the Project Recipe or create a new recipe and import only the labels relevant to a specific task, staying consistent at the project level while keeping flexibility at the task level. [Learn more](https://docs.dataloop.ai/docs/recipes-overview#project-recipe) ### AI Support #### Ask Donna Donna is DDOE's built-in AI assistant. Ask questions, get answers, and get guided help, without leaving the platform. Whether you're exploring a feature, tracing a workflow, or looking for the right SDK call, Donna is there in the interface. No tab switching. No doc search. Just ask. [Learn more](https://docs.dataloop.ai/docs/ask-donna) --- ## March 30th, 2026 We're kicking off this release with a major milestone: a new name, a new identity, and a wave of powerful updates across the platform. ### Introducing "Dell Data Orchestration Engine" We're proud to introduce Dell Data Orchestration Engine, or DDOE — our new name that reflects our continued evolution as part of Dell Technologies. The platform UI has been updated with the new name, logo, and Dell's look and feel. Everything you know and rely on is still here — now under a name that better represents where we're headed. ### Data Management #### New Item Details Panel with Preview No more jumping into the Studio just to check what's inside a file. The new Item Details panel now includes a built-in Preview section so you can view your files' content directly from the dataset browser. Need more space? Expand the preview into full-screen mode for a closer look without leaving your workflow. ### Marketplace #### DDOE Hub: "Getting Started" Tab Explore what DDOE can do — without starting from scratch. The new Getting Started tab in the DDOE Hub features a curated set of ready-to-use solutions built on real-world use cases. Available now: **Pipelines** - Active Learning for Agriculture — continuous model improvement with human-in-the-loop labeling - RAG over Transactions Data (OpenAI) — grounded Q&A over financial PDFs - RAG Knowledge Base for Banking (NVIDIA NIM) — compliance-aware document assistant - Metadata Enrichment for Autonomous Vehicles — automated image quality scoring and routing - PII Detection & Redaction (Microsoft Presidio) — secure document processing for healthcare - Multimodal Video Indexing (NVIDIA Blueprint) — searchable video archives across vision, audio, and text **Dataset / Data Exploration** Data Indexing & Understanding — embedding-based clustering and semantic search for datasets Deploy any solution directly from the Hub and use them as reference implementations or starting points for your own production workflows. ### GenAI Evaluation Studio & Multimodal Recipes #### GenAI Evaluation Studio The GenAI Evaluation Studio is a flexible workspace for evaluating generative AI model outputs. Design custom layouts and forms, collect structured feedback at scale, and support multimodal evaluations — all in one interface. #### Build Multimodal Evaluation Workflows You can now create Multimodal Recipes using the new Multimodal Recipe Editor, to use with the GenAI Evaluation Studio. Define interactive layouts using a JSON structure, add sample data, and preview exactly what your annotators will see — before any task goes live. Multimodal Recipes support combinations of text, images, audio, video, and more in a single interface. ### **Tasks Management** - Improved recipe selection during task creation, enabling smarter recommendations based on task type and data modality. - Users can now create, edit, or clone recipes directly within the recipe selection step, streamlining the task setup workflow. [Learn more](https://docs.dataloop.ai/docs/labeling-tasks1) ### Recipe Creation #### Choose Your Data Type and ML Task Creating a recipe is now smarter and more focused. When creating a new recipe, you'll first select a Data Type — the type of data you plan to label or evaluate. This narrows the recipe to only the tools, settings, and studio relevant to that data type. Next, choose your ML Task to get pre-configured initial settings that match your annotation goal. The result: a recipe that's purpose-built for your workflow, with no irrelevant clutter. [Learn more](https://docs.dataloop.ai/docs/recipes-overview) ## February 18th, 2026 ### Enhancements #### **Tasks Management** This release introduces multiple updates and enhancements, including new views and an improved tab-based layout to enhance task visibility and management. - **Renaming Update**: The **Labeling** page has been renamed to **Tasks**. - **New Task-Level Tabs**: Each task now includes dedicated tabs for better organization and navigation: - **Assignments**: View and manage task assignments - **Item Browser**: Browse and access task items - **Analytics**: Monitor task performance and metrics - **Issues**: Track and review task-related issues These tabs allow you to quickly access detailed information for a specific task. **Annotator View Updates** The **My Assignments** section for annotator roles now includes two tabs: - **Assignments** - **Issues** This improves clarity and simplifies task and issue tracking for annotators. [Read more](https://docs.dataloop.ai/docs/tasks-overview) --- ## June 30th, 2025 ### New Features **Azure Container Registry (ACR)** DDOE now supports integration with Azure Container Registry (ACR), allowing users to securely deploy and manage containerized pipelines and applications. This streamlines DevOps workflows by using ACR as a private, efficient container registry within the DDOE platform. [Read more](https://docs.dataloop.ai/docs/azure-container-registry) **Transliteration Settings in Audio Studio** The Audio Studio now supports Transliteration Settings, allowing you to type transcripts phonetically in your native language. As you enter text in English, it is automatically converted into the selected script (e.g., Hindi, Tamil, Assamese). This simplifies transcription in regional languages without the need to switch keyboard layouts. [Read more](https://docs.dataloop.ai/docs/audio-guide#transliteration-settings) **LiDAR Studio: Fisheye Camera** DDOE’s LiDAR Annotation Studio now supports annotation workflows involving fisheye camera images, enabling seamless integration of wide-angle visual data into 3D annotation projects. [Read more](https://docs.dataloop.ai/docs/lidar-20#fisheye-camera-images) ### Enhancements **Instance Segmentation Studio** - **SAM-Box Enhancements**: Add/remove areas inside a drawn box; new boxes auto-approve the previous instance and create a new one. - **Auto Instance Creation**: No more "Add Instance" button—new drawings auto-create or update instances. Press Enter to finalize. - **Instance Merging**: Merge annotations with the new Merge button. [Read more](https://docs.dataloop.ai/docs/instance-segmentation-studio) **Dataset Browser - Pipeline Task Enhancement** You can now add items directly to an existing pipeline task from the Dataset Browser, making it easier to update ongoing workflows without recreating tasks. [Read more](https://docs.dataloop.ai/docs/manage-your-tasks#add-items-to-pipeline-tasks) #### Marketplace We’ve expanded our marketplace with new models and upgrades: **Large Language Models (LLM)** Apple DCLM-Baseline-7B – New 7B model for general-language text generation and understanding. **Vision-Language Models (VLM)** NVIDIA VILA – Enables querying and summarizing images and videos. **LLaMA 3.2 (Vision Support)** – Now with vision capabilities. **Object Detection** YoloX – Upgraded with improved metrics support for more accurate evaluation. **RF-DETR** – High-performance model for object detection by Roboflow. --- ## April 28th, 2025 ### New Features **AI Playground** Introducing the AI Playground – an interactive chat tool that lets users test AI models and pipelines using text prompts and image uploads. Easily experiment, validate, and refine AI workflows directly within the DDOE platform. [Read more](https://docs.dataloop.ai/docs/playground-overview) **Export Summary file** Developers and higher roles can now generate an Export Summary to quickly review the contents of a downloaded dataset. The summary includes details about the items and annotations exported. Learn more to export the summary file via [UI](https://docs.dataloop.ai/docs/manage-your-datasets#download-an-export-summary-file) and [SDK](https://developers.dataloop.ai/tutorials/data_management/datasets_and_versioning/chapter#exporting-datasets) **Lock datasets during export** A new option allows you to lock datasets during export, ensuring data consistency by preventing modifications throughout the download process. When this option is enabled, the dataset switches to Read-Only Mode until the ZIP compression is completed. While locked: - Any updates to the dataset items will not be saved. - Any execution (pipeline/application) that tries to modify the dataset will fail. - Project Owners/Developers can manually unlock the dataset. - Auto-unlock occurs after a 2-hour timeout (configurable via SDK). Enable this feature via: The Dataset page → Actions menu → Download data → Enable Lock during export checkbox. [UI](https://docs.dataloop.ai/docs/manage-your-datasets#lock-datasets-during-export) and [SDK](https://developers.dataloop.ai/tutorials/data_management/datasets_and_versioning/chapter#exporting-datasets). [Read more](https://docs.dataloop.ai/docs/datasets-overview#lock-datasets-during-export) **Cards View & Bulk Classification in Assignment Browser** We’ve introduced a new Cards View in the Assignment Browser, featuring a visual, card-style layout designed for fast and efficient bulk classification. Annotators can easily classify multiple items directly from the assignment browser—no need to open the Annotation Studio. - To activate, enable **Bulk Classification Mode** in the recipe settings. Once enabled, the Open Assignments in Browse View setting will be automatically turned on. This means that when annotators double-click an assignment, they’ll be directed to the assignment browser instead of the Annotation Studio. [Read more](https://docs.dataloop.ai/docs/assignment-browser#bulk-classification-items) ### Enhancements **Enhanced Task Creation Flow** The task creation experience has been redesigned for clarity and ease of use by splitting the existing unified task creation dialog into dedicated dialogs for labeling, review (QA), and quality type tasks. [Read more](https://docs.dataloop.ai/docs/tasks#manage-tasks) **Folder Tree: Select Sub-folders** You can now select a folder and all its nested content by clicking the checkbox next to it. Clicking on the folder name will only select the folder’s items, not its sub-folders or their contents. [Read more](https://docs.dataloop.ai/docs/organize-your-data#:~:text=Select%20a%20folder%20and%20all%20its%20nested) **Marketplace** We’ve expanded our marketplace with new models and upgrades: - **DinoV2 Image Embedder** for high-quality image embedding is now available. - **DeepSeek NIM** – Supports advanced data processing capabilities. - **Llama3** family is now available for fine-tuning with images or text data. - **Mistral 7B Instruct v0.3** – Latest version now available. ### Deprecations **Attributes 1.0 in SDK** The use of list-based attributes (Attributes 1.0) in the DDOE SDK is deprecated as of SDK version 1.99.0 and has been non-functional since version 1.90.39. [Read more](https://docs.dataloop.ai/docs/deprecations#deprecation-attributes-10) --- ## February 27th, 2025 ### New Features **Integrations: Docker Hub - Container Registry** DDOE now supports seamless integration with Docker Hub, enabling users to pull, store, and manage Docker images for AI model deployment, automation pipelines, and custom processing tasks. [Learn more](https://docs.dataloop.ai/docs/dockerhub-container-registry) ### Enhancements **Data Browser: Left-Side Filter Panel - Expanded Tabbed Format** The Left-Side Filter Panel now supports an expandable tabbed format for improved navigation. Click the Expand Panel icon to access filtering features such as Folders, Collections, Tasks, and ML Subsets, all organized into distinct tabs. [Learn more](https://docs.dataloop.ai/docs/data-browser#folders:~:text=the%20folder%20icon,-,a%20tabbed%20view%20will%20appear,-,%20presenting%20file%20items) **Video Studio** The Disable Auto-Grouping for 50+ Annotations option in the Recipe, allowing users to manage large annotation sets in video studio more flexibly. When enabled, annotations in frames with more than 50 items will remain separate instead of being automatically grouped. [Learn more](https://docs.dataloop.ai/docs/instructions#video-settings:~:text=Disable%20auto-grouping%20for%2050+%20annotations) --- ## January 31st, 2025 ### New Features **1. PDF Studio** The new PDF Labeling Studio is a powerful tool designed to streamline the annotation process for PDF documents within DDOE. This feature enables annotators to interact with PDF files and perform structured labeling tasks efficiently. [Learn more](https://docs.dataloop.ai/docs/pdf-studio) **2. Integrate AWS and GCP Private Docker Registries** Easily set up Docker Registry integrations with AWS or GCP from the DDOE UI to seamlessly connect and use private Docker images in your applications. [Learn more](https://docs.dataloop.ai/docs/integrations-overview#docker-registry-integration). **3. ML Subset filters** A new feature that allows users to classify and filter dataset items based on their ML Subset assignments (Train, Validation, Test), enhancing dataset organization for machine learning model development and evaluation. [Learn more](https://docs.dataloop.ai/docs/data-browser#ml-subsets) **4. Find Similar Items** Easily discover similar items using feature vectors to enhance data exploration, cleaning, and ML subset optimization. [Learn more](https://docs.dataloop.ai/docs/data-browser#similarity) ### Enhancements **1. Pipelines: Adjust Service Configuration at Runtime** You can now adjust pipeline node service configurations (e.g., machine type, concurrency, timeout) during runtime, enabling easy adaptation to changing workloads without pausing or disrupting the pipeline. [Learn more](https://docs.dataloop.ai/docs/node-actions#how-to-edit-service-settings-of-the-model). **2. Unassigned Items** A new feature that allows users to identify dataset items that are not assigned to any [collection](https://docs.dataloop.ai/docs/data-browser#collections:~:text=the%20list%20of-,Unassigned%20Items,-%28the%20items%20that) or [ML subsets](https://docs.dataloop.ai/docs/data-browser#ml-subsets:~:text=their%20subset%20classification:-,Unassigned%20items,-:%20Items%20that%20have), ensuring better dataset organization and completeness. **3. Label Selection with Keyboard Shortcuts** Easily select labels using the keyboard shortcuts to improved your efficiency. Available only in **Image** and **Video** Studios: - **Quick Access**: Use `Shift` + `1` to `9` to select the first 9 labels. - **Sub-labels**: Press `Tab` to enter sub-level, Shift+Tab to level up. - **Navigate**: Move through the labels using Up/Down arrows. - **Search**: Press Shift+L to instantly go to the labels search bar. [Learn more](https://docs.dataloop.ai/docs/image-studio#image-studio-keyboard-shortcuts:~:text=Search%20Labels,and%20Down%20arrows). **4. GIS Studio: Supports Large GeoTIFF Files** Users can now upload and work with large GeoTIFF (.tif) files in GIS Studio without performance issues, ensuring smooth annotation and data handling. [Learn more](https://docs.dataloop.ai/docs/platform-recommended#supported-file-formats:~:text=EML,%20and%20PDF-,GIS,-COG,%20GeoTIFF,%20OSM) **5. Labeling: Review Approved Issues** Users can now filter and view a list of approved issues directly from the Issues tab for better tracking and management. [Learn more](https://docs.dataloop.ai/docs/issues#:~:text=corrected%20for%20review.-,Approved,-This%20displays%20the) **6. SDK: Manage Collections** Leverage the DDOE SDK to create, update, delete, and manage collections at both the dataset and item levels. 1. Dataset Level Methods: These methods operate on collections within a dataset. 2. Item Level Methods: These methods work at the individual item level for managing collections. [Learn more](https://docs.dataloop.ai/docs/data-browser#collections-view:~:text=Use%20SDK%20to%20Perform%20Collection's%20Features%20in%20the%20Dataloop) **7. SDK: Assign and Manage ML Subsets** Utilize the SDK to select items and either manually assign them to a subset or automatically distribute them. [Learn more](https://docs.dataloop.ai/docs/data-browser#ml-subsets:~:text=Use%20SDK%20Code%20for%20ML%20Subset%20Splitting%20and%20Assignment) --- ## December 31st, 2024 ### New Annotation Tools #### GIS Studio for Geospatial Data We’re excited to introduce the GIS Studio, a solution for geospatial data annotation. Designed to handle data from satellites and aircraft, GIS Studio integrates with a dynamic map view interface, and Geographic Information Systems features, enabling precise and efficient labeling. [Learn more](https://docs.dataloop.ai/docs/gis-studio) #### Instance Segmentation For Images Another exciting new addition to our platform is the Instance Segmentation Studio – the ultimate tool for annotating multiple object instances with precision. This new solution is tailored for advanced semantic segmentation tasks. With built-in SAM tools, this studio enables rapid segmentation of multiple instances of the same object type within an image, significantly boosting efficiency and accuracy in your annotation workflows. - Currently, it's not supported to labeling the same item in both the Image Studio and the Instance Segmentation Studio. [Learn more](https://docs.dataloop.ai/docs/instance-segmentation-studio) ### New: Embeddings Tab in Data Management The Data Management module now features an Embeddings tab, designed to simplify the management of the feature sets within your project. Whether you’ve uploaded your feature sets or extracted embeddings using DDOE's tools, this tab provides detailed information including the associated models, connected datasets, and feature dimensions. [Learn more](https://docs.dataloop.ai/docs/embeddings-1) ### Data Browser Enhancements **View Items by Task Assignment** The updated dataset browser now includes the ability to filter items by tasks, streamlining your workflow and improving task management. [Learn more](https://docs.dataloop.ai/docs/data-browser#tasks-view) **Organize Your Data With Collections** We’re thrilled to introduce Collections, a new feature designed to help you efficiently tag, group, and organize your data. With Collections, you can streamline labeling tasks, manage large datasets more effectively, and prepare cleaner, well-structured data for model training to ensure better control over your data. [Learn more](https://docs.dataloop.ai/docs/data-browser#collections-view) **Split Items Into Subsets: Train, Validation, Test** Easily organize your dataset into three subsets – train, validation, and test using the Split Data into Subsets feature. Choose automatic splitting (random) or manually distribute items to fit your specific requirements. [Learn more](https://docs.dataloop.ai/docs/manage-datasets#how-to-split-items-into-subsets-test-train-or-validation-in-the-dataset-browser:~:text=Split%20Data%20Into%20Subsets%20feature%20allows) ### Annotation Studio Enhancements **LiDAR Studio: Zoom** The LiDAR Studio side panel now supports Zoom to Mouse Cursor, enabling annotators to zoom directly to the mouse cursor in the 3D Camera View. [Learn more](https://docs.dataloop.ai/docs/lidar-20#zoom-to-mouse-cursor-in-lidar-side-panel) **Audio Studio: Peak Amplitude (dB)** Peak Amplitude in Decibel is now available in the Audio studio, allowing annotators to identify the maximum sound level within a selected segment (annotation) for more accurate audio insights. [Learn more](https://docs.dataloop.ai/docs/audio-guide#:~:text=on%20the%20waveform.-,Peak%20Amplitude%20%28dB%29,-%28by%20default%29:%20Indicates) **Image Studio – New Segmentation Restriction** To prevent duplication and ensure accuracy in semantic segmentations, points already present in a segmentation instance are skipped when adding new layers, with notifications provided for skipped points. [Learn more](https://docs.dataloop.ai/docs/instance-semantic-3d#create-instance-semantic-annotation-with-brush-applicator:~:text=When%20adding%20a%20segmentation%20layer%20to%20point%20clouds) **New Keyboard Shortcut: Search Labels** Easily search and pick through the label list using the Shift + L shortcut. Navigate the results with the up and down arrow keys for quick access. ### Security Enhancements: Password Rotation Enforcement As part of our 90-day password rotation policy, all username/password accounts, including M2M (machine-to-machine) accounts, must update their passwords regularly. As the current cycle is ending soon, a password update will be required. Note: To ensure uninterrupted operations, we recommend transitioning M2M accounts to API keys. [Learn more](https://docs.dataloop.ai/docs/signup-login#enforce-password-change-for-usernamepassword-authentication) --- ## August 15th, 2024 ### New Models Available in the DDOE Marketplace - **Generative Models**: Easily install generative models from top providers like Meta, Mistral, OpenAI, and more. - **Embedding Models**: Install embedding models from the DDOE Marketplace to extract features from your data, enabling applications like semantic search, data cleanup, and clustering analysis. Easily integrate these models into your pipeline automations to create powerful workflows like Chatbot, Retrieval-Augmented Generation (RAG), RLHF, Content Summarization, and more. [Learn more](https://docs.dataloop.ai/docs/startline-models) ### Quickly Access and Install pre-built Datasets within DDOE Marketplace Whether you're new to DDOE or exploring its diverse solutions, the 'Datasets' tab on the DDOE Marketplace has everything you need. Browse and install datasets from top providers like Meta and Hugging Face directly into your project with just a few clicks. [Learn more](https://docs.dataloop.ai/docs/marketplace-datasets) ### Enhanced Data Management Resource Creation Interface We've enhanced the process of creating datasets and connecting them to your external storage. Integrating your external storage with DDOE is now simpler and more efficient. You can now manage and create everything in one place: - Cloud Integrations: Seamlessly connect with external storage providers (AWS, GCP, Azure). - Storage Drivers: Easily configure and manage storage drivers. - Datasets: Simplified dataset creation. [Learn more](https://docs.dataloop.ai/docs/data-management-features-overview#data-management-resource-creation) ### Labeling Tasks: Status Filtering Tasks can now be easily filtered by their status, enabling quick viewing of tasks that are in progress or completed with just a few clicks. [Learn more](https://docs.dataloop.ai/docs/labeling-tasks#filter-criteria-for-tasks) ### Annotation Studio: Status Log The new Status Logs view in the annotation studio lets you to track and monitor your team's actions on data items. You can now easily follow task status changes, see who applied each status ('Completed,' 'Discard,' or custom status), and when it was done. [Learn more](https://docs.dataloop.ai/docs/annotation-studio-basics#item-details) ### Task Analytics: Annotation Time Metric A new metric, “Task Browser annotation time,” is now available on the Task Analytics page. It offers insights into labeling time in the Task Browser view (such as bulk classification), allowing you to monitor team efficiency and productivity. You can easily find it under: Task Analytics > Performance Tab > User Stats. [Learn more](https://docs.dataloop.ai/docs/analytics-performance#assignment-stats) ### CloudOps: Service Initialization Logs Real-time progress updates are now provided during service initialization, reflecting the environment installation status in the service logs. This offers greater transparency and a smoother setup experience. [Learn more](https://docs.dataloop.ai/docs/logs#overview) --- ## June 19th, 2024 ### Semantic Search with CLIP Model Utilize the CLIP model to query your files using semantic search for both images and documents. More embedding models can be found and installed on the DDOE marketplace. [Learn more](/1-3-8/docs/schema-based-search) ### Data Cleanup Tab Use embeddings for similarity searches to clean your training datasets from overly similar items, whether they are documents, images, or other unstructured data. For images, also filter out those that are too dark, too bright, or blurred to maintain high data quality. [Learn More](https://docs.dataloop.ai/docs/cleanup?) ### Dataset Node Updates #### Trigger pipeline over existing data It is now possible to execute pipelines on existing data within a selected dataset when the pipeline is activated. This enhancement complements the ability to trigger pipelines on new data, offering increased flexibility. ### Dataset Node Variables Introducing the ability to set variables, enabling dynamic dataset selection within your pipeline using Pipeline Variable. [Learn More](https://docs.dataloop.ai/docs/pipeline-nodes) ### Labeling Task and Workflow Templates You can now create labeling tasks and use workflow templates to accelerate your labeling process. For instance, use the microtasks workflow to manage a series of small labeling tasks, enabling labelers to focus on specific tasks according to their expertise. [Learn more](https://docs.dataloop.ai/docs/create-task?) ### Hiding Annotations Functionality Change We've updated the keyboard shortcuts for hiding annotations to enhance performance and streamline the user experience. Now, use "J" to hide all annotations and "H" to hide/unhide selected annotations. [Learn More](https://docs.dataloop.ai/docs/en/annotation-studio-basics?highlight=hide%20) ### Attributes Search Users can now search for attribute values to streamline the annotation process, making it quicker and easier. [Learn More](https://docs.dataloop.ai/docs/en/annotation-studio-basics?highlight=attributes%20) ### Secret Management We have improved the interface for secrets management. Continue to securely store and manage your sensitive information, such as API keys and database credentials, within DDOE, ensuring your AI building process remains protected and efficient. [Learn More](https://docs.dataloop.ai/docs/manage-secrets?) --- ## May 1st, 2024 ### Video Performance Enhancements Major performance changes were made to improve the user experience and allow for a smooth annotation flow, read [more](https://docs.dataloop.ai/docs/video-annotation-studio#working-with-video-annotations) about it. ### Labeling Tasks: Layout Deprecation As recently announced, Labeling Task now features a unified page for managing tasks, assignments, and issues, replacing the old multipage view. Please note that we no longer support the old layout at this time. To explore the new layout, on the navigation menu, click **Labeling**, read [more](https://docs.dataloop.ai/docs/labeling-tasks) about it. ### Simplified File Export and Download We've streamlined our data [export](https://docs.dataloop.ai/docs/manage-datasets#export-datasets-or-items-as-json-file) and [download](https://docs.dataloop.ai/docs/manage-datasets#download-files) processes for enhanced ease of use. --- ## April 15th, 2024 ### Lidar Annotation Studio We're excited to announce significant upgrades to our Lidar Annotation Studio, designed to boost performance and enrich your annotation capabilities. This update includes: - **Timeline Feature**: Effortlessly traverse your project's point cloud data (PCD) timeline. This feature not only allows you to visualize the start and end times of annotations but also provides a clearer insight into the scope and details of the annotations. - **Instance Segmentation**: This annotation tool is designed for intricate segmentation of point clouds, allowing for a deeper level of detail and precision in your annotations. - **Polyline and Spline**: This update introduces precise marking of linear elements, coupled with the ability to fine-tune their curvature. Read more about [LiDAR](https://docs.dataloop.ai/docs/lidar-20). ### Task 2.0 in now the default task interface We are excited to announce that the Task 2.0 interface is now the default tool for managing your tasks, designed to enhance your workflow and boost efficiency. The option to revert to the classic interface is available until May 1st. Simply click on **Switch to Classic Labeling Tasks** to make the switch. We encourage you to explore the new interface and share your feedback! Read more about [Labeling Tasks 2.0](https://docs.dataloop.ai/docs/labeling-tasks-20). ### Data Management - **New Clustering Tab**: We're excited to introduce the Clustering Tab, a new feature designed to simplify the analysis of large data sets by using feature vector projections to organize similar items. This tool not only streamlines your data, but also highlights anomalies by identifying data points that stand out from the rest. Embrace enhanced insight and efficiency as you explore the hidden patterns and relationships within your data. - **New Insights Tab**: The Insights Tab provides deep visibility into your annotations, offering features like an annotation location heat map, a histogram of annotation labels, and detailed attributes per label, among others. Accessing comprehensive information about your data with just a quick glance has never been simpler. To experience the new insights and clustering tabs: Go to the Marketplace > Applications > Install The **Clustering App** / **Insights App** > Go to one of your datasets> the new tabs now available, check them out! Read more about [Clustering](https://docs.dataloop.ai/docs/clustering) tab and [Insights](https://docs.dataloop.ai/docs/insights) tab. ### DDOE Marketplace **Filter by Trainability** In models tab, users can easily filter trainable model, to quickly get started with the model fine-tuning process. Read more about DDOE [Marketplace](https://docs.dataloop.ai/docs/startline). ### Model Management Possible to filter models in the model versions tab by model status, application, dataset, and creator for easy finding of models. - **Enhanced Model Filtering**: Users can now seamlessly filter models within the Model Versions tab based on criteria such as model status, application, dataset, and creator, facilitating quick and efficient model identification. - **Use Data Tags to Simplify Model Training & Evaluation**: Initiate model training and evaluation effortlessly by selecting data subsets tagged as Train, Test, or Validation from your dataset. This enhancement simplifies the process of initiating model training or evaluation, streamlining the workflow for users. Read more about [Model Management](https://docs.dataloop.ai/docs/model-management-overview) and [Model Evaluation](https://docs.dataloop.ai/docs/model-evaluation). --- ## February 28th, 2024 ### Marketplace - Your ML applications marketplace Marketplace revolutionizes the way developers of all skill levels and expertise build AI, providing them with models, applications, pipelines, and out-of-the-box end-to-end AI and GenAI workflows. #### Pipelines Expand your pipeline possibilities with our broad selection of new node types — the essential building blocks for ML projects. Install them via Marketplace and craft your pipelines effortlessly, using either built-in templates or starting from scratch with intuitive drag-and-drop features. #### Models Access and deploy models from popular & leading providers like Google, Microsoft, Nvidia, and more, enriching your project with cutting-edge AI capabilities. #### Applications Integrate FaaS applications with public and applications in your model development, streamlining workflows and simplifying project advancements. To learn more about it, refer to the [Marketplace](https://docs.dataloop.ai/docs/startline) article. ### Evolving from Application Hub to CloudOps The switch from the Applications page to CloudOps marks a significant upgrade in managing application services. To learn more about it, refer to the [CloudOps](https://docs.dataloop.ai/docs/cloudops-overview) article. ### Dataset Browser - Filtering by Task The Tasks filter in the Dataset Browser introduces granular data sorting by Task Name and Task ID for advanced dataset management. To learn more about it, refer to the [Schema-Based Search](https://docs.dataloop.ai/docs/schema-based-search#search-items-by-tasks-id-or-name) article. ### Pipelines The new nodes panel makes it straightforward to discover and configure the wide variety of nodes available, supported by an organized layout and enhanced categorization. In addition, new node types have been added to allow you to customize your pipeline exactly as you wish. To learn more about it, refer to the [Pipelines Nodes](https://docs.dataloop.ai/docs/pipeline-nodes) article. ### Model Management - Simplified AI Deployment - **Foundation Models Installation**: Foundation models are now easily installable on the Model Management page with just one click! - **Workflow Enhancements**: Improved cloning, training, evaluating, and deploying models, especially with a more user-friendly cloning experience. - **Pre-trained Model Integration**: New Pre-trained status enhances AI model utility across workflows, enabling better predictions, further training, easy deployment, and performance evaluation. - **Model Creation Update**: Introduced a nuanced status flow for model creation and cloning, accurately reflecting each stage of a model's life cycle. To learn more about it, refer to the [Model Management](https://docs.dataloop.ai/docs/model-management-overview) article. --- ## January 8th, 2024 ### Dataset Browser 2.1 The updated dataset browser features an enhanced interface and introduces new impactful functionalities. With Dataset Browser 2.1, you can elevate your browsing experience. #### Smart Search with DDOE Explore the upgraded data filtering experience with Smart Search on the DDOE data browser. Uncover an intelligent search feature that simplifies obtaining the precise data results you're looking for. Now, you can filter by different operators such as annotation labels, item media types, and more. Click [here](https://docs.dataloop.ai/docs/schema-based-search) to discover more about the enhanced functionalities of Smart Search. #### Enhanced User Interface Now, you can effortlessly access all the item information you need in one glance. Easily discover item metadata, basic information, and Function Executions. Find more about it in the [Dataset Browser](https://docs.dataloop.ai/docs/data-browser) documentation. ### Audio Annotation Studio - Transcription Finder Introducing the Transcription Finder feature in the Audio Annotation Studio, designed to help users efficiently "find and replace" specific text within transcriptions. This tool supports annotators in ensuring data quality with ease. Find more about it in the [Audio Studio](https://docs.dataloop.ai/docs/audio-guide#find-and-replace-transcription) documentation. ### Task Management 2.0 Enhancements Continuing our commitment to an improved user experience, we've enhanced the Task Management interface. Now, effortlessly review issues and annotations marked "For Review." We've also streamlined access to the Task Browser and Task Analytics for a smoother user journey. Find more about it on the [Labeling Tasks](https://docs.dataloop.ai/docs/labeling-tasks-20) documentation. ### Rerun Pipeline Cycles Addressing pipeline errors is now more efficient. Users can rerun cycles at the execution or cycle level, removing item status from tasks and facilitating reinsertion into assignments. Find more about it on the [Running and Monitoring pipelines](https://docs.dataloop.ai/docs/running-and-monitoring-pipelines#rerun-pipeline-cycles) documentation. --- ## December 19th, 2023 ### Service Page New tabs were added to the service page to streamline the process and make it easier for users to access useful features. Now, users can analyze their service executions, logs, triggers, and instances conveniently through distinct tabs. Find more about it in the [Service Analytics](https://docs.dataloop.ai/docs/service-analytics) documentation. ### Data Governance Exciting news! Our latest update introduces enhanced data governance, offering a centralized hub with dedicated tabs for Integrations and Secrets, providing a streamlined and provider-focused view for efficient data asset management and protection. Try out the new Data Governance page by clicking **Switch to V2**. Read more about it on the [Data Governance](https://docs.dataloop.ai/docs/overview-1) documentation page. ### Annotator Issue Creation in QA Task Annotators can now create issues directly within the QA task, providing them with the ability to address concerns as needed. For more information, see the [Roles of Users in the QA Process](https://docs.dataloop.ai/docs/create-qa-task#roles-of-users-in-the-qa-process) article. ### Pipeline Tasks **New UI for task node** The redesigned task node interface ensures a streamlined and organized view of task details, offering users instant access to all relevant information. **Edit tasks from the pipeline page when the pipeline is running** Introducing a new level of flexibility! Users can now edit pipeline tasks, even while the pipeline is running ensuring a dynamic workflow where any changes made seamlessly reflect in the ongoing pipeline process, guaranteeing an uninterrupted experience. To learn more about editing a task, see the [Annotation and QA tasks](https://docs.dataloop.ai/docs/create-task) article. **Edit workload of pipeline Distribution tasks - New feature** You can now edit the workload item within the distribution of a pipeline task, providing more granular control over specific task details within the broader distribution process. To learn more about editing the Workload Item, see the [Edit the Workload Item](https://docs.dataloop.ai/docs/task-assignments-and-reassignments#edit-the-workload-item-in-the-distribution-task-of-a-pipeline-task) article. --- ## November 20th, 2023 ### My Projects The My Projects user interface (UI) has been enhanced to include the ability to display additional information on project cards, the option to sort projects by name and creation date, and the introduction of a page refresh functionality. For more information, see the [My Projects](https://docs.dataloop.ai/docs/my-projects-1) article. #### Task Management UI - The Task Management page has undergone significant enhancements, with the introduction of tabs for Tasks, Assignments, and Issues, offering a more provider-centric perspective. - Additionally, a detailed view of Tasks and Assignments has been incorporated into the new UI, featuring information such as Actions, General Information, Status, Metadata, and more in a right-side panel. This empowers users to access a wealth of information on a single page. - Furthermore, the Search and Filter functionalities have been improved to provide more precise and effective search results. For more information, see the [Labeling Tasks 2.0](https://docs.dataloop.ai/docs/labeling-tasks-20) article. --- #### Pipeline Page and Nodes - The Pipeline user interface (UI) has undergone a series of enhancements, offering a fresh perspective on the pipeline table list, buttons, menus, pipeline counters, popup messages, and search fields. - The detailed view of Pipeline Nodes, including FaaS Node and Code Node, on the right-side panel has been enriched with new fields, information text, view and edit mode information, and node validation errors. - The Action Menu feature for nodes on the right-side panel empowers users to perform various actions, such as opening datasets, accessing FaaS services, and viewing analytics, etc. according to the nodes. - The Node Service Status feature allows users to monitor the current status of their nodes, including states like Active, Initializing, Inactive, or Failed. For more information, see the [Pipeline Overview](https://docs.dataloop.ai/docs/pipelines-overview) and the [Pipeline Nodes](https://docs.dataloop.ai/docs/pipeline-nodes) articles. --- ## October 15th, 2023 ### New Features #### VS Code Application Introducing the new built-in VS Code for enhanced FaaS development. With its integrated code editor, you can easily view, edit, and debug code in the same environment where FaaS applications are built and deployed, instead of locally. - For more information, see the [Service Debugging](https://docs.dataloop.ai/docs/service-debugging) article. ![image.png](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28625%29.png) #### Pipeline: Evaluate Model Node The new "Evaluate Model" node seamlessly integrates with your pipeline, allowing you to easily create model evaluations, fully automating the training process in conjunction with our "Train Model" and "Predict" nodes. Assess model performance, compare predictions to ground truth, and access results effortlessly. - For more information, see the [Evaluate Model](https://docs.dataloop.ai/docs/pipeline-nodes#evaluate-model) article. ![image.png](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28626%29.png) ### Enhancements #### New Data Management UI - The Data Management page has been enhanced to show resources by tabs and enable a more provider-focused view. - Users can now easily copy and paste the SDK script for the relevant resource that has been added to the Data Management page. - The recipe feature has been moved out of the Data Management page. For more information, see the [Storage Drivers](https://docs.dataloop.ai/docs/storage-drivers-overview) and [Dataset](https://docs.dataloop.ai/docs/dataset-overview) topics. --- ## September 20th, 2023 ### New Features **Logs of User Logins** User login logs are records that track and document when users access the DDOE platform. It records the user login email, organization role, and timestamp. For more information, see the [Logins](https://docs.dataloop.ai/docs/audit-log#logins) topic. **Apple Quicktime Videos (MOV format) are Now Automatically Converted** DDOE converts MOV video files to WebM format for frame-accurate annotations. For more information, see the [Supported File Formats](https://docs.dataloop.ai/docs/platform-recommended#supported-file-formats) topic. **Redistribute items with Metadata (key & value) Numbers** Use numbers as metadata (key and value) to filter items while redistributing tasks. For more information, see the [To redistribute a task](https://docs.dataloop.ai/docs/task-assignments-and-reassignments#redistribute-a-task-or-an-assignment-of-a-task) topic. **Rerun Pipeline Cycles** Users can rerun pipeline cycles by selecting executions for rerun starting points or entire cycles for rerun to avoid regenerating the cycles again in the case of failure. Rerunning the cycle in the pipeline removes the item status from a task, so it will be reinserted into an assignment. For more information, see the [Supported File Formats](https://docs.dataloop.ai/docs/platform-recommended#supported-file-formats) topic. ### Enhancements **New Data Management UI** - The Data Management page has been enhanced to show resources by tabs and enable a more provider-focused view. - Users can now easily copy and paste the SDK script for the relevant resource that has been added to the Data Management page. - The recipe feature has been moved out of the Data Management page. For more information, see the [Storage Drivers](https://docs.dataloop.ai/docs/storage-drivers-overview) and [Dataset](https://docs.dataloop.ai/docs/dataset-overview) topics. **The Term "Contributors" is Now "Users"** The terminology **Contributors** is now replaced with the term **Users**, and the menu will indicate **Team**. There is no functionality change, only a change of terminology. For example, - Menu term changed to **Team** - The Contributors' title page becomes **Project Team**. - Buttons and search boxes are now **Users**. - Project contributors are now **Project Users**, etc. For more information, see the [Project Team](https://docs.dataloop.ai/docs/project-team) topic. --- ## August 9th, 2023 ### New Widget in Project Overview: Task Management A task management widget is now available on the Project Overview page, where you can: - View a list of available tasks. - Create a task. - Create a workflow via Pipeline. For more information, see [Project Dashboard](https://docs.dataloop.ai/docs/project-dashboard#project-dashboard). ### Data Management #### New Storage Integration Type: GCP Cross Project From now on, DDOE allows users with a GCP project to integrate their GCS bucket with the DDOE platform and create datasets. The GCP Cross Project integration stands as the best choice for GCP due to its significant advantages: - **Least Privilege Principle:** By leveraging GCP Cross Project integration, fine-tuned IAM roles and permissions can be assigned, providing stringent access and reducing risk of third-party compromises. - **Rotational Capability:** This integration allows the straightforward rotation of IAM roles, ensuring better security by refreshing third-party service credentials periodically. - **Better Audibility:** GCP Cross Project integration aids in generating detailed action logs, facilitating more effective monitoring and detection of any unusual activity. - **Separation of Responsibilities:** This feature enables precise control over GCP projects and task delegation to third parties without the need to expose credentials, thereby mitigating risk of misuse. For more information, see the [Cross Project Integration](https://docs.dataloop.ai/docs/cross-project-integration). #### Integration Validation for all Supported Cloud Integrations - All supported integration types will now be validated for actual resource creation on the customer's side. - This does not impact existing storage integrations. Only applicable to new integrations. - Supported on: - GCP - Cross Project, Private Key - Azure - Client Secret - AWS - Cross Account, STS For more information, see the [Integration Validation for all Supported Cloud Integrations](https://docs.dataloop.ai/docs/overview-1#integration-validation-for-all-supported-cloud-integrations). #### Supports Duplicate Naming Enforcement for New Integrations & Secrets - Newly created integrations and secrets will now be enforced to be unique by name - This does not impact existing integrations and secrets. Only applicable to new integrations and secrets. For more information, see the [Supports Duplicate Naming Enforcement for New Integrations & Secrets](https://docs.dataloop.ai/docs/overview-1#supports-duplicate-naming-enforcement-for-new-integrations-secrets). ### Data Management Deprecations: #### Support for Elastic Container Registry (AWS) Support for ECR in DDOE platform Integrations has deprecated and no longer available. For more information, see [Deprecate AWS ECR](https://docs.dataloop.ai/docs/deprecate-aws-ecr). #### Legacy API Calls and Introducing Replacements For more information about deprecated API calls and the replacements, see [Deprecate annotations JSON and ZIP endpoints](https://docs.dataloop.ai/docs/deprecate-annotations-json-and-zip-endpoints). --- ## July 19th, 2023 ### Widget Management You can now manage the widgets on the Project Overview page, enable or disable, and drag & drop widgets to rearrange. For more information, see [Manage Widgets](https://docs.dataloop.ai/docs/project-dashboard#manage-widgets). ### Annotation Studio #### Video Keyframe Shortcuts Keyframe setting keyboard shortcuts are now available for selected annotations, such as the annotation end-keyframe, Annotation start-keyframe, and toggling occlusion on the current frame. For more information, see the [Keyboard shortcuts](https://docs.dataloop.ai/docs/annotation-studio-basics#keyboard-shortcuts). #### Bulk attribute update for annotations from different labels You can bulk select and update the attribute for annotations from different labels. For more information, see the [Annotation Controls](https://docs.dataloop.ai/docs/instructions#annotation-controls). #### Remove item status before you edit All users must require to remove the item status to edit the item. You have an **Enable/disable force all users to remove status before editing annotations** option in the Project settings to enable or disable this feature. For more information, see the [Status Overlay On Completed Items](https://docs.dataloop.ai/docs/tasks-assignments#status-overlay-on-completed-items). ### Inactive user management Inactive users are grayed out and are not available for redistribution, but are available for reassignment. You can view an inactive message next to the user who is no longer in the project. For more information, see the [Inactive users](https://docs.dataloop.ai/docs/task-assignments-and-reassignments#reassigning-tasks). ### Quality Tasks Scoring Quality task scoring is available now for items, annotations, contributors, Avg. IOU, and Avg. Labels Consensus. When you activate a Consensus, Qualification, or Honeypot task while creating a task, a score function is enabled. All contributors, items, and annotations are scored via the function. The new **Scores** tab on the **Analytics** page gives you more information. For more information, see the [Score Analytics](https://docs.dataloop.ai/docs/score-analytics). #### Scores tab - New The new **Scores** tab on the **Analytics** page is added to provide new information, such as: 1. Number of contributors, items, and annotators. 2. Average percentage of IoU and Label Consensus. 3. Items consensus 4. Annotations consensus 5. Labels confusion matrix 6. Contributors' confusion matrix --- ## July 10th, 2023 ### Annotation Task - New UI 1. A new UI has been introduced for the following Annotation task management features: 1. Create or edit a task. 2. Create a QA task for the labeling task. 3. Add items to an existing task. For more information, see [Annotation and QA tasks](https://docs.dataloop.ai/docs/create-task). ### Annotation Studio Filters The filter in the annotation studio has been enhanced to include the following sections and updates: 1. **Annotation type**: It displays annotations based on the annotation tool types, such as box, point, polygon, note, etc. 2. **Annotation status**: It displays annotations based on the annotation status, such as no status, issue, for review, etc. 3. **Annotation properties**: It displays annotations based on the annotation properties, such as object ID, description, etc. 4. By default, all the filter are unchecked. When you select a filter, it displays only the annotations according to the selection you made. For more information, see [Annotations Filter](https://docs.dataloop.ai/docs/annotation-studio-basics#annotations-filter). ### Redistribute tasks by Metadata or DQL query You can now redistribute items to the assignees based on the specific context of their workload. The items that do not meet the criteria will not be listed for redistribution and will remain with the current assignee. For more information, see [Redistribute a Task or an Assignment of a Task](https://docs.dataloop.ai/docs/task-assignments-and-reassignments#redistribute-a-task-or-an-assignment-of-a-task). ### Notification Center #### Notification Channel - New A new **Notification Channel** feature has been introduced in the Organization Overview page of the DDOE platform to expand the notification capabilities by allowing users to connect additional channels, such as **email**, to receive notifications based on their notification subscription. For more information, see [Notification Channels](https://docs.dataloop.ai/docs/org-setup#notification-channels). ![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28579%29.png) #### Notification Subscription The notification subscription section in the **User Settings** dialog has been updated to include the option to unsubscribe from Task/Assignment (workflow) updates. For more information, see [Notification Subscriptions](https://docs.dataloop.ai/docs/notification-system). ![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/notifi-subscription%281%29.png) --- ## June 7th, 2023 ### Filter Out Annotations in the annotation-studio New recipe settings to filter annotations in the annotation-studio: Task scope and Recipe scope. Task scope only shows annotations created in the current task, while recipe scope only shows annotations that match the current recipe. These settings can help annotators and reviewers focus on their work and hide unnecessary annotations. ![image.png](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28565%29.png) ### Qualification Tasks Introducing qualification tasks in the annotation-studio to assess annotator skills. Create test tasks with hidden ground-truth answers and compare annotations to calculate scores for performance evaluation. Learn more qualification tasks [here](https://docs.dataloop.ai/docs/qualification-honeypot#qualification). ### Honeypot Tasks Introducing honeypot tasks for ongoing monitoring of annotator performance. Embed known ground-truth items into tasks with raw data to estimate annotation accuracy and ensure quality standards. Effectively monitor and maintain the desired level of annotation accuracy and quality by strategically placing honeypot items within tasks. Learn more honeypot tasks [here](https://docs.dataloop.ai/docs/qualification-honeypot#honeypot). --- ## May 24th, 2023 ### Workflow Notifications The platform will send users notifications regarding workflow events. Managers are notified when tasks are created, updated and completed. Annotators are notified when they have new work assignments, new issues were created, additional items added to their assignments and more. For more information, read [here](https://docs.dataloop.ai/docs/notification-system#workflow). ### Status Overlay Setting a status (e.g. Completed) on items in annotation or QA tasks flags the item as done, and often triggers further automation steps in Pipelines and FaaS. It is therefore important to ensure no changes to items/annotations are done after setting the status, else those changes won't be in effect for the automation steps. This situation is now resolved enforcing **annotators** to remove the status from the item, before they can make changes to annotations, even as part of fixing issues opened during a QA task. For more information, [read here](https://docs.dataloop.ai/docs/tasks-assignments#status-overlay-on-completed-items). ### Editing Integrations While we encourage using cross-account integrations for better security, customers using standard IAM integrations can now update the key/password values when credentials expire. ### Pipeline Variables Pipeline Variables bring enhanced flexibility and control to pipelines. With Pipeline Variables, it is possible to easily set and manage dynamic parameters as node input, access them simultaneously across multiple nodes, and update their values during runtime. The variables can be managed easily from the pipeline interface. SDK support will be added on the next SDK release. ![image.png](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28562%29.png) ### Pipeline Widget A new widget is now available in the project overview page, providing information about running pipelines, pending pipeline cycles and errors. ### Pipeline “Action” Filters - Supported by FaaS Now available for FaaS nodes as well! Enhance your pipeline flows by setting actions on function outputs when creating your FaaS. Use the actions as filters in the Pipeline to selectively process the returned outputs of a FaaS node. Action filters empower you to create smarter pipeline flows with increased flexibility and control. ![image.png](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/image%28561%29.png) --- ## May 1st, 2023 ### Auto Segmentation With SAM Model The [Auto-segmentation](https://docs.dataloop.ai/docs/create-semantic-segmentation#auto-segmentation) feature has been updated to use Meta's latest [Segment Anything Model](https://segment-anything.com/), delivering amazingly accurate segmentation right from the first shot. ![](https://cdn.document360.io/53f32fe9-1937-4652-8526-90c1bc78d3f8/Images/Documentation/da_ss_auto-seg.gif) ### AWS Policy S3 Folder Limited Access Storage integrations for AWS now support policies restricting access to specific folders. ### Azure Datalake Gen2 Storage integration for Azure Datalake Gen2 is now available from the platforms' user-interface. When selecting the storage integration type of Azure, it is not possible to further refine and select between Azure Blob and Azure DataLake Gen2. ### Ignoring WebM Format Warnings For Linked Items Linked items do not reside on the platform, and therefore cannot be converted to WebM format (which would require storing the WebM file on the platform). Projects using linked items can now set a project-wide configuration from the Project settings to hide and WebM format related messages. ### Pipeline: Support Consensus Tasks in Pipeline Consensus Tasks for QA workflow is now supported in pipeline as well - reviewers can work on items containing work from multiple contributors and flag issues on annotations or create note annotations to trigger correction work, and increase the quality of the annotations. To read more about consensus tasks, click [here](https://docs.dataloop.ai/docs/consensus). ### Model Management: Define Train & Validation Subsets on Model Train & validation subset were updated to be defined on the model itself. When setting a train/validation subset (as DQL filter), it will be added to the **model** metadata under metadata.system.subsets.train metadata.system.subsets.validation --- ## April 4th, 2023 ### New Dataset Technology A technological breakthrough enables DDOE to offer Dataset supporting sub-second queries for hot storage of 30M items. Query by item attributes, item and annotations metadata, and user metadata that represents the customer’s project context, all at sub-second queries at industry-leading capacity. The new technology now applies to all new Datasets. Migration of existing ones is available on-demand. ### Multi-Labeling Service Vendors Working with multiple labelling service providers is now inherently supported in the DDOE platform. Labeling companies are added to a project, confined to view only their scope of the project and unaware of other companies. Managers from such labeling companies can easily import the workforce from their organization, individually or in bulk as groups. Combined with the 'Pulling' work allocation method in annotation tasks, the workforce in tasks can dynamically be scaled according to budget and deadlines. Employ multi-labeling service providers to achieve 24/7 work continuity, cross-company quality validation, or domain expertise. ### New Project Dashboard/Overview Completely redesigned the project overview/dashboard, and it is now based on widgets, the first 2 covering what the old one did, Contributors and Datasets. The new widgets provide more information and more functionality. Additional widgets will be added in future releases. ### Consensus tasks - Reassignments A new feature now allows reassigning Consensus task assignments from one contributor to another, ensuring work continuity. For more information and to learn how to reassign a consensus assignment, read here. ### Video - delete keyframe from right-bottom keyframes list When annotating video files, a new feature makes it possible to delete key-frames. This is very useful when needs to undo an annotation change, or when working with uploaded model annotations, containing many key-frames. ### Consensus Support In QA Workflows Consensus work is supported in the QA workflow - reviewers can work on items containing work from multiple contributors and flag issues on annotations or create note annotations to trigger correction work, and increase the quality of the annotations. When annotators correct their work and apply 'Complete' status to the item, their annotations are removed from the master item, and the new, corrected ones are copied over to represent the corrected higher-quality work. Reviewers can then review their work and set the 'Approve' status on the item. ### Pipeline & FaaS ### Service Shutdown - Docker Image Our FaaS service restart loop is expanded to include monitoring for Docker image errors. Once an `ImagePullBackOff` error is detected, preventing your service from scaling up, DDOE system will shut the service down to save on computational costs. Also, you are notified of this action through the notification center. ### Merge Flows in a pipeline cycle Use the explicit Merge option which is now supported to bring flows within the pipeline back together into a single flow, as long they are all running in a single pipeline cycle. ### Pipeline ML Nodes: New Pipeline nodes are now available for you, supporting our Model Management tool directly from the pipeline. #### Model Predict node: Integrate the Model Predict node into your pipeline to invoke the `predict()` function of your model in your pipeline automation flow. The node gets an item as input and completes running once the prediction is completed. For more information, read [here](https://docs.dataloop.ai/docs/pipeline-nodes#model-predict-). #### Model Train node: The Model Train node can integrate into your pipeline to execute your model’s `train()` function during the pipeline run, allowing for seamless model training within your flow. For more information, read [here](https://docs.dataloop.ai/docs/pipeline-nodes#model-train-). #### Model-Related Trigger Events: To streamline your work with Model Management tool, DDOE introduces new trigger events that can use to automatically invoke your FaaS and Pipeline based on the model-related actions. The newly supported events are `model_create`, `model_update`, and `model_delete`. A system event is fired with every create, update, or delete action of your models. --- ## March 20th, 2023 ### Organization Members & Groups Redesigned The DDOE user interface is redesigned to manage organization members and groups. Now it is easier to invite members into your organization, assign roles, and view the organization project in which they are members. The interface for creating groups is enhanced to allow the reusing of groups and subgroups while showing the actual number of users in the group (handling duplicates). ### New storage driver - Azure Datalake Gen2 A new cloud storage driver is now available to support Azure Datalake Gen2. To learn more, [read here](https://docs.dataloop.ai/docs/azure-datalake-gen2-1). ### Cross-Account integration for AWS Create cross-account integration for AWS, yielding higher security. Eliminating the need for users to create and maintain IAM for use with the DDOE platform, you can now assume the role of a DDOE ARN. For more information and detailed setup instructions , [read here](https://docs.dataloop.ai/docs/aws-cross-account-integration). ### Improved Data Sampling The new data-sampling mechanism now samples data in a better-distributed way. When you create annotation and QA tasks, the data is evenly sampled from all task contributors (all assignments) while sampling data from the annotation task. ---