Video Studio

Overview

Dataloop's Video Annotation Studio provides a user-friendly interface for annotating videos with precision, supporting frame-by-frame labeling, object tracking, and advanced tools like interpolation. It allows for seamless collaboration, customizable workflows, and integrates with Dataloop’s data management and automation systems for efficient project scaling. Ideal for large-scale video datasets, the studio is perfect for industries like autonomous driving, surveillance, and media analytics.

Dataloop’s Video Tool brings pixel-accurate frame annotations to videos, and as such - requires annotation to take place while using the WebM file format. For more information, see the WebM and Frame-Accurate Annotation article.

Video Annotation Concepts

• Standard video playback controls: Play, pause, jump frames, adjust play-speed
• Selecting labels and creating annotations (much like in Image Studio): annotations are shown at frame-level in the Annotation Bar area.
• Annotation Bar controls: Enables you to set the annotation start and end frames, as well as toggle occlusion (hidden annotation) on/off. All these states are visualized in the annotations bar.
• The frames on which changes in an annotation occur are considered as Key-frames (annotation creation, position changes, label change, attribute change, occlusion on/off, annotation ending).
• The ranges between every 2 keyframes (from-to) are listed on the right-side panel. Click such a range to have the video jump to the first keyframe.
• The video timeline displays the mapping of annotations according to their number on each frame and allows you to jump to areas of interest.

Video Studio User-Interface

Video Controls

• Adjust volume
• Take a snapshot
• Adjust play speed - from X0.5 to X8
• Jump to start
• Jump back X frames (enter the number of frames into the text box, default is 1)
• Play/pause
• Text box to enter the number of frames for jumping backward/forward
• Jump to the end of the video
• Change annotation position automation
      * Linear interpolation (default)
      * None (off)
      * AI Tracker (may imply computation costs)
• Playback time - current position/total video length

Video Timeline

The video timeline includes:

• A marker showing the current playback position
• A window surrounding the marker
      * The window determines the scope of frames presented in the annotation bar.
      * The window can be resized to increase or decrease the number of frames.
• The marker or window can be grabbed and moved anywhere on the timeline.
• Click anywhere on the timeline to have the marker set at that position.
• Use the mouse scroll wheel to zoom in and out of the timeline and better manage longer video files.

Annotations Bar

The Annotations Bar shows a line with frames per annotation that’s in the current frame of the video. A marker shows the current playback position, placed on the exact current frame.

• Drag & drop the marker, or click anywhere on the timeline to set it as the current playback time.

• Use the controls on the left to set a keyframe at the current position.
      * Set Start Frame (removes previous Start Frame).
      * Set End Frame: The annotation will end on this frame.
      * Set occlusion on/off: Set ON when the annotation is hidden and Off when it's back. Changes are visualized over frames with different opacities.

• Use the controls that are just over the AnnotationBar to set these start, end, or occlusion to all annotations at once.

KeyFrames

Every change or action performed on an annotation occurs in a certain frame, making it a keyframe. Keyframes are created when first creating an annotation, whenever an annotation is manually moved, when setting occlusion on or off, and when ending an annotation.

The range between two keyframes is listed on the right-side panel. The ranges are sorted by the order of frames.

Click any range of keyframes to jump to the first frame; this allows you to easily navigate between areas of interest in your annotated video.

Before You Begin

Supported Video formats

Refer to the Supported Video formats section to view the supported image formats in the Video Studio.

Task Prerequisites

Labeling work is performed in the context of a labeling task. Create your task with the intended data (video files), the Recipe (labels, attributes, and labeling instructions), and annotate team members.

Supported Video Annotation Tools and Types

The Video Annotation platform supports the following annotation tools: classification, point (with pose), box, note, polygon, polyline, cuboid and auto annotation tools.

Video Studio Keyboard Shortcuts

Working With Video Annotations

1. Create an annotation

   1. Use the label picker to search for labels and select them. Use the pin-to-top option to access frequently used labels.
   2. Use the default tool or select one. For example, classification, box, etc.
   3. Create an annotation on the frame you need.

2. Adjust an annotation: A common action is setting a new start frame for an annotation after it was created, usually just a few frames back.

   1. Use the marker in the annotation bar to move a few frames back, until you find the right frame.
   2. Click the Start-frame button - that’s it, a new start frame is set.

3. Working on annotations: As the video continues, change the annotation as needed. Every change is reflected as a keyframe, visualized in the annotation bar and in the keyframes list.

   1. Move an annotation to adjust it to the tracked object's location.
   2. Set the annotation to hidden when the object is not visible.
      1. Use the ‘Hidden’ action button in the Annotations Bar.
      2. Use the ‘Hidden’ action button from the annotation controller (in the canvas itself).
   3. Resume the annotation's visibility by clicking the button again.
   4. Change annotation labels.
      1. Click the label name in the annotation controller.
      2. Select a different label and approve.
      3. The label color will change in the annotation bar, and a keyframe is created.
   5. Set or change an attribute: Select an annotation and set or change its attribute. A new keyframe is created on that frame.

4. Ending annotations: When a new annotation is created, it ends by default on the last frame of the video. Set a new end-frame by clicking the button (from the annotation controller or the Annotations Bar) to change the end-frame to be in the current play position.

Pre-Processed and Blocked Video Files

Video Studio does not allow annotation work while video files are being converted to WebM format. The duration of this pre-processing depends on the number and size of the video files.

If the conversion process fails for any reason or if a media-attribute discrepancy between the WebM and the original file is detected, the video studio will prevent annotation work on the file, as annotations cannot be guaranteed to be frame-accurate.

Changing Video Annotations

The process of changing video annotations (labels and attributes) enables faster and more accurate work, especially when correcting models' pre-annotations. When performing such an update, select one of the following duration options:

• Next keyframe
• Specific frames: Set the requested range of frames
• End of annotation

For more actions available on the Image Studio, see the Annotation Studio Basics article.