Annotator - Getting Started Guide

Dataloop annotation platform (also referred to as 'annotation-studio' and 'data-application') is designed to provide annotators a smooth and streamlined experience, to ensure pixel-accurate work at high velocity. The platform's performances, responsiveness and user-friendliness is highly important to us, as it all directly reflects on annotation work throughput.

Getting started

1. Annotators are usually joined by an invite to a project. If you're expecting an invite, check with the project manager. Our customers also have the option to manage their workforce in Organizations, and from there assign to different projects - in such case, expect an invite to the organization, but you'll need to be added to a project before you begin annotating
2. The project you are currently working on is referred to as the 'active project'. It is indicated as a 'breadcrumb' at the header of the platform. It's straight forward if you have only 1, otherwise click the 'My dashboard' icon to see available projects and switch between them.
   
3. In your active project there's a list of 'My assignments' - your work assignment with progress indication, number of items and more. Double-click an assignment to start working on it.

Annotation Work Process

These are the main concepts of annotation work in the Dataloop platform:

1. Your work in a project is contained in 'Assignments'. A manager can create multiple assignments for you in the same project.
2. You work on 'files' or 'items'. Assignment can have all the files allocated to you in advance, or it can contain only a few, and more are added when those are completed.
3. You need to declare finishing work on a file by setting a status on it from the buttons in the bottom-right. The default ones for annotation work are
   1. Complete - Annotation work is completed
   2. Discard - File is disqualified for annotation work

4. You can browse between files in your assignment using arrow keys at the bottom of the annotation studio. This can help in learning about edge cases or when waiting for information from your managers, reflecting new knowledge on uncompleted items. However you are still expected to either Complete or Discard all of the files.
   

QA Process

QA Process Overview

Many projects make use of Dataloop QA Workflow, which allows managers to take the work done by annotators and have it reviewed. If reviewers decide that corrections needs be made, they have several options:

1. They can do the corrections by themselves - you will not be aware of this, as this does not affect you
2. They can raise an Issue on an annotation - raising an Issue on an annotation in a QA assignment will automatically remove the 'Complete' status, and the item will reappear in the annotation assignment. Annotators are expected to review all items with issues, correct them, and flag them with For-review , so reviewers can decide if correction is good and the item is finalized.
3. They can create an Annotation-Note - if an annotator misses an annotation, for example placed a bounding-box over 3 cars and skipped the 4th car, then the reviewer sees a problem but doesn't have the annotation to flag with an Issue . The tool for that is the annotation-note, an annotation tool that puts an icon of a bug over the image/video, with a label (e.g. 'Car'), and an optional free-text comment (e.g. 'you missed this car'). This note=annotation is created with an Issue , and will trigger the same process. The annotator should correct the situation (e.g. create a box over the 4th car), flag the annotation note with For-review*, and press the 'Complete' button again.

Working With QA

Assignments List

From the assignments page, annotators can see assignments that have issues, and address those by double-clicking the assignment.

Annotations List

In the annotation studio, annotations with issues are marked with the red exclamation mark

Marking 'For-Review'

After correcting the annotation, click the magnifier's icon from the annotation list, or select all the annotations with issues and do a bulk operation from a similar icon at the top icons line.