Annotate Queue Items¶

This guide shows how to work through an annotation queue as a reviewer: submit annotations, edit them, skip items and flag items. To learn what the statuses mean, read Annotating. For the bigger picture, see the Annotations overview.

Prerequisites¶

An annotation queue that contains items. See Create an Annotation Queue and Add Items.
Permission to add annotations, and either assignment to the queue or an empty assignee list. See User Groups.

Open Your Queue¶

In the left sidebar, open Annotations.
Click the queue you want to work on.

Annotate Items¶

On the queue detail page, click Start Annotating. Items open one at a time, oldest first.
Read the item in the left panel. For a session, it shows the conversation history with roles and timestamps, a Participant Data tab and a Session State tab.
Fill in the annotation form in the right panel. The fields appear in the order the queue's author set. See Annotation Schema for the field types.
Click Submit.

The next item opens automatically. The page shows how many items you have reviewed out of the total in the queue.

Edit Your Annotation¶

Open the item you annotated.
In the annotations list, click Edit next to your annotation. The form opens with your previous answers.
Change your answers and click Save. Click Cancel to discard your changes.

Saving updates your annotation and recalculates the queue's aggregate scores.

Skip an Item¶

Click Skip to leave an item for later. Skipped items stay in the queue and reappear after you have been through the other available items.

Flag an Item¶

Flag an item when something prevents you from annotating it, such as missing content, a corrupted session, or content that needs an admin to review.

Click Flag on the annotation form.
Enter the reason. It is required.
Click Flag Item.

The item's status changes to Flagged and it no longer appears in your annotation sequence. The reason is recorded with your name and the time. Later flags are added to the list instead of replacing it. A queue manager can unflag the item, which returns it to the review workflow.

Expected Outcome¶

Items that have all required reviews become Completed, or Awaiting resolution on queues that need more than one review. See Item Status Lifecycle. To resolve an Awaiting resolution item, see Resolving Multi-Reviewer Conflicts.
The queue's progress and aggregate scores on its detail page update after each submission.