Once you have created labels, take the time to define the annotation guidelines. This step is of high importance, especially when working in a team. It allows to explain in a central place how annotation shall take place such that every involved person can follow the same principles.
Take for instance the annotation of a person’s name: Do we take the academic title into the entity or not? “Dr. Susan Miller” or “Susan Miller”? There are countless such questions that different annotators may treat differently if they don’t have access to a set of guidelines. And this will create confusion in the corpus and suboptimal performance of the resulting models.
So we recommend you take the time and define annotation guidelines for your team (even for yourself!).
In order to do so:
- Go to the Labels view
- Click on a labelset
- Write the general annotation guidelines in the right panel with the text editor. The annotation guidelines will be automatically shared with Data Analysts & Annotators that will have access to this project. Following annotation guidelines are key to annotate consistently a project!
- Don’t forget to save before moving to another menu, otherwise you’ll have to rewrite the content!
- Click on a label name
- Write the specific guidelines on this label (if any)
- Don’t forget to save before moving to another menu, otherwise you’ll have to rewrite the content!
Let’s move on to the next step: