If you are not fully satisfied with the default segmentation, you can use and experiment other off-the-shelf segmenter.
- Go to the Processing view
- Create a new segmenter
- Give a name to the segmenter
- Select an off-the-shelf segmenter:
- blingfire: a sentence segmenter from Microsoft (this is the one used by default in the platform)
- pySBD: another sentence segmenter
- regex: a regex segmenter
- rules-segmenter: a rule-based segmenter from Spacy
- Configure the segmenter in the right panel
- Save and activate the segmenter on your project.
- Note that all existing annotations will be kept.
- The yellow star shows the default segmenter.
- Note this segmenter will be used as soon as you upload new documents in the project.
- Go to the Segments view
- Check the new segmentation (below is an example to segment using the comma with a regex segmenter)
- If you want to configure and use a segmentation pipeline: