Explicit Boundary Markers for Subword Vocabularies
This work introduces an alternative to standard whitespace conventions using an explicit word boundary marker, which prevents duplication in subword tokenizers, and suggests that duplication carries a cost that compression does not capture.