SentencePiece tokenizer

Enter text in the box - tokenization is done as you type.

This is a SentencePiece tokenizer implemented in pure Go and compiled to WebAssembly. The vocabulary and settings are taken from the Google AI Gemma open model.