tokenization (Q2438971)
Jump to navigation
Jump to search
breaking a stream of text up into chunks for analysis or further processing
- tokenisation
| Language | Label | Description | Also known as |
|---|---|---|---|
| English | tokenization |
breaking a stream of text up into chunks for analysis or further processing |
|
Statements
Identifiers
Sitelinks
Wikipedia(4 entries)
- dewiki Tokenisierung
- enwiki Tokenization (lexical analysis)
- idwiki Tokenisasi
- ruwiki Токен (лексический анализ)