quarkdown-core/com.quarkdown.core.lexer/Lexer/tokenize

tokenize

abstract fun tokenize(): List<Token>

Disassembles some raw string into smaller tokens.

Return

the ordered list of tokens