quarkdown-core/com.quarkdown.core.lexer.scan.block/BlockScanLexer/tokenize

tokenize

open override fun tokenize(): Sequence<Token>

Disassembles some raw string into smaller tokens.

Return

a lazy sequence of tokens, produced on demand