quarkdown-core/com.quarkdown.core.lexer/AbstractLexer

AbstractLexer

abstract class AbstractLexer(val source: CharSequence) : Lexer

A Lexer that expects iterations through source content.

Parameters

source

the content to be tokenized

Inheritors

Constructors

Link copied to clipboard
constructor(source: CharSequence)

Properties

Link copied to clipboard
abstract val currentIndex: Int

Index of the latest scanned character within source.

Link copied to clipboard
open override val source: CharSequence

Functions

Link copied to clipboard
abstract fun createFillToken(position: IntRange): Token?
Link copied to clipboard
abstract fun tokenize(): List<Token>

Disassembles some raw string into smaller tokens.