quarkdown-core/com.quarkdown.core.lexer

Package-level declarations

Types

abstract class AbstractLexer(val source: CharSequence) : Lexer

A Lexer that expects iterations through source content.

interface Lexer

A scanner that transforms raw string data into a list of token. For instance, the Markdown code Hello _Quarkdown_ is tokenized by its implementation into Hello , _, Quarkdown, _. See lexing_examples.txt for further information.

Token

abstract class Token(val data: TokenData)

A wrapper of a TokenData that may be parsed in order to extract information. A token can be parsed into a com.quarkdown.core.ast.Node.

TokenCoordinates

data class TokenCoordinates(val line: Int, val column: Int)

The position of a token within the source code.

TokenData

data class TokenData(val text: String, val position: IntRange, val groups: Sequence<String> = emptySequence(), val namedGroups: Map<String, String> = emptyMap(), val walkerResult: WalkerParsingResult<*>? = null)

Data of a single, usually small, substring of the source code that stores a chunk of information. For instance, the Markdown code Hello _Quarkdown_ contains the tokens Hello , _, Quarkdown, _.

Functions

acceptAll

fun <T> Iterable<Token>.acceptAll(visitor: TokenVisitor<T>): List<T>

Accepts a list of tokens to a shared visitor.

toCoordinates

fun IntRange.toCoordinates(source: CharSequence): TokenCoordinates

Converts a range of indexes within a string to its (x, y) coordinates.