TokenData
data class TokenData(val text: String, val position: IntRange, val groups: Sequence<String> = emptySequence(), val namedGroups: Map<String, String> = emptyMap(), val walkerResult: WalkerParsingResult<*>? = null)
Data of a single, usually small, substring of the source code that stores a chunk of information. For instance, the Markdown code Hello _Quarkdown_ contains the tokens Hello , _, Quarkdown, _.
Parameters
text
the substring extracted from the source code, also known as lexeme.
position
location of the token within the source code
groups
capture groups values for this token
namedGroups
capture groups that hold a name. groups does not contain groups from namedGroups