StandardRegexLexer
constructor(source: CharSequence, patterns: List<TokenRegexPattern>, fillTokenType: (TokenData) -> Token? = null)
Parameters
source
the content to be tokenized
patterns
the patterns to search for, in descending order of priority
fillTokenType
type of token to assign to uncaptured groups in order to fill the gaps. A token will not be created if null and empty gaps will be present in the output