BlockScanLexer

A Lexer that scans block-level tokens line by line, trying each recognizer in order at every line start.

The scanned text is the source followed by one newline, and scanning stops once a block has ended at or past the source's own length. That trailing newline is what lets a construct whose terminator expects one still close at the end of input, and it is why a source that already ends in a newline yields one final token for the empty line after it.

Parameters