TokenCache

Memoizes tokens by source, so that a source tokenized repeatedly, such as the body of a lambda invoked in a loop, is lexed only once.

Lexing is a pure function of the source, hence tokens are safe to share between parses. Since tokens are memoized by source alone, a cache must always be used with the same kind of lexer.

Thread-safe.