FunctionCallTokenizer

Tokenizes function calls in text content.

This class is responsible for parsing text to identify function calls and their components, such as function names, parameters, and arguments. It uses a lexer to tokenize the text and then processes the tokens to create FunctionCall objects.

This is a lightweight approach to function call tokenization, more efficient than using a full parser.