Same as Surface.Lexer.token_list_language_agnostic, but with tokens specialized to French.

Main lexing function used in code blocks

Main lexing function used outside code blocks

Entry point of the lexer, distributes to lex_code_fr or lex_law_fr depending of Surface.Lexer.is_code.