Function tokenize

  • Tokenizes an input text using the ANTLR4 tokenizer

    Returns

    • Array of AntlrTokens containing the tokens in the text

    Parameters

    • input: string

      Text to be tokenized

    Returns AntlrToken[]

Settings

Member Visibility

Theme

Generated using TypeDoc