Tokenizer

Tokenizer

new Tokenizer()

Source:

tokenizer.js, line 6

Methods

(static) tokenize(input) → {Array.<Token>}

Source:

tokenizer.js, line 17

Parameters:

Name	Type	Description
`input`	string

Returns:

Type: Array.<Token>