Types of tokens
Warning
uCalc API Preview Release Notice:The uCalc engine has successfully transitioned to modern cross-platform environments.The next phase envolves some structural changes, performance optimizations, and API refinements.The API is subject to breaking changes prior to the stable release. Please evaluate the preview version thoroughly before production use.
| Name ▲ ▼ | Value ▲ ▼ | Description ▲ ▼ |
|---|---|---|
| None | ||
| Generic | A token defined without any specified property is generic (default) | |
| Empty | Represents the absence of any character (token not found) | |
| AlphaNumeric | A token that behaves as alphanumeric (works with word boundaries) | |
| ArgSeparator | A token that serves as argument separator (typically a comma) | |
| Bracket | A token that starts a bracketed group; typically things like opening curly braces {, square brackets [, or parenthesis ( | |
| BracketMatch | The closing token that matches a Bracket token, such as }, ], or ) | |
| MemberAccess | A token that separates an object from its member, typically a . (a dot) | |
| Literal | A token that represents a literal value, typically a quoted string or number | |
| StatementSep | A token that represents a statement separator, typically a ; (semicolon) | |
| Whitespace | A token that represents whitespace, typically tab or space characters | |
| Reducible | A token that is reduced to the minimum number of characters necessary to produce a match. See remarks | |
| Eof | Non-character that represents the end of a stream of text | |
| Escape | Escapes the next token so it's not interpreted as special (like part of a replacement command or variable) | |
| Command | Not implemented (yet) | |
| LocalVar | Not implemented | |
| StaticVar | Not implemented | |
| DefineNow | StaticVar | Not implemented |
| TempDef | Not implemented | |
| EvalInsert | Not implemented | |
| EvalDuringParse | Not implemented | |
| TransformInsert | Not implemented | |
| FileInclude | Not implemented | |
| CurrentSpace | Not implemented | |
| DefSpace | Not implemented | |
| EvalSpace | Not implemented | |
| TokenTransform | Partially implemented. Used internally to allow string interpolation |
This enum is used when creating tokens with [topic: uCalc.Tokens.Add], [topic: uCalc.Tokens.Define], or [topic: uCalc.Tokens.Insert].
A Reducible token is typically used for operator symbols like +, -, *, etc. Unlike alphanumeric tokens where the match includes all consecutive characters, a the minimum number of characters to produce a match is what is used. For example, "aaaaa" would be considered one alphanumeric token, whereas consecutive minus signs "-----" might be considered as 5 separate "-" tokens.