Token

processing.app.syntax

## Class Token

- Object
- - Token

-     public class Token
      extends Object

  A linked list of tokens. Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by `SyntaxDocument.getColors()` to get a color value, a length value which is the length of the token in the text, and a pointer to the next token in the list.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static byte` | `COMMENT1` This can be used to mark a comment.
    `static byte` | `COMMENT2` This can be used to mark a comment.
    `static byte` | `END` The token type, that along with a length of 0 marks the end of the token list.
    `static byte` | `FUNCTION1` Functions
    `static byte` | `FUNCTION2` Methods (functions inside a class)
    `static byte` | `FUNCTION3` Loop/function-like blocks (for, while, etc.)
    `static byte` | `FUNCTION4` Built-in Processing functions (setup, draw, mouseDragged).
    `byte` | `id` The id of this token.
    `static byte` | `ID_COUNT` The total number of defined token ids.
    `static byte` | `INTERNAL_FIRST` The first id that can be used for internal state in a token marker.
    `static byte` | `INTERNAL_LAST` The last id that can be used for internal state in a token marker.
    `static byte` | `INVALID` Invalid token id.
    `static byte` | `KEYWORD1` Keywords (void, int, boolean, etc.)
    `static byte` | `KEYWORD2` Fields [variables within a class]
    `static byte` | `KEYWORD3` Loop/function-like blocks (for, while, etc.)
    `static byte` | `KEYWORD4` Processing variables (width, height, focused, etc.)
    `static byte` | `KEYWORD5` Datatypes (int, boolean, etc.)
    `static byte` | `LABEL` Label token id.
    `int` | `length` The length of this token.
    `static byte` | `LITERAL1` Strings in quotes
    `static byte` | `LITERAL2` Constants (QUARTER_PI, CORNERS, etc.)
    `Token` | `next` The next token in the linked list.
    `static byte` | `NULL` Normal text token id.
    `static byte` | `OPERATOR` Operator token id.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Token(int length, byte id)` Creates a new token.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `String` | `toString()` Returns a string representation of this token.

    - ### Methods inherited from class Object

      `equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### NULL

          public static final byte NULL

      Normal text token id. This should be used to mark normal text.
      See Also:
        Constant Field Values

    - #### COMMENT1

          public static final byte COMMENT1

      This can be used to mark a comment.
      See Also:
        Constant Field Values

    - #### COMMENT2

          public static final byte COMMENT2

      This can be used to mark a comment.
      See Also:
        Constant Field Values

    - #### LITERAL1

          public static final byte LITERAL1

      Strings in quotes
      See Also:
        Constant Field Values

    - #### LITERAL2

          public static final byte LITERAL2

      Constants (QUARTER_PI, CORNERS, etc.)
      See Also:
        Constant Field Values

    - #### LABEL

          public static final byte LABEL

      Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)
      See Also:
        Constant Field Values

    - #### KEYWORD1

          public static final byte KEYWORD1

      Keywords (void, int, boolean, etc.)
      See Also:
        Constant Field Values

    - #### KEYWORD2

          public static final byte KEYWORD2

      Fields [variables within a class]
      See Also:
        Constant Field Values

    - #### KEYWORD3

          public static final byte KEYWORD3

      Loop/function-like blocks (for, while, etc.)
      See Also:
        Constant Field Values

    - #### KEYWORD4

          public static final byte KEYWORD4

      Processing variables (width, height, focused, etc.)
      See Also:
        Constant Field Values

    - #### KEYWORD5

          public static final byte KEYWORD5

      Datatypes (int, boolean, etc.)
      See Also:
        Constant Field Values

    - #### FUNCTION1

          public static final byte FUNCTION1

      Functions
      See Also:
        Constant Field Values

    - #### FUNCTION2

          public static final byte FUNCTION2

      Methods (functions inside a class)
      See Also:
        Constant Field Values

    - #### FUNCTION3

          public static final byte FUNCTION3

      Loop/function-like blocks (for, while, etc.)
      See Also:
        Constant Field Values

    - #### FUNCTION4

          public static final byte FUNCTION4

      Built-in Processing functions (setup, draw, mouseDragged).
      See Also:
        Constant Field Values

    - #### OPERATOR

          public static final byte OPERATOR

      Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)
      See Also:
        Constant Field Values

    - #### INVALID

          public static final byte INVALID

      Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.
      See Also:
        Constant Field Values

    - #### ID_COUNT

          public static final byte ID_COUNT

      The total number of defined token ids.
      See Also:
        Constant Field Values

    - #### INTERNAL_FIRST

          public static final byte INTERNAL_FIRST

      The first id that can be used for internal state in a token marker.
      See Also:
        Constant Field Values

    - #### INTERNAL_LAST

          public static final byte INTERNAL_LAST

      The last id that can be used for internal state in a token marker.
      See Also:
        Constant Field Values

    - #### END

          public static final byte END

      The token type, that along with a length of 0 marks the end of the token list.
      See Also:
        Constant Field Values

    - #### length

          public int length

      The length of this token.

    - #### id

          public byte id

      The id of this token.

    - #### next

          public Token next

      The next token in the linked list.

  - ### Constructor Detail

    - #### Token

          public Token(int length,
               byte id)

      Creates a new token.
      Parameters:
        `length` - The length of the token
        `id` - The id of the token

  - ### Method Detail

    - #### toString

          public String toString()

      Returns a string representation of this token.
      Overrides:
        `toString` in class `Object`