antlr
public class TokenBuffer extends Object
A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.
See Also: Token antlr.Tokenizer TokenQueue
Field Summary | |
---|---|
protected TokenStream | input |
int | markerOffset |
int | nMarkers |
int | numToConsume |
TokenQueue | queue |
Constructor Summary | |
---|---|
TokenBuffer(TokenStream input_) Create a token buffer |
Method Summary | |
---|---|
void | consume() Mark another token for deferred consumption |
TokenStream | getInput() return the Tokenizer (needed by ParseView) |
int | LA(int i) Get a lookahead token value |
Token | LT(int i) Get a lookahead token |
int | mark() Return an integer marker that can be used to rewind the buffer to
its current state. |
void | reset() Reset the input buffer to empty state |
void | rewind(int mark) Rewind the token buffer to a marker. |
Parameters: mark Marker returned previously from mark()