Backtracking control -- tokens

regex TOP     { <integer> <addop> <integer> {*} }
regex integer { \d+: {*} }
regex addop   { [ '+' | '-' ] {*} }

For modern one-pass languages, non-backtracking regexes are very common

The keyword "token" indicates a regex that does no backtracking by default

Examples:

token integer  { \d+ }

token ident    { <alpha> \w* }

token variable { [ '$' | '@' | '%' ] <ident> }