Lexical grammar

Lexical notes

In the lexical grammar, the various elements that come together to form a single token on the right-hand sides of rules must not be separated by white-space, so that the end result will be a single token. This is in contrast to the phrase grammar, where each element is already a complete token or a series of complete tokens.

Arbitrary white-space is permitted between tokens, but it is required only as necessary to separate tokens that might otherwise blend together.

Case is not significant except within character and string literals. The grammars do not reflect this, using one case or the other, but it is still true.

Comments

comment:

// ...the rest of the line

/* ...everything even across lines... */

Tokens

token:

SYMBOL

KEYWORD

LITERAL

STRING

UNARY-OPERATOR

BINARY-OPERATOR

reserved-word

punctuation

#-word

LITERAL:

number

character-literal

punctuation:

one of ( ) , . ; [ ] { } :: - = == => #( #[ ? ?? ...

#-word:

one of #t #f #next #rest #key #all-keys

Reserved words

reserved-word:

core-word

begin-word

intermediate-word

DEFINE-WORD

begin-word:

SIMPLE-BEGIN-WORD

EXPR-BEGIN-WORD

BEGIN-WORD

intermediate-word:

SIMPLE-INTERMEDIATE-WORD

EXPR-INTERMEDIATE-WORD

INTERMEDIATE-WORD

core-word:

one of define end generic handler let

one of local method macro otherwise

Symbols and keywords

SYMBOL:

any symbol that's not also a reserved-word

\ operator-symbol

KEYWORD:

symbol :

# STRING

symbol:

leading-alphabetic

leading-numeric alphabetic-character leading-alphabetic

leading-graphic leading-alphabetic

leading-alphabetic:

alphabetic-character

leading-alphabetic any-character

leading-numeric:

numeric-character

leading-numeric any-character

leading-graphic:

graphic-character

leading-graphic any-character

any-character:

alphabetic-character

numeric-character

graphic-character

special-character

alphabetic-character:

one of a b c d e f g h i j k l m n o p q r s t u v w x y z

numeric-character:

one of 0 1 2 3 4 5 6 7 8 9

graphic-character:

one of ! & * < = > | ^ $ % @ _

special-character:

one of - + ~ ? /

Operators

UNARY-OPERATOR:

unary-operator

BINARY-OPERATOR:

binary-operator

special-operator

operator-symbol:

unary-operator

binary-operator

unary-operator:

one of - ~

binary-operator:

one of + - * / ^ = == ~= < <= > >=

special-operator:

one of & | :=

Character and string literals

character-literal:

' character '

character:

any printing character (including space) except for ' or \

\ escape-character

\ '

STRING:

" more-string

more-string:

string-character more-string

"

string-character:

any printing character (including space) except for " or \

\ escape-character

\ "

escape-character:

one of \ a b e f n r t 0

Numbers

number:

integer

ratio

floating-point

integer:

binary-integer

octal-integer

signopt decimal-integer

hex-integer

binary-integer:

#b binary-digit

binary-integer binary-digit

octal-integer:

#o octal-digit

octal-integer octal-digit

decimal-integer:

decimal-digit

decimal-integer decimal-digit

hex-integer:

#x hex-digit

hex-integer hex-digit

binary-digit:

one of 0 1

octal-digit:

one of 0 1 2 3 4 5 6 7

decimal-digit:

one of 0 1 2 3 4 5 6 7 8 9

hex-digit:

one of 0 1 2 3 4 5 6 7 8 9 A B C D E F

ratio:

signopt decimal-integer / decimal-integer

floating-point:

signopt decimal-integeropt . decimal-integer exponentopt

signopt decimal-integer . decimal-integeropt exponentopt

signopt decimal-integer exponent

exponent:

E signopt decimal-integer

sign:

one of + -