Previous section: Appendix B: Dylan Syntax BNF

Dylan reference manual -- Lexical grammar

Lexical grammar

Lexical notes

In the lexical grammar, the various elements that come together to form a single token on the right-hand sides of rules must not be separated by white-space, so that the end result will be a single token. This is in contrast to the phrase grammar, where each element is already a complete token or a series of complete tokens.

Arbitrary white-space is permitted between tokens, but it is required only as necessary to separate tokens that might otherwise blend together.

Case is not significant except within character and string literals. The grammars do not reflect this, using one case or the other, but it is still true.

Comments

comment:
// ...the rest of the line
/* ...everything even across lines... */

Tokens

token:
SYMBOL
KEYWORD
LITERAL
STRING
UNARY-OPERATOR
BINARY-OPERATOR
reserved-word
punctuation
#-word
LITERAL:
number
character-literal
punctuation:
one of ( ) , . ; [ ] { } :: - = == => #( #[ ? ?? ...
#-word:
one of #t #f #next #rest #key #all-keys

Reserved words

reserved-word:
core-word
begin-word
intermediate-word
DEFINE-WORD
begin-word:
SIMPLE-BEGIN-WORD
EXPR-BEGIN-WORD
BEGIN-WORD
intermediate-word:
SIMPLE-INTERMEDIATE-WORD
EXPR-INTERMEDIATE-WORD
INTERMEDIATE-WORD
core-word:
one of define end generic handler let
one of local method macro otherwise

Symbols and keywords

SYMBOL:
any symbol that's not also a reserved-word
\ operator-symbol
KEYWORD:
symbol :
# STRING
symbol:
leading-alphabetic
leading-numeric alphabetic-character leading-alphabetic
leading-graphic leading-alphabetic
leading-alphabetic:
alphabetic-character
leading-alphabetic any-character
leading-numeric:
numeric-character
leading-numeric any-character
leading-graphic:
graphic-character
leading-graphic any-character
any-character:
alphabetic-character
numeric-character
graphic-character
special-character
alphabetic-character:
one of a b c d e f g h i j k l m n o p q r s t u v w x y z
numeric-character:
one of 0 1 2 3 4 5 6 7 8 9
graphic-character:
one of ! & * < = > | ^ $ % @ _
special-character:
one of - + ~ ? /

Operators

UNARY-OPERATOR:
unary-operator
BINARY-OPERATOR:
binary-operator
special-operator
operator-symbol:
unary-operator
binary-operator
unary-operator:
one of - ~
binary-operator:
one of + - * / ^ = == ~= < <= > >=
special-operator:
one of & | :=

Character and string literals

character-literal:
' character '
character:
any printing character (including space) except for ' or \
\ escape-character
\ '
STRING:
" more-string
more-string:
string-character more-string
"
string-character:
any printing character (including space) except for " or \
\ escape-character
\ "
escape-character:
one of \ a b e f n r t 0

Numbers

number:
integer
ratio
floating-point
integer:
binary-integer
octal-integer
signopt decimal-integer
hex-integer
binary-integer:
#b binary-digit
binary-integer binary-digit
octal-integer:
#o octal-digit
octal-integer octal-digit
decimal-integer:
decimal-digit
decimal-integer decimal-digit
hex-integer:
#x hex-digit
hex-integer hex-digit
binary-digit:
one of 0 1
octal-digit:
one of 0 1 2 3 4 5 6 7
decimal-digit:
one of 0 1 2 3 4 5 6 7 8 9
hex-digit:
one of 0 1 2 3 4 5 6 7 8 9 A B C D E F
ratio:
signopt decimal-integer / decimal-integer
floating-point:
signopt decimal-integeropt . decimal-integer exponentopt
signopt decimal-integer . decimal-integeropt exponentopt
signopt decimal-integer exponent
exponent:
E signopt decimal-integer
sign:
one of + -

Next section: Phrase Grammar