The Px Language

A Px grammar is a sequence of rules. Each rule is a name, an =, a PEG expression, and an optional capture list, and is terminated by a blank line. The capture list indicates which of the named items in the expression carry semantic content that should be made available to the calling program.

rule_name =
	expression
	-> captured, names

Rules and highlighting scope annotations

Right after the name, before the =, a rule can optionally carry a parenthesised (as scope.name) — this doesn’t change what the rule matches at all; it’s metadata read only by the syntax-highlighting generator:

comment (as comment.line.px) =
	'//' *[^\n]

Atoms

Every expression is built from these six kinds of atom:

Atom	Meaning
`.`	any single character (including a newline)
`name`	call another rule
`\x`	a character property (see below)
`'text'`	a literal string
`[chars]`	a character class
`(expr)`	a parenthesised group

Operators

Operators go in prefix position, before the atom they apply to — the opposite of a regex, where x* follows the x. In Px it’s *x.

Operator	Meaning
`?A`	zero or one of A
`*A`	zero or more of A
`+A`	one or more of A
`\\|A\\|B`	either A, or B, etc (ordered choice — see the PEG primer)
`&A`	succeed here (consuming nothing) if A would match
`!A`	succeed here (consuming nothing) if A would not match
`{n}A`	exactly n of A; `{n,name}` and other count forms reference a captured value

&/! are lookahead assertions, the main subject of Pitfalls & Lookahead.

Note: {n} is not yet implemented in the underlying Pegexp library.

Character classes and properties

[...] matches one character from the defined set. A - between two characters gives a range, exactly like a regex class. A ^ at the start negates the whole set, succeeding on any character not in the set. Any single character may be any valid Unicode character. Inside a class or a literal string, a backslash escape can be:

a C-style escape: \n \t \r \b \f, or \' \" \\ for those characters themselves;
a numeric escape: \177 (octal), \xHH or \x{H...} (hex), \uHHHH or \u{H...} (Unicode)
Except in literal strings, a character property: \a alpha, \d digit, \h hex digit, \s whitespace, \w alpha-or-digit, \L lowercase, \U uppercase. These work inside a character class, or as a standalone atom (\d on its own means “any digit”; note that the underlying Unicode library recognises many kinds of digits), but not in a literal string;

Captures

A rule followed by -> a, b, c captures whatever matched a, b and c into an ordered array in the parse tree, even when one of them appears more than once in the rule (handy for a comma-separated list). Anything not named in the list is parsed but discarded — only what you capture is kept. Note that if multiple items of the same name are matched more than once, the capture is an Array of all the matches.

A label (:name right after any atom, including a rule call) gives that specific occurrence a name to capture by, independent of the rule’s name:

object = '{' *(|(string:k ':' TOP:v *(',' string:k ':' TOP:v)) | s) '}'
	-> k, v

Here string:k and TOP:v both appear twice (once, and again inside the repeated ,-separated tail) — every match of either gets appended to the k or v array respectively, so object’s capture is two parallel arrays of keys and values.

A worked example

Px’s own JSON grammar uses everything above in about twenty lines:

s	= *[ \t\r\n]				// Zero or more space characters

TOP	= s (| object |array |string |'true':v |'false':v |'null':v |number ) s
	-> object, array, string, v, number

object	= '{' *(|(string:k ':' TOP:v *(',' string:k ':' TOP:v)) | s) '}'
	-> k, v

array	= '[' (|(TOP:e *(',' TOP:e)) |s) ']'
	-> e

string	= s ["] *(|[^"\\\u{0}-\u{1F}]:c |escape:c) ["] s
	-> c

escape	= [\\] (|["\\/bfnrt] | 'u' [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] [0-9a-fA-F] )

number	= ?'-' (|'0' | [1-9] *[0-9]) ?('.' +[0-9]) ?([eE] ?[-+] +[0-9])

TOP’s ordered choice tries object, then array, then string, then the three literal keywords, then number — and because a JSON value can only ever start with one of those, order doesn’t matter here. string’s body is exactly the “unbounded repetition needs a stop condition” shape from pitfalls.md: every iteration either takes a plain character that isn’t ", \ or a control character, or an escape, until the closing " ends it. The syntax for numeric character escapes is derived from Unicode 2 (which uses two 16-bit surrogate characters to encode a larger set), so it differs from Px’s native grammar.

See this grammar turned into a real parser on the C++ generator page, and into a highlighter on the syntax highlighting page.