ANTLR - How to extact units from a dimension - c++

Question

I'm using ANTLR4 and the CSS grammar from https://github.com/antlr/grammars-v4/tree/master/css3. The grammar defines the following ( pared down a little for brevity )
dimension
: ( Plus | Minus )? Dimension
;
fragment FontRelative
: Number E M
| Number E X
| Number C H
| Number R E M
;
fragment AbsLength
: Number P X
| Number C M
| Number M M
| Number I N
| Number P T
| Number P C
| Number Q
;
fragment Angle
: Number D E G
| Number R A D
| Number G R A D
| Number T U R N
;
fragment Length
: AbsLength
| FontRelative
;
Dimension
: Length
| Angle
;
The matching works fine but I don't see an obvious way to extract the units. The parser creates a DimensionContext which has 3 TerminalNode members - Dimension, Plus and Minus. I'd like to be able to extract the unit during parse without having to do additional string parsing.
I know that one issue that the Length and Angle are fragments. I changed the grammar not use fragments
Unit
: 'em'
| 'ex'
| 'ch'
| 'rem'
| 'vw'
| 'vh'
| 'vmin'
| 'vmax'
| 'px'
| 'cm'
| 'mm'
| 'in'
| 'pt'
| 'q'
| 'deg'
| 'rad'
| 'grad'
| 'turn'
| 'ms'
| 's'
| 'hz'
| 'khz'
;
Dimension : Number Unit;
And things still parse but I don't get any more context about what the units are - the Dimension is still a single TerminalNode. Is there a way to deal with this without having to pull apart the full token string?

Answer 1

You will want to do as little as possible in the lexer:
NUMBER
: Dash? Dot Digit+ { atNumber(); }
| Dash? Digit+ ( Dot Digit* )? { atNumber(); }
;
UNIT
: { aftNumber() }?
( 'px' | 'cm' | 'mm' | 'in'
| 'pt' | 'pc' | 'em' | 'ex'
| 'deg' | 'rad' | 'grad' | '%'
| 'ms' | 's' | 'hz' | 'khz'
)
;
The trick is to produce the NUMBER and UNIT as separate tokens, yet limited to the required ordering. The actions in the NUMBER rule just set a flag and the UNIT predicate ensures that a UNIT can only follow a NUMBER:
protected void atNumber() {
_number = true;
}
protected boolean aftNumber() {
if (_number && Character.isWhitespace(_input.LA(1))) return false;
if (!_number) return false;
_number = false;
return true;
}
The parser rule is trivial, but preserves the detail required:
number
: NUMBER UNIT?
;
Use a tree-walk, parse the NUMBER to a Double and an enum (or equivalent) to provide the semantic UNIT characterization:
public enum Unit {
CM("cm", true, true), // 1cm = 96px/2.54
MM("mm", true, true),
IN("in", true, true), // 1in = 2.54cm = 96px
PX("px", true, true), // 1px = 1/96th
PT("pt", true, true), // 1pt = 1/72th
EM("em", false, true), // element font size
REM("rem", false, true), // root element font size
EX("ex", true, true), // element font x-height
CAP("cap", true, true), // element font nominal capital letters height
PER("%", false, true),
DEG("deg", true, false),
RAD("rad", true, false),
GRAD("grad", true, false),
MS("ms", true, false),
S("s", true, false),
HZ("hz", true, false),
KHZ("khz", true, false),
NONE(Strings.EMPTY, true, false), // 'no unit specified'
INVALID(Strings.UNKNOWN, true, false);
public final String symbol;
public final boolean abs;
public final boolean len;
private Unit(String symbol, boolean abs, boolean len) {
this.symbol = symbol;
this.abs = abs;
this.len = len;
}
public boolean isAbsolute() { return abs; }
public boolean isLengthUnit() { return len; }
// call from the visitor to resolve from `UNIT` to Unit
public static Unit find(TerminalNode node) {
if (node == null) return NONE;
for (Unit unit : values()) {
if (unit.symbol.equalsIgnoreCase(node.getText())) return unit;
}
return INVALID;
}
#Override
public String toString() {
return symbol;
}
}