module lang::box::util::Tree2Box
The default formatting rules for any parse tree.
Usage
import lang::box::util::Tree2Box;
Dependencies
import ParseTree;
import lang::box::\syntax::Box;
import String;
Description
This module is meant to be extended to include rules specific for a language.
The main goal of this module is to minimize the number of necessary specializations for any specific programming language.
This module is a port of the original default formatting rules, implemented in C + ATerm library + APIgen, of the "Pandora" in The ASF+SDF Meta-Environment, as described in
M.G.J. van den Brand, A.T. Kooiker, Jurgen J. Vinju, and N.P. Veerman. A Language Independent Framework for Context-sensitive Formatting. In CSMR '06: Proceedings of the Conference on Software Maintenance and Reengineering, pages 103-112, Washington, DC, USA, 2006. IEEE Computer Society Press.
However, due to the more powerful pattern matching available in Rascal, than in C with the ATerm library, we can specialize for more cases more easily than in the original paper. For example, single and multi-line comment styles are automatically recognized.
The current algorithm, not extended, additionally guarantees that no comments are lost as long as their grammar
rules have been tagged with @category="comment" or the legacy @category="Comment"
Another new feature is the normalization of case-insensitive literals. By providing To Upper or To Lower the mapping algorithm will change every instance of a case-insensitive literal accordingly before translating it to an L box expression. In case of As Is, the literal will be printed as it occurred in the source code.
Examples
rascal>import lang::box::\syntax::Box;
ok
rascal>extend lang::box::util::Tree2Box;
ok
Notice how we used extend and not import, which will be important in the following.
rascal>import lang::pico::\syntax::Main;
ok
First, let's get an example program text
rascal>example = "begin
|1 >>>> '%% this is an example Pico program
|2 >>>> ' declare
|3 >>>> ' a : %inline comment% natural,
|4 >>>> ' b : natural;
|5 >>>> ' a := a + b;
|6 >>>> ' b := a - b;
|7 >>>> ' a := a - b
|8 >>>> 'end";
str: "begin\n%% this is an example Pico program\n declare\n a : %inline comment% natural,\n b : natural;\n a := a + b;\n b := a - b;\n a := a - b\nend"
───
begin
%% this is an example Pico program
declare
a : %inline comment% natural,
b : natural;
a := a + b;
b := a - b;
a := a - b
end
───
Now we parse it:
rascal>program = [start[Program]] example;
value: appl(
prod(
start(sort("Program")),
[
layouts("Layout"),
label(
"top",
sort("Program")),
layouts("Layout")
],
{}),
[appl(
prod(
layouts("Layout"),
[conditional(
\iter-star(lex("WhitespaceAndComment")),
{\not-follow(\char-class([
range(9,10),
range(13,13),
range(32,32),
range(37,37)
]))})],
{}),
[appl(
regular(\iter-star(lex("WhitespaceAndComment"))),
[],
src=|prompt:///|(0,0,<1,0>,<1,0>))],
src=|prompt:///|(0,0,<1,0>,<1,0>)),appl(
prod(
label(
"program",
sort("Program")),
[
lit("begin"),
layouts("Layout"),
label(
"decls",
sort("Declarations")),
layouts("Layout"),
label(
"body",
\iter-star-seps(
sort("Statement"),
[
layouts("Layout"),
lit(";"),
layouts("Layout")
])),
layouts("Layout"),
lit("end")
],
{}),
[appl(
prod(
lit("begin"),
[
\char-class([range(98,98)]),
\char-class([range(101,101)]),
\char-class([range(103,103)]),
\char-class([range(105,105)]),
\char-class([range(110,110)])
],
{}),
[char(98),char(101),char(103),char(105),char(110)]),appl(
prod(
layouts("Layout"),
[conditional(
\iter-star(lex("WhitespaceAndComment")),
{\not-follow(\char-class([
range(9,10),
range(13,13),
range(32,32),
range(37,37)
]))})],
{}),
[appl(
regular(\iter-star(lex("WhitespaceAndComment"))),
[appl(
prod(
lex("WhitespaceAndComment"),
[\char-class([
range(9,10),
range(13,13),
range(32,32)
])],
{}),
[char(10)],
src=|prompt:///|(5,1,<1,5>,<2,0>)),appl(
prod(
lex("WhitespaceAndComment"),
[
lit("%%"),
conditional(
\iter-star(\char-class([
range(1,9),
range(11,1114111)
])),
{\end-of-line()})
],
{tag("category"("comment"))}),
[appl(
prod(
lit("%%"),
[
\char-class([range(37,37)]),
\char-class([range(37,37)])
],
{}),
[char(37),char(37)]),appl(
regular(\iter-star(\char-class([
range(1,9),
range(11,1114111)
]))),
[char(32),char(116),char(104),char(105),char(115),char(32),char(105),char(115),char(32),char(97),char(110),char(32),char(101),char(120),char(97),char(109),char(112),char(108),char(101),char(32),char(80),char(105),char(99),char(111),char(32),char(112),char(114),char(111),char(103),char(114),char(97),char(109)],
src=|prompt:///|(8,32,<2,2>,<2,34>))],
src=|prompt:///|(6,34,<2,0>,<2,34>)),appl(
prod(
lex("WhitespaceAndComment"),
[\char-class([
...
Then we can convert it to a Box tree:
rascal>b = toBox(program);
Box: HV_(
[
U_([]),
V_(
[
H_(
[L("begin")],
hs=1),
I_([V_(
[
U_([]),
V_(
[
H_(
[L("declare")],
hs=1),
I_([V_(
[
U_([]),
HV_(
[
H_(
[
HV_(
[
L("a"),
U_([]),
L(":"),
U_([]),
L("natural")
],
hs=1,
vs=0),
L(",")
],
hs=0),
H_(
[HV_(
[
L("b"),
U_([]),
L(":"),
U_([]),
L("natural")
],
hs=1,
vs=0)],
hs=0)
],
hs=1,
vs=0),
U_([])
],
vs=0)]),
L(";")
],
vs=0),
U_([]),
V_(
[
H_(
[
HV_(
[
L("a"),
U_([]),
L(":="),
U_([]),
HV_(
[
L("a"),
U_([]),
L("+"),
U_([]),
L("b")
],
hs=1,
vs=0)
],
hs=1,
vs=0),
L(";")
],
hs=0),
H_(
[
HV_(
[
L("b"),
U_([]),
L(":="),
U_([]),
HV_(
[
L("a"),
U_([]),
L("-"),
U_([]),
L("b")
],
hs=1,
vs=0)
],
hs=1,
vs=0),
L(";")
],
hs=0),
H_(
[HV_(
[
L("a"),
U_([]),
L(":="),
U_([]),
HV_(
[
L("a"),
U_([]),
L("-"),
U_([]...
Finally, we can format the box tree to get a prettier format:
rascal>import lang::box::util::Box2Text;
ok
rascal>format(b)
str: "begin\n declare\n a : natural, b : natural\n ;\n a := a + b;\n b := a - b;\n a := a - b\nend\n"
───
begin
declare
a : natural, b : natural
;
a := a + b;
b := a - b;
a := a - b
end
───
If you are not happy, then you should produce a specialization:
rascal>Box toBox((Program) `begin <Declarations decls> <{Statement ";"}* body> end`, FormattingOptions opts=formattingOptions())
|1 >>>> = V([
|2 >>>> L("begin"),
|3 >>>> I([
|4 >>>> toBox(decls)
|5 >>>> ], is=2),
|6 >>>> I([
|7 >>>> toBox(body)
|8 >>>> ], is=4),
|9 >>>> L("end")
|10 >>>> ]);
Box (Program, FormattingOptions opts = ...): function(|prompt:///|(0,285,<1,0>,<11,7>))
and we see the result here:
rascal>format(toBox(program));
str: "begin\n declare\n a : natural, b : natural\n ;\n a := a + b;\n b := a - b;\n a := a - b\nend\n"
───
begin
declare
a : natural, b : natural
;
a := a + b;
b := a - b;
a := a - b
end
───
data FormattingOptions
Configuration options for toBox
data FormattingOptions (
CaseInsensitivity ci = asIs()
)
= formattingOptions()
;
data CaseInsensitivity
Normalization choices for case-insensitive literals.
data CaseInsensitivity
= toLower()
| toUpper()
| toCapitalized()
| asIs()
;
function toBox
This is the generic default formatter
default Box toBox(t:appl(Production p, list[Tree] args), FO opts = fo())
This generic formatter is to be overridden by someone constructing a formatter tools
for a specific language. The goal is that this toBox default rule maps
syntax trees to plausible Box expressions, and that only a minimal amount of specialization
by the user is necessary.
function toBox
For ambiguity clusters an arbitrary choice is made.
default Box toBox(amb({Tree t, *Tree _}), FO opts=fo())
function toBox
When we end up here we simply render the unicode codepoint back.
default Box toBox(c:char(_), FormattingOptions opts=fo() )
function toBox
Cycles are invisible and zero length
default Box toBox(cycle(_, _), FO opts=fo())
function toClusterBox
Create a V box of V boxes where the inner boxes are connected and the outer boxes are separated by an empty line.
Box toClusterBox(list[Tree] lst, FO opts=fo())
Box toClusterBox(&T* lst, FO opts=fo())
Box toClusterBox(&T+ lst, FO opts=fo())
This function learns from the input trees how vertical clusters were layout in the original tree.
The resulting box maintains the original clustering.
For example, such lists of declarations which are separated by a newline, remain separated after formatting with toClusterBox
int a1 = 1;
int a2 = 2;
int b1 = 3;
int b2 = 4;
Benefits
- many programmers use vertical clustering, or "grouping statements", to indicate meaning or intent, by not throwing this away we are not throwing away the documentative value of their grouping efforts.
Pitfalls
- To Cluster Box is one of the (very) few Box functions that use layout information from the input tree to influence the layout of the output formatted code. It replaces a call to To Box for that reason.
- To Cluster Box does not work on separated lists, yet.
function toExpBox
Reusable way of dealing with large binary expression trees
Box toExpBox(Box prefix, Tree expression, Box wrapper=HOV(), FO opts=fo())
- the default
toBoxwill flatten nested binary expressions to U lists. - the G box groups each operator with the following expression on the right hand-side,
- given an initial element (usually L("=") or L(":=")) for the assignment operators
- the entire list is indented in case the surrounding context needs more space
- the net result is usually in vertical mode:
= operand1
+ operand2
+ operand3
or in horizontal mode:
= operand1 + operand2 + operand3
By default To Exp Box wraps it result in a HOV context, but you can pass
in a different wrapper if you like.
function toExpBox
Reusable way of dealing with large binary expression trees
Box toExpBox(Tree expression, Box wrapper=HV(), FO opts=fo())
- the default
toBoxwill flatten nested binary expressions to U lists. - the G box groups each operator horizontally with the following expression on the right hand-side.
- the net result is usually in vertical mode:
operand1 + operand2
+ operand3
or in horizontal mode:
operand1 + operand2 + operand3
By default To Exp Box wraps it result in a HV context, but you can pass
in a different wrapper if you like.
alias FO
Private type alias for legibility's sake
FormattingOptions
function fo
This is a short-hand for legibility's sake
FO fo()
function ci
Implements normalization of case-insensitive literals
str ci(toLower(), str word)
str ci(toUpper(), str word)
str ci(toCapitalized(), str word)
str ci(asIs(), str word)
function delabel
Removing production labels helps with case distinctions on ((Symbol)) kinds.
Production delabel(prod(Symbol s, list[Symbol] syms, set[Attr] attrs))
Production delabel(regular(Symbol s))
function delabel
Removing symbol labels helps with case distinctions on ((Symbol)) kinds.
Symbol delabel(label(_, Symbol s))
Symbol delabel(conditional(Symbol s, _))
default Symbol delabel(Symbol s)