Skip to main content

module analysis::grammars::ParserTesting

rascal-Not specified

Reusable test functions to run (generated) parsers on a test corpus

Usage​

import analysis::grammars::ParserTesting;

Dependencies​

extend lang::rascal::\syntax::Rascal;
import Exception;
import Grammar;
import IO;
import List;
import Message;
import ParseTree;
import String;
import analysis::grammars::Ambiguity;
import lang::rascal::grammar::definition::Modules;
import lang::rascal::grammar::definition::Symbols;
import util::Monitor;
import util::Reflective;

syntax ParserTestModule​

start syntax ParserTestModule
= @synopsis{A syntax for files that can contain many small test cases for a parser}
Tags tags "module" QualifiedName name Import* imports ParserTestCase* cases;

syntax ParserTestCase​

syntax ParserTestCase
= "syntax" "test" Name? label "(" Sym symbol ")" "expects" ParserTestExpect expect TagString content;

syntax ParserTestExpect​

syntax ParserTestExpect
= ok: "ok"
| ambiguous:"ambiguous"
| error: "error"
;

alias FileParser​

&T (loc org)

function reportParserTestModule​

Run the tests embedded in a parser test module file and return true iff all the tests succeeded.

bool reportParserTestModule(loc m, PathConfig pcfg=pathConfig())

Benefits​

  • all failing tests are reported in standard fashion on stdout
  • when grammars remain unchanged on disk, the tests reuse parsers from the previous run

Pitfalls​

  • parser test module files have to be loaded, as well as Rascal module files, and converted to a parser, which involves critical file IO and the right Path Config instance.

function runParserTestModule​

Run the tests embedded in a parser test module file and return true iff all the tests succeeded.

list[Message] runParserTestModule(loc m, PathConfig pcfg=pathConfig())

Benefits​

  • all failing tests are reported in standard fashion as a list[Message], which can be processed further downstream by communicating to IDE diagnostics views, or as editor annotations, for example.
  • when grammars remain unchanged on disk, the tests reuse parsers from the previous run

Pitfalls​

  • parser test module files have to be loaded, as well as Rascal module files, and converted to a parser, which involves critical file IO and the right Path Config instance.

function doRunParserTestModule​

The workhorse parses the test specifications, composes the main grammar, and iterates over each individual test

list[Message] doRunParserTestModule(ParserTestModule m, PathConfig pcfg=pathConfig())

function deslash​

str deslash(str input)

function collectRascalModules​

set[Module] collectRascalModules(Import* imports, set[str] done, PathConfig pcfg=pathConfig())

set[Module] collectRascalModules((Import) `extend <QualifiedName name>;`, set[str] done, PathConfig pcfg=pathConfig())

set[Module] collectRascalModules((Import) `import <QualifiedName name>;`, set[str] done, PathConfig pcfg=pathConfig())

default set[Module] collectRascalModules(Import _, set[str] done, PathConfig pcfg=pathConfig())

function runParserTestCase​

Message runParserTestCase(ParserTestCase testCase, &T (type[&T <: Tree], str) theParser)

function constructParsingTestCorpus​

Collect files of a specific extension from a folder recursively, while filtering ignored files and folders

list[loc] constructParsingTestCorpus(loc root, set[str] extensions, set[loc] ignores={})

function testCorpusForParseErrorsAndAmbiguity​

Run a generated parser on a corpus and print errors, diagnostics and summary statistics

bool testCorpusForParseErrorsAndAmbiguity(type[&T <: Tree] grammar, list[loc] corpus, str jobLabel="Testing <grammar>", bool abort=false)

This functions returns true if and only if none of the files in the corpus exhibit a parse error or an ambiguity for the given grammar

Pitfalls​

  • the grammar is best if it's a start symbol (since we are parsing files we need spaces before and after the top sort).

function testCorpusForParseErrorsAndAmbiguity​

Run a parser on a corpus and print errors, diagnostics and summary statistics

bool testCorpusForParseErrorsAndAmbiguity(FileParser[&T] parser, list[loc] corpus, str jobLabel="Testing parser ", bool abort=false)

This functions returns true if and only if none of the files in the corpus exhibit a parse error or an ambiguity

Benefits​

  • this is a typical simple task for every project that has a grammar
  • ambiguity reported here can be analyzed using analysis::grammars::Ambiguity or the drambiguity project

Pitfalls​

  • the parser must not allowAmbiguity=true
  • the parser must not allowRecovery=true either
  • the parser must accept whitespace and comments at the beginning and end of the file