Parsing Service

rascal-0.42.0

Synopsis

A parser is the first and foremost thing you need for constructing an LSP server/IDE.

Examples

Type in a Syntax Definition, also known as a context-free grammar, for your language.

Here is an example that defines a very small programming language called "Pico". We will use this demo language throughout all recipes for your Language Server.

module demo::lang::Pico::Syntax

import ParseTree;

lexical Id = [a-z][a-z0-9]* !>> [a-z0-9];      ❶  

lexical Natural = [0-9]+ ;      ❶  

lexical String = "\"" ![\"]*  "\"";      ❶  

layout Layout = WhitespaceAndComment* !>> [\ \t\n\r%];      ❷  

lexical WhitespaceAndComment      ❷  
   = [\ \t\n\r]
   | @category="Comment" ws2: "%" ![%]+ "%"      ❸  
   | @category="Comment" ws3: "%%" ![\n]* $      ❸  
   ;

start syntax Program      ❹  
   = program: "begin" Declarations decls {Statement  ";"}* body "end" ;

syntax Declarations 
   = "declare" {Declaration ","}* decls ";" ;  
 
syntax Declaration = decl: Id id ":" Type tp;

syntax Type 
   = natural:"natural" 
   | string :"string" 
   ;

syntax Statement 
   = asgStat: Id var ":="  Expression val 
   | ifElseStat: "if" Expression cond "then" {Statement ";"}*  thenPart "else" {Statement ";"}* elsePart "fi"
   | whileStat: "while" Expression cond "do" {Statement ";"}* body "od"
  ;  
     
syntax Expression      ❺  
   = id: Id name
   | strCon: String string
   | natCon: Natural natcon
   | bracket "(" Expression e ")"
   > left conc: Expression lhs "||" Expression rhs
   > left ( add: Expression lhs "+" Expression rhs
          | sub: Expression lhs "-" Expression rhs
          )
  ;

start[Program] program(str s) {
  return parse(#start[Program], s);
}

    ❻     start[Program] program(str s, loc l) {
  return parse(#start[Program], s, l);
} 

It's important that you import the ParseTree module to be able to call the Parse function:

import ParseTree;

and then you can write your own parser function that wraps the #start[Program] non-terminal:

import demo::lang::Pico::Syntax;
start[Program] parsePico(str contents, loc origin) 
    = parse(#start[Program], contents, origin);

The parse function must take a start non-terminal as parameter (so not just #Program), because otherwise spaces, newlines and comments before and after the main Program content will lead to parse errors.

With the above function every parse error will lead to an error diagnostic in the editor, and syntax highlighting only works if there is no parse error. To help your users a bit, you can activate parse error recovery:

import ParseTree;
import demo::lang::Pico::Syntax;
start[Program] parsePicoWithRecovery(str contents, loc origin) 
    = parse(#start[Program], contents, origin, allowRecovery=true);

Now syntax highlighting will indicate which part of the file has been recognized and which part of the file has not. The parse errors will still appear in the Diagnostics view.

It is always a good idea to test your parser in the terminal:

rascal>parsePico("begin declare a: natural; a := 42 end", |demo:///|)
start[Program]: (start[Program]) `begin declare a: natural; a := 42 end`

And to find out what a parse error looks like:

rascal>parsePico("begin declare a: natural; a = 42 end", |demo:///|)
|TODO:///|: ParseError(|demo:///|(28,1,<1,28>,<1,29>))

Or you could write a test function for it, for future reference:

test bool testPicoParser() {
    return start[Program] _ := parsePico("begin declare a: natural; a := 42 end", |demo:///|);
}
test bool testErrorPicoParser() {
    try {
         parsePico("begin declare a: natural; a = 4$2 end", |demo:///|);
         return false;
    }
    catch ParseError(_): {
        return true;
    }
}

In general a Parsing Service is simply a function that satisfies the LanguageServer-Parser signature.

Now let's move on to registering your language with the IDE and run your own language server.

Benefits

• you can always test any service function in the terminal. This is highly recommended because simple errors and output can sometimes be hard to find in the IDE.
• you can always write Rascal test functions to add to the stability of your LSP services.
• the Trees produced by your Parsing Service will be the input of all other services later.