Skip to main content

module demo::lang::pico::LanguageServer

rascal-0.43.0
org.rascalmpl.rascal-lsp-2.23.0

Demonstrates the API for defining and registering IDE language services for Programming Languages and Domain Specific Languages.

Usage​

import demo::lang::pico::LanguageServer;

Dependencies​

import util::LanguageServer;
import util::IDEServices;
import ParseTree;
import util::ParseErrorRecovery;
import util::Reflective;
extend demo::lang::pico::Extensions;
import DateTime;
import IO;
import List;
import Location;
import String;

Description​

The core functionality of this module is built upon these concepts:

  • Register Language for enabling your language services for a given file extension in the current IDE.
  • Language is the data-type for defining a language, with meta-data for starting a new LSP server.
  • A Language Service is a specific feature for an IDE. Each service comes with one Rascal function that implements it.

function picoParser​

private Tree (str _input, loc _origin) picoParser(bool allowRecovery) {
    return ParseTree::parser(#start[Program], allowRecovery=allowRecovery, filters=allowRecovery ? {createParseErrorFilter(false)} : {});
}

function picoLanguageServer​

A language server is simply a set of ((LanguageService))s.


set[LanguageService] picoLanguageServer() = {
    parsing(picoParser(true), usesSpecialCaseHighlighting = false),
    documentSymbol(picoDocumentSymbolService),
    codeLens(picoCodeLenseService),
    execution(picoExecutionService),
    inlayHint(picoInlayHintService),
    definition(picoDefinitionService),
    codeAction(picoCodeActionService),
    rename(picoRenamingService, prepareRenameService = picoRenamePreparingService),
    didRenameFiles(picoFileRenameService),
    selectionRange(picoSelectionRangeService),
    callHierarchy(picoPrepareCallHierarchy, picoCallsService),
    completion(picoCompletionService, additionalTriggerCharacters = ["="])
};

Each Language Service for pico is implemented as a function. Here we group all services such that the LSP server can link them with the Language definition later.

function picoLanguageServerSlowSummary​

This set of contributions runs slower but provides more detail.


set[LanguageService] picoLanguageServerSlowSummary() = {
    parsing(picoParser(true), usesSpecialCaseHighlighting = false),
    analysis(picoAnalysisService, providesImplementations = false),
    build(picoBuildService)
};

Language Services can be registered asynchronously and incrementally, such that quicky loaded features can be made available while slower to load tools come in later.

function picoDocumentSymbolService​

The documentSymbol service maps pico syntax trees to lists of DocumentSymbols.


list[DocumentSymbol] picoDocumentSymbolService(start[Program] input)
  = [symbol("<input.src>", DocumentSymbolKind::\file(), input.src, children=[
      *[symbol("<var.id>", var is function ? \function() : \variable(), var.src) | /IdType var := input, var.id?]
  ])];

Here we list the symbols we want in the outline view, and which can be searched using symbol search in the editor.

function picoAnalysisService​

The analyzer maps pico syntax trees to error messages and references


Summary picoAnalysisService(loc l, start[Program] input) = picoSummaryService(l, input, analyze());

function picoBuildService​

The builder does a more thorough analysis then the analyzer, providing more detail


Summary picoBuildService(loc l, start[Program] input) = picoSummaryService(l, input, build());

data PicoSummarizerMode​

A simple "enum" data type for switching between analysis modes

data PicoSummarizerMode  
     = analyze()
     | build()
     ;

function findDefinitions​

rel[DocumentSymbolKind, loc, Id, str] findDefinitions(Tree input, bool funcScope = false) {
    rel[DocumentSymbolKind, loc, Id, str] defs = {};
    top-down-break visit (input) {
        case var:(IdType) `<Id id>: <Type _>`: defs += <funcScope ? constant() : variable(), var.src, id, typeOf(var)>;
        case func:(IdType) `<Id id>(<{IdType ","}* args>): <Type _> := <Expression _>`: {
            defs += <function(), func.src, id, typeOf(func)>;
            defs += findDefinitions(args, funcScope = true);
        }
    }
    return defs;
}

data Summary​

data Summary (rel[DocumentSymbolKind, loc, Id, str] definitionsByKind = {})

function picoSummaryService​

Translates a pico syntax tree to a model (Summary) of everything we need to know about the program in the IDE.


Summary picoSummaryService(loc l, start[Program] input, PicoSummarizerMode mode) {
    Summary s = summary(l);

    // definitions of variables
    s.definitionsByKind = findDefinitions(input);
    rel[str, loc] defs = {<"<id>", d> | <d, id> <- s.definitionsByKind<1, 2>};

    // uses of identifiers
    rel[loc, str] uses = {<id.src, "<id>"> | /Id id := input, id notin s.definitionsByKind<2>};

    // documentation strings for identifier uses
    rel[loc, str] docs = {<var.src, "*variable* <var>"> | /IdType var := input};

    // Provide errors (cheap to compute) both in analyze mode and in build mode.
    s.messages += {<src, error("<id> is not defined", src, fixes=prepareNotDefinedFixes(src, defs))>
                  | <src, id> <- uses, id notin defs<0>};

    // "references" are links for loc to loc (from def to use)
    s.references += (uses o defs)<1,0>;

    // "definitions" are also links from loc to loc (from use to def)
    s.definitions += uses o defs;

    // "documentation" maps locations to strs
    s.documentation += (uses o defs) o docs;

    // Provide warnings (expensive to compute) only in build mode
    if (build() := mode) {
        rel[loc, str] asgn = {<id.src, "<id>"> | /Statement stmt := input, (Statement) `<Id id> := <Expression _>` := stmt};
        s.messages += {<src, warning("<id> is not assigned", src)> | <src, id, _> <- s.definitionsByKind[variable()], "<id>" notin asgn<1>};
    }

    return s;
}

function picoDefinitionService​

Looks up the declaration for any variable use using a list match into a ((Focus))


set[loc] picoDefinitionService([*_, Id use, *_, start[Program] input]) = { def.src | /IdType def := input, use := def.id};

Pitfalls​

This demo actually finds the declaration rather than the definition of a variable in Pico.

function prepareNotDefinedFixes​

If a variable is not defined, we list a fix of fixes to replace it with a defined variable instead.


list[CodeAction] prepareNotDefinedFixes(loc src,  rel[str, loc] defs)
    = [CodeAction::action(title="Change to <existing<0>>", edits=[changed(src.top, [replace(src, existing<0>)])]) | existing <- defs];

function picoCodeActionService​

Finds a declaration that the cursor is on and proposes to remove it or add a TODO to it.


list[CodeAction] picoCodeActionService([*_, IdType x, *_, start[Program] program])
    = [CodeAction::action(command=removeDecl(program, x, title="remove <x>"))];

default list[CodeAction] picoCodeActionService(Focus _focus) = [];

data Command​

data Command  
     = renameAtoB(start[Program] program)
     | removeDecl(start[Program] program, IdType toBeRemoved)
     ;

function picoCodeLenseService​

Adds an example lense to the entire program.


lrel[loc,Command] picoCodeLenseService(start[Program] input)
    = [<input.src, renameAtoB(input, title="Rename variables a to b.")>];

function picoInlayHintService​

Generates inlay hints that explain the type of each variable usage.


list[InlayHint] picoInlayHintService(start[Program] input) {
    typeLookup = ( "<name>" : "<tp>" | /(IdType)`<Id name> : <Type tp>` := input);

    return [
        hint(name.src, " : <typeLookup["<name>"]>", \type(), atEnd = true)
        | /(Expression)`<Id name>` := input
        , "<name>" in typeLookup
    ];
}

function getAtoBEdits​

Helper function to generate actual edit actions for the renameAtoB command


list[DocumentEdit] getAtoBEdits(start[Program] input)
   = [changed(input.src.top, [replace(id.src, "b") | /id:(Id) `a` := input])];

function picoExecutionService​

Command handler for the renameAtoB command


value picoExecutionService(renameAtoB(start[Program] input)) {
    applyDocumentsEdits(getAtoBEdits(input));
    return ("result": true);
}

function picoExecutionService​

Command handler for the removeDecl command


value picoExecutionService(removeDecl(start[Program] program, IdType toBeRemoved)) {
    applyDocumentsEdits([changed(program.src.top, [replace(toBeRemoved.src, "")])]);
    return ("result": true);
}

function picoRenamePreparingService​

Prepares the rename service by checking if the id can be renamed


loc picoRenamePreparingService(Focus _:[Id id, *_]) {
    if ("<id>" == "fixed") {
        throw "Cannot rename id <id>";
    }
    return id.src;
}

function picoRenamingService​

Renaming service implementation, unhappy flow.


tuple[list[DocumentEdit], set[Message]] picoRenamingService(Focus focus, "error") = <[], {error("Test of error detection during renaming.", focus[0].src.top)}>;

function picoRenamingService​

Renaming service implementation, happy flow.


default tuple[list[DocumentEdit], set[Message]] picoRenamingService(Focus focus, str newName) = <[changed(focus[0].src.top, [
    replace(id.src, newName)
    | cursor := focus[0]
    , /Id id := focus[-1]
    , id := cursor
])], {}>;

function picoFileRenameService​


tuple[list[DocumentEdit],set[Message]] picoFileRenameService(list[DocumentEdit] fileRenames) {
    // Iterate over fileRenames

    list[DocumentEdit] edits = [];
    for (renamed(loc from, loc to) <- fileRenames) {
        // Surely there is a better way to do this?
        toBegin = to[offset=0][length=0][begin=<1,0>][end=<1,0>];
        edits = edits + changed(to, [insertBefore(toBegin, "%% File moved from <from> to <to> at <now()>\n", separator="")]);
    }
    return <edits, {info("<size(edits)> moves succeeded!", |unknown:///|)}>;
}

function picoSelectionRangeService​

list[loc] picoSelectionRangeService(Focus focus)
    = dup([t.src | t <- focus]);

function picoPrepareCallHierarchy​

list[CallHierarchyItem] picoPrepareCallHierarchy(Focus focus: [*_, e:(Expression) `<Id callId>(<{Expression ","}* _>)`, *_, start[Program] prog]) {
    s = picoSummaryService(prog.src.top, prog, analyze());
    return [ callHierarchyItem(prog, id, d, tp)
        | d <- s.definitions[callId.src]
        , <id, tp> <- s.definitionsByKind[function(), d]
    ];
}

list[CallHierarchyItem] picoPrepareCallHierarchy(Focus _: [*_, d:(IdType) `<Id _>(<{IdType ","}* _>): <Type _> := <Expression _>`, *_, start[Program] prog])
    = [callHierarchyItem(prog, d)];

default list[CallHierarchyItem] picoPrepareCallHierarchy(Focus _) = [];

function callHierarchyItem​

CallHierarchyItem callHierarchyItem(start[Program] prog, Id id, loc decl, str tp)
    = callHierarchyItem("<id>", function(), decl, id.src, detail = tp, \data = \data(prog));

CallHierarchyItem callHierarchyItem(start[Program] prog, d:(IdType) `<Id id>(<{IdType ","}* _>): <Type _> := <Expression _>`)
    = callHierarchyItem("<id>", function(), d.src, id.src, detail = typeOf(d), \data = \data(prog));

data CallHierarchyData​

data CallHierarchyData  
     = \data(start[Program] prog)
     ;

function typeOf​

str typeOf((IdType) `<Id _>: <Type t>`) = "<t>";

function picoCallsService​

lrel[CallHierarchyItem, loc] picoCallsService(CallHierarchyItem ci, CallDirection dir) {
    s = picoSummaryService(ci.\data.prog.src.top, ci.\data.prog, analyze());
    calls = [];
    for (<d, id, t> <- s.definitionsByKind[function()]) {
        newItem = callHierarchyItem(ci.\data.prog, id, d, t);
        <caller, callee> = dir is incoming
            ? <newItem, ci>
            : <ci, newItem>
            ;
        for (use <- s.references[callee.src], isContainedIn(use, caller.src)) {
            calls += <newItem, use>;
        }
    };

    return calls;
}

function picoCompletionService​

list[CompletionItem] picoCompletionService(Focus focus, int cursorOffset, CompletionTrigger trigger) {
    t = focus[0];
    str prefix = "<t>"[..cursorOffset];
    cc = t.src.begin.column + cursorOffset;
    list[CompletionItem] items = [];

    isTypingId = false;
    try {
        if (prefix != "" && trim(prefix) == prefix) {
            parse(#Id, prefix);
            isTypingId = true;
        }
    } catch ParseError(_): {;}

    top-down-break visit (focus[-1]) {
        case IdType def: {
            name = "<def.id>";
            if (!isTypingId || startsWith(name, prefix)) {
                e = isTypingId && !trigger is character
                    ? completionEdit(t.src.begin.column, cc, t.src.end.column, name)
                    : completionEdit(cc, cc, cc, name);
                items += completionItem(def is function ? function() : variable(), e, name, labelDetail = ": <typeOf(def)>");
            }
        }
    }

    return sort(items, bool(CompletionItem i1, CompletionItem i2) {return i1.label < i2.label; });
}

function main​

The main function registers the Pico language with the IDE


void main() {
    pcfg = getProjectPathConfig(|project://rascal-lsp|, mode=interpreter_external()); // Replace `rascal-lsp` with the name of the DSL project (`artifactId` in pom.xml)
    registerLanguage(
        language(
            pcfg,
            "Pico",
            {"pico", "pico-new"},
            "demo::lang::pico::LanguageServer",
            "picoLanguageServer"
        )
    );
    registerLanguage(
        language(
            pcfg,
            "Pico",
            {"pico", "pico-new"},
            "demo::lang::pico::LanguageServer",
            "picoLanguageServerSlowSummary"
        )
    );
}

Register the Pico language and the contributions that supply the IDE with features.

Register Language is called twice here:

  1. first for fast and cheap contributions
  2. asynchronously for the full monty that loads slower

The errorRecovery parameter can be set to true to enable error recovery in the parser. When enabled, all the contributions in this file will mostly work when parse errors are present in the input because the contributions are written to be robust in the presence of error trees. See LanguageServer for more details.

Benefits​

  • You can run each contribution on an example in the terminal to test it first. Any feedback (errors and exceptions) is faster and more clearly printed in the terminal.