module util::PathConfig

rascal-Not specified

Standard intermediate storage format for configuring language processors (such as interpreters, type-checkers and compilers)

Usage

import util::PathConfig;

Dependencies

import Exception;
import IO;
import Location;
import Message;
import String;
import util::UUID;

Description

The module offers the Path Config datatype which standardizes the configuration of source code projects. Together with Language File Config automatic mappings from source files to target files, and back, can be computed.

The following reusable invertible mapping functions are provided for the sake of convenience and for the sake of consistent interpretation of a Path Config instance. We map fully qualified module names to their corresponding file locations on disk:

qualified module name (str) to file path (loc)	file path (loc) to qualified module name (str)	Path Config field used
Srcs File	Srcs Module	srcs
Bin File	Bin Module	bin
Libs File	Libs Module	libs

data PathConfig

General configuration (via locations) of a language processor.

data PathConfig  
     = pathConfig(
    loc projectRoot        = |unknown:///|,
    list[loc] srcs         = [],  
    list[loc] ignores      = [],  
    loc bin                = |unknown:///|,
    list[loc] resources    = [],
    list[loc] libs         = [],          
    list[Message] messages = []
)
     ;

A PathConfig is the result of dependency resolution and other configuration steps. Typically, IDEs produce the information to fill a PathConfig, such that language tools can consume it transparantly. A PathConfig is also a log of the configuration process. Typically a single Path Config instance configures the language processor for a single source project tree.

• projectRoot is the root directory of the source project tree that is being configured.
• srcs list of root directories to search for source files; to interpret or to compile.
• ignores list of directories and files to not compile or not interpret (these are typically subtracted from the srcs tree, and/or skipped when the compiler arrives there.)
• bin is the target root directory for the output of a compiler. Typically this directory would be linked into a zip or a jar or some other executable archive later.
• libs is a list of binary dependencies (typically jar files or bin folders) on other projects, for checking and linking purposes. Each entry is expected to return true for Is Directory.
• resources is a list of files or folders that will be copied by the compiler to the bin folder, synchronized with its other (binary) output files..
• messages is a list of info, warning and error messages informing end-users about the quality of the configuration process. Typically missing dependencies would be reported here, and clashing versions.

Benefits

• main functions which have a keyword parameter of type Path Config are automatically augmented with commandline parameters for every field of Path Config
• messages can be printed in a standard way using Main Message Handler
• Path Config is a reusable bridge between language processing functions and different execution environments such as VScode, the commandline or Maven.
• Path Config makes all configuration processors of file-based language processors explicit and transparent
• Path Config is programming language and domain-specific language independent
• This module contains bidirectional transformation functions between fully qualified module names and their file locations in source folders and library dependencies.

data LanguageFileConfig

Defines the parameters of mappings between qualified module names and source, target, and library files.

data LanguageFileConfig  
     = fileConfig(
    str packageSep = "::",
    str binExt     = "tpl",
    str targetRoot = "rascal", 
    str targetEsc  = "$",
    str srcsExt    = "rsc"
)
     ;

For most languages a single fileConfig() instance is enough to define:

• the mapping from source files and source folders to fully qualified module names, and back: Srcs Module and Srcs File
• the mapping from binary library files to fully qualified module names and back: Libs Module and Libs File
• the mapping from source files to target files in the bin folder, and back: Bin File and Bin Module

Together with a Path Config instance, the above six functions can be re-used to build a language processor that supports:

• execution (testing) of generated files from the bin folder, using libs as run-time dependencies
• using binary compile-time libraries, using libs to find binary interfaces to previously generated targets
• packaging binary (generated) files as jar files to be re-used later as libs dependencies
• modular language processors that work incrementally per changed source file or changed dependency

Benefits

• one File Config constant can be reused for configure all six different mapping functions.
• a simple fileConfig() constant is configured for the Rascal compiler by default (.tpl files as binary extension).
• the mapping functions that use Language File Config can always use the same Path Config instance for one project.
• more complex mappings can be made by combining the six functions. For example first retrieving the module name using srcsModule and then seeing if it exists also in one of the libraries using libsFile.

Pitfalls

• If a compiler produces multiple target files from a single source file, then you might have to configure different instances of File Config for every target binExt.
• If the mapping between qualified module names and source files or binary files is different ---it has more parameters than defined by Language File Config--- then you have to write your own versions of Srcs Module, Srcs File, Libs Module, Libs File, Bin File and Bin Module.

function latestModuleFile

Produces the latest up-to-date file to load for a given module name, searching in the bin folder, the srcs folder and the libraries.

loc latestModuleFile(str qualifiedModuleName, PathConfig pcfg, LanguageFileConfig fcfg) throws PathNotFound

We find the right file to source for the given moduleName:

1. If the binary target file is younger than the source file, the binary target wins
2. If a binary target is found, without a corresponding source unit, we try the libraries instead because a source module can have been deleted.
3. If a source file is found, without a binary target, this source file is returned.
4. Otherwise we search in the libraries for a binary file and return it.
5. We throw Path Not Found if a module can not be resolved using either the bin, srcs, or libs, and also if the only place we found the module in was a bin folder (signifying a deleted source module).

In other words, Latest Module File prefers newer binaries over older source files, and source files over library modules. If a module is present in both libraries and the current project, then the current project's sources shadow the libraries.

This function is based on the core features of Srcs File, Bin File, and Libs File.

Benefits

• Finicky issues with file IO are dealt with here in a language parametric way, based on Language File Config and Relativize.
• Provides the basic setup for creating a programming language or DSL with independent libraries/components to depend on.
• Use returnValue.extension == fcfg.binExt to detect if you need to read a binary or parse the source code.
• The PathNotFound exception should be caught by the code that processes import statements in your language.

function srcsModule

Compute a fully qualified module name for a module file, relative to the source roots of a project

str srcsModule(loc moduleFile, PathConfig pcfg, LanguageFileConfig fcfg) throws PathNotFound

str srcsModule(loc moduleFile, list[loc] srcs, LanguageFileConfig fcfg) throws PathNotFound

• Srcs Module is the inverse of Srcs File

function libsModule

Compute a fully qualified module name for a library file, relative to the library roots of a project

str libsModule(loc libsFile, PathConfig pcfg, LanguageFileConfig fcfg) throws PathNotFound

str libsModule(loc libsFile, list[loc] libs, LanguageFileConfig fcfg) throws PathNotFound

• Libs Module is the inverse of Libs File

function libsFile

Find out in which library file a module was implemented.

loc libsFile(str qualifiedModuleName, PathConfig pcfg, LanguageFileConfig fcfg) throws PathNotFound

loc libsFile(str qualifiedModuleName, list[loc] libs, LanguageFileConfig fcfg) throws PathNotFound

• Libs File is the inverse of Libs Module
• the computed file has to exist in at least one of the library modules. Otherwise Path Not Found is thrown.

function srcsFile

Find out in which source file a module was implemented.

loc srcsFile(str qualifiedModuleName, PathConfig pcfg, LanguageFileConfig fcfg, bool force = false) throws PathNotFound

loc srcsFile(str qualifiedModuleName, list[loc] srcs, LanguageFileConfig fcfg, bool force = false) throws PathNotFound

• Srcs File is the inverse of Srcs Module
• throws Path Not Found if the designated source file does not exist, unless force == true.
• if force then the first element of the srcs path is used as parent to the new module file.

function binFile

Compute the binary file location for a fully qualified source module name

loc binFile(str srcsModule, PathConfig pcfg, LanguageFileConfig fcfg)

• Bin File is the inverse of Bin Module.
• the returned target location does not have to exist yet.

function binFile

Compute a target file name for a generated folder with a given extension

loc binFile(str srcsModule, loc generated, LanguageFileConfig fcfg)

function binModule

Computing a fully qualified module name back from a file in the bin folder

str binModule(loc binFile, PathConfig pcfg, LanguageFileConfig fcfg) throws PathNotFound

• Bin Module is the inverse of Bin File

function binModule

Recovers the original module name back from a file that was generated.

str binModule(loc targetFile, loc bin, LanguageFileConfig fcfg) throws PathNotFound

Tests

test inverseBinFileModule

test bool inverseBinFileModule() {
    pcfg = pathConfig(
        bin=testLibraryLoc + "target/classes",
        srcs=[|project://rascal/src/org/rascalmpl/library/|]
    );
    fcfg = fileConfig();

    tgt = binFile("util::Monitor", pcfg, fcfg);
    writeFile(tgt, "blabla");

    return binModule(tgt, pcfg, fcfg) == "util::Monitor";
}

test inverseSrcsFileModule

test bool inverseSrcsFileModule() {
    pcfg = pathConfig(
        bin=testLibraryLoc + "target/classes",
        srcs=[testLibraryLoc + "src/main/rascal"]
    );

    writeFile(pcfg.srcs[0] + "util/Monitor.rsc", "module util::Monitor");
    fcfg = fileConfig();

    src = srcsFile("util::Monitor", pcfg, fcfg);

    return srcsModule(src, pcfg, fcfg) == "util::Monitor";
}

test inverseLibsFileModule

test bool inverseLibsFileModule() {
    pcfg = pathConfig(
        bin=testLibraryLoc + "target/classes",
        libs=[testLibraryLoc + "libs"]
    );
    fcfg = fileConfig(binExt="tpl");

    writeFile(testLibraryLoc + "libs" + "rascal/util/$Monitor.tpl", "blabla");
    lib = libsFile("util::Monitor", pcfg, fcfg);

    return libsModule(lib, pcfg, fcfg) == "util::Monitor";
}

test moduleExceptionWithSrc

test bool moduleExceptionWithSrc() {
    pcfg = pathConfig(srcs=[|project://rascal/src/org/rascalmpl/library/|]);
    fcfg = fileConfig();
    return srcsModule(|project://rascal/src/org/rascalmpl/library/Exception.rsc|, pcfg, fcfg) 
        == "Exception";
}

test moduleReflectiveWithSrc

test bool moduleReflectiveWithSrc() {
    pcfg = pathConfig(srcs=[|project://rascal/src/org/rascalmpl/library/|]);
    fcfg = fileConfig();

    return srcsModule(|project://rascal/src/org/rascalmpl/library/util/Reflective.rsc|, pcfg, fcfg) 
        == "util::Reflective";
}

test moduleExceptionOnlyTplModule

test bool moduleExceptionOnlyTplModule() {
    tplFile = testLibraryLoc + "/lib/rascal/$Exception.tpl";
    writeFile(tplFile, "$Exception.tpl (only file matters, content irrelevant)");
    pcfg = pathConfig(libs=[testLibraryLoc + "/lib/"]);
    fcfg = fileConfig();

    return libsModule(tplFile, pcfg, fcfg) == "Exception";
}

test moduleExceptionOnlyTplFile

test bool moduleExceptionOnlyTplFile() {
    tplFile = testLibraryLoc + "/lib/rascal/$Exception.tpl";
    writeFile(tplFile, "$Exception.tpl (only file matters, content irrelevant)");
    pcfg = pathConfig(libs=[testLibraryLoc + "/lib/"]);
    fcfg = fileConfig(binExt="tpl");

    return libsFile("Exception", pcfg, fcfg) == tplFile;
}

test moduleReflectiveOnlyTplModule

test bool moduleReflectiveOnlyTplModule() {
    writeFile(testLibraryLoc + "/libs/rascal/util/$Reflective.tpl",
        "util::Reflective (only file matters, content irrelevant)");
    pcfg = pathConfig(srcs = [],
                    libs=[testLibraryLoc + "libs"]
                     );
    fcfg = fileConfig();

    return libsModule(testLibraryLoc + "libs/rascal/util/$Reflective.tpl", pcfg, fcfg) 
            == "util::Reflective";
}

test moduleReflectiveOnlyTplFile

test bool moduleReflectiveOnlyTplFile() {
    libFile = testLibraryLoc + "/libs/rascal/util/$Reflective.tpl";
    writeFile(libFile, "util::$Reflective.tpl (only file matters, content irrelevant)");
    pcfg = pathConfig(srcs = [],
                    libs=[testLibraryLoc + "libs"]
                     );
    fcfg = fileConfig(binExt="tpl");

    return libsFile("util::Reflective", pcfg, fcfg) == libFile;
}

test longestModuleReflectiveOnlyTpl

test bool longestModuleReflectiveOnlyTpl() {
    writeFile(testLibraryLoc + "/1/libs/rascal/$Reflective.tpl", "$Reflective.tpl at top level (only file matters, content irrelevant)");
    writeFile(testLibraryLoc + "/2/libs/rascal/util/$Reflective.tpl",
        "util::$Reflective.tpl in subdir util (only file matters, content irrelevant)");
    pcfg = pathConfig(srcs= [], 
                      libs=[testLibraryLoc + "1/libs", testLibraryLoc + "2/libs"]
                     );
    fcfg = fileConfig(binExt="tpl");
    return libsFile("util::Reflective", pcfg, fcfg) == testLibraryLoc + "2/libs/rascal/util/$Reflective.tpl";
}

test moduleOnlyInSecondSrc

test bool moduleOnlyInSecondSrc() {
    testLibrarySrc = testLibraryLoc + "src/org/rascalmpl/library/";
    ESrc = testLibrarySrc + "E.rsc";
    writeFile(ESrc,  "module E");

    pcfg = pathConfig(srcs=[|project://rascal/src/org/rascalmpl/library/|, testLibrarySrc]);
    fcfg = fileConfig();
    return srcsModule(ESrc, pcfg, fcfg) == "E";
}