KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #ai-agent #cli #claude-code #open-source #deepseek-harness #dsh-plugin #python #mcp

mlscript

★ 226 Open GitHub ↗

The MLscript programming language. Functional and object-oriented; structurally typed and sound; with powerful type inference. Soon to have full interop with TypeScript!

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MLscript

What would TypeScript look like if was designed with type inference, soundness, and pattern matching in mind?

This original question lead to the creation of the MLscript programming language,
a modern object-oriented and functional programming language for the Web.

Since then, the goals of the language have evolved,
and MLscript has become a more ambitious effort at defining a next-generation high-level programming language,
aiming to maximize reliability, expressiveness, and performance.
We are notably adding new backends, such as WASM, so this is no longer "just" a scripting language.
Hence, the name of the next iteration of the language will also most likely change.

This repository contains a few iterations of the MLscript experimental compiler,
and most notably:

- [MLscript Compiler Version 1](#mlscript-compiler-version-1)
- [MLscript Compiler Version 2 (hkmc2)](#mlscript-compiler-version-2-hkmc2)

The latter is in active development, on the `hkmc2` branch.
The continuous integration (CI) has been set up to only test this version of the compiler.
The `mlscript` branch contains the last commit where the CI was set up to test the old version-1 compiler.

The webpage at https://hkust-taco.github.io/mlscript/ still demonstrates the old version-1 compiler.
An online demo of hkmc2 (already somewhat outdated) can be found at https://mlscript.fun/.

# MLscript Compiler Version 1

This legacy version of the compiler provided basic JavaScript code-gen along with advanced type checking for object-oriented and functional programming with records, generic classes, mix-in traits, first-class unions and intersections, instance matching, and ML-style principal type inference.
These features can be used to implement expressive class hierarchies as well as extensible sums and products.

The approach supports union, intersection, and complement (or negation) connectives, making sure they form a Boolean algebra, and adds enough structure to derive a sound and complete type inference algorithm.

Many extensions were also added over time and splintered off into supporting research paper artifacts, such as more advanced extensible programming support, expressive pattern matching syntax and compilation, and first-class polymorphism (see: [Publications](#publications)).

## Getting Started

### Project Structure

- The `shared/src/main/scala/mlscript` directory contains the sources of the MLscript compiler.

- The `shared/src/test/scala/mlscript` directory contains the sources of the testing infrastructure.

- The `shared/src/test/diff` directory contains the actual tests.

- The `ts2mls/js/src/main/scala/ts2mls` directory contains the sources of the ts2mls module.

- The `ts2mls/js/src/test/scala/ts2mls` directory contains the sources of the ts2mls declaration generation test code.

- The `ts2mls/jvm/src/test/scala/ts2mls` directory contains the sources of the ts2mls diff test code.

- The `ts2mls/js/src/test/typescript` directory contains the TypeScript test code.

- The `ts2mls/js/src/test/diff` directory contains the declarations generated by ts2mls.

- The `ts2mls` experimental sub-project allows one to use TypeScript libraries in MLscript. It can generate libraries' declaration information in MLscript by parsing TypeScript AST, which can be used in MLscript type checking.

### Prerequisites

You need [JDK supported by Scala][supported-jdk-versions], [sbt][sbt], [Node.js][node.js], and TypeScript to compile the project and run the tests.

We recommend you to install JDK and sbt via [coursier][coursier]. The versions of Node.js that passed our tests are from v16.14 to v16.17, v17 and v18. **Run `npm install` in the repository root** to install required npm packages (including TypeScript). **Note that ScalaJS cannot find globally installed TypeScript.** We explicitly support TypeScript v4.7.4.

[supported-jdk-versions]: https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
[sbt]: https://www.scala-sbt.org/
[node.js]: https://nodejs.org/
[coursier]: https://get-coursier.io/

### Running the tests

Running the main MLscript tests only requires the Scala Build Tool installed.
In the terminal, run `sbt mlscriptJVM/test`.

Running the ts2mls MLscript tests requires NodeJS, and TypeScript in addition.
In the terminal, run `sbt ts2mlsTest/test`.

You can also run all tests simultaneously.
In the terminal, run `sbt test`.

### Running tests individually

Individual tests can be run with `-z`.
For example, `~mlscriptJVM/testOnly mlscript.DiffTests -- -z parser` will watch for file changes and continuously run all parser tests (those that have "parser" in their name).

You can also indicate the test you want in `shared/src/test/scala/mlscript/DiffTests.scala`:

```scala
  // Allow overriding which specific tests to run, sometimes easier for development:
  private val focused = Set[Str](
    // Add the test file path here like this:
    "shared/src/test/diff/mlscript/Methods.mls"
  ).map(os.RelPath(_))
```

To run the tests in ts2mls sub-project individually,
you can indicate the test you want in `ts2mls/js/src/test/scala/ts2mls/TSTypeGenerationTests.scala`:

```scala
private val testsData = List(
    // Put all input files in the `Seq`
    // Then indicate the output file's name
    (Seq("Array.ts"), "Array.d.mls")
  )
```

### Running the web demo locally

To run the demo on your computer, compile the project with `sbt fastOptJS`, then open the `local_testing.html` file in your browser.

You can make changes to the type inference code
in `shared/src/main/scala/mlscript`,
have it compile to JavaScript on file change with command
`sbt ~fastOptJS`,
and immediately see the results in your browser by refreshing the page with `F5`.

# MLscript Compiler Version 2 (hkmc2)

This is the second iteration of the MLscript compiler,
nicknamed _hkmc2_ (Hong Kong MLscript Compiler v2).

## Getting Started

### Project Structure

Most of the important code of the new compiler is in the `hkmc2` folder.

### Prerequisites

You need [JDK supported by Scala][supported-jdk-versions], [sbt][sbt], [Node.js][node.js], and TypeScript to compile the project and run the tests.

We recommend you to install JDK and sbt via [coursier][coursier]. The versions of Node.js that passed our tests are from v16.14 to v16.17, v17 and v18.

**Run `npm install` in the repository root** to install the required npm packages.
This installs TypeScript (used by the JS backend tests) as well as [Binaryen][binaryen] (used by the WASM backend tests in `hkmc2/shared/src/test/mlscript/wasm/`).
If you skip this step, **WASM tests will fail** at runtime because the `binaryen` module cannot be found by Node.js.

**Note that ScalaJS cannot find globally installed TypeScript.** We explicitly support TypeScript v4.7.4.

[supported-jdk-versions]: https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
[sbt]: https://www.scala-sbt.org/
[node.js]: https://nodejs.org/
[coursier]: https://get-coursier.io/
[binaryen]: https://github.com/WebAssembly/binaryen

Some tests in the `compiler` subproject generate and compile C++ while making use of some libraries.
You can run these by installing `nix` (for MacOS, we recommend https://determinate.systems/posts/graphical-nix-installer/)
and running `nix develop` before launching SBT.
If you don't want to use nix, you can install the dependencies manually as follows, but this has not been tested on non-MacOS systems:
```bash
brew install mimalloc boost gmp
```

### Running the tests

Running the tests requires the Scala Build Tool (SBT) installed.

We recommend running all tests in the SBT shell,
i.e., do not restart SBT every time,
but launch it in shell mode (with command `sbt`)
and then use one of the following commands.

#### Main SBT test commands (use these for validating changes before pushing)

- `hkmc2JVM/test` for running only the main compilation tests, in `hkmc2/shared/src/test/mlscript-compile`.
- `hkmc2DiffTests/test` for running only the main diff-tests, in `hkmc2/shared/src/test/mlscript`.
- `hkmc2MainTests/test` for running the above two.
- `hkmc2AppsTests/test` for running the applications compile and diff tests.
- `hkmc2NofibTests/test` for running the nofib compile and diff tests.
- `hkmc2WasmTests/test` for running the wasm compile and diff tests.
- `hkmc2MostTests/test` for running all of the above.
- `hkmc2AllTests/test` for running all default hkmc2 tests, including ScalaJS-compiled tests.
- `~hkmc2DiffTests/Test/run` for running the test watcher,
  which updates test files as you save them and recompiles the Scala sources automatically on change.
- `test` for compiling all JVM and JS subprojects
  and running every single test in the repository.

#### Useful SBT commands aliases (use these for more pleasant development)

The most useful command alias for development is `watch` for continuously running the test watcher, which:

- watches for any file-save event in compilation and diff- tests, updating them accordingly
- recompiles the Scala sources automatically on change

The command aliases below let you either specify no arguments,
in which case all corresponding tests are run,
or specify a string to filter the tests to run.

- `ctest` aka "main compilation tests" for running specific tests in `hkmc2/shared/src/test/mlscript-compile`
- `dtest` aka "main diff-tests" for running only the main diff-tests in `hkmc2/shared/src/test/mlscript`
- You can find other similar command aliases in `build.sbt`.

The command aliases below are shorthand for running aggregated tests; they take no arguments.

- `qtest` aka "quick tests" for `ctest` and `dtest` together.
- `mtest` aka "main tests" for runnign most tests, excluding JS compilation + diff- tests (usually sufficient)
- `atest` aka "all tests" for running all tests, including JS compilation + diff- tests

Example usages:

- `dtest opt/` for running the optimization diff-tests in `mlscript/opt/`.
- `catest pars` to run all application compilation tests with "pars" in their name, such as `mlscript-compile/apps/parsing/Lexer.mls`.

Another useful SBT incantation is `; qtest; watch`.
This command runs all quick tests tests once and then starts the test watcher.
This is a useful command to use periodically while making changes to the compiler,
to check that you haven't broken anything.

Note that when saved, the special file `ChangedTests.cmd` will trigger the test watcher to run
all tests that currently have unstaged changes in git.
This is useful when you have a working subset of tests that you want to run periodically.

# Publications

The following research publications notably used various iterations of the MLscript compiler for their implementations.
Links to the individual paper artifact repositories are provided at the corresponding paper webpages.

* [**OOPSLA '22**] [MLstruct: Principal Type Inference in a Boolean](https://cse.hkust.edu.hk/~parreaux/publication/oopsla22a/)
  used v1 with the old ML-like syntax.
* [**ECOOP '23**] [super-Charging Object-Oriented Programming Through Precise Typing of Open Recursion](https://cse.hkust.edu.hk/~parreaux/publication/ecoop23/) (🏆 Distinguished Artifact!)
  used v1 with the current Scala-like syntax and a new object system.
* [**OOPSLA '23**] [Getting into the Flow: Towards Better Type Error Messages for Constraint-Based Type Inference](https://cse.hkust.edu.hk/~parreaux/publication/oopsla23/)
  used v1 with an OCaml-syntax parser and heavily modified type checker.
* [**POPL '24**] [When Subtyping Constraints Liberate: A Novel Approach to Type Inference for First-Class Polymorphism](https://cse.hkust.edu.hk/~parreaux/publication/popl24/)
  used v1 with the old ML-like syntax.
* [**OOPSLA '24**] [The Ultimate Conditional Syntax](https://cse.hkust.edu.hk/~parreaux/publication/oopsla24b/) (🏆 Distinguished Paper!)
  used v1 with the current Scala-like syntax;
  the parser had many problems – the reimplementation of the Ultimate Conditional Syntax in hkmc2 is of much higher quality.
* [**GPCE '24**] [Seamless Scope-Safe Metaprogramming Through Polymorphic Subtype Inference](https://cse.hkust.edu.hk/~parreaux/publication/gpce24/)
  used v1 with the current Scala-like syntax.
* [**OOPSLA '25**] [A Lightweight Type-and-Effect System for Invalidation Safety](https://cse.hkust.edu.hk/~parreaux/publication/oopsla25/)
  used hkmc2.

mlscript

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README