KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #ai-agent #cli #claude-code #open-source #deepseek-harness #dsh-plugin #python #mcp

c-compiler

★ 50 Open GitHub ↗

C compiler in one JS file

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# c-compiler

C-to-WebAssembly compiler in one file.

I occasionally post videos about this project at [youtube.com/@groundupcoder](https://www.youtube.com/@groundupcoder) — design decisions, problems I've hit, and parts of the implementation I find interesting. This repo isn't structured as a tutorial; it's the actual code I'm writing as I go.

**compiler.js** — JavaScript, runs on Node.js and browser.

A frozen C++20 port (**compiler.cc**) is preserved in `old/` along with its own test runner and unit tests. The two compilers produce identical output for all unit tests (verified by equiv tests in `old/`).

### Node.js version

Requires **Node.js 24** with `--experimental-wasm-jspi`, or **Node.js 25+** (no flag needed). Older versions can still produce valid `.wasm` bytes (codegen is version-independent), but can't run them locally because their V8 lacks WASM GC and/or `WebAssembly.Suspending`. See [docs/NODE_VERSIONS.md](docs/NODE_VERSIONS.md) for the full compatibility matrix.

There are three ways to run the compiled programs:

### WASM files (`.wasm`) — run with Node.js

Compile to a `.wasm` file and run it with **host.js**, the WASM runtime that provides libc, filesystem, and terminal support:

```bash
node compiler.js hello.c -o hello.wasm
node host.js hello.wasm
```

### JS files (`.js`) — run directly with Node.js

Compile to a self-contained `.js` file with the WASM binary and runtime embedded. The result is a standalone Node.js script (with a `#!/usr/bin/env node` shebang) — no separate `host.js` needed:

```bash
node compiler.js hello.c -o hello.js
node hello.js
```

### HTML files (`.html`) — run in a browser

Compile to a self-contained `.html` file with everything embedded (WASM binary, runtime, xterm.js terminal, data files). Runs in any modern browser with support for graphics (SDL/canvas), audio (SharedArrayBuffer), and interactive terminal programs:

```bash
node compiler.js hello.c -o hello.html
```

## Project files

Both compilers accept JSON project files as positional arguments. Each vendor project uses `lib.json` (for libraries) or `bin.json` (for executables). A project file expands inline as if its `compilerArgs`, `sources`, and `dataFiles` were passed directly at that position:

```bash
# These are equivalent:
node compiler.js vendor/doom/bin.json -o doom.html
node compiler.js -Ivendor/doom/Nuked-OPL3 vendor/doom/src/*.c ... --opfs-file vendor/doom/data/doom1.wad:/doom1.wad -o doom.html

# Project files mix freely with explicit args:
node compiler.js -DFOO vendor/lua/bin.json extra.c -o out.wasm
```

### Project file format

```json
{
  "type": "lib",
  "name": "mylib",
  "description": "Optional description",
  "includes": ["src"],
  "compilerArgs": ["-DNDEBUG"],
  "sources": ["src/util.c", "src/core.c"]
}
```

Libraries (`"type": "lib"`) cannot be compiled directly — they must be referenced via `deps` from a binary project. Binary projects omit `type` or set it to `"bin"`. All paths are resolved relative to the JSON file's directory. `dataFiles` maps local files to virtual filesystem paths (used for HTML output via OPFS).

## Compiler flags

| Flag | Description |
|------|-------------|
| `-o <file>` | Output file (`.wasm`, `.js`, or `.html`) |
| `-D<name>[=val]` | Define preprocessor macro |
| `-I<path>` | Add include search path |
| `-a <action>` | Stop at stage: `lex`, `parse`, `link`, `compile` |
| `--opfs-file <src:dest>` | Embed data file in HTML output (accessible via `fopen`) |
| `--run-arg <arg>` | Pass argument to program's `argv` |
| `--gc-sections` | Remove unused code sections |
| `--no-undefined` | Error on undefined symbols |
| `--no-wasm-validate` | Skip the codegen-time `WebAssembly.validate()` backstop. Lets older Node versions emit `.wasm` for cross-compile. Implies `--no-version-check`. |
| `--no-version-check` | Suppress the Node.js runtime-support warning at startup. |
| `--no-irreducible-lowering` | Disable the loop-switch fallback for functions with cross-block gotos (the structured codegen will then surface a "target label not in scope" diagnostic). On by default. |
| `-v` / `--verbose` | Print warnings to stderr — currently lists which functions required loop-switch lowering. |
| `--require-source <file>` | Require a source file to be present |
| `--allow-old-c` | Enable all legacy C compatibility flags |
| `--allow-implicit-int` | Allow implicit int in declarations |
| `--allow-empty-params` | Allow empty parameter lists |
| `--allow-knr-definitions` | Allow K&R-style function definitions |
| `--allow-implicit-function-decl` | Allow implicit function declarations |
| `--allow-undefined` | Allow undefined symbols |
| `--no-xterm` | Disable xterm.js terminal in HTML output |
| `--time-report` | Print compilation timing breakdown |
| `-W<name>` | Enable warning (`pointer-decay`, `circular-dependency`) |

## Standard library support

The compiler provides a built-in standard library with headers including:

- **Core**: `stdio.h`, `stdlib.h`, `string.h`, `math.h`, `stdint.h`, `stdbool.h`, `stdarg.h`, `stddef.h`, `ctype.h`, `assert.h`, `errno.h`, `limits.h`, `float.h`
- **Memory/strings**: `malloc`, `free`, `realloc`, `memcpy`, `memset`, `strlen`, `strcmp`, `sprintf`, `snprintf`, `printf`, `fprintf`
- **Files**: `fopen`, `fclose`, `fread`, `fwrite`, `fseek`, `ftell`
- **Time**: `time.h`, `clock()`, `time()`, `usleep()`, `nanosleep()`
- **Terminal**: `termios.h` (`tcgetattr`, `tcsetattr`, `cfmakeraw`), `sys/ioctl.h` (`ioctl`, `TIOCGWINSZ`)
- **I/O multiplexing**: `sys/select.h` (`select`, `FD_SET`, `FD_CLR`, `FD_ISSET`, `FD_ZERO`)
- **Graphics/Audio**: SDL2 subset (video, events, audio)

Terminal and timing primitives use JSPI (WebAssembly JavaScript Promise Integration) for async operations on both Node.js and browser backends.

## WebAssembly GC types

The compiler supports Wasm GC heap-allocated structs and arrays managed by the engine's garbage collector. These live on the GC heap (not linear memory) and can be passed to/from JavaScript without serialization.

### Preferred syntax

A GC struct **value** is a reference to a heap-allocated object. To match C's pointer idioms (and stay friendly to clang IDE tooling), **always spell GC struct refs with `*`** and access fields with `->`:

```c
__struct Point { int x; int y; };

__struct Point *p = __new(__struct Point, 3, 7);
p->x = 99;
printf("%d\n", p->x);
```

For GC arrays, **never** add `*` — arrays don't have a "pointer to" idiom in C, and the compiler rejects `__array(T) *`:

```c
__array(int) arr = __array_new(int, 5);    // OK
arr[0] = 42;
__array_len(arr);
```

The `*` convention applies to type-arg intrinsics like `__ref_test` / `__ref_test_null`, `__ref_cast` / `__ref_cast_null`, `__ref_null`. (Exception: `__extends(__struct Foo)` stays bare because it names a parent class, never an array.) `__new` takes `__struct Foo` (no `*`, since the result is always a struct ref). The array allocation intrinsics (`__array_new`, `__array_of`) take the bare element type directly:

| Allocation | Always write |
|---|---|
| Struct | `__new(__struct Foo, args...)` |
| Array (default-init) | `__array_new(T, n)` |
| Array (filled) | `__array_new(T, n, val)` |
| Array (literal values) | `__array_of(T, v1, v2, ...)` |

The bare form (`__struct Foo` and `.`) also works — both spellings produce the same WASM type — but the `*`/`->` form is the documented preferred style.

### `__struct`

```c
__struct Node {
    int v;
    __struct Node *next;       // recursive ref — '*' form
    __array(int) children;     // array — no '*'
};

__struct Animal { int id; };
__struct Dog {
    __extends(__struct Animal);
    int id;       // must repeat parent's fields, in order, same names + types
    int paws;
};
```

Single inheritance via `__extends`. All `__struct` types are emitted as open for subtyping. Structurally identical types unify across translation units; mutually-recursive types share a rec group.

### `__array(T)`

GC-managed arrays with a fixed element type and runtime length:

```c
__array(int) scores = __array_new(int, 100);  // 100 zero-initialized
scores[0] = 42;
int len = __array_len(scores);

__array(int) vals = __array_of(int, 1, 2, 3, 4, 5);  // literal element list
__array(int) ones = __array_new(int, 5, 1);           // 5 elements, all = 1
```

When the element type is a GC struct, spell *that* with `*` too:

```c
__array(__struct Point *) pts = __array_of(__struct Point *,
    __new(__struct Point, 1, 2),
    __new(__struct Point, 3, 4));
pts[0]->x;
```

Bulk operations: `__array_fill(arr, off, val, n)` and `__array_copy(dst, dstOff, src, srcOff, n)`.

### Reference intrinsics

| Intrinsic | Description |
|-----------|-------------|
| `__ref_is_null(ref)` | Null check (`ref.is_null`) |
| `__ref_eq(a, b)` | Reference identity (`ref.eq`) |
| `__ref_null(__struct Foo *)` | Typed null reference |
| `__ref_test(__struct Foo *, ref)` | Type test, false on null (`ref.test`) |
| `__ref_test_null(__struct Foo *, ref)` | Type-lattice test, true on null (`ref.test null`) |
| `__ref_cast(__struct Foo *, ref)` | Downcast, traps on null (`ref.cast`) |
| `__ref_cast_null(__struct Foo *, ref)` | Downcast, null passes through (`ref.cast null`) |

### Boolean / null sugar

Refs can be used in boolean / null contexts as sugar for the explicit intrinsics:

| Sugar | Equivalent |
|---|---|
| `if (ref)`, `while (ref)`, `ref ? a : b` | `if (!__ref_is_null(ref))` etc |
| `!ref` | `__ref_is_null(ref)` |
| `ref == 0`, `ref == NULL` | `__ref_is_null(ref)` |
| `ref1 == ref2` | `__ref_eq(ref1, ref2)` (identity) |
| `__struct Foo *p = NULL;` | `= __ref_null(__struct Foo *)` |

Both forms are valid; pick per IDE-friendliness vs source readability.

### Universal `__eqref` + `__cast`

`__eqref` is the GC-universe supertype of all reference types (struct, array, boxed primitives) — analogous to `void *` for the GC heap. `__cast(TargetType, expr)` is the universal conversion intrinsic that dispatches based on the source/target type combo.

**Implicit conversions to `__eqref` work everywhere**, just like `int *p = NULL;` or `void *q = some_ptr;`:

```c
void describe(__eqref x);

describe(42);                          // implicit box int → __eqref
describe(3.14);                        // implicit box double → __eqref
describe(some_struct);                 // implicit upcast (no opcode)

__eqref store = 100;                   // implicit box on init
__eqref r = some_function();           // implicit box if return type mismatches
```

The 0/NULL convention is preserved — `__eqref e = 0;` and `__eqref e;` both produce null (not boxed-zero). Use `__cast(__eqref, 0)` if you specifically want a boxed-zero distinct from null.

For unboxing (and for explicit conversions), use `__cast`:

```c
int v = __cast(int, store);                                // unbox
__struct Point *p2 = __cast(__struct Point *, store);      // downcast (ref.cast)

// Discriminated union
if (__ref_test(__struct Point *, store)) { ... }
else if (__ref_test(__struct Color *, store)) { ... }
```

Supported `__cast(target, source)` combos: prim ↔ prim (numeric), prim ↔ `__eqref` (box/unbox), GC ref ↔ `__eqref` (subtype upcast / ref.cast downcast), GC ref → GC ref (downcast/sidecast), GC ref ↔ externref (extern bridges).

`==` works directly between two `__eqref` values (identity comparison via `ref.eq`).

### Extern bridge

GC refs cross the JS/Wasm boundary via `__eqref` + the extern conversions:

| Intrinsic | Description |
|---|---|
| `__ref_as_extern(ref)` | GC ref → externref (`extern.convert_any`) |
| `__ref_as_eq(ext)` | externref → eqref (`any.convert_extern` + `ref.cast eq`; traps if not eq-compatible) |

### Auto + GC

C23 `auto` pairs naturally with GC types — the type spelling stays on the right of the `=`:

```c
auto p = __new(__struct Point, 7, 11);
auto arr = __array_of(int, 1, 2, 3);
for (auto cur = head; cur; cur = cur->next) printf("%d\n", cur->v);
```

### Constraints

- No `&` on GC refs (no address)
- No embedding in C structs/unions
- No `sizeof` on ref types
- No casts to/from integers (use `__ref_*` intrinsics)
- `__array(T) *` is rejected — arrays don't take the `*` sugar
- `__new` takes `__struct Foo` (no `*`): `__new(__struct Foo, ...)`. Typedefs of GC structs work too. `__struct_new` is an alias.

For the full GC design doc see [todos/WASM_GC.md](todos/WASM_GC.md).

## The OS (os/) — gucOS

The repo's north star ([todos/OS.md](todos/OS.md)) is **gucOS** (groundupcoder OS): a wasm-native, almost-POSIX OS in a browser tab — every binary a real wasm module from this compiler. It boots:

```bash
node serve.js .            # then open http://localhost:8080/os/os.html
# → a real shell (busybox hush) + userland (28 busybox applets: ls, grep,
#   sed, … and vi as /bin/vi) over a persistent filesystem (BlockFS on
#   OPFS): pipelines, $( ), redirections, here-docs, job control — and
#   `cc hello.c && ./a.out` compiles and runs IN the OS.

echo 'ls / | cat' | node os/boot.js   # the same OS headless, tty on stdio
```

Under it sits **kernel.js**, the owner-side process control plane ([todos/KERNEL.md](todos/KERNEL.md)): process table, posix_spawn (deliberately **not** fork — see OS.md for the decision), async signal delivery with EINTR, a kernel-side tty line discipline (Ctrl-C means SIGINT), kernel-owned fd tables over one shared BlockFS, pipes with real cross-process blocking + SIGPIPE, and job control (stop/continue, WUNTRACED/WCONTINUED, SIGTTIN). First boot compiles the OS's own userland — including the shell itself — from sources listed in `os/image.json`; there is no build step. `/bin/sh` is **busybox hush**, ported to the no-fork world through a vfork-on-`__spawn` journaling shim ([vendor/busybox/README.md](vendor/busybox/README.md)) — `popen()`/`system()` work, and the port is the kernel design's acceptance test ([todos/KERNEL.md](todos/KERNEL.md)).

## Vendored projects

The compiler is tested against real-world C projects:

- **QuickJS 2025-09-13** — Fabrice Bellard's small JavaScript engine. The full engine + libc + REPL entry point (`qjs.c`) compiles to a 737 KB wasm and runs JavaScript end-to-end. Including the self-host loop: **`compiler.js` runs inside that QuickJS wasm and produces bit-identical wasm output compared to a native build.** See `vendor/quickjs/README.md`, the [Self-hosting](#self-hosting-bootstrap) section below, and [`demos/self-host/`](demos/self-host/).
- **Lua 5.5.0** — Full interpreter, compiles and passes the official test suite
- **DOOM** — doomgeneric port with Nuked-OPL3 music synthesis, runs in the browser
- **Snake** — Terminal-based snake game using termios raw mode, ANSI escape codes, and `select()` for input handling
- **TinyEMU** — Fabrice Bellard's RISC-V emulator. The headless 32-bit build boots Linux 4.15 to a BusyBox shell with interactive stdin. The guest userland is minimal (no compiler); x86 isn't supported because TinyEMU's open source ships only a KVM frontend for x86, not a software interpreter. See `vendor/tinyemu/README.md`

### Building vendored projects

```bash
# Lua interpreter
node compiler.js vendor/lua/bin.json -o lua.wasm
node host.js lua.wasm

# DOOM (HTML with embedded WAD)
node compiler.js vendor/doom/bin.json -o doom.html

# FreeType text rendering demo
node compiler.js vendor/freetype/demo/bin.json -o freetype-demo.js
node freetype-demo.js

# Snake (terminal game)
node compiler.js vendor/snake/main.c -o snake.html
node compiler.js vendor/snake/main.c -o snake.wasm && node host.js snake.wasm

# TinyEMU — boots Linux RISC-V to a BusyBox shell
node compiler.js vendor/tinyemu/bin.json -o /tmp/tinyemu.js
node /tmp/tinyemu.js          # run in a real terminal for interactive stdin
```

### Serving HTML output

```bash
node serve.js [dir] [port]   # defaults: build/, 8080
```

`serve.js` adds COOP/COEP headers needed for `SharedArrayBuffer` (used by audio). Any HTTP server works for rendering, but without those headers DOOM runs silently.

## Self-hosting bootstrap

QuickJS is the path to running `compiler.js` on its own output. The pipeline today:

1. **Stage 1 — C to wasm.** Our compiler builds QuickJS (~64 KLOC of C) into a 737 KB wasm.
2. **Stage 2 — JS engine alive.** That wasm runs JavaScript: `qjs -e '1+1'` → `2`. Recursion, classes, regex, JSON, all working.
3. **Stage 3 — Load self.** `compiler.js` itself (~870 KB of JS) loads and executes inside QuickJS-on-our-wasm, exposing `globalThis.CompilerJS`.
4. **Stage 4 — Compile from inside.** `compiler.js` running inside QuickJS-on-wasm reads C source via `std.loadFile`, drives the full lex → parse → link → codegen pipeline, and writes a wasm file via `std.open(..., "wb").write(...)`. **The output is bit-identical to a native build of the same C source** (verified with `cmp`). See [`demos/self-host/`](demos/self-host/).

```bash
scripts/quickjs-bootstrap.sh
# → builds qjs.wasm, runs smoke test, loads compiler.js inside qjs.wasm

node host.js /tmp/qjs.wasm --std demos/self-host/selfhost.js
# → compiles demos/self-host/hello.c inside qjs.wasm
node host.js /tmp/selfhost-demo/hello-rebuilt.wasm
# → "Hello from a wasm built INSIDE wasm!" (exit 42)
```

Equivalent stage-1–3 manual commands:

```bash
node compiler.js -o /tmp/qjs.wasm vendor/quickjs/bin.json
node host.js /tmp/qjs.wasm -e 'console.log(1+1)'        # → 2
node host.js /tmp/qjs.wasm --std -e \
  "std.evalScript(std.loadFile('compiler.js')); console.log('OK');"
# → OK
```

### Patches to upstream

QuickJS sources in `vendor/quickjs/` are upstream verbatim with three small patches:
- `quickjs.c` and `libregexp.c` — added `#include <alloca.h>` (QuickJS calls `alloca()` directly relying on glibc's transitive include via `<stdlib.h>`).
- `quickjs-libc.c` — commented out `#define USE_WORKER` (the `os.Worker` API needs real OS threads).

Plus one new compiler flag introduced along the way: `--allow-zero-length-arrays` (for the GCC legacy `arr[0]` extension QuickJS leans on for `JSString`'s type-aliased trailing buffer).

### What's left

- Build `qjsc.c` (QuickJS's AOT compiler) on our wasm and feed `repl.js` through it → real interactive REPL.
- Compile QuickJS *itself* through stage 4 (~64 KLOC of C inside the wasm sandbox) — heavier memory/time test of the same pipeline.

## Tests

`node tests/run.js` is the unified entry point over the whole estate — it
dispatches to the individual runners and, given a diff, runs exactly the
suites that diff needs:

```bash
node tests/run.js all                    # The entire estate, one summary
node tests/run.js unit kernel            # Named suites (--list to see them)
node tests/run.js --diff                 # Suites the working-tree diff needs
node tests/run.js --diff main --dry-run  # Print the plan vs `main`, run nothing
node tests/run.js --list                 # Suites + the path→suite rule table
```

The individual runners stay independently invocable:

```bash
python3 tests/run.py                                  # Unit tests (default)
python3 tests/run.py --all                             # Everything
python3 tests/run.py --types=unit,extra                # Multiple categories
python3 tests/run.py --types=lua                       # Lua test suite
python3 tests/run.py --filter=struct                   # Filter by name
```

Test categories:
- **unit** — Core C language features and standard library (compile + run, check stdout)
- **extra** — Additional compile + run tests
- **lua** — Compile the Lua VM and run the official Lua test suite

The frozen C++ compiler snapshot in `old/` has its own test runner with equiv tests:

```bash
python3 old/tests/run.py                              # Unit tests, JS compiler
python3 old/tests/run.py --all                         # Unit (both compilers) + equiv + sourcemap
python3 old/tests/run.py --types=equiv --compiler=all  # JS vs C++ equivalence
```

## License

Apache License, Version 2.0 — see [LICENSE](LICENSE).

## Contributing

Bug reports as issues are very welcome. For pull requests, please open an issue first to discuss. See [CONTRIBUTING.md](CONTRIBUTING.md) for details.

c-compiler

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README