c-web-modules - GitLite

KO | EN

gitlite — search

#javascript #python #hacktoberfest #go #android #machine-learning #linux #docker #nodejs #awesome #kubernetes #mcp #openai #open-source

c-web-modules

(★ 209)

A modular web framework for C, inspired by kernel modules. Dynamically upload and compile C code at runtime to build REST APIs, WebSocket services, and more. Includes support for JSON (Jansson), SQLite, OpenSSL, and streaming large data efficiently.

File Explorer

Download Latest Version (.zip)

- - build.yml
- index.html
- chat.c
- counter.c
- cweb
- deploy.bat
- job_sum.c
- json.c
- readme.md
- routes.ini
- static.c
- websocket.c
- active_conn.h
- container.h
- crypto.h
- cweb.h
- cweb_config.h
- db.h
- defer.h
- engine.h
- http.h
- jobs.h
- libevent.h
- list.h
- map.h
- pool.h
- queue.h
- router.h
- scheduler.h
- server.h
- shutdown.h
- uuid.h
- ws.h
- .keep
- http.c
- libevent.c
- module.c
- .keep
- active_conn.c
- config.c
- container.c
- crypto.c
- db.c
- engine.c
- jobs.c
- list.c
- manage.c
- map.c
- modules.c
- net.c
- persist.c
- pool.c
- queue.c
- router.c
- routes.c
- scheduler.c
- server.c
- shutdown.c
- uuid.c
- worker.c
- ws.c
- .keep
- hello.txt
- index.html
- test.h
- test_db.c
- test_helpers.c
- test_jobs.c
- test_main.c
- test_scheduler.c
- test_stubs.c
.gitignore
CODE_OF_CONDUCT.md
docker-compose.yml
Dockerfile
LICENSE
Makefile
production.sh
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# c-web-modules: Modules for the Web

> **Note:**  
> This project is currently a **proof of concept** and represents the **bare minimum viable product (MVP)**.  
> It is not designed for production use, and there may be bugs, limitations, or incomplete features.  
> Use at your own discretion.

> **Alert:** Rework in progress. The “web framework” direction isn’t a good fit for C because HTML/templates add too much complexity. I’m refocusing this on compute-first modules where C is a better fit.

Welcome to **c-web-modules**, a C plugin host for CPU bound web workloads. Inspired by kernel modules and AWS Lambda, this project lets you upload C code directly to the server, compile it at runtime, and hot-swap the logic without restarting the host.

## Documentation

For more detailed information, please refer to the [official documentation](https://joexbayer.github.io/c-web-modules/).

## What it's for

c-web-modules is focused on **compute + streaming**, not traditional frontend work. It is designed for workloads where you want C-level performance and low-latency streaming.

### Core model: HTTP endpoints, WebSockets, Jobs

- HTTP endpoints: synchronous request/response routes defined by modules.
- WebSockets: long-lived connections for streaming, interactive updates, and push.
- Jobs: asynchronous work units triggered via server-owned `/jobs` APIs.
- Jobs stream events over `/jobs/ws` and expose durable status via `/jobs/:uuid`.
- Hot-swap: routes, websockets, and jobs can be updated without restarting the server.

### Good use cases

- CPU-bound transforms: image/video frames, audio DSP, compression, hashing.
- Streamed analytics: log parsing, metrics aggregation, JSON/CSV crunching.
- Real-time pipelines: ingest -> transform -> stream results.

---

# Getting started

## TLS

The production environment uses TLS for encrypting, by default it expects there to be a server.crt and server.key which can be generated by using this command:

```bash
openssl req -x509 -newkey rsa:2048 -keyout server.key -out server.crt -days 365 -nodes
```

## Examples (by core part)

### HTTP endpoints
Modules register routes with method + path. Paths must be unique across modules and can use regex.

### WebSockets
Modules register WebSocket handlers and keep connections alive across hot-swaps.

### Jobs
Modules register jobs and the server triggers them via `/jobs/*`.

## Examples

[Counter](example/counter.c)

[WebSocket Echo](example/websocket.c)

[Chat](example/chat.c)

[Static File Server](example/static.c)

[JSON](example/json.c)

[Job Sum](example/job_sum.c)

---

## Why Use c-web-modules?

1. **Compute-first runtime**: Designed for CPU-bound work and streaming outputs.  
2. **Hot-swappable modules**: Deploy new code without restarting the server.  
3. **Stable job semantics**: Running jobs stick to the old module; new jobs use the updated module.  
4. **Traceable results**: Job status includes `module_hash` for provenance.  
5. **WebSocket streaming**: Keep long-lived result streams alive through module updates.  
6. **Built-In Features**: Includes a cross-module cache and scheduler for deferred tasks.  
7. **Regex in Paths**: Define routes using regular expressions for more flexible and powerful URL matching.

### Deployment

Jobs deploy the same way as HTTP/WS modules: upload a C module to `/mgnt`. A single module can define routes, WebSockets, and jobs at the same time.

### Reserved paths

`/jobs/*` is a protected namespace owned by the server. Modules cannot register routes under `/jobs/`.

### Endpoints

- `POST /jobs` - Trigger a job with `{ "module": "mod", "job": "name", "payload": { ... } }`
- `GET /jobs/:uuid` - Job status, including `module_hash`
- `POST /jobs/:uuid/cancel` - Cancel a job
- `GET /jobs?state=running` - List jobs by state

### WebSocket

- `WS /jobs/ws?id=:uuid&since_event_id=456`
- Streams `job_events` and supports replay from `since_event_id`.

### Authentication

- Set `CWEB_ADMIN_KEY` to require an admin key for `/mgnt`, `/jobs`, and `/jobs/ws`.
- Send `X-API-Key: <key>` or `Authorization: Bearer <key>`.
- In DEV builds, requests from `127.0.0.1` are allowed without a key.

### What is implemented

- Server-owned `/jobs` HTTP API with UUIDs, status, cancel, and list.
- SQLite schema for durable job state + event log with replay.
- Jobs defined inside modules (`module_t.jobs[]`) and triggered by name.
- Module hash computed from uploaded `.so` for provenance in job status.
- Hot-swap semantics: running jobs keep the old module; new jobs use the updated module.
- WebSocket job event stream at `/jobs/ws` with `id` and `since_event_id` query params.

### SQLite schema

```
jobs(
  id INTEGER PRIMARY KEY,
  module_name TEXT,
  job_name TEXT,
  state TEXT,
  module_hash TEXT,
  created_at INTEGER,
  updated_at INTEGER,
  payload_json TEXT,
  result_json TEXT,
  error_text TEXT
)

job_events(
  id INTEGER PRIMARY KEY,
  job_id INTEGER,
  ts INTEGER,
  type TEXT,
  data_json TEXT
)
```

Currently supported external libraries:  
- **OpenSSL**: Currently only for hashing, but later for secure communication.  
- **SQLite3**: Shared by all modules for lightweight database needs.  
- **Jansson**: For easy JSON parsing and manipulation.

---

# Deployment

Deploying code to the server is simple and can be done in multiple ways, depending on your workflow.

### 1. Basic Deployment with `curl`

At its core, deploying code to the server involves sending a POST request with the C file attached. Here’s an example using `curl`:

`curl -X POST -F "code=@path/to/yourcode.c" http://localhost:8080/mgnt`

### 2. Using the cweb script and .ini config
The script handles:  
- Sending the file to the server using `curl`.  
- Parsing responses for success or failure.  
- Providing helpful logs and error messages.

#### Deploying Multiple Files with a Config File  
`./cweb deploy path/to/yourcode.c`

You can deploy multiple modules in one go using a configuration file. By default, the script looks for a file named `routes.ini`.

Example `routes.ini` file:  
```ini
server_url=http://localhost:8080/mgnt

[modules]
example1.c
example2.c
```

When using the .ini files you run: `./cweb deploy`

## Windows

For Windows there currently only is a very primitve deploy.bat script:

```bash
.\deploy.bat file.c
```

The server needs to be specified inside the .bat file, default is: http://localhost:8080/mgnt

### Errors

Error messages are forwarded back to you over http.

---

# Build it yourself!

> **Note:**  
>  MacOS support is not guaranteed!

The project depends on:

```bash
# Debian
sudo apt-get install libssl-dev
sudo apt-get install libsqlite3-dev
sudo apt-get install libjansson-dev

# Arch
sudo pacman -S openssl
sudo pacman -S sqlite
sudo pacman -S jansson

# Bug with archlinux and fanitizser, ref: https://github.com/joexbayer/c-web-modules/issues/12
sudo sysctl vm.mmap_rnd_bits=30

# MacOS
brew install openssl@3
brew install sqlite
brew install jansson
```

Run make to compile and make run to start the server.

```bash
make
make run
```

### Server flags

- `--module-dir <path>` (or `-m <path>`): Store compiled modules and `routes.dat` in a specific directory. Defaults to `modules`.
- `--purge-modules` (or `-P`): Delete compiled `.so` modules on shutdown. Off by default.

## Docker

```bash
docker-compose up --build
```

# How is c-web-modules different from Apache Modules and ISAPI Extensions?

Unlike Apache modules and ISAPI extensions, which are tightly integrated into the server and require configuration changes followed by a server restart or reload, **c-web-modules** offers runtime flexibility.

**Key Differences:**
- **Dynamic Deployment**:  
  c-web-modules allows you to upload raw C code directly to the server, which compiles and integrates it into the running application without restarts. Neither Apache modules nor ISAPI extensions support this level of runtime modification.

- **Module Isolation**:  
  Each module in c-web-modules is independently managed, minimizing the risk of crashing the entire server.

- **WebSocket Updates**:  
  WebSocket handlers can be updated at runtime without breaking existing connections, a feature not typically available in Apache modules or ISAPI.

This makes **c-web-modules** suitable for rapid experimentation and modular design, especially in scenarios requiring frequent updates without disrupting service.