omnifs

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 84 Open GitHub ↗

a projected filesystem that turns APIs, services, and data sources into local files and directories

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center"><b>omnifs</b></h1>
<h4 align="center">open a path, read the world.</h4>
<p align="center"><a href="#quickstart">quickstart</a> | <a href="https://omnifs.dev">homepage</a> | <a href="https://omnifs.dev/start">docs</a> | <a href="#things-to-try">things to try</a> | <a href="#providers">providers</a> | <a href="#how-it-works">how it works</a></p>
</div>

omnifs projects external systems into local filesystem paths. GitHub, DNS, arXiv, Docker, Linear, SQLite, Kubernetes, web pages, and Oura become directories and files you can `cd`, `ls`, `cat`, `grep`, `find`, `jq`, and script against.

The goal is simple: if a tool can read files, it can read the outside world without learning another SDK, auth flow, pagination model, or response schema.

> Alpha status: omnifs is real and usable, but the read surface is still early. The daemon runs natively on Linux and macOS. Named filesystems are independent whole-namespace access surfaces managed with `omnifs fs`. Docker and libkrun deliver FUSE only. Filesystems attach over the wire protocol and carry no providers or credentials.

## Quickstart

omnifs is written in Rust. We ship prebuilt Linux and macOS binaries through the npm registry, so the npm installation path needs Node.js and npm. Docker or OrbStack is needed only for the optional Docker-hosted FUSE filesystem; macOS can use libkrun instead.

```bash
npm install -g @0xff-ai/omnifs
omnifs setup
omnifs status
```

`omnifs setup` boots the host-native daemon, shows what is already running, and lists every embedded provider with an honest auth label. It then offers two default-yes confirms: mount every provider that needs no sign-in in one atomic batch, and attach the platform-recommended filesystem. `--yes` accepts both without asking; `--no-input` declines both and still exits 0. Anything that needs a sign-in or a config value goes through `omnifs mount add`, or use the commands below for separate operations.

---

For a direct, scriptable path, create mounts one at a time with `omnifs mount add <provider>`. Mounts, credentials, and provider artifacts are daemon-owned state changed through local typed RPC. The CLI starts the daemon when a command needs it; there is no separate apply or offline mode.

```bash
omnifs mount add github
omnifs mount add dns
omnifs mount update github --config-json '{"owner":"0xff-ai"}'
omnifs status
omnifs fs create --name local --protocol fuse --runtime host --location "$HOME/omnifs"
omnifs fs attach --name local
omnifs status
cd "$HOME/omnifs"
```

---

Useful commands:

```bash
omnifs status      # runtime, mount, and auth state
omnifs fs ls        # configured filesystems and attachment state
omnifs logs -f     # follow daemon logs
omnifs inspect     # live TUI for namespace, provider, cache, and callout activity
omnifs fs detach --name local # stop one named filesystem
omnifs down        # stop the daemon
```

Filesystem specs persist, but their running state does not. Host locations must be absolute; Docker and libkrun own their guest location.

```bash
omnifs fs create --name local --protocol nfs --runtime host --location "/Users/me/omnifs"
omnifs fs create --name guest --runtime docker
omnifs fs attach --name local
omnifs fs attach --name guest
omnifs fs restart --name guest
omnifs fs shell --name guest
omnifs fs shell --name guest -- ls -la /omnifs/github
omnifs fs detach --name guest
```

Every attached filesystem exposes every configured mount. `omnifs mount add` resolves and retains one exact content-addressed provider artifact through daemon RPC; the daemon validates the mount before serving it.

`omnifs mount update NAME` changes only named auth, config, or limits fields and rejects a stale concurrent edit. Use `--no-auth`, `--clear-config`, or `--clear-limits` for explicit removal.

For automation, select one invocation-owned output contract. JSON prints one envelope and keeps resource collections plural; JSONL uses the same terminal result or error envelope with its stream-record discriminator. Live logs and Inspector records remain line streams.

```bash
omnifs --output json status | jq '.result.filesystems[] | {id, protocol, runtime, location, state}'
omnifs --output json mount ls | jq '.result.mounts[]'
```

On an interactive terminal, `omnifs inspect` shows a timestamped operation stream, path activity, per-mount rates, and the selected provider/cache/callout stages. Press `?` for the current keys. `omnifs inspect --plain` emits human lines and `omnifs --output jsonl inspect` emits typed records without opening terminal mode.

## Things to try

Once you are inside any filesystem mount, use normal shell tools.

```bash
# GitHub
cd /github/ollama/ollama
ls
cat repo.json
cat issues/open/12959/title
cat pulls/all/9585/diff.patch
# repository trees are cloned on demand
cd repo && ls

# DNS
cat /dns/cloudflare.com/A
cat /dns/@google/google.com/AAAA
cat /dns/openai.com/TXT
cat /dns/reverse/1.1.1.1

# arXiv -- "Attention is all you need"
ls /arxiv/papers/1706.03762
cat /arxiv/papers/1706.03762/@latest/paper.json | jq .title

# Docker
cat /docker/system/version.json | jq .
cat /docker/containers.json | jq .
ls /docker/containers/running

# Linear -- requires a Linear API key
ls /linear/teams
cat /linear/teams/ENG/issues/open/ENG-123/title

# SQLite -- download an example db and explore the data
wget -O /tmp/chinook.sqlite https://github.com/lerocha/chinook-database/raw/refs/heads/master/ChinookDatabase/DataSources/Chinook_Sqlite.sqlite
omnifs mount add db # provide path: /tmp/chinook.sqlite
ls /db/tables
cat /db/tables/Album/schema.sql
cat /db/tables/Album/sample.json | jq .
```

<details>
<summary>SSH agent troubleshooting</summary>

Check the host before opening repo tree paths:

```bash
echo "$SSH_AUTH_SOCK"
ssh-add -L
ssh -T git@github.com
```

</details>

## Why paths

APIs are good boundaries for applications. They are a bad default interface for every script, terminal session, CI job, editor, and agent that only needs to read state.

omnifs makes the path the interface:

```text
/github/ollama/ollama/issues/open/12959/title
/docker/containers/running/{name}/state
/arxiv/papers/1706.03762/@latest/paper.json
/dns/cloudflare.com/TXT
```

That gives existing tools a common substrate. `grep -r`, `find`, `jq`, `tar`, `diff`, `head`, `tail`, and editors can all operate without provider-specific clients. Agents get the same benefit: open a path and read bytes.

The projected namespace is read-only. Mount commands change daemon-owned state through local RPC; projected issue, PR, container, and DNS files are never direct mutation controls.

## Providers

| Provider | Mount | What it projects |
| --- | --- | --- |
| GitHub | `/github` | Users, orgs, repos, issues, pull requests, Actions runs, diffs, and repo trees cloned on demand |
| DNS | `/dns` | DNS-over-HTTPS records, resolver-scoped queries, raw answers, and reverse lookups |
| arXiv | `/arxiv` | Paper version families, PDFs, source archives, metadata, and category paper listings |
| Docker | `/docker` | Docker daemon system state, container listings, per-container inspect output, state, and summaries |
| Linear | `/linear` | Teams and issues, with title, state, priority, assignee, and description files |
| SQLite | `/db` | Read-only SQLite metadata, table schemas, indexes, row counts, and samples |
| Kubernetes | `/k8s` | Live namespaces, cluster resources, manifests, status, events, and pod logs |
| Web | `/web` | Allowed HTTPS pages as readable Markdown and raw response bytes |
| Oura | `/oura` | Daily health, sleep, readiness, workout, heart-rate, and ring-battery data |

### GitHub

| Path | Content |
| --- | --- |
| `/github/{owner}` | Repositories for a user or organization |
| `/github/{owner}/{repo}` | Repository surface |
| `/github/{owner}/{repo}/repo/` | Source tree, cloned on demand via SSH |
| `/github/{owner}/{repo}/issues/{open,all}/` | Issue listings |
| `/github/{owner}/{repo}/issues/{filter}/{n}/title` | Issue title |
| `/github/{owner}/{repo}/issues/{filter}/{n}/body` | Issue body |
| `/github/{owner}/{repo}/pulls/{filter}/{n}/diff.patch` | Pull request diff |
| `/github/{owner}/{repo}/actions/runs/{id}/status` | Actions run status |
| `/github/{owner}/{repo}/actions/runs/{id}/log` | Actions run log |

### DNS

| Path | Content |
| --- | --- |
| `/dns/{domain}/A` | A records |
| `/dns/{domain}/AAAA` | AAAA records |
| `/dns/{domain}/MX` | MX records |
| `/dns/{domain}/TXT` | TXT records |
| `/dns/{domain}/all` | Common record types |
| `/dns/{domain}/raw` | Dig-style output |
| `/dns/@{resolver}/{domain}/{record}` | Query through a named or IP resolver |
| `/dns/reverse/{ip}` | Reverse lookup |
| `/dns/resolvers` | Configured resolvers |

### arXiv

| Path | Content |
| --- | --- |
| `/arxiv/papers/{id}/` | Paper version family |
| `/arxiv/papers/{id}/@latest/paper.pdf` | Latest version PDF |
| `/arxiv/papers/{id}/@latest/source.tar.gz` | Latest version source bundle |
| `/arxiv/papers/{id}/@latest/paper.atom` | Raw upstream Atom feed |
| `/arxiv/papers/{id}/@latest/paper.json` | Rendered metadata |
| `/arxiv/papers/{id}/v{n}/paper.pdf` | Version-pinned PDF |
| `/arxiv/categories/{cat}/papers/` | Recent papers in a category |
| `/arxiv/categories/{cat}/papers/{id}/@latest/...` | Category alias for the same paper version family |

### Docker, Linear, and SQLite

| Path | Content |
| --- | --- |
| `/docker/system/version.json` | Docker daemon version |
| `/docker/containers.json` | Container listing |
| `/docker/containers/{by-name,by-id,running,stopped}/` | Container indexes |
| `/docker/containers/running/{name}/state` | Live container state |
| `/docker/containers/running/{name}/inspect.json` | Docker inspect JSON |
| `/linear/teams/` | Linear teams by key |
| `/linear/teams/{KEY}/issues/{open,all}/` | Team issue listings |
| `/linear/teams/{KEY}/issues/{filter}/{KEY-N}/description.md` | Issue description |
| `/db/meta/info.json` | SQLite database metadata |
| `/db/tables/{table}/schema.sql` | Table schema |
| `/db/tables/{table}/sample.json` | Sample rows |

## How it works

The host-native daemon owns providers, credentials, caching, callouts, and one shared namespace. FUSE and NFS filesystems in host, Docker, or libkrun runtimes all expose that same tree via the wire protocol.

Providers are WebAssembly components implementing the [`omnifs:provider` WIT interface](crates/omnifs-wit/wit/provider.wit). Providers are self-contained: they declare identity, capabilities, config metadata, and auth via `#[omnifs_sdk::provider]` annotations, which the build assembles into the `omnifs.provider-metadata.v1` Wasm custom section. A provider's main job is to answer filesystem operations via entrypoint methods `lookup_child`, `list_children`, and `read_file`.

Providers do not hold tokens, open sockets, or run Git themselves. They await typed host callouts such as HTTP fetches, blob downloads, archive opens, or repo tree handoffs. The host executes those imports, attaches credentials at the boundary, enforces declared capabilities, and owns caching.

The cache is host-owned plain byte storage. Providers can return canonical upstream bytes and derived filesystem entries together, so one upstream payload can populate multiple files and child entries. Invalidations come from explicit provider effects and runtime events.

## Development workflows

Use `just dev` when working from this repository. It builds provider WASM and the native CLI, renders the built-in dev mounts and credentials under `~/.omnifs-dev`, starts the host-native daemon and fixtures, attaches the slim Docker-hosted FUSE filesystem, and opens a shell at `/omnifs`.

```bash
git clone https://github.com/0xff-ai/omnifs
cd omnifs
just dev -y
# opens the attached filesystem shell at /omnifs
```

Refresh generated and formatted artifacts:

```bash
just refresh
```

For runtime behavior, validate through the host daemon and attached filesystem:

```bash
just dev -y
omnifs status
tail -n 80 ~/.omnifs-dev/daemon-state/logs/daemon.log
```

## Current status

- FUSE (Linux) and read-only NFSv4 loopback (macOS) filesystems in host, Docker, or libkrun runtimes; attached explicitly with the filesystem lifecycle commands.
- A host CLI on npm that handles mounts, auth, lifecycle, logs, status, and inspection.
- Sandboxed Wasm providers that can only reach the network, Git, sockets, and files the host hands them.
- Host-held credentials, layered caching, and `omnifs inspect` for a live view of what the runtime is doing.
- Daemon-owned SQLite state and typed local RPC for mounts, credentials, providers, logs, and attachments.
- Nine live providers: GitHub, DNS, arXiv, Docker, Linear, SQLite, Kubernetes, Web, and Oura.

## License

MIT OR Apache-2.0

omnifs

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README