emacs-backbone

KO | EN

gitlite — search

#python #java #python3 #arduino #golang #machine-learning #rust #html #flask #javascript #seismology #nodejs

★ 19 Open GitHub ↗

Emacs framework for deterministic, dependency-ordered configuration management inspired by NixOS principles.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Emacs Backbone

An external orchestration program (written in Gleam) that manages Emacs configuration through bidirectional JSON-RPC over stdio, providing deterministic, dependency-aware package installation and configuration management inspired by NixOS principles.

## Motivation

After over a decade of using Emacs, I grew frustrated with a persistent problem: **my configuration would break unexpectedly**, and I'd have to debug it before I could actually get work done.

**The root cause: no true dependency resolution**

Traditional Emacs configuration tools don't actually resolve dependencies. `with-eval-after-load` is just a convenience macro that defers code execution until a library loads - it doesn't guarantee execution order across multiple deferred blocks. Similarly, use-package's `:after` keyword delays loading until dependencies are available, but doesn't provide deterministic ordering. There's no built-in way to declare "configure B only after A is fully ready" and have the system guarantee it.

This leads to non-deterministic behavior where the same configuration could behave differently across restarts or machines. Race conditions and timing issues create bugs that are nearly impossible to track down.

**The lazy loading dilemma**

I've been a Doom Emacs user for years with heavily customized config. Doom's lazy loading is excellent for fast startup, but I kept hitting the same frustrating pattern:

1. Make a mistake in my config or custom elisp
2. Error only appears when that feature loads - could be hours into my session
3. Hard to debug because the context is lost

The problem isn't lazy loading itself - it's that **errors hide until runtime**. Traditional approaches optimize for startup speed but defer error discovery.

**The fail-fast philosophy**

That's why Emacs Backbone takes the opposite approach: load everything at startup in deterministic order and encounter all errors immediately. Fix them right away while the context is fresh. Once Emacs starts successfully, you can be confident nothing will break hours into your session. If you comment out a package, its config-unit automatically skips - no manual cleanup needed.

This trades a few extra seconds at startup for reliability. Since Emacs is a long-lived process that runs for days or weeks, I'd rather fail fast at startup than risk hours of lost work from a mid-session crash.

**What Emacs Backbone provides**

Inspired by NixOS principles, Emacs Backbone treats configuration as a dependency graph that's resolved deterministically using topological sorting:

- **True dependency resolution** - Declare dependencies explicitly, the system figures out the order
- **Deterministic execution** - Same declarations always produce the same order
- **Immediate feedback** - All configuration errors surface at startup
- **Reproducible behavior** - Your config works the same way every time, on every machine

Every package and configuration unit declares its dependencies explicitly, and the system ensures they execute in the correct order - every single time.

**Why Gleam?**

I happened to be learning Gleam when this idea crystallized, and it turned out to be the perfect choice. Gleam's strong static type system and compiler give me confidence that **once it compiles, it will work**. The compiler catches potential issues before runtime, which is exactly the kind of reliability I was seeking for my Emacs configuration. This compile-time safety complements the deterministic runtime behavior - together they ensure that configuration problems are caught early and behavior remains consistent.

<summary>Why not just `use-package :after`?</summary>

A common question is: *If `use-package` has the `:after` keyword, why do we need `config-unit!` and an external orchestrator?*

The short answer is that `use-package` is an excellent **package loader**, but a very limited **state manager**. When your configuration scales, relying solely on `:after` for dependency management breaks down in a few critical ways.

### 1. The Cross-Package Integration Problem (N-to-N)

The most glaring limitation of `use-package :after` is handling configurations that depend on *multiple* packages being loaded and configured first.

Take the `evil-org` package as an example. To configure it properly, you need both `evil` and `org` to be fully initialized. Where does the integration code go? With `use-package`, you are forced into awkward, tightly coupled nesting:

```elisp
;; The use-package way: Awkward and conceptually messy
(use-package org
  :config
  ;; Do org stuff...
  (use-package evil-org
    :after evil
    :config
    ;; Wait, does this belong in the 'org' block or the 'evil' block?
    (evil-org-set-key-theme)))
```

This forces your integration logic to live inside the namespace of a single, arbitrary package.

With `emacs-backbone` and `config-unit!`, you decouple the integration entirely. You create a distinct, independent unit that explicitly declares its upstream dependencies. It sits perfectly at the intersection of the two, keeping your base package configs clean:

```elisp
;; The emacs-backbone way: Clean, decoupled dependency declaration
(config-unit! evil-org-integration
  :requires (evil-org)
  :after (org-base evil-base)
  :config
  (evil-org-set-key-theme))
```

### 2. Reactivity vs. Determinism

Under the hood, `use-package :after` is just syntactic sugar for Emacs's built-in `with-eval-after-load`. This is a **reactive** system. It tells Emacs: *"Keep an eye out, and if package X ever happens to load, run this code."*

Because it is reactive, your configuration's execution order is highly vulnerable to the physical order of your `init.el` file, autoload triggers, or user actions. This is a common source of silent failures, race conditions, and infinite loading loops.

`emacs-backbone` is **deterministic**. The Gleam backend reads your `config-unit!` declarations, parses them into a strict Directed Acyclic Graph (DAG), and uses topological sorting to guarantee mathematically perfect execution order over JSON-RPC. The physical location of the code in your files no longer matters; the dependencies dictate the flow.

### 3. Granularity of State

`use-package` operates strictly at the *package* level—it waits for a package to call `(provide 'some-package)`.

However, in advanced Emacs setups, you often need to depend on a *logical state*, not just a package. For example, you might want your customized modeline to load only after your entire "UI Framework" (fonts, themes, and fringes) is fully configured. `config-unit!` allows you to depend on arbitrary blocks of configuration state to resolve, giving you orchestration granularity that `use-package` simply cannot provide.

</details>

## Overview

Emacs Backbone separates the configuration framework from user configuration, similar to Doom Emacs:

- **Core System** (`~/.config/emacs`): Gleam backend, Emacs Lisp framework, macros, and orchestration logic
- **User Configuration** (`~/.config/backbone`): Your personal `config.org`, package declarations, and config units

This separation allows you to:

- Version control your user config independently from the core
- Receive core updates without affecting your personal configuration
- Share your configuration with others easily

## Core Concepts

The heart of Emacs Backbone lies in two declarative macros that replace traditional Emacs configuration patterns:

### `package!` - Declarative Package Management

The `package!` macro declares packages with explicit dependencies, eliminating load-order guesswork:

```elisp
;; Simple package declaration
(package! magit)

;; Package from a specific repository
(package! evil
  :repo "emacs-evil/evil"
  :host "github")

;; Package with dependencies - Emacs Backbone ensures correct installation order
(package! evil-collection
  :repo "emacs-evil/evil-collection"
  :deps (evil))

;; Local package with subdirectories
(package! lsp-bridge
  :local "~/projects/lsp-bridge"
  :files ("*.el" "*.py" "acm" "core" "langserver")
  :no-compilation t
  :deps (markdown-mode yasnippet))
```

**Key features:**

- **Dependency resolution**: Use `:deps` to declare package dependencies - the system guarantees they install first
- **Repository flexibility**: Install from GitHub, GitLab, or any git host with `:repo` and `:host`
- **Local packages**: Use `:local` for packages from local filesystem, with `:files` to include subdirectories
- **Version pinning**: Use `:branch`, `:tag`, or `:ref` for reproducible builds
- **No manual ordering needed**: Declare packages anywhere in your config - dependencies are resolved automatically

### `config-unit!` - Dependency-Ordered Configuration

The `config-unit!` macro replaces `use-package`, `with-eval-after-load`, and scattered configuration blocks with explicit, ordered units:

```elisp
;; Basic configuration unit
(config-unit! evil-config
  :requires (evil)  ; Require evil before running this unit
  :config
  (evil-mode 1)
  (setq evil-want-C-u-scroll t))

;; Configuration with ordering dependencies
(config-unit! evil-collection-config
  :requires (evil-collection)
  :after (evil-config)  ; Guaranteed to run after evil-config
  :config
  (evil-collection-init))

;; Complex dependency graph - automatically resolved
(config-unit! my-keybindings
  :after (evil-config evil-collection-config which-key-config)
  :config
  (global-set-key (kbd "C-c g") 'magit-status))

;; Runtime prerequisites: require env vars and executables before running
(config-unit! llm-tools
  :requires (gptel)
  :env (OPENAI_API_KEY ANTHROPIC_API_KEY)
  :executable (rg git)
  :config
  (message "LLM tooling is ready"))
```

**Key features:**

- **`:requires`** - Require listed features before running the unit body
- **`:after`** - Execution ordering between `config-unit!` names (not `package!` names); unlike `eval-after-load`, this is **guaranteed**
- **`:env`** - Require environment variables to be set before the unit runs
- **`:executable`** - Require external binaries to exist on `PATH` before the unit runs
- **`:config`** - Configuration body that runs in deterministic order
- **Topological sorting** - Complex dependency graphs are automatically resolved
- **Circular dependency detection** - Fails fast if your configuration has cycles

### How They Work Together

1. **Declaration phase**: You declare `package!` and `config-unit!` in any order in your `config.org`
2. **Dependency resolution**: Gleam backend extracts all declarations and builds a dependency graph
3. **Package installation**: Packages install in dependency order (packages with no deps first)
4. **Configuration execution**: After all packages install, config units execute in topologically sorted order
5. **Guaranteed determinism**: Same declarations always produce the same execution order

This replaces the traditional Emacs configuration chaos with a **predictable, reproducible system**.

## Installation

### Prerequisites

- **Emacs** 29+ (with native compilation support recommended)
- **Bun** (runtime for `generated/emacs-backbone.mjs`)

### Core Installation

1. **Clone the core repository:**

```bash
   git clone git@github.com:nohzafk/emacs-backbone.git ~/.config/emacs
   cd ~/.config/emacs
   ```

### User Configuration Setup

1. **Create your user configuration directory:**

```bash
   mkdir -p ~/.config/backbone
   ```

2. **Create your configuration:**

Create `~/.config/backbone/config.org` with your Emacs configuration using `package!` and `config-unit!` macros. For example:

```org
   * Package Declarations
   #+begin_src emacs-lisp
   (package! evil
     :repo "emacs-evil/evil")

(package! which-key
     :repo "justbur/emacs-which-key")
   #+end_src

* Configuration Units
   #+begin_src emacs-lisp
   (config-unit! evil-setup
     :requires (evil)
     :config
     (evil-mode 1))

(config-unit! which-key-setup
     :requires (which-key)
     :after (evil-setup)
     :config
     (which-key-mode 1))

(config-unit! ai-setup
     :requires (gptel)
     :after (which-key-setup)
     :env OPENAI_API_KEY
     :executable (git rg)
     :config
     (message "AI tooling enabled"))
   #+end_src
   ```

3. **Start Emacs:**

```bash
   emacs
   ```

4. **Tangle and reload from inside Emacs:**

- `C-h r e` to tangle `~/.config/backbone/config.org` into `config.el`
   - `C-h r r` to reload the freshly tangled config

On first startup, Emacs Backbone may warn that `config.el` does not exist
   yet. That is expected until you tangle it for the first time.

### Example Configuration

For a complete, real-world example of an Emacs Backbone configuration, see:

**[nohzafk's literate-darwin config](https://github.com/nohzafk/literate-darwin/blob/main/modules/emacs/backbone/config.org)**

## Configuration

### Customization Variables

#### `emacs-backbone-user-directory`

**Default:** `~/.config/backbone`

Directory containing your user configuration. Customize if you want to use a different location:

```elisp
(setq emacs-backbone-user-directory "~/my-custom-config")
```

#### `emacs-backbone-bun-executable`

**Default:** Auto-detected (`/opt/homebrew/bin/bun` or from PATH)

Path to the Bun executable used for `generated/emacs-backbone.mjs`:

```elisp
(setq emacs-backbone-bun-executable "/opt/homebrew/bin/bun")
```

#### `emacs-backbone-enable-debug`

**Default:** `nil`

Enable verbose logging and debugging output:

```elisp
(setq emacs-backbone-enable-debug t)
```

## Usage

### Interactive Commands

- `M-x emacs-backbone-start` - Initialize Emacs Backbone (auto-invoked in init.el)
- `M-x emacs-backbone-tangle-config` - Tangle `config.org` to `config.el`
- `M-x emacs-backbone-reload-config` - Full reload: tangle + reinitialize

The Gleam backend is managed over stdio, so it exits with Emacs. There is no separate manual shutdown command.

### Keybindings

- `C-h r e` - Tangle configuration
- `C-h r r` - Reload configuration

## Development

### Building the Project

```bash
bun run build            # Regenerate generated/emacs-backbone.mjs
gleam test               # Run tests
```

### Generated Bun Backend

The repo ships a committed Bun bundle so end users do not need Gleam
installed at runtime. Emacs always starts Backbone with:

```elisp
(list emacs-backbone-bun-executable
      (expand-file-name "generated/emacs-backbone.mjs" user-emacs-directory))
```

Regenerate that bundle during development with:

```bash
bun run build
```

This script:

1. Runs `gleam build`
2. Bundles the generated Gleam JavaScript into `generated/emacs-backbone.mjs`
3. Produces the stable backend entrypoint Emacs runs with Bun

Run the committed bundle directly with:

```bash
bun generated/emacs-backbone.mjs
```

If Emacs cannot find Bun in its environment, set
`emacs-backbone-bun-executable` to an absolute path.

### Developer Workflow

Developers need both **Gleam** and **Bun**.

When you change backend code under `src/`:

1. Edit the Gleam source
2. Run `bun run build`
3. Run `gleam test`
4. Commit both the source changes and the updated `generated/emacs-backbone.mjs`

When you change only the Emacs Lisp side under `lisp/`, you usually do not need
to rebuild the generated backend bundle.

### Normal User Workflow

Normal users need only **Bun**.

1. Install Bun
2. Clone this repo to `~/.config/emacs`
3. Keep `generated/emacs-backbone.mjs` from the repo checkout
4. Start Emacs

No Gleam installation is required unless the user intends to modify the Gleam
backend and regenerate the committed bundle.

### Architecture

#### Two-Directory Structure

```text
~/.config/emacs/                           # Core system (this repository)
├── src/                                   # Gleam backend
│   ├── emacs_backbone.gleam              # Entry point
│   ├── jsonrpc.gleam                      # JSON-RPC communication over stdio
│   ├── resolver.gleam                     # Dependency resolution
│   └── ...
├── lisp/                                  # Core Emacs Lisp framework
│   ├── backbone.el                       # JSON-RPC client & lifecycle
│   ├── macro-package.el                   # package! macro
│   ├── macro-config-unit.el               # config-unit! macro
│   └── ...
├── init.el                                # Core initialization
├── early-init.el                          # Early initialization
├── gleam.toml                             # Gleam project config
└── package.json                           # JavaScript module metadata

~/.config/backbone/                        # User configuration (you create this)
├── early-init.el                          # Your early initialization (optional)
├── config.org                             # Your literate configuration
├── config.el                              # Generated from config.org
├── config/                                # Your helper utilities
└── clis/                                  # Your CLI configurations
```

#### Elpaca Compatibility

Emacs Backbone currently bootstraps against **Elpaca installer version `0.12`** in
[lisp/package-manager.el](/Users/randall/projects/emacs-backbone/lisp/package-manager.el).

This integration currently depends on some Elpaca implementation details, not
just its public user-facing configuration surface. In particular, Backbone has
workarounds for **split packages in a shared monorepo checkout**, such as
`magit-section` living inside the shared `magit` repository checkout, and
packages like `embark-consult` living inside the shared `embark` checkout.

That means Elpaca upgrades should be treated as compatibility work: if Elpaca's
shared-source, clone-skip, or dependency-detection internals change, Backbone's
integration may need to be updated as well.

### Project Structure

- `src/` - Gleam backend source code
- `lisp/` - Core Emacs Lisp framework
- `test/` - Gleam tests
- `bin/` - Utility scripts

## Troubleshooting

### Config not found

If you see "Emacs Backbone user config not found", ensure:

1. `~/.config/backbone/config.el` exists
2. You've tangled your `config.org` file
3. The `emacs-backbone-user-directory` variable points to the correct location

### Backbone won't start

Check:

1. Bun is installed: `which bun`
2. The `emacs-backbone-bun-executable` variable is correct
3. `generated/emacs-backbone.mjs` exists in the repo
4. If you changed Gleam source, you rebuilt the bundle with `bun run build`

### Package installation fails

1. Check the `*emacs-backbone*` buffer for errors
2. Enable debug mode: `(setq emacs-backbone-enable-debug t)`
3. Verify your package declarations syntax in `config.el`

emacs-backbone

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README