mdex - GitLite

KO | EN

gitlite — search

#javascript #python #ai #typescript #golang #deep-learning #docker #macos #nodejs #awesome #kubernetes #awesome-list #ios #open-source

mdex

(★ 445)

Markdown for Elixir. Fast, Extensible, Phoenix-native. AI-ready. Built on top of comrak, ammonia, and lumis.

File Explorer

Download Latest Version (.zip)

- - bug_report.md
- - elixir.yml
  - publish.yml
  - release.yml
- dependabot.yml
- FUNDING.yml
- - dockyard_logo.svg
  - mdex.svg
  - mdex_card.png
  - mdex_favicon.png
  - mdex_icon.png
  - mdex_logo.png
  - mdex_logo.svg
- config.exs
- - 2025-02-07-example-post.md
- code_block_decorators.livemd
- codefence_renderers.livemd
- custom_theme.livemd
- gfm.livemd
- highlight_words.livemd
- light_dark_theme.livemd
- liquid.livemd
- lumis.livemd
- mermaid.livemd
- nimble_publisher.livemd
- phoenix_live_view_heex.livemd
- sigil_md.livemd
- streaming.exs
- syntax_highlight.livemd
- syntect.livemd
- code_block_decorators.md
- compilation.md
- earmark_to_mdex.livemd
- heex.md
- plugins.md
- safety.md
- streaming.md
- - comrak_converter.ex
  - delta_converter.ex
  - document.ex
  - errors.ex
  - fragment_parser.ex
  - sigil.ex
  - slack_converter.ex
  - tree.ex
- mdex.ex
- - comrak_converter_test.exs
  - delta_converter_test.exs
  - delta_integration_test.exs
  - delta_property_test.exs
  - document_test.exs
  - errors_test.exs
  - fragment_parser_test.exs
  - heex_format_test.exs
  - html_format_test.exs
  - inspect_document_test.exs
  - json_format_test.exs
  - parse_test.exs
  - plugin_test.exs
  - sigil_test.exs
  - slack_converter_test.exs
  - streaming_test.exs
  - xml_format_test.exs
- mdex_test.exs
- test_helper.exs
.credo.exs
.formatter.exs
.gitignore
benchmark.exs
CHANGELOG.md
Dockerfile
justfile
LICENSE.md
mix.exs
mix.lock
README.md
SECURITY.md
usage-rules.md

# Installation Guide

1. Get the code

git clone https://github.com/leandrocp/mdex

Downloads the entire project code from GitHub to your computer.

cd mdex

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Docker Desktop Needed to build and run containers. Install it and keep it running in the background.

docker build -t mdex .

Builds a runnable image based on the Dockerfile.

docker run -p 8080:80 mdex

Runs the built image as an actual container.

✅ Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MDEx

<div align="center">
  <img src="https://raw.githubusercontent.com/leandrocp/mdex/main/assets/images/mdex_logo.png" width="360" alt="MDEx logo" />
  <br>

<a href="https://hex.pm/packages/mdex">
    <img alt="Hex Version" src="https://img.shields.io/hexpm/v/mdex">
  </a>

<a href="https://hexdocs.pm/mdex">
    <img alt="Hex Docs" src="https://img.shields.io/badge/hex.pm-docs-green.svg?style=flat">
  </a>

<a href="https://opensource.org/licenses/MIT">
    <img alt="MIT" src="https://img.shields.io/hexpm/l/mdex">
  </a>

<p align="center">Fast and Extensible Markdown for Elixir.</p>
</div>

## Features

- Fast
- Compliant with the [CommonMark spec](https://commonmark.org)
- [Plugins](https://hexdocs.pm/mdex/plugins.html)
- Formats:
  - Markdown (CommonMark)
  - HTML
  - [Phoenix HEEx](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.Rendered.html)
  - JSON
  - XML
  - [Quill Delta](https://quilljs.com/docs/delta/)
  - [Slack mrkdwn](https://docs.slack.dev/messaging/formatting-message-text)
- Floki-like [Document AST](https://hexdocs.pm/mdex/MDEx.Document.html)
- Req-like [Document pipeline API](https://hexdocs.pm/mdex/MDEx.Document.html)
- [GitHub Flavored Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax)
- [GitLab Flavored Markdown](https://docs.gitlab.com/user/markdown)
- Discord Flavored Markdown (Partial)
- Wiki-style links
- Phoenix HEEx components and expressions
- [Streaming](https://hexdocs.pm/mdex/streaming.html) incomplete fragments
- [Emoji](https://www.webfx.com/tools/emoji-cheat-sheet) shortcodes
- Built-in Syntax Highlighting with [Lumis](https://mdex.hexdocs.pm/lumis.html) or [Syntect](https://mdex.hexdocs.pm/syntect.html)
- [Code Block Decorators](https://hexdocs.pm/mdex/code_block_decorators-2.html)
- HTML sanitization
- [~MD Sigil](https://hexdocs.pm/mdex/MDEx.Sigil.html) for Markdown, HTML, HEEx, JSON, XML, and Quill Delta

## Plugins

- [mdex_gfm](https://hex.pm/packages/mdex_gfm) - Enable [GitHub Flavored Markdown](https://github.github.com/gfm) (GFM)
- [mdex_mermaid](https://hex.pm/packages/mdex_mermaid) - Render [Mermaid](https://mermaid.js.org) diagrams in code blocks
- [mdex_katex](https://hex.pm/packages/mdex_katex) - Render math formulas using [KaTeX](https://katex.org)
- [mdex_video_embed](https://hex.pm/packages/mdex_video_embed) - Privacy-respecting video embeds from code blocks
- [mdex_custom_heading_id](https://hex.pm/packages/mdex_custom_heading_id) - Custom heading IDs for markdown headings
- [mdex_mermex](https://hex.pm/packages/mdex_mermex) - Render [Mermaid](https://mermaid.js.org) diagrams server-side using Mermex (Rust NIF)
- [mdex_multiline_cells](https://hex.pm/packages/mdex_multiline_cells) - Multi-line cells in Markdown tables with inline/block Markdown rendering

## Installation

Add `:mdex` dependency:

```elixir
def deps do
  [
    {:mdex, "~> 0.12"}
  ]
end
```

Or use [Igniter](https://hexdocs.pm/igniter):

```sh
mix igniter.install mdex
```

## Usage

#### Convert to HTML
```elixir
iex> MDEx.to_html!("# Hello :smile:", extension: [shortcodes: true])
"<h1>Hello 😄</h1>"
```

#### Syntax Highlighting

Syntax highlight code blocks using either [Lumis](https://mdex.hexdocs.pm/lumis.html) or [Syntect](https://mdex.hexdocs.pm/syntect.html),
for example to use Lumis:

```elixir
def deps do
  [
    {:mdex, "~> 0.12"},
    {:lumis, "~> 0.1"}
  ]
end
```

```elixir
config :mdex_native, syntax_highlighter: :lumis
```

#### GitHub Flavored Markdown (GFM)

_Using the `:html_multi_themes` syntax highlighting formatter is not required but you get light/dark using it._

````elixir
Mix.install([
  {:mdex_gfm, "~> 0.1"}
])

markdown = """
- [x] Set up project
- [ ] Write docs

```elixir
spawn(fn -> send(current, {self(), 1 + 2}) end)
```
"""

MDEx.new(
  markdown: markdown,
  syntax_highlight: [
    engine: :lumis,
    opts: [
      formatter: {:html_multi_themes,
        themes: [light: "github_light", dark: "github_dark"],
        default_theme: "light-dark()"}
    ]
  ]
)
|> MDExGFM.attach()
|> MDEx.to_html!()
````

#### Sigils
```elixir
iex> import MDEx.Sigil
iex> ~MD[# Hello :smile:]HTML
"<h1>Hello 😄</h1>"
```

```elixir
iex> import MDEx.Sigil
iex> assigns = %{project: "MDEx"}
iex> ~MD[# {@project}]HEEX
%Phoenix.LiveView.Rendered{...}
```

```elixir
iex> import MDEx.Sigil
iex> ~MD[# Hello :smile:]
#MDEx.Document(3 nodes)<
├── 1 [heading] level: 1, setext: false
│   ├── 2 [text] literal: "Hello "
│   └── 3 [short_code] code: "smile", emoji: "😄"
>
```

#### Streaming
```elixir
iex> MDEx.new(streaming: true)
...> |> MDEx.Document.put_markdown("**Install")
...> |> MDEx.to_html!()
"<p><strong>Install</strong></p>"
```

## Examples and Guides

In docs you can find [Livebook examples](https://hexdocs.pm/mdex/gfm.html) covering options and usage, and [Guides](https://hexdocs.pm/mdex/plugins.html) for more info.

## Foundation

The library is built on top of:

- [comrak](https://crates.io/crates/comrak) - a fast Rust port of [GitHub's CommonMark parser](https://github.com/github/cmark-gfm)
- [ammonia](https://crates.io/crates/ammonia) for HTML Sanitization
- [lumis](https://crates.io/crates/lumis) for Syntax Highlighting

## Used By

- [00](https://github.com/technomancy-dev/00)
- [Algora](https://github.com/algora-io/algora)
- [Ash AI](https://github.com/ash-project/ash_ai)
- [BeaconCMS](https://github.com/BeaconCMS/beacon)
- [BeamLens Web](https://github.com/beamlens/beamlens_web)
- [Bonfire](https://github.com/bonfire-networks/bonfire-app)
- [Canada Navigator](https://github.com/canada-ca/navigator)
- [Conpipe](https://github.com/andyl/conpipe)
- [Exmeralda](https://github.com/bitcrowd/exmeralda)
- [Hexpm](https://github.com/hexpm/hexpm)
- [Jido](https://github.com/agentjido/jido)
- [Loomkin](https://github.com/bleuropa/loomkin)
- [Phoenix SEO](https://github.com/dbernheisel/phoenix_seo)
- [Phoenix Storybook](https://github.com/phenixdigital/phoenix_storybook)
- [Plural Console](https://github.com/pluralsh/console)
- [Prosody](https://github.com/halostatue/prosody)
- [Reposit](https://github.com/reposit-bot/reposit)
- [Sagents Live Debugger](https://github.com/sagents-ai/sagents_live_debugger)
- [Sayfa](https://github.com/furkanural/sayfa)
- [Tableau](https://github.com/elixir-tools/tableau)
- [Teiserver](https://github.com/beyond-all-reason/teiserver)
- [TermUI](https://github.com/pcharbon70/term_ui)
- [Tuist](https://github.com/tuist/tuist)
- [Valentine](https://github.com/maxneuvians/valentine)
- [Wraft](https://github.com/wraft/wraft)
- And [more...](https://github.com/search?q=lang%3Aelixir+%3Amdex&type=code)

_Are you using MDEx and want to list your project here? Please send a PR!_

## Sponsors

💜 **Support MDEx Development**

If you or your company find MDEx useful, please consider sponsoring its development.

➡️ **[GitHub Sponsors](https://github.com/sponsors/leandrocp)**

Your support helps maintain and improve MDEx for the entire Elixir community!

**Current and previous sponsors**

<a href="https://dockyard.com" target="_blank"><img src="assets/images/dockyard_logo.svg" width="200" alt="DockYard" /></a>

## Motivation

MDEx was born out of the necessity of parsing CommonMark files, to parse hundreds of files quickly, and to be easily extensible by consumers of the library.

- [earmark](https://hex.pm/packages/earmark) is extensible but [can't parse](https://github.com/RobertDober/earmark_parser/issues/126) all kinds of documents and is slow to convert hundreds of markdowns.
- [md](https://hex.pm/packages/md) is very extensible but the doc says "If one needs to perfectly parse the common markdown, Md is probably not the correct choice" and CommonMark was a requirement to parse many existing files.
- [markdown](https://hex.pm/packages/markdown) is not precompiled and has not received updates in a while.
- [cmark](https://hex.pm/packages/cmark) is a fast CommonMark parser but it requires compiling the C library, is hard to extend, and was archived on Apr 2024.

## Comparison

|Feature|MDEx|mdex_native|Earmark|md|cmark|erlang-markdown|
| --- | --- | --- | --- | --- | --- | --- |
|Active|✅|✅|⚠️⁵|✅|❌|✅|
|Pure Elixir|⚠️⁶|❌|✅|✅|❌|✅|
|Extensible|✅|❌|✅|✅|❌|⚠️⁹|
|Syntax Highlighting|✅|✅|❌|❌|❌|❌|
|Code Block Decorators|✅|❌|❌|❌|❌|❌|
|Streaming (fragments)|✅|❌|❌|❌|❌|❌|
|Phoenix HEEx components|✅|❌|❌|❌|❌|❌|
|MDX (JS)|❌|❌|❌|❌|❌|✅|
|Kramdown attributes|❌|❌|✅⁸|❌|❌|❌|
|EEx support|✅|❌|✅|❌|❌|❌|
|Custom syntax DSL|❌|❌|❌|✅|❌|❌|
|AST|✅|✅|✅|✅|❌|✅|
|AST to Markdown|✅|✅|⚠️⁷|❌|❌|❌|
|To HTML|✅|✅|✅|✅|✅|✅|
|To JSON|✅|❌|❌|❌|❌|❌|
|To XML|✅|✅|❌|❌|✅|❌|
|To Manpage|❌|❌|❌|❌|✅|❌|
|To LaTeX|❌|❌|❌|❌|✅|❌|
|To Quill Delta|✅|❌|❌|❌|❌|❌|
|To Slack|✅|❌|❌|❌|❌|❌|
|Emoji|✅|✅|❌|❌|❌|❌|
|GFM²|✅|✅|✅|❌|❌|✅|
|GLFM³|✅|✅|❌|❌|❌|❌|
|Discord⁴|⚠️¹|⚠️¹|❌|❌|❌|❌|

1. Partial support
2. GitHub Flavored Markdown
3. GitLab Flavored Markdown
4. Discord Flavored Markdown
5. Earmark is currently marked as retired
6. MDEx depends on [mdex_native](https://hex.pm/packages/mdex_native), which uses Rustler
7. Possible with [earmark_reversal](https://hex.pm/packages/earmark_reversal)
8. EarmarkParser supports Kramdown-style attributes
9. No custom plugin API

## Benchmark

A [benchmark](benchmark.exs) is available to compare existing libs:

```text
Name                      ips        average  deviation         median         99th %
cmark                 8681.81       0.115 ms    ±50.21%       0.111 ms       0.161 ms
mdex_native           6560.90       0.152 ms     ±7.26%       0.149 ms       0.199 ms
mdex                   395.26        2.53 ms    ±10.58%        2.49 ms        2.98 ms
md                     278.94        3.59 ms     ±8.62%        3.55 ms        3.91 ms
earmark                 69.86       14.32 ms     ±6.02%       14.24 ms       15.41 ms
erlang-markdown          4.80      208.43 ms     ±0.75%      208.19 ms      213.13 ms

Comparison:
cmark                 8681.81
mdex_native           6560.90 - 1.32x slower +0.0372 ms
mdex                   395.26 - 21.96x slower +2.41 ms
md                     278.94 - 31.12x slower +3.47 ms
earmark                 69.86 - 124.28x slower +14.20 ms
erlang-markdown          4.80 - 1809.56x slower +208.32 ms

Memory usage statistics:

Name                    average  deviation         median         99th %
cmark                0.00006 MB     ±0.00%     0.00006 MB     0.00006 MB
mdex_native          0.00008 MB     ±0.00%     0.00008 MB     0.00008 MB
mdex                    2.36 MB     ±0.00%        2.36 MB        2.36 MB
md                     11.77 MB     ±0.00%       11.77 MB       11.77 MB
earmark                 7.91 MB     ±0.00%        7.91 MB        7.91 MB
erlang-markdown       956.16 MB     ±0.00%      956.16 MB      956.16 MB

Comparison:
cmark                0.00006 MB
mdex_native          0.00008 MB - 1.38x memory usage +0.00002 MB
mdex                    2.36 MB - 38615.38x memory usage +2.36 MB
md                     11.77 MB - 192838.89x memory usage +11.77 MB
earmark                 7.91 MB - 129581.63x memory usage +7.91 MB
erlang-markdown       956.16 MB - 15665687.31x memory usage +956.16 MB
```

The most performance gain is using the `~MD` sigil to compile the Markdown instead of parsing it at runtime,
prefer using it when possible.

To finish, a friendly reminder that all libs have their own strengths and trade-offs so use the one that better suits your needs.

## Acknowledgements

- [comrak](https://crates.io/crates/comrak) crate for all the heavy work on parsing Markdown and rendering HTML
- [Floki](https://hex.pm/packages/floki) for the AST
- [Req](https://hex.pm/packages/req) for the pipeline API
- Logo based on [markdown-mark](https://github.com/dcurtis/markdown-mark)