mdex - GitLite

KO | EN

gitlite — search

#llm #go #android #rust #macos #nodejs #java #windows #awesome #cli #kubernetes #awesome-list #security #open-source

mdex

(★ 445)

Markdown for Elixir. Fast, Extensible, Phoenix-native. AI-ready. Built on top of comrak, ammonia, and lumis.

File Explorer

Download Latest Version (.zip)

- - bug_report.md
- - elixir.yml
  - publish.yml
  - release.yml
- dependabot.yml
- FUNDING.yml
- - dockyard_logo.svg
  - mdex.svg
  - mdex_card.png
  - mdex_favicon.png
  - mdex_icon.png
  - mdex_logo.png
  - mdex_logo.svg
- config.exs
- - 2025-02-07-example-post.md
- code_block_decorators.livemd
- codefence_renderers.livemd
- custom_theme.livemd
- gfm.livemd
- highlight_words.livemd
- light_dark_theme.livemd
- liquid.livemd
- lumis.livemd
- mermaid.livemd
- nimble_publisher.livemd
- phoenix_live_view_heex.livemd
- sigil_md.livemd
- streaming.exs
- syntax_highlight.livemd
- syntect.livemd
- code_block_decorators.md
- compilation.md
- earmark_to_mdex.livemd
- heex.md
- plugins.md
- safety.md
- streaming.md
- - comrak_converter.ex
  - delta_converter.ex
  - document.ex
  - errors.ex
  - fragment_parser.ex
  - sigil.ex
  - slack_converter.ex
  - tree.ex
- mdex.ex
- - comrak_converter_test.exs
  - delta_converter_test.exs
  - delta_integration_test.exs
  - delta_property_test.exs
  - document_test.exs
  - errors_test.exs
  - fragment_parser_test.exs
  - heex_format_test.exs
  - html_format_test.exs
  - inspect_document_test.exs
  - json_format_test.exs
  - parse_test.exs
  - plugin_test.exs
  - sigil_test.exs
  - slack_converter_test.exs
  - streaming_test.exs
  - xml_format_test.exs
- mdex_test.exs
- test_helper.exs
.credo.exs
.formatter.exs
.gitignore
benchmark.exs
CHANGELOG.md
Dockerfile
justfile
LICENSE.md
mix.exs
mix.lock
README.md
SECURITY.md
usage-rules.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# MDEx

<div align="center">
  <img src="https://raw.githubusercontent.com/leandrocp/mdex/main/assets/images/mdex_logo.png" width="360" alt="MDEx logo" />
  <br>

<a href="https://hex.pm/packages/mdex">
    <img alt="Hex Version" src="https://img.shields.io/hexpm/v/mdex">
  </a>

<a href="https://hexdocs.pm/mdex">
    <img alt="Hex Docs" src="https://img.shields.io/badge/hex.pm-docs-green.svg?style=flat">
  </a>

<a href="https://opensource.org/licenses/MIT">
    <img alt="MIT" src="https://img.shields.io/hexpm/l/mdex">
  </a>

<p align="center">Fast and Extensible Markdown for Elixir.</p>
</div>

## Features

- Fast
- Compliant with the [CommonMark spec](https://commonmark.org)
- [Plugins](https://hexdocs.pm/mdex/plugins.html)
- Formats:
  - Markdown (CommonMark)
  - HTML
  - [Phoenix HEEx](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.Rendered.html)
  - JSON
  - XML
  - [Quill Delta](https://quilljs.com/docs/delta/)
  - [Slack mrkdwn](https://docs.slack.dev/messaging/formatting-message-text)
- Floki-like [Document AST](https://hexdocs.pm/mdex/MDEx.Document.html)
- Req-like [Document pipeline API](https://hexdocs.pm/mdex/MDEx.Document.html)
- [GitHub Flavored Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax)
- [GitLab Flavored Markdown](https://docs.gitlab.com/user/markdown)
- Discord Flavored Markdown (Partial)
- Wiki-style links
- Phoenix HEEx components and expressions
- [Streaming](https://hexdocs.pm/mdex/streaming.html) incomplete fragments
- [Emoji](https://www.webfx.com/tools/emoji-cheat-sheet) shortcodes
- Built-in Syntax Highlighting with [Lumis](https://mdex.hexdocs.pm/lumis.html) or [Syntect](https://mdex.hexdocs.pm/syntect.html)
- [Code Block Decorators](https://hexdocs.pm/mdex/code_block_decorators-2.html)
- HTML sanitization
- [~MD Sigil](https://hexdocs.pm/mdex/MDEx.Sigil.html) for Markdown, HTML, HEEx, JSON, XML, and Quill Delta

## Plugins

- [mdex_gfm](https://hex.pm/packages/mdex_gfm) - Enable [GitHub Flavored Markdown](https://github.github.com/gfm) (GFM)
- [mdex_mermaid](https://hex.pm/packages/mdex_mermaid) - Render [Mermaid](https://mermaid.js.org) diagrams in code blocks
- [mdex_katex](https://hex.pm/packages/mdex_katex) - Render math formulas using [KaTeX](https://katex.org)
- [mdex_video_embed](https://hex.pm/packages/mdex_video_embed) - Privacy-respecting video embeds from code blocks
- [mdex_custom_heading_id](https://hex.pm/packages/mdex_custom_heading_id) - Custom heading IDs for markdown headings
- [mdex_mermex](https://hex.pm/packages/mdex_mermex) - Render [Mermaid](https://mermaid.js.org) diagrams server-side using Mermex (Rust NIF)
- [mdex_multiline_cells](https://hex.pm/packages/mdex_multiline_cells) - Multi-line cells in Markdown tables with inline/block Markdown rendering

## Installation

Add `:mdex` dependency:

```elixir
def deps do
  [
    {:mdex, "~> 0.12"}
  ]
end
```

Or use [Igniter](https://hexdocs.pm/igniter):

```sh
mix igniter.install mdex
```

## Usage

#### Convert to HTML
```elixir
iex> MDEx.to_html!("# Hello :smile:", extension: [shortcodes: true])
"<h1>Hello 😄</h1>"
```

#### Syntax Highlighting

Syntax highlight code blocks using either [Lumis](https://mdex.hexdocs.pm/lumis.html) or [Syntect](https://mdex.hexdocs.pm/syntect.html),
for example to use Lumis:

```elixir
def deps do
  [
    {:mdex, "~> 0.12"},
    {:lumis, "~> 0.1"}
  ]
end
```

```elixir
config :mdex_native, syntax_highlighter: :lumis
```

#### GitHub Flavored Markdown (GFM)

_Using the `:html_multi_themes` syntax highlighting formatter is not required but you get light/dark using it._

````elixir
Mix.install([
  {:mdex_gfm, "~> 0.1"}
])

markdown = """
- [x] Set up project
- [ ] Write docs

```elixir
spawn(fn -> send(current, {self(), 1 + 2}) end)
```
"""

MDEx.new(
  markdown: markdown,
  syntax_highlight: [
    engine: :lumis,
    opts: [
      formatter: {:html_multi_themes,
        themes: [light: "github_light", dark: "github_dark"],
        default_theme: "light-dark()"}
    ]
  ]
)
|> MDExGFM.attach()
|> MDEx.to_html!()
````

#### Sigils
```elixir
iex> import MDEx.Sigil
iex> ~MD[# Hello :smile:]HTML
"<h1>Hello 😄</h1>"
```

```elixir
iex> import MDEx.Sigil
iex> assigns = %{project: "MDEx"}
iex> ~MD[# {@project}]HEEX
%Phoenix.LiveView.Rendered{...}
```

```elixir
iex> import MDEx.Sigil
iex> ~MD[# Hello :smile:]
#MDEx.Document(3 nodes)<
├── 1 [heading] level: 1, setext: false
│   ├── 2 [text] literal: "Hello "
│   └── 3 [short_code] code: "smile", emoji: "😄"
>
```

#### Streaming
```elixir
iex> MDEx.new(streaming: true)
...> |> MDEx.Document.put_markdown("**Install")
...> |> MDEx.to_html!()
"<p><strong>Install</strong></p>"
```

## Examples and Guides

In docs you can find [Livebook examples](https://hexdocs.pm/mdex/gfm.html) covering options and usage, and [Guides](https://hexdocs.pm/mdex/plugins.html) for more info.

## Foundation

The library is built on top of:

- [comrak](https://crates.io/crates/comrak) - a fast Rust port of [GitHub's CommonMark parser](https://github.com/github/cmark-gfm)
- [ammonia](https://crates.io/crates/ammonia) for HTML Sanitization
- [lumis](https://crates.io/crates/lumis) for Syntax Highlighting

## Used By

- [00](https://github.com/technomancy-dev/00)
- [Algora](https://github.com/algora-io/algora)
- [Ash AI](https://github.com/ash-project/ash_ai)
- [BeaconCMS](https://github.com/BeaconCMS/beacon)
- [BeamLens Web](https://github.com/beamlens/beamlens_web)
- [Bonfire](https://github.com/bonfire-networks/bonfire-app)
- [Canada Navigator](https://github.com/canada-ca/navigator)
- [Conpipe](https://github.com/andyl/conpipe)
- [Exmeralda](https://github.com/bitcrowd/exmeralda)
- [Hexpm](https://github.com/hexpm/hexpm)
- [Jido](https://github.com/agentjido/jido)
- [Loomkin](https://github.com/bleuropa/loomkin)
- [Phoenix SEO](https://github.com/dbernheisel/phoenix_seo)
- [Phoenix Storybook](https://github.com/phenixdigital/phoenix_storybook)
- [Plural Console](https://github.com/pluralsh/console)
- [Prosody](https://github.com/halostatue/prosody)
- [Reposit](https://github.com/reposit-bot/reposit)
- [Sagents Live Debugger](https://github.com/sagents-ai/sagents_live_debugger)
- [Sayfa](https://github.com/furkanural/sayfa)
- [Tableau](https://github.com/elixir-tools/tableau)
- [Teiserver](https://github.com/beyond-all-reason/teiserver)
- [TermUI](https://github.com/pcharbon70/term_ui)
- [Tuist](https://github.com/tuist/tuist)
- [Valentine](https://github.com/maxneuvians/valentine)
- [Wraft](https://github.com/wraft/wraft)
- And [more...](https://github.com/search?q=lang%3Aelixir+%3Amdex&type=code)

_Are you using MDEx and want to list your project here? Please send a PR!_

## Sponsors

💜 **Support MDEx Development**

If you or your company find MDEx useful, please consider sponsoring its development.

➡️ **[GitHub Sponsors](https://github.com/sponsors/leandrocp)**

Your support helps maintain and improve MDEx for the entire Elixir community!

**Current and previous sponsors**

<a href="https://dockyard.com" target="_blank"><img src="assets/images/dockyard_logo.svg" width="200" alt="DockYard" /></a>

## Motivation

MDEx was born out of the necessity of parsing CommonMark files, to parse hundreds of files quickly, and to be easily extensible by consumers of the library.

- [earmark](https://hex.pm/packages/earmark) is extensible but [can't parse](https://github.com/RobertDober/earmark_parser/issues/126) all kinds of documents and is slow to convert hundreds of markdowns.
- [md](https://hex.pm/packages/md) is very extensible but the doc says "If one needs to perfectly parse the common markdown, Md is probably not the correct choice" and CommonMark was a requirement to parse many existing files.
- [markdown](https://hex.pm/packages/markdown) is not precompiled and has not received updates in a while.
- [cmark](https://hex.pm/packages/cmark) is a fast CommonMark parser but it requires compiling the C library, is hard to extend, and was archived on Apr 2024.

## Comparison

|Feature|MDEx|mdex_native|Earmark|md|cmark|erlang-markdown|
| --- | --- | --- | --- | --- | --- | --- |
|Active|✅|✅|⚠️⁵|✅|❌|✅|
|Pure Elixir|⚠️⁶|❌|✅|✅|❌|✅|
|Extensible|✅|❌|✅|✅|❌|⚠️⁹|
|Syntax Highlighting|✅|✅|❌|❌|❌|❌|
|Code Block Decorators|✅|❌|❌|❌|❌|❌|
|Streaming (fragments)|✅|❌|❌|❌|❌|❌|
|Phoenix HEEx components|✅|❌|❌|❌|❌|❌|
|MDX (JS)|❌|❌|❌|❌|❌|✅|
|Kramdown attributes|❌|❌|✅⁸|❌|❌|❌|
|EEx support|✅|❌|✅|❌|❌|❌|
|Custom syntax DSL|❌|❌|❌|✅|❌|❌|
|AST|✅|✅|✅|✅|❌|✅|
|AST to Markdown|✅|✅|⚠️⁷|❌|❌|❌|
|To HTML|✅|✅|✅|✅|✅|✅|
|To JSON|✅|❌|❌|❌|❌|❌|
|To XML|✅|✅|❌|❌|✅|❌|
|To Manpage|❌|❌|❌|❌|✅|❌|
|To LaTeX|❌|❌|❌|❌|✅|❌|
|To Quill Delta|✅|❌|❌|❌|❌|❌|
|To Slack|✅|❌|❌|❌|❌|❌|
|Emoji|✅|✅|❌|❌|❌|❌|
|GFM²|✅|✅|✅|❌|❌|✅|
|GLFM³|✅|✅|❌|❌|❌|❌|
|Discord⁴|⚠️¹|⚠️¹|❌|❌|❌|❌|

1. Partial support
2. GitHub Flavored Markdown
3. GitLab Flavored Markdown
4. Discord Flavored Markdown
5. Earmark is currently marked as retired
6. MDEx depends on [mdex_native](https://hex.pm/packages/mdex_native), which uses Rustler
7. Possible with [earmark_reversal](https://hex.pm/packages/earmark_reversal)
8. EarmarkParser supports Kramdown-style attributes
9. No custom plugin API

## Benchmark

A [benchmark](benchmark.exs) is available to compare existing libs:

```text
Name                      ips        average  deviation         median         99th %
cmark                 8681.81       0.115 ms    ±50.21%       0.111 ms       0.161 ms
mdex_native           6560.90       0.152 ms     ±7.26%       0.149 ms       0.199 ms
mdex                   395.26        2.53 ms    ±10.58%        2.49 ms        2.98 ms
md                     278.94        3.59 ms     ±8.62%        3.55 ms        3.91 ms
earmark                 69.86       14.32 ms     ±6.02%       14.24 ms       15.41 ms
erlang-markdown          4.80      208.43 ms     ±0.75%      208.19 ms      213.13 ms

Comparison:
cmark                 8681.81
mdex_native           6560.90 - 1.32x slower +0.0372 ms
mdex                   395.26 - 21.96x slower +2.41 ms
md                     278.94 - 31.12x slower +3.47 ms
earmark                 69.86 - 124.28x slower +14.20 ms
erlang-markdown          4.80 - 1809.56x slower +208.32 ms

Memory usage statistics:

Name                    average  deviation         median         99th %
cmark                0.00006 MB     ±0.00%     0.00006 MB     0.00006 MB
mdex_native          0.00008 MB     ±0.00%     0.00008 MB     0.00008 MB
mdex                    2.36 MB     ±0.00%        2.36 MB        2.36 MB
md                     11.77 MB     ±0.00%       11.77 MB       11.77 MB
earmark                 7.91 MB     ±0.00%        7.91 MB        7.91 MB
erlang-markdown       956.16 MB     ±0.00%      956.16 MB      956.16 MB

Comparison:
cmark                0.00006 MB
mdex_native          0.00008 MB - 1.38x memory usage +0.00002 MB
mdex                    2.36 MB - 38615.38x memory usage +2.36 MB
md                     11.77 MB - 192838.89x memory usage +11.77 MB
earmark                 7.91 MB - 129581.63x memory usage +7.91 MB
erlang-markdown       956.16 MB - 15665687.31x memory usage +956.16 MB
```

The most performance gain is using the `~MD` sigil to compile the Markdown instead of parsing it at runtime,
prefer using it when possible.

To finish, a friendly reminder that all libs have their own strengths and trade-offs so use the one that better suits your needs.

## Acknowledgements

- [comrak](https://crates.io/crates/comrak) crate for all the heavy work on parsing Markdown and rendering HTML
- [Floki](https://hex.pm/packages/floki) for the AST
- [Req](https://hex.pm/packages/req) for the pipeline API
- Logo based on [markdown-mark](https://github.com/dcurtis/markdown-mark)