rebar3_osv

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 46 Open GitHub ↗

Turn an Erlang/OTP application into a Unikernel image

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

rebar3_osv
==========

Turn an Erlang/OTP application release into a OSv-based Unikernel image (with a few notable exceptions).

[![asciicast](https://asciinema.org/a/B7y3tS4tu8b50tpzTuCbU5SvF.png)](https://asciinema.org/a/B7y3tS4tu8b50tpzTuCbU5SvF)

Use
---

Add the plugin to your rebar config:

{plugins, [
        { rebar3_osv, ".*", {git, "https://github.com/OtoloNetworks/rebar3_osv", {tag, "OTP-21.0"}}}
    ]}.

You require an Erlang/OTP installation compatible with OSv. The simplest way to get that is to check out the pre-compiled version from Github:

# An example, use whatever location suites you - reference
    # it in the rebar.config!
    cd /usr/local/packages
    git clone --branch OTP-21.0 https://github.com/OtoloNetworks/otp-osv.git
 
You need to specifiy a couple of things for relx, such as where the
OSv specific ERTS is located. rebar3_osv includes a base OSv image,
but you can specify your own if required, as well as tweaking the
command line used to start applications, turn on 'verbose' output
etc. See below for how

{profiles,
     [
      {osv_build,
       [
        {relx, [
                {dev_mode, false}
               , {include_src, false}
               , {include_erts, "/usr/local/packages/otp-osv"}
               , {system_libs, "/usr/local/packages/otp-osv/lib"}
               ]
        {osv, [
               %% {osv_image, "overidden_base.img"}
               %% , {verbose, true}
               ]
        }]
      }
     ]}.

Then just call your plugin directly in an existing application:

$ rebar3 as osv_build osv generate
    ===> Fetching rebar3_osv
    ===> Compiling rebar3_osv
    ===> Verifying dependencies...
    ===> Compiling cowboy_example
    ===> Starting relx build process ...
    ===> Resolving OTP Applications from directories:
              /home/rickp/src/demo/cowboy_example/_build/osv_build/lib
              /usr/local/packages/osp-osv/usr/lib64/erlang
              /home/rickp/src/demo/cowboy_example/_build/osv_build/rel
    ===> Resolved cowboy_example-0.1.0
    ===> rendering builtin_hook_status hook to "/home/rickp/src/demo/cowboy_example/_build/osv_build/rel/cowboy_example/bin/hooks/builtin/status"
    ===> Including Erts from /usr/local/packages/otp-osv/usr/lib64/erlang
    ===> release successfully created!
    ===> OSv image built: /home/rickp/src/demo/cowboy_example/_build/osv_build/cowboy_example.img
    ===> Command Line: /start-otp.so /otp/releases/0.1.0/cowboy_example /otp/releases/0.1.0/vm.args /otp/releases/0.1.0/sys.config

To run the resulting image, I use a script like this:

#!/bin/sh
    IMAGE=_build/osv_build/cowboy_example.img
    qemu-system-x86_64 -m 6G -smp 2 --nographic -gdb tcp::1234,server,nowait \
        -device virtio-blk-pci,id=blk0,bootindex=0,drive=hd0,scsi=off \
        -drive file=$IMAGE,if=none,id=hd0,cache=none,aio=native \
        -netdev bridge,id=hn0,br=br0,helper=/usr/lib/qemu/qemu-bridge-helper \
        -device virtio-net-pci,mac=00:11:11:11:11:01,netdev=hn0,id=nic0 \
        -device virtio-rng-pci -enable-kvm -cpu host,+x2apic \
        -chardev stdio,mux=on,id=stdio,signal=off -mon chardev=stdio,mode=readline,default \
        -device isa-serial,chardev=stdio

The base OSv image included with rebar_osv has the monitoring
webservice built in on port 8000 of whatever IP address the VM gets by
DHCP.

Additional Info
===============

OSv is an open-source Unikernel for various virtual machines, designed
to exectute a single application on top of a hypervisor. Designed to
run Linux software unmodified, it seems an ideal choice to allow an
Erlang/OTP application to be packaged with the bare minimum needed to
run on a hypervisor.

There was some work done to allow Erlang to run, which involves some
patches to the Erlang ERTS - mainly because OSv does not support the
notion of fork(), plus a few of the os_mon items are not
supported. Some alterations have been made to run epmd as a thread
rather than a separate process. (We want to move to use the erlang
native epmd at somepoint). Along with this is the fact that ports will
not work, which means some of the os_mon applications do not work
properly.

I've taken the patches generated by the OSv contributors (myself
included) and applied them to an OTP tree which we maintain on
GitHub. We also maintain a compiled version so you do not need to
download and compile Erlang/OTP to get going with this.

Limitations for Erlang under OSv
================================

1. rebar3_osv must be run on Linux x86_64, as the release is uploaded by running the image under kvm/qemu and transfering in the release files.
 2. Needs to use the modified OTP (currently based on 21.0) and this needs to match that installed on the system
 3. Ports are not supported
 4. NIFs are supported but may need compiling -fpie (the default on many distributions)

Building your own OSv Image
===========================

Building your own OSv image is relatively straightforward. Checkout
the OSv code from https://github.com/cloudius-systems/osv and update
the submodules.

Then its just a case of compiling:

$ scripts/build -j4 fs_size_mb=200 image=cloud-init,httpserver,openssl,libz,ncurses

If you want a debug image, use:

$ scripts/build -j4 mode=debug fs_size_mb=200 image=cloud-init,httpserver,openssl,libz,ncurses

The fs_size_mb sets the size of the image FS, so if you need more
storage, increase the size. The file you need is in the
build/*/usr.img - you can use either the debug or the release image,
depending on what you need.

Once built, you can reference this image from the rebar3 configuration using the osv_image variable:

{osv_image, "/home/rickp/osv/build/debug.x64/usr.img"}
               %% , {verbose, true}

How is the final image built
============================

The way the image is built is as follows:

1. A release is generated using the OSv compatible ERTS. As this is
generated using the host Erlang install, this probably needs to match
the OSv specific ERTS (otherwise the boot scripts may be worng).

2. The osv_image is copied to a new file (called after the application
name) and started with the command that starts the cpio application.

3. The release built in stepa 1 is CPIO-ed into the running image,
along with a small starter program which takes the place of 'erl' -
sets the appropriate environment variables, then loads erlexec and
invokes main() to start erlang.

rebar3_osv

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README