mythos

KO | EN

gitlite — search

#javascript #python #llm #golang #go #deep-learning #linux #docker #nodejs #windows #cli #vue #pytorch #mcp

(★ 19)

Many Threads Operating System

mythos Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# About MyThOS

MyThOS is a micro-kernel for many-core processors with cache-coherent
shared memory. Its design aims for high-throughput operation for
highly-dynamic workloads. Examples are task-based high-performance
computing frameworks and service clouds with very fine-grained
application tasks.

Scalability to a high degree of parallelism is achieved as follows:
All kernel objects are allocated explicitly by the application from
freely partitioned memory pools in order to eliminate hidden
synchronisation. Any kernel object can be accessed concurrently from
any core in order to enable parallel system management. Concurrent
operations on kernel objects are serialised through delegation-based
object monitors. Granting and revoking access rights (object
capabilities) affects just a small set of rights in order to minimise
synchronisation.

* https://manythreads.github.io/mythos/

## Current State of Implementation

* Supports x86-64 aka amd64 on following platforms:
    * emulators like QEMU and Bochs (Gem5 still requires additional patches)
    * as co-kernel next Linux via Riken's IHK on multi-core machines
    * Intel Xeon Phi Knights Corner processor
* Starts an embedded init application on the first core and this
  application can create and configure further kernel objects.
* Creating and starting application threads (aka Execution Contexts)
  on other hardware threads (aka Scheduling Contexts).
* Implemented kernel objects: page maps, frames, capability maps,
  untyped memory for allocation of kernel objects,
  portals for outgoing system calls and messages,
  execution contexts for application threads.
* musl libc, llvm/clang libcxx, llvm libomp integration mostly usable
    * user-mode futex within the same capability space, uses the execution context's binary semaphore for notifications
    * `pthread_create`, `pthread_join`, `pthread_mutex` are working
* x87 FPU support including AVX and AVX512F (needs more testing though)

## Work in Progress

* actual memory management support for mmap
* more complete pthreads and openmp support
* basic benchmarks, testing, performance tuning
* endpoints for receiving incoming messages
* integrating a tracing framework for system-wide performance analysis

## Future Work

* access to the performance counters
* KNC: vector unit support (if needed by users)
* KNC: CPUHOT interrupt and related events (if needed by users)

# Quick Usage Guide

## Running Mythos in QEMU within docker

Qemu is using X11 to display the virtual display. Thus we need to activate X11 output for docker. Install and start XQuartz. In the XQuartz settings under security enable "Allow connections from network clients" and reboot the laptop, not just restart XQuartz neither just log out and in again.

```bash
git clone https://github.com/ManyThreads/mythos.git
cd mythos
git submodule update --init --recursive
docker build -f Dockerfile -t mythos --progress=plain .

xhost +localhost
XAUTH=$HOME/.Xauthority
docker run --rm -it \
    -e DISPLAY=host.docker.internal:0 \
    -e XAUTHORITY=$XAUTH \
    -v $XAUTH:$XAUTH \
    mythos
```

## Running in the QEMU virtual machine

* First, run `git submodule update --init --recursive` to pull the needed projects
* Run `3rdparty/mcconf/install-python-libs.sh` in order to install the build configuration tool. This requires python and pip (a widespread package installer for python).
* Run `3rdparty/install-libcxx.sh` to compile the musl libc and llvm/clang libstdc++
* Now you can run `make` in the root folder. This will assemble the source code into the subfolders `kernel-amd64`, `kernel-knc` and `host-knc`.
* Change into the `kernel-amd64` folder and run `make qemu`. This will compile the init application, the kernel, and finally boot the kernel image inside the qemu emulator. The debug and application output will be written to the console.
* Whenever you add or remove files from modules (the `mcconf.module` files in the `kernel` folder), rerun `make` in the root folder and then `make clean` in the target-specific folder.

## Running on the Intel XeonPhi KNC

In order to run MyThOS on an Intel XeonPhi Knights Corner processor, a recent version of Intel's MPSS software stack is needed. In addition a not too old C++ compiler is needed for the host tools. After loading the respective environment variables:
* First, run `git submodule update --init --recursive` to pull the needed projects
* Run `3rdparty/mcconf/install-python-libs.sh` in order to install the build configuration tool. This requires python and pip (a widespread package installer for python).
* Run `3rdparty/install-libcxx.sh` to compile the musl libc and llvm/clang libstdc++
* Now you can run `make` in the root folder. This will assemble the source code into the subfolders `kernel-amd64`, `kernel-knc` and `host-knc`.
* Change into the `host-knc` folder and run `make` in order to compile the `xmicterm` application. This is used to receive the debugging and log messages from the MyThOS kernel and applications.
* Then, change into the `kernel-knc` folder and run `make` in order to compile the init application and the kernel.
* In order to boot the system on the XeonPhi, you need root access (sudo). Run `make micrun` in order to stop any running coprocessor OS and boot MyThOS. If everything goes well, the script will start the `xmicterm` and you see the debug output. Stop it with CTRL-c and use `make micstop` to shut down the coprocessor. You can enable more detailed debug messages by editing `Makefile.user` and recompiling the kernel.

## Running as IHK cokernel next to CentOS GNU/Linux

* First, run `git submodule update --init --recursive` to pull the needed projects
* Run `3rdparty/mcconf/install-python-libs.sh` in order to install the build configuration tool. This requires python and pip (a widespread package installer for python).
* Run `3rdparty/install-libcxx.sh` to compile the musl libc and llvm/clang libstdc++
* Disable SELinux: `vim /etc/selinux/config` and change the file to SELINUX=disabled
* Reboot the machine: `sudo reboot`
* Install required packages: `sudo yum install cmake kernel-devel binutils-devel systemd-devel numactl-devel`
* Grant read permission to the System.map file of your kernel version: ``sudo chmod a+r /boot/System.map-`uname -r` ``
* Run `3rdparty/install-ihk.sh`
* Now you can run `make` in the root folder. This will assemble the source code into the subfolders `kernel-amd64`, `kernel-ihk`, `kernel-knc` and `host-knc`.
* `cd kernel-ihk`
* `make run`

## Running as IHK cokernel next to Ubuntu GNU/Linux

* `sudo apt install python python-pip python-virtualenv curl cmake`
* `git submodule update --init --recursive` to pull the needed projects.
* Run `3rdparty/mcconf/install-python-libs.sh` in order to install the build configuration tool.
* Run `3rdparty/install-libcxx.sh`
* `sudo apt install linux-headers-$(uname -r) binutils-dev libsystemd-dev libnuma-dev libiberty-dev libudev-dev`
* Perhaps you have to disable AppArmor by entering `sudo systemctl stop apparmor.service` and `sudo update-rc.d -f apparmor remove`
* Grant read permission to the System.map file of your kernel version: ``sudo chmod a+r /boot/System.map-`uname -r` ``
* Run `3rdparty/install-ihk.sh`
* Now you can run `make` in the root folder. This will assemble the source code into the subfolders `kernel-amd64`, `kernel-ihk`, `kernel-knc` and `host-knc`.
* `cd kernel-ihk`
* `make run`

## Problems and Solutions
* If `make qemu` fails with `qemu: could not load PC BIOS 'bios-256k.bin'` on CentOS 7. Fix: `yum install seabios-bin`.
* If `make qemu` fails with `qemu-system-x86_64: Unable to find CPU definition: max`, try to change "-cpu max" into "-cpu SandyBridge" in /mythos/kernel/build/emu-qemu-amd64/mcconf.module

# Acknowledgements

The MyThOS project was funded by the Federal Ministry of Education and Research (BMBF) under Grant No. 01IH13003 from October 2013 to September 2016. The grant was part of the 3rd HPC-Call of the Gauss Allianz.

The high-level kernel design is inspired by the many good ideas from seL4, Nova, Fiasco, and L4. The delegation-based monitors are inspired by delegation locks.

mythos

mythos Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

mythos

mythos Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README