make_cabana_route

KO | EN

gitlite — search

#javascript #python #typescript #llm #golang #machine-learning #rust #docker #nodejs #java #windows #awesome #cli #mcp

make_cabana_route

(★ 17)

Convert local CAN log files to "routes" suitable for Cabana

make_cabana_route Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Make Cabana Route

Utility that takes CSV formatted CAN log files and (optionally) accompanying videos, convert them to "routes" that can be opened in comma.ai's Cabana for CAN reverse engineering.

## Motivation

The company [comma.ai](https://comma.ai/) has an open source CAN log viewer and reverse engineering tool called [Cabana](https://docs.comma.ai/tools/cabana/README.html). This is actually technically "[New Cabana](https://blog.comma.ai/090release/#new-cabana)", a Qt desktop application distributed on GitHub in their [openpilot repo](https://github.com/commaai/openpilot/tree/master/tools/cabana).

Although Cabana is built to work with comma.ai's unique hardware and infrastructure, it has a very nice workflow for finding signals in CAN captures and creating DBC files to document them. One of its features is synchronising a video with the CAN log, which can make it easy to correlate events as they happen.

![Screenshot of the Cabana UI](https://raw.githubusercontent.com/projectgus/make_cabana_route/main/screenshots/cabana-ui.webp)

Normally, Cabana integrates directly with other comma.ai tools and their cloud "route" storage. The `make_cabana_route` tool is for when you have some random CAN log files and corresponding videos, and you want to load these into Cabana instead.

For background of how I personally came to want this, see [this blog post](https://projectgus.com//2023/10/kona-can-decoding/).

## IMPORTANT

This **isn't** a way to use comma.ai's driver assistance software without buying their hardware. It's a way to use the Cabana workflow for otherwise unrelated projects.

If you think you've found an issue with Cabana using a route generated by this tool, consider carefully whether it **might be an issue with this tool** before reporting it to comma.ai. This tool is entirely unofficial and unsupported by comma.ai.

## Terminology

* In comma.ai's world, a "route" is a combination of a detailed vehicle log (including CAN messages) and one or more video streams that cover the same period in time.

## Prerequisites

* CAN log files in the "gvret/SavvyCAN" CSV format, as supported by [SavvyCAN](https://savvycan.com/). For examples, [see here](https://github.com/projectgus/hyundai-kona-ev-can-logs/). As far as I know there's no official written specification for this format. PRs to add more CAN log input formats would be welcome.
* (Optional) video file that matches the CAN log.
* Currently this tool is only tested on Linux. It should work out of the box on macOS, but will require at least some small patch to work on Windows. PRs welcome!

## Building

* You'll need to [install Rust and its build tool Cargo](https://doc.rust-lang.org/book/ch01-01-installation.html).
* Clone the code from GitHub.
* Run `git submodule update --init` to get the "Cereal" submodule.
* `cargo build --release` to build an optimised version.
* If the build succeeds, you can find the binary at `target/release/make_cabana_route` (or use `cargo run --release -- [...arguments...]`).

You'll also need to build the Cabana program, I don't believe comma.ai distribute built versions of it. It is built as part of [openpilot](https://github.com/commaai/openpilot/tree/master/tools#openpilot-tools).

## Usage

### YAML file

Create a YAML file with the details of your CAN log inputs. Here's an example:

```yml
- car: Kona Electric 2021
  fingerprint: pcan-ccan-drive-modes-abs-traction
  logfile: 221217-3-2021-pcan-ccan-drive-modes-abs-traction.csv
  video: 221217-3-2021-pcan-ccan-drive-modes-abs-traction.mp4
  sync:
    video_s: 1.291
    log_us: 1469445700
```

* The YAML file can contain as many of these entries as you like.
* `car` and `fingerprint` are arbitrary strings, `fingerprint` is displayed above the video so you can put some identifying information here (the "route" itself is identified by its timestamp, so it's not very descriptive by itself.)
* `logfile` is the path to the CSV log file (relative to the YAML file).
* `video` is the path to the video file (relative to the YAML file). This is optional, if your log has no video then leave it and the `sync` key out.
* The `sync` values synchronise the video to the CAN log. Provide a timestamp for the video (in seconds) that corresponds to a particular microsecond timestamp in the log file. It's best to do this by recording a shot of the screen where the log is being captured at the start of each video, then you can step through frame by frame ([mpv](https://mpv.io/) and [VLC](vlc.org/) both let you do this) and find the exact timestamp of the frame when a particular CAN message timestamp first appears.

An additional optional key (not shown in the example) is `route_timestamp` that allows you to manually set the timestamp used to identify the route. If not found, `make_cabana_route` will use the video modification time (if there is a video file) or the CSV log file modification time otherwise.

### Run make_cabana_route

Run the tool as follows:

```
make_cabana_route --yaml-path logs.yml --data-dir ./data_dir
```

The `--data-dir` argument provides the path to a "data directory" (created if missing) for Cabana. The route is created there as a series of sub-directories with names like `2022-12-17--09-35-30--0`, `2022-12-17--09-35-30--1`, `2022-12-17--09-35-30--2`, etc.

The first part of each sub-directory name (before `--`) is the timestamp that uniquely identifies the "route" to Cabana. The final part (after the `--`) is the "segment" index, comma.ai splits each route into segments (presumably to save bandwidth when streaming them from their server).

Processing logs is pretty slow as it includes transcoding the video content.

You can also specify a filter on the command line in order to only process some logs:

```
make_cabana_route --yaml-path logs.yml --data-dir ./data_dir pcan-ccan-drive-modes
```

If a filter string like this is provided, only entries in the YAML file which contain this string in either the `fingerprint`, `logfile`, or `video` fields will be processed.

### Run Cabana

To run Cabana and load a route, use a command line such as:

```
cabana --data_dir /path/to/data_dir --dbc optional/path/to/this.dbc "2022-12-17--09-35-30"
```

The `--data_dir` path is the same one as above. The `--dbc` argument is an optional one to open the Route in Cabana with a DBC file associated. Finally, the route name is the name of one of the directories inside the data directory, minus the final `--0` for the segment number.

If the route doesn't contain any associated video, you need to add `--no-vipc` on the command line as well, or the route won't load.

Cabana should open immediately and start playing back the "route".

(Note, Cabana also has a UI for opening local routes like this, but I find it very difficult to get the right settings there compared to the command line.)

### Shortcuts

To make loading Cabana easier, `make_cabana_route` also generates some shell script segments in the `data_dir`. These are named the same as the CSV log file, but with a `.sh` extension.

To run one, first ensure that `cabana` is on the `PATH`. Then you can run the script directly, like this:

```
path/to/data_dir/221217-2-2021-pcan-bcan-drive-modes.sh --dbc dbc/hyundai_kona.dbc
```

Additional arguments (like `--dbc` here) are passed to cabana as well. It's not necessary to manually pass `--no-vipc` here, the shell script will add it if/when it's needed.

## Support

*As mentioned above, please don't report issues to comma.ai until you're certain they're not issues introduced by this tool.*

* PRs are welcome. If they seem well-written and don't make the project harder to maintain then I'm likely to merge them.
* Like every random thing published on GitHub, this project otherwise clearly comes without warranty!
* I'm not sure how much I intend to support this project into the future. There's a lot of "polish" that could be added, but at this stage it's useful enough for my purposes and I need to spend time on [the project it was intended to assist](https://www.projectgus.com/tag/ev-conversion-project.html).
* Nevertheless, feel free to open issues if you run into them. However, please don't be surprised if they don't get fixed (at least not by me).

## Known Limitations

* Video transcoding is CPU only. This boils down to two things: the libavcodec API for hardware-assisted transcoding is relatively complex (as it's a complex concept), and [rust-ffmpeg](https://github.com/meh/rust-ffmpeg) does not appear to fully support it.

make_cabana_route

make_cabana_route Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

make_cabana_route

make_cabana_route Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README