KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #claude-code #codex #typescript #ai-agents #deepseek-harness #mcp #cli #rust

ffmpeg_image_transport

★ 43 Open GitHub ↗

image transport that uses libavcodec for compression

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ROS image transport for FFmpeg encoding

NOTE: This repo has reached End-Of-Life. All future development will
be on the [ROS2 repository here](https://github.com/berndpfrommer/ffmpeg_image_transport/).

This ROS image transport supports encoding/decoding with the FFMpeg
library. With this transport, you can encode h264 and h265, using
nvidia hardware acceleration when available.

Also have a look at [this repo tools to help with decoding](https://github.com/daniilidis-group/ffmpeg_image_transport_tools).

## Downloading

Create a catkin workspace (if not already there) and download the
ffmpeg image transport and other required packages:

mkdir -p catkin_ws/src
    cd catkin_ws/src
    git clone https://github.com/daniilidis-group/ffmpeg_image_transport_msgs.git
    git clone https://github.com/daniilidis-group/ffmpeg_image_transport.git

Optionally get the additional set of ffmpeg tools:

git clone git@github.com:daniilidis-group/ffmpeg_image_transport_tools.git

## Requirements: FFmpeg v4.0 or later, and a "reasonable" resolution

At some point this software worked on the following systems:

- Ubuntu 16.04 + ROS kinetic (not tested in a long while, maybe broken
  by now)
- Ubuntu 18.04 + ROS melodic
- Ubuntu 20.04 + ROS noetic (recently tested, should work with stock ffmpeg)

If you have ffmpeg 4.0 or later you may be able to compile against default header files:

sudo apt install ffmpeg

If you don't have ffmpeg 4.0 or later, or you hit compile errors
(supposedly even happens now on Ubuntu 18.04), 
[follow these instructions to compile ffmpeg from scratch](docs/compile_ffmpeg.md), and point the
transport to the right place:

ffmpeg_dir=<here the ffmpeg_dir>
    catkin config -DCMAKE_BUILD_TYPE=Release -DFFMPEG_LIB=${ffmpeg_dir}/build/lib -DFFMPEG_INC=${ffmpeg_dir}/build/include

Note that not all resolutions are supported in hardware accelerated
mode for h264 encoding/decoding, so you may get strange results if you
have cameras with resolutions other than those supported by
e.g. nvenc. Plain 1920x1200 works, probably also VGA, but other
resolutions need to be verified. Experiments indicate that the line
width must be a multiple of 32!

## Compiling

This should be easy as running the following inside your catkin workspace

catkin config -DCMAKE_BUILD_TYPE=Release

or, if you have your own version of ffmpeg installaed under ``${ffmpeg_dir`` (see above)

catkin config -DCMAKE_BUILD_TYPE=Release -DFFMPEG_LIB=${ffmpeg_dir}/build/lib -DFFMPEG_INC=${ffmpeg_dir}/build/include

then compile should be as easy as this:

catkin build ffmpeg_image_transport

## How to use

Usage should be transparent, but the ffmpeg image transport plugin
needs to be available on both the publishing and the subscribing
node. You also *need to add the location to your custom built ffmpeg
library* to ``LD_LIBRARY_PATH``, like so (change path to match yours):
```
export LD_LIBRARY_PATH=$HOME/catkin_ws/ffmpeg/build/lib:$LD_LIBRARY_PATH
```
Once done, you should get the following output when you run
``list_transports``:
```
rosrun image_transport list_transports 
Declared transports:
... some stuff ...
image_transport/ffmpeg
... some stuff ...

Details:
----------
... some stuff ...
"image_transport/ffmpeg"
 - Provided by package: ffmpeg_image_transport
 - Publisher: 
      This plugin encodes frame into ffmpeg compressed packets
    
 - Subscriber: 
      This plugin decodes frame from ffmpeg compressed packets
```

If it says something about "plugins not built", that means the
``LD_LIBRARY_PATH`` is not set correctly.

### republishing
Most ROS nodes that publish images use an image transport for that. If
not you can use a republish node to convert images into an ffmpeg
stream. The following line will start a republish node in ffmpeg format:

```
rosrun image_transport republish raw in:=/webcam/image_raw ffmpeg out:=/webcam/image_raw/ffmpeg _/webcam/image_raw/ffmpeg/encoding:=x264
```

### encoder parameters

The image transport has various parameters that you can configure
dynamically via ``rqt_reconfigure``, or specify at startup by
prepending the topic name. For instance to use the unaccelerated
``libx264`` encoding, start a republish node like this:
```
rosrun image_transport republish raw in:=/webcam/image_raw ffmpeg __name:=repub out:=~/image_raw  _/image_raw/ffmpeg/encoder:=libx264
```
Note: I had to ``__name`` the node to get the ros parameter show up under the topic name.

Parameters (see ffmpeg h264 documentation for explanation):

- ``encoder``: ffmpeg encoding to use. Allowed values: ``h265_nvenc``,
  ``hevc_nvenc``, ``libx264`` (no GPU). Default: ``hevc_nvenc``.
- ``profile``: ffmpeg profile. Allowed values: ``main``, ``main10``,
  ``rext``, ``high``. Default: ``main``.
- ``qmax``: maximum allowed quantization, controls quality. Range 0-63, default: 10.
- ``bit_rate``: bit rate in bits/sec. Range 10-2000000000, default: 8242880.
- ``gop_size``: gop size (number of frames): Range 1-128, default: 15
- ``measure_performance``: switch on performance debugging output. Default: false.
- ``performance_interval``: number of frames between performance printouts. Default: 175.

### encoder parameters

Usually the decoder infers from the received encoding type what decoding scheme to use.
You can overwrite it with a node-level parameter though:

- ``decoder_type``:
  - ``h264``: used if encoding is ``libx264`` or ``h264_nvenc``.
  - ``hevc_cuvid``: first tried if encoding is ``hevc_nvenc``.
  - ``hevc``: second tried if encoding is ``hevc_nvenc``.

For example to force ``hevc`` decoding, run a node like this:
```
rosrun image_transport republish ffmpeg in:=/repub/image_raw raw out:=~/image_raw _/ffmpeg/decoder_type:=hevc
```

## Trouble shooting:

On e.g. Ubuntu 16.04, you need a newer version of ffmpeg. If you get an error like this one,
you need a newer version of ffmpeg:

In member function ‘bool ffmpeg_image_transport::FFMPEGDecoder::decodePacket(const ConstPtr&)’:
    /home/pfrommer/Documents/birds/src/ffmpeg_image_transport/src/ffmpeg_decoder.cpp:104:47:
    error: ‘avcodec_send_packet’ was not declared in this scope
    int ret = avcodec_send_packet(ctx, &packet);

If the build still fails, make sure you start from scratch:

catkin clean ffmpeg_image_transport

## License

This software is issued under the Apache License Version 2.0.

ffmpeg_image_transport

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README