Franka-vision-grasping

KO | EN

gitlite — search

#javascript #react #ai #typescript #llm #golang #machine-learning #linux #macos #nodejs #windows #cli #awesome-list #mcp

Franka-vision-grasping

(★ 20)

Vision grasping task on Franka arm with RealSense D435 and Jetson AGX Xavier

Franka-vision-grasping Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Franka vision grasping

This repo is for grasping task on the following setup.

**Arm**: Franka Emika Panda

**Vision Sensor**: Intel RealSense D435

**Computing Platform**: Jetson AGX Xavier

## Requirements

* [Jetpack 4.4 with Realtime Kernel](./docs/Jetpack4.4-with-realtime-kernel.md)
* [Set user task priority for realtime kernel](https://frankaemika.github.io/docs/installation_linux.html)
* [libfranka version == 0.7.1](https://frankaemika.github.io/docs/installation_linux.html)
* [opencv with contrib (for calibration with aruco marker)](https://github.com/AastaNV/JEP/blob/master/script/install_opencv4.3.0_Jetson.sh)
* [jetson-inference](https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md)
* [librealsense2](https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md)
* [pyrealsense2 (build from source)](https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python)
* [pybind11](https://github.com/pybind/pybind11)
* Eigen3
* Python3
  * numpy
  * yaml
  * matplotlib (for calibration visualization)
  * pytransform3d (for calibration visualization)
  
## TODO

* complete README (currently the usage info is not fully correct and it requires additional setup related with Jetson-inference)
* Add yolov5 example

> The requirement info is correct

## Usage

### DetectNet trained on COCO-bottle

```bash
python3 vision_grasping.py --network=coco-bottle
```

### Self-trained ONNX model

```bash
python3 vision_grasping.py --model=<path-to-your-model> --labels=<path-to-your-label> --input-blob=input_0 --output-cvg=scores --output-bbox=boxes
```

## Troubleshooting

* After flash the Jetson Xavier with JetPack 4.4 and realtime kernel patch, `sudo jetson_clocks` may not able to turn on the fan, which will lead to severe heat accumulation. You can use `echo 255 | sudo tee /sys/devices/pwm-fan/target-pwm` to manually set the fan at maximum speed.

> Reference: https://forums.developer.nvidia.com/t/fan-management-xavier/70166/9

Franka-vision-grasping

Franka-vision-grasping Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Franka-vision-grasping

Franka-vision-grasping Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README