KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #deepseek-harness #typescript #ai-agent #codex #developer-tools #claude-code #windows #mcp #python #desktop-app

CrackNex

★ 35 Open GitHub ↗

Official code of ICRA 2024 paper: CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# [ICRA2024] CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections
[Zhen Yao](https://scholar.google.com/citations?user=8-IhrB0AAAAJ&hl=en&oi=sra), [Jiawei Xu](https://scholar.google.com/citations?user=b3XkcPkAAAAJ&hl=en&oi=ao), Shuhang Hou, [Mooi Choo Chuah](https://scholar.google.com/citations?hl=en&user=SZBKvksAAAAJ).

The codebase contains the official code of our paper [CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections](https://arxiv.org/abs/2403.03063), ICRA 2024.

## Abstract
Routine visual inspections of concrete structures are imperative for upholding the safety and integrity of critical infrastructure. Such visual inspections sometimes happen under low-light conditions, e.g., checking for bridge health. Crack segmentation under such conditions is challenging due to the poor contrast between cracks and their surroundings. However, most deep learning methods are designed for well-illuminated crack images and hence their performance drops dramatically in low-light scenes. In addition, conventional approaches require many annotated low-light crack images which is time-consuming. In this paper, we address these challenges by proposing CrackNex, a framework that utilizes reflectance information based on Retinex Theory to learn a unified illumination-invariant representation. Furthermore,  we utilize few-shot segmentation to solve the inefficient training data problem. In CrackNex, both a support prototype and a reflectance prototype are extracted from the support set. Then, a prototype fusion module is designed to integrate the features from both prototypes. CrackNex outperforms the SOTA methods on multiple datasets. Additionally, we present the first benchmark dataset, LCSD, for low-light crack segmentation. LCSD consists of 102 well-illuminated crack images and 41 low-light crack images.

## LCSD Dataset
We have established a crack segmentation dataset, LCSD. It comprises 102 normal light crack images and 41 low-light crack images with fine-grained annotations.

You can download the datasets from [the link](https://huggingface.co/datasets/Chrisathy/LCSD).

## Requirements

- Python 3.7
- PyTorch 1.9.1
- cuda 11.1
- pillow 8.4.0
- opencv
- sklearn

Conda environment settings:
```bash
conda create -n cracknex python=3.7 -y
conda activate cracknex

pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html
conda install -c conda-forge cudatoolkit-dev==11.1.1
pip install opencv-python==4.5.2.54
pip install scikit-learn
pip install tqdm
pip install pillow==8.4.0
```

## Data preparation
**Pretrained model:** [ResNet-50&ResNet-101](https://huggingface.co/Chrisathy/CrackNex)

### File Organization

To prepare the LCSD datasets, you need to download all images and labels from the above link. Put all images in ``JPEGImages/`` subfolder and all labels in ``SegmentationClassAug/`` subfolder, and two split ``txt`` file into ``ImageSets/`` subfolder. We also provide a zip file of our processed [LCSD](https://huggingface.co/Chrisathy/CrackNex) dataset.
```
    ../                         # parent directory
    ├── ./CrackNex              # current (project) directory
    |   ├── codes               # various codes
    |   └── ./pretrained        # pretrained model directory
    |            ├── resnet50.pth
    |            └── resnet101.pth
    └── Datasets_CrackNex/
        ├── LCSD/            
        │   ├── ImageSets/
        │   ├── JPEGImages/
        │   └── SegmentationClassAug/
```

## Run the code

You can adapt the scripts of `train.sh` and `test.sh` to train and evaluate your models.

You may change the ``backbone`` from ``resnet50`` to ``resnet101`` or change the ``shot`` from ``1`` to ``5`` for other settings.

```
bash train.sh
```

```
bash test.sh
```

Remember to change the path to dataset and checkpoint.

## Evaluation and Trained Models

### LCSD

### Visualization in LCSD

### Visualization of videos in the wild

 <img src="docs/vis_wild.jpg">

## Acknowledgment
This codebase is built based on [SSP's baseline code](https://github.com/fanq15/SSP/). We thank SSP and other FSS works for their great contributions.

## Citation
```bibtex
@inproceedings{yao2024cracknex,
  title={Cracknex: a few-shot low-light crack segmentation model based on retinex theory for uav inspections},
  author={Yao, Zhen and Xu, Jiawei and Hou, Shuhang and Chuah, Mooi Choo},
  booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={11155--11162},
  year={2024},
  organization={IEEE}
}
```

CrackNex

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README