KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #ai-agents #dsh-plugin #codex #developer-tools #claude-code #windows #desktop-app #typescript #rust #ai-agent #cli

FoL

★ 46 Open GitHub ↗

Official repository for the AAAI 2025 paper "Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition".

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition

This is the official repository for the AAAI 2025 paper "FoL" available at [AAAI Paper Page](https://ojs.aaai.org/index.php/AAAI/article/view/32811). In addition, our paper and its extensive supplementary materials can be found on [arXiv](https://arxiv.org/abs/2504.09881).

## 📑 Contents

- [Summary](#summary) · [Live Demo](#live-demo) · [Setup](#setup)
- [Torch Hub](#torch-hub) · [Pretrained Weights](#pretrained-weights)
- [Evaluation](#evaluation) · [Training](#training) · [Visualization](#visualization)
- [Performance](#performance) · [Related Work](#related-work) · [Citation](#citation)

## 📝 Summary

We introduce Focus on Local **(FoL)**, a two-stage Visual Place Recognition (VPR) approach that enhances image retrieval and re-ranking by identifying and leveraging **reliable discriminative local regions**.

## 🤗 Live Demo

Try FoL in our [Hugging Face Space](https://huggingface.co/spaces/shunpeng/fol-visual-place-recognition). Upload a query image and database image, or select a provided pair. The demo visualizes reliable discriminative regions, local correspondences, and geometrically verified matches, while also reporting the global cosine similarity, local match count, and RANSAC inlier count.

## 🛠️ Setup & Requirements
**Quick install:**
```bash
# create and activate conda env
conda create -n fol python=3.9.19 -y
conda activate fol

# install dependencies
pip install -r requirements.txt
```
**Key dependencies:**
```
torch==2.0.0
torchvision==0.15.1
faiss-gpu==1.7.2
scikit-learn==1.3.0
numpy==1.26.4
opencv-python==4.10.0.84
```
> **Note — reproducibility:** The reranking step is sensitive to small numerical differences across `faiss-gpu`, `torch`, and `numpy` versions. Use the exact versions (in [requirements.txt](https://github.com/chenshunpeng/FoL/blob/main/requirements.txt)) to match paper results.

## ⚡ Quick Start with Torch Hub

To quickly use our model without cloning the repository, you can easily load it via Torch Hub:

```python
import torch

# Load the default FoL model (ViT-L)
model = torch.hub.load("chenshunpeng/FoL", "FoL", pretrained=True, trust_repo=True)

# Alternatively, load specific backbones explicitly
model_vitl = torch.hub.load("chenshunpeng/FoL", "fol_vitl14", pretrained=True, trust_repo=True)
model_vitb = torch.hub.load("chenshunpeng/FoL", "fol_vitb14", pretrained=True, trust_repo=True)
```

## ⬇️ Download Pretrained Weights

You can download our pretrained FoL model either via Google Drive or directly from Hugging Face.

- **Google Drive:** [FoL (ViT-L)](https://drive.google.com/file/d/1-7LE_4Q0zL3S8lGVEH0Ob1NCFXq4KfJ8/view?usp=sharing) | [FoL (ViT-B)](https://drive.google.com/file/d/1Z05ZLFliQXOPJMH1YPdXqYjzC15-0nam/view?usp=sharing) | [All Models Folder](https://drive.google.com/drive/folders/1d3uEHdnzgWbGnj2g1ffLzLLI3pKuV7Vz?usp=sharing)

- **Hugging Face Hub:** Install the client (`pip install huggingface_hub`) and download weights directly in:
  ```python
  from huggingface_hub import hf_hub_download
  # Download ViT-L and ViT-B respectively
  fol_large_path = hf_hub_download(repo_id="shunpeng/FoL", filename="FoL_large.pth")
  fol_base_path = hf_hub_download(repo_id="shunpeng/FoL", filename="FoL_base.pth")
  ```
---

## 📊 Evaluation

### Recommended Dataset Layout

FoL expects standard datasets to contain `database/` and `queries/` under `<dataset_name>/images/test/`. SF-XL and SVOX use shared database/gallery images with multiple query subsets:

```text
datasets/
├── pitts30k/
│   └── images/test/
│       ├── database/
│       └── queries/
├── amstertime/
│   └── images/test/
│       ├── database/
│       └── queries/
├── <other_dataset_name>/
│   └── images/test/
│       ├── database/
│       └── queries/
├── SF_XL/
│   └── test/
│       ├── database/
│       ├── queries_v1/
│       ├── queries_v2/
│       ├── queries_night/
│       └── queries_occlusion/
└── svox/
    └── images/test/
        ├── gallery/
        ├── queries/
        ├── queries_night/
        ├── queries_overcast/
        ├── queries_rain/
        ├── queries_snow/
        └── queries_sun/
```

Assuming you have your datasets under `./datasets/` and the ViT-L checkpoint at `./weights/FoL_large.pth`:

```bash
python eval.py --eval_datasets_folder=./datasets/ --dataset_names pitts30k amstertime --resume=./weights/FoL_large.pth
```

### SF-XL, SVOX, and Exact Low-Memory Reranking

`SF_XL` and `SVOX` evaluate multiple query subsets against one shared database. `--dataset_names SF_XL` reports `v1`, `v2`, `night`, and `occlusion`, while `--dataset_names SVOX` reports `SVOX`, `night`, `overcast`, `rain`, `snow`, and `sun`; the shared database/gallery is extracted and searched only once.

For memory-limited evaluation, add `--efficient_ram_testing`. It preserves float32 features, exact chunked `IndexFlatL2` retrieval, and the original local reranking, while caching only query and retrieved-candidate local features on SSD. `--efficient_ram_max_cache_gib` and `--efficient_ram_min_free_gib` default to 500 GB, and reranking depth follows `--recall_values`.

For [SF-XL](https://github.com/gmberton/cosplace) at 322×322, regular mode would require about $\color{red}{\mathbf{4816.54\ GB}}$ RAM for database local features and $\color{red}{\mathbf{88.30\ GB}}$ for database global features. Low-memory top-100 reranking uses at most about $\color{red}{\mathbf{371.03\ GB}}$ temporary SSD cache ($\color{red}{\mathbf{3.67\ GB}}$ query local + $\color{red}{\mathbf{367.36\ GB}}$ candidate local) and only a few GB extra CPU RAM.

```bash
# SF-XL: four query subsets
python eval.py --eval_datasets_folder=./datasets/ --dataset_names SF_XL --resume=./weights/FoL_large.pth --resize 322 322 --efficient_ram_testing --efficient_ram_cache_dir=/data_nvme/VPR/FoL --efficient_ram_max_cache_gib=500 --efficient_ram_min_free_gib=500

# SVOX: six query subsets
python eval.py --eval_datasets_folder=./datasets/ --dataset_names SVOX --resume=./weights/FoL_large.pth --resize 322 322 --efficient_ram_testing --efficient_ram_cache_dir=/data_nvme/VPR/FoL --efficient_ram_max_cache_gib=500 --efficient_ram_min_free_gib=500
```

## 🚀 Train

```bash
python train.py --eval_datasets_folder=./datasets/ --eval_dataset_name pitts30k --epochs_num=8 --train_batch_size=60 --lr=6e-5 --optim=adamw --resize 322 322 --save_dir train_log/
```

## 🎨 Visualization [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1CxchBdFYxzwtCf5UOUgjMt8FxKMo2A-5?usp=sharing)

A quick way to explore FoL is via our [Colab demo](https://colab.research.google.com/drive/1CxchBdFYxzwtCf5UOUgjMt8FxKMo2A-5?usp=sharing) and the included [notebook](https://github.com/chenshunpeng/FoL/blob/main/FoL_demo.ipynb).

We provide a standalone visualization script, [FoL_visualize_match.py](https://github.com/chenshunpeng/FoL/blob/main/visualize_pairs/FoL_visualize_match.py), to demonstrate keypoint matching guided by <b><span style="color:red">Discriminative Region Guidance</span></b> with **optional** geometric verification.

> **🌟 Extension Work Notice:**
> The visualization of local feature matching in the reranking stage shown here corresponds to the extended analysis in our journal version: [**FoL++ (Region Matters)**](https://arxiv.org/abs/2604.22390).

You can run it directly with the default sample images included in the repository:

```bash
cd visualize_pairs
python FoL_visualize_match.py --resume ../weights/FoL_large.pth
```

To visualize matching on your own custom image pair, simply specify the image paths:

```bash
cd visualize_pairs
python FoL_visualize_match.py --image_path0 path/to/your/query_image.jpg --image_path1 path/to/your/database_image.jpg --resume /weights/FoL_large.pth --device cuda
```

Below we show a comparison of matching results:

<table>
<tr>
<td>
<img src="image/foL_visualize.jpg" width="100%">
<br>
<em>Raw matching results (FoL + similarity filtering)</em>
</td>

<td>
<img src="image/foL_visualize_ransac.jpg" width="100%">
<br>
<em>Geometry-consistent matches after RANSAC (outliers removed)</em>
</td>
</tr>
</table>
Geometric verification effectively removes outliers and improves match consistency (Matches: 29 → 21).

## 📈 Performance
 
This table lists R1, R5, and R10 for FoL across common VPR datasets, showing both **FoL-global** and **FoL-reranking** results. The best R1 value within each dataset row is typeset in $\color{red}{\mathbf{bold}}$. All results are reported for [ViT-L](https://drive.google.com/file/d/1-7LE_4Q0zL3S8lGVEH0Ob1NCFXq4KfJ8/view?usp=sharing) at 504×504.

For Nordland variants: `Nordland*` uses 2,760 summer queries against a 27,592-image winter database, while `Nordland**` uses the full 27,592 winter queries against a 27,592-image summer database.

<table style="width:100%; border-collapse: collapse; font-size: 12px;">
  <thead>
    <tr>
      <th rowspan="2" style="text-align:left;">Dataset</th>
      <th>global</th>
      <th>re-ranking</th>
      <th rowspan="2" style="text-align:left;">Dataset</th>
      <th>global</th>
      <th>re-ranking</th>
    </tr>
    <tr>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
    </tr>
  </thead>
  <tbody>
    <tr><td>Pitts250k-test</td><td>96.5 99.1 99.5</td><td>$\color{red}{\mathbf{97.0}}$ 99.2 99.5</td><td>MSLS-val</td><td>93.1 96.9 97.4</td><td>$\color{red}{\mathbf{93.5}}$ 96.9 97.6</td></tr>
    <tr><td>MSLS-challenge</td><td>78.7 90.8 93.0</td><td>$\color{red}{\mathbf{80.0}}$ 90.9 93.0</td><td>Tokyo24/7</td><td>96.2 98.7 98.7</td><td>$\color{red}{\mathbf{98.4}}$ 99.1 99.4</td></tr>
    <tr><td>Pitts30k</td><td>93.9 97.2 98.1</td><td>$\color{red}{\mathbf{94.5}}$ 97.4 98.2</td><td>Sped</td><td>$\color{red}{\mathbf{92.1}}$ 96.5 98.0</td><td>91.8 96.5 97.4</td></tr>
    <tr><td>Amstertime</td><td>64.6 84.3 88.2</td><td>$\color{red}{\mathbf{70.1}}$ 89.0 91.8</td><td>Eynsham</td><td>91.7 95.3 96.2</td><td>$\color{red}{\mathbf{92.4}}$ 95.8 96.6</td></tr>
    <tr><td>Nordland*</td><td>78.3 90.8 94.0</td><td>$\color{red}{\mathbf{85.8}}$ 94.9 96.8</td><td>Nordland**</td><td>87.8 94.5 96.4</td><td>$\color{red}{\mathbf{92.6}}$ 96.9 98.0</td></tr>
    <tr><td><kbd>SF-XL</kbd>-testv1</td><td>91.5 94.8 95.6</td><td>$\color{red}{\mathbf{95.0}}$ 96.5 96.8</td><td><kbd>SF-XL</kbd>-testv2</td><td>$\color{red}{\mathbf{95.0}}$ 98.0 98.3</td><td>$\color{red}{\mathbf{95.0}}$ 97.8 98.5</td></tr>
    <tr><td><kbd>SF-XL</kbd>-night</td><td>53.4 65.9 71.7</td><td>$\color{red}{\mathbf{60.5}}$ 72.8 75.8</td><td><kbd>SF-XL</kbd>-occlusion</td><td>51.3 65.8 73.7</td><td>$\color{red}{\mathbf{61.8}}$ 77.6 77.6</td></tr>
    <tr><td><kbd>SVOX</kbd></td><td>98.4 99.4 99.6</td><td>$\color{red}{\mathbf{98.9}}$ 99.6 99.7</td><td><kbd>SVOX</kbd>-sun</td><td>98.1 99.4 99.5</td><td>$\color{red}{\mathbf{98.8}}$ 99.8 99.9</td></tr>
    <tr><td><kbd>SVOX</kbd>-night</td><td>98.3 99.6 99.6</td><td>$\color{red}{\mathbf{98.8}}$ 99.8 99.9</td><td><kbd>SVOX</kbd>-snow</td><td>99.1 99.7 99.8</td><td>$\color{red}{\mathbf{99.3}}$ 99.8 99.9</td></tr>
    <tr><td><kbd>SVOX</kbd>-overcast</td><td>97.9 99.2 99.3</td><td>$\color{red}{\mathbf{98.3}}$ 99.3 99.7</td><td><kbd>SVOX</kbd>-rain</td><td>96.5 99.6 99.7</td><td>$\color{red}{\mathbf{98.2}}$ 99.9 99.9</td></tr>
  </tbody>
</table>

### ✨ Additional Results at 322×322

The following table reports results at 322×322 on six datasets for both backbones.

<table style="width:100%; border-collapse: collapse; font-size: 12px;">
  <thead>
    <tr>
      <th rowspan="3" style="text-align:left;">Dataset</th>
      <th colspan="2"><a href="https://drive.google.com/file/d/1-7LE_4Q0zL3S8lGVEH0Ob1NCFXq4KfJ8/view?usp=sharing">ViT-L</a></th>
      <th colspan="2"><a href="https://drive.google.com/file/d/1Z05ZLFliQXOPJMH1YPdXqYjzC15-0nam/view?usp=sharing">ViT-B</a></th>
    </tr>
    <tr>
      <th>global</th>
      <th>re-ranking</th>
      <th>global</th>
      <th>re-ranking</th>
    </tr>
    <tr>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
      <th>R@1 @5 @10</th>
    </tr>
  </thead>
  <tbody>
    <tr><td>Pitts30k-test</td><td>93.6 96.9 97.9</td><td>$\color{red}{\mathbf{93.9}}$ 96.9 98.1</td><td>92.1 96.4 97.6</td><td>$\color{red}{\mathbf{93.1}}$ 96.9 97.7</td></tr>
    <tr><td>MSLS-val</td><td>$\color{red}{\mathbf{92.8}}$ 96.9 97.2</td><td>90.1 95.7 96.9</td><td>91.1 95.7 96.4</td><td>$\color{red}{\mathbf{91.5}}$ 96.2 96.8</td></tr>
    <tr><td>Nordland**</td><td>83.8 92.6 95.1</td><td>$\color{red}{\mathbf{87.9}}$ 94.8 96.6</td><td>72.7 85.5 89.6</td><td>$\color{red}{\mathbf{85.4}}$ 92.7 94.8</td></tr>
    <tr><td>Tokyo24/7</td><td>96.5 98.1 98.4</td><td>$\color{red}{\mathbf{97.1}}$ 97.8 98.7</td><td>94.6 96.5 96.8</td><td>$\color{red}{\mathbf{97.5}}$ 98.1 98.4</td></tr>
    <tr><td>Nordland*</td><td>74.1 88.8 92.2</td><td>$\color{red}{\mathbf{80.8}}$ 92.0 94.7</td><td>62.5 80.3 85.0</td><td>$\color{red}{\mathbf{78.2}}$ 90.2 92.9</td></tr>
    <tr><td>Eynsham</td><td>91.5 95.1 96.1</td><td>$\color{red}{\mathbf{91.7}}$ 95.4 96.4</td><td>$\color{red}{\mathbf{91.3}}$ 95.2 96.0</td><td>$\color{red}{\mathbf{91.3}}$ 95.1 96.1</td></tr>
    <tr><td><kbd>SF-XL</kbd>-testv1</td><td>90.4 93.8 94.7</td><td>$\color{red}{\mathbf{93.4}}$ 95.6 96.1</td><td>86.1 91.4 92.7</td><td>$\color{red}{\mathbf{90.0}}$ 93.4 94.4</td></tr>
    <tr><td><kbd>SF-XL</kbd>-testv2</td><td>$\color{red}{\mathbf{94.1}}$ 98.2 98.5</td><td>93.5 97.7 98.5</td><td>94.1 97.7 98.5</td><td>$\color{red}{\mathbf{94.8}}$ 97.2 98.0</td></tr>
    <tr><td><kbd>SF-XL</kbd>-night</td><td>53.4 67.0 70.2</td><td>$\color{red}{\mathbf{56.0}}$ 70.0 75.8</td><td>44.8 56.7 61.8</td><td>$\color{red}{\mathbf{52.8}}$ 64.8 67.6</td></tr>
    <tr><td><kbd>SF-XL</kbd>-occlusion</td><td>44.7 67.1 71.1</td><td>$\color{red}{\mathbf{51.3}}$ 72.4 78.9</td><td>$\color{red}{\mathbf{48.7}}$ 56.6 60.5</td><td>46.1 61.8 67.1</td></tr>
    <tr><td><kbd>SVOX</kbd></td><td>98.3 99.4 99.6</td><td>$\color{red}{\mathbf{98.7}}$ 99.5 99.7</td><td>98.3 99.4 99.5</td><td>$\color{red}{\mathbf{98.4}}$ 99.3 99.5</td></tr>
    <tr><td><kbd>SVOX</kbd>-night</td><td>$\color{red}{\mathbf{98.4}}$ 99.5 99.8</td><td>98.2 99.6 99.9</td><td>93.1 98.2 99.0</td><td>$\color{red}{\mathbf{96.5}}$ 98.4 99.3</td></tr>
    <tr><td><kbd>SVOX</kbd>-overcast</td><td>97.7 99.2 99.4</td><td>$\color{red}{\mathbf{98.2}}$ 99.4 99.7</td><td>$\color{red}{\mathbf{97.7}}$ 99.2 99.4</td><td>$\color{red}{\mathbf{97.7}}$ 99.3 99.4</td></tr>
    <tr><td><kbd>SVOX</kbd>-rain</td><td>97.4 99.6 99.6</td><td>$\color{red}{\mathbf{98.6}}$ 99.7 99.8</td><td>96.2 99.1 99.6</td><td>$\color{red}{\mathbf{97.9}}$ 99.7 99.8</td></tr>
    <tr><td><kbd>SVOX</kbd>-snow</td><td>98.3 99.7 99.8</td><td>$\color{red}{\mathbf{98.7}}$ 99.5 99.8</td><td>$\color{red}{\mathbf{98.7}}$ 99.5 99.5</td><td>97.7 99.7 99.7</td></tr>
    <tr><td><kbd>SVOX</kbd>-sun</td><td>97.8 99.4 99.6</td><td>$\color{red}{\mathbf{97.9}}$ 99.4 99.5</td><td>$\color{red}{\mathbf{96.7}}$ 98.9 99.3</td><td>96.6 98.9 99.5</td></tr>
  </tbody>
</table>

## 📚 Related Work
Our ICLR 2026 work (single-stage VPR based on DINOv2) [SAGE](https://openreview.net/forum?id=DCpbEXqPvS) achieved SOTA performance on several datasets. The code is released at [here](https://github.com/chenshunpeng/SAGE).

## 🙏 Acknowledgements

This code is based on the excellent work of [SelaVPR](https://github.com/Lu-Feng/SelaVPR), [CricaVPR](https://github.com/Lu-Feng/CricaVPR), [SALAD](https://github.com/serizba/salad), [Visual Geo-localization benchmark](https://github.com/gmberton/deep-visual-geo-localization-benchmark), [VPR-datasets-downloader](https://github.com/gmberton/VPR-datasets-downloader), [GSV-Cities](https://github.com/amaralibey/gsv-cities), and [MixVPR](https://github.com/amaralibey/MixVPR).

## 📌 Citation

If you find this repo useful for your research, please consider citing our original conference paper ([FoL](https://arxiv.org/abs/2504.09881)) and the extended journal version ([FoL++](https://arxiv.org/abs/2604.22390)):

```bibtex
@inproceedings{FoL,
  title={Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition},
  author={Wang, Changwei and Chen, Shunpeng and Song, Yukun and Xu, Rongtao and Zhang, Zherui and Zhang, Jiguang and Yang, Haoran and Zhang, Yu and Fu, Kexue and Du, Shide and others},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={39},
  number={7},
  pages={7536--7544},
  year={2025}
}

@article{FoL++,
  title={Region Matters: Efficient and Reliable Region-Aware Visual Place Recognition},
  author={Chen, Shunpeng and Song, Yukun and Wang, Changwei and Xu, Rongtao and Fu, Kexue and Gao, Longxiang and Guo, Li and Wang, Ruisheng and Xu, Shibiao},
  journal={arXiv preprint arXiv:2604.22390},
  year={2026}
}
```

FoL

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README