KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

MasKD

★ 72 Open GitHub ↗

Official implementation of paper "Masked Distillation with Receptive Tokens", ICLR 2023.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Masked Distillation with Receptive Tokens (MasKD)
Official implementation of paper "[Masked Distillation with Receptive Tokens](https://arxiv.org/abs/2205.14589)" (MasKD), ICLR 2023.

By Tao Huang*, Yuan Zhang*, Shan You, Fei Wang, Chen Qian, Jian Cao, Chang Xu.

:fire: **MasKD: better and more general feature distillation method for dense prediction tasks (e.g., detection and segmentation).**

## Updates  
### March 04, 2023  
Configs for KD are available in `mmrazor` folders and student checkpoints are provided.
### May 30, 2022  
Code for mask learning and KD is available in `mmdetection` and `mmrazor` folders.

## Reproducing our results

### Train students with pretrained masks  
We provide the learned pretrained mask tokens in our experiments at [release](https://github.com/hunto/MasKD/releases/tag/v0.0.1).

This repo uses [MMRazor](https://github.com/open-mmlab/mmrazor) as the knowledge distillation toolkit. For environment setup, please see [mmrazor/README.md](mmrazor/README.md).

**Train student:**  
```shell
cd mmrazor
sh tools/mmdet/dist_train_mmdet.sh ${CONFIG} 8 ${WORK_DIR}
```

Example for reproducing our `cascade_mask_rcnn_x101-fpn_r50` result:
```shell
sh tools/mmdet/dist_train_mmdet.sh configs/distill/maskd/cascade_mask_rcnn_x101-fpn_x50_coco.py 8 work_dirs/maskd_cmr_x101-fpn_x50
```

### Results  
* Baseline settings:

* Stronger teachers:

### Semantic segmentation  
For semantic segmentation, please see [segmentation](./segmentation/) folder.

## License  
This project is released under the [Apache 2.0 license](LICENSE).

## Citation  
```
@inproceedings{
huang2023masked,
title={Masked Distillation with Receptive Tokens},
author={Tao Huang and Yuan Zhang and Shan You and Fei Wang and Chen Qian and Jian Cao and Chang Xu},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=mWRngkvIki3}
}
```

MasKD

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README