KO
|
EN
gitlite — search
Search
#javascript
#python
#hacktoberfest
#react
#ai
#typescript
#llm
#go
#golang
#android
#machine-learning
#rust
#deep-learning
#linux
TRM
★ 9
Open GitHub ↗
No description available.
Download README (.md)
Explore Similar Repositories
nuclei-c-templates
:
Custom nuclei templates
birds-detection-with-yolov8
:
No description available.
terraform-example-deploy-java-gke
:
Deploy a Legacy Java App GKE
open-economics
:
It aims to predict the optimal time for a country to buy oil from OPEC in order to obtain more oil at a cheaper rate and prevent economic recession. The prediction is done using logistic regression and statistical method
chat-gpt-demo
:
No description available.
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
TRM
?
Download (.md)
This repo provides code: Phrase-level Temporal Relationship Mining for Temporal Sentence Localization (AAAI2023). [paper](https://ojs.aaai.org/index.php/AAAI/article/view/25478) Large-Scale Pre-trained Models Empowering Phrase Generalization in Temporal Sentence Localization (IJCV2025). [paper](https://link.springer.com/article/10.1007/s11263-025-02599-w) # TRM: Phrase-level and Sentence-level Video Temporal Grounding ## Abstract In this paper, we address the problem of video temporal sentence localization, which aims to localize a target moment from videos according to a given language query. We observe that existing models suffer from a sheer performance drop when dealing with simple phrases contained in the sentence. It reveals the limitation that existing models only capture the annotation bias of the datasets but lack sufficient understanding of the semantic phrases in the query. To address this problem, we propose a phrase-level Temporal Relationship Mining (TRM) framework employing the temporal relationship relevant to the phrase and the whole sentence to have a better understanding of each semantic entity in the sentence. Specifically, we use phrase-level predictions to refine the sentence-level prediction, and use Multiple Instance Learning to improve the quality of phrase-level predictions. We also exploit the consistency and exclusiveness constraints of phrase-level and sentence-level predictions to regularize the training process, thus alleviating the ambiguity of each phrase prediction. The proposed approach sheds light on how machines can understand detailed phrases in a sentence and their compositions in their generality rather than learning the annotation biases. Experiments on the ActivityNet Captions and Charades-STA datasets show the effectiveness of our method on both phrase and sentence temporal localization and enable better model interpretability and generalization when dealing with unseen compositions of seen concepts. ## Pipeline  ## Requiments - pytorch - h5py - yacs - terminaltables - tqdm - transformers ## Main Results ### Charades-STA Dataset | Method | | Sentence | | | Phrase | | | :-----: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | | | Rank1@0.3 | Rank1@0.5 | Rank1@0.7 | Rank1@0.3 | Rank1@0.5 | Rank1@0.7 | | TRM | 60.67 | 47.77 | 28.01 | 57.03 | 33.69| 11.86 | | TRM-PT | 61.57 | 48.13 | 28.97 | 58.21 | 34.65| 12.85 | ### ActivityNet Captions Dataset | Method | | Sentence | | | Phrase | | | :-----: | :-------: | :-------: | :-------: | :-------: | :-------: | :-------: | | | Rank1@0.3 | Rank1@0.5 | Rank1@0.7 | Rank1@0.3 | Rank1@0.5 | Rank1@0.7 | | TRM | 66.41 |50.44 |31.18 | 52.46 | 42.84 | 33.68 | | TRM-PT | 66.92 | 51.54 | 31.85 | 53.79 | 44.01 | 34.21 | ## Quick Start ### Data Preparation We use the C3D feature for the ActivityNet Captions dataset. Please download from [here](http://activity-net.org/challenges/2016/download.html) and save as `dataset/ActivityNet/sub_activitynet_v1-3.c3d.hdf5`. We use the VGG feature provided by [2D-TAN](https://github.com/microsoft/VideoX) for the Charades-STA dataset, which can be downloaded from [here](https://rochester.app.box.com/s/8znalh6y5e82oml2lr7to8s6ntab6mav/folder/137471415879). Please save it as `dataset/Charades-STA/vgg_rgb_features.hdf5`. ### Training To train on the ActivityNet Captions dataset: ```bash sh scripts/anet_train.sh ``` To train on the Charades-STA dataset: ```bash sh scripts/charades_train.sh ``` You can change the options in the shell scripts, such as the GPU id, configuration file, et al. ### Inference Run the following commands for evaluation: ```bash sh scripts/eval.sh ``` Please change the configuration file and the directory of the saved weight in the shell script.
