clifs - GitLite

KO | EN

gitlite — search

#python #ai #typescript #llm #golang #machine-learning #rust #deep-learning #macos #windows #cli #awesome-list #security #mcp

clifs

(★ 484)

Contrastive Language-Image Forensic Search allows free text searching through videos using OpenAI's machine learning model CLIP

파일 탐색기

최종 버전 다운로드 (.zip)

- bicyclist.jpg
- bmw.jpg
- jcn.jpg
- odwalla.jpg
- smart.jpg
- - clifs.py
  - requirements.txt
  - server.py
- .DS_Store
- Dockerfile
- - style.css
- - __init__.py
  - asgi.py
  - manage.py
  - settings.py
  - urls.py
  - wsgi.py
- - style.css
- Dockerfile
- entrypoint.sh
- requirements.txt
.gitignore
docker-compose-gpu.yml
docker-compose.yml
LICENSE
README.md
setup.sh

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/johanmodin/clifs

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd clifs

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Docker

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Docker Desktop 컨테이너를 빌드하고 실행하려면 필요합니다. 설치 후 실행해서 백그라운드에 켜두세요.

docker-compose build && docker-compose up

compose 설정 파일에 정의된 서비스들을 대상으로 명령을 실행합니다.

docker-compose build && docker-compose -f docker-compose-gpu.yml up

compose 설정 파일에 정의된 서비스들을 대상으로 명령을 실행합니다.

✅ 터미널에 docker compose ps 를 입력해 컨테이너들이 Up 상태인지 확인하세요. README에 포트 번호가 적혀있다면 브라우저에서 http://localhost:포트번호 로 접속해보세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r search/app/requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

파이썬 스크립트(또는 모듈)를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Contrastive Language-Image Forensic Search

## Overview

CLIFS is a proof-of-concept for free text searching through videos for video frames with matching contents.
This is done using [OpenAI's CLIP](https://openai.com/blog/clip/) model, which is trained to match images with the corresponding captions and vice versa.
The searching is done by first extracting features from video frames using the CLIP image encoder and then
getting the features for the search query through the CLIP text encoder. The features are then matched by similarity
and the top results are returned, if above a set threshold.

To allow easy use of the CLIFS backend, a simple web server running django is used to provide an interface to the search engine.

## Examples
To give an idea of the capability of this model, a few examples are shown below, with the search query in bold and the result below.
These search queries are done against the 2 minute Sherbrooke video from the [UrbanTracker Dataset](https://www.jpjodoin.com/urbantracker/dataset.html).
Only the top image result for each query is shown. Note that the model is in fact quite capable of OCR.

#### A truck with the text "odwalla"
![alt text](https://raw.githubusercontent.com/johanmodin/clifs/master/media/odwalla.jpg)
======

#### A white BMW car
![alt text](https://raw.githubusercontent.com/johanmodin/clifs/master/media/bmw.jpg)
======

#### A truck with the text "JCN"
![alt text](https://raw.githubusercontent.com/johanmodin/clifs/master/media/jcn.jpg)
======

#### A bicyclist with a blue shirt
![alt text](https://raw.githubusercontent.com/johanmodin/clifs/master/media/bicyclist.jpg)
======

#### A blue SMART car
![alt text](https://raw.githubusercontent.com/johanmodin/clifs/master/media/smart.jpg)
======

## Setup
1. Run the setup.sh script to setup the folders and optionally download a video file for testing:
```sh
./setup.sh
```

2. Put your own video files that you want to index in the `data/input` directory

3. Build and start the search engine and web server containers through docker-compose:
```sh
docker-compose build && docker-compose up
```

Optionally, a docker-compose file with GPU support can be used if the host environment has a NVIDIA GPU and is setup for docker GPU support:

```sh
docker-compose build && docker-compose -f docker-compose-gpu.yml up
```
 
4. Once the features for the files in the `data/input` directory have been encoded, as shown in the log, navigate to [127.0.0.1:8000](http://127.0.0.1:8000) and search away.