emotion2vec-speech-emotion-detection-api

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

emotion2vec-speech-emotion-detection-api

★ 14 Open GitHub ↗

This API utilizes a pre-trained model for emotion recognition from audio files. It accepts audio files as input, processes them using the pre-trained model, and returns the predicted emotion along with the confidence score. The API leverages the FastAPI framework for easy development and deployment.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  <img src="docs/assets/readme-icon.svg" width="128" alt="Emotion2Vec Speech Emotion API icon with an audio waveform" />
  <h1>Emotion2Vec Speech Emotion API</h1>
  <p><strong>Upload audio to FastAPI and receive the highest-scoring label from a pinned ModelScope emotion2vec pipeline.</strong></p>
  <p>Experimental inference wrapper with local execution after the model is downloaded.</p>

[![FastAPI](https://img.shields.io/badge/API-FastAPI-009688?logo=fastapi&logoColor=white)](https://raw.githubusercontent.com/jayanth-mkv/emotion2vec-speech-emotion-detection-api/main/requirements.txt)
  [![emotion2vec v2.0.4](https://img.shields.io/badge/model-emotion2vec%20v2.0.4-B54A32)](https://raw.githubusercontent.com/jayanth-mkv/emotion2vec-speech-emotion-detection-api/main/app.py)
  [![Status: experimental](https://img.shields.io/badge/status-experimental-69655E)](https://raw.githubusercontent.com/jayanth-mkv/emotion2vec-speech-emotion-detection-api/main/)

[What is it?](#what-is-it) · [Quick start](#quick-start) · [API](#api) · [Architecture](#architecture) · [Runtime notes](#runtime-notes) · [Limitations](#limitations)
</div>

## What is it?

This repository is a small FastAPI wrapper for developers experimenting with utterance-level speech-emotion inference. At process startup, [app.py](app.py) creates a ModelScope pipeline for <code>iic/emotion2vec_base_finetuned</code> at revision <code>v2.0.4</code>.

Requests run on the host machine: uploaded bytes are passed to the downloaded model, NumPy selects the largest score, and a fixed label map is returned as JSON. ModelScope access is needed when the model is not already cached; the repository does not call a separately hosted inference API.

## Quick start

Create an isolated Python environment. The first install and first model load can be large because the dependency set includes FunASR, ModelScope, TorchAudio, and dataset tooling.

~~~bash
git clone https://github.com/jayanth-mkv/emotion2vec-speech-emotion-detection-api.git
cd emotion2vec-speech-emotion-detection-api
python -m venv .venv
python -m pip install -r requirements.txt
uvicorn app:app --reload
~~~

Open <http://localhost:8000/docs> for FastAPI's generated interface.

## API

### POST /emotion_recognition

Send one multipart field named <code>audio_file</code>:

~~~bash
curl -X POST \
  -F "audio_file=@/path/to/sample.wav" \
  http://localhost:8000/emotion_recognition
~~~

| Response field | Meaning |
| --- | --- |
| <code>emotion</code> | Label at the index of the largest score. |
| <code>confidence</code> | The pipeline score at that index; the repository does not document calibration. |

The hard-coded index order is:

~~~text
angry, disgust, fear, happy, neutral, other, sad, surprised, unknown
~~~

That order must remain compatible with the pinned model revision. The API does not currently expose the full score vector.

## Architecture

## Runtime notes

- The model pipeline is created while <code>app.py</code> imports, so server readiness waits for model initialization.
- Inference writes to <code>./outputs</code> through the pipeline's <code>output_dir</code> argument.
- The endpoint uses utterance granularity and disables embedding extraction.
- [vercel.json](vercel.json) expresses a planned Vercel Python deployment, but no deployed URL or serverless verification is included.

## Limitations

- There is no audio MIME, codec, duration, sample-rate, channel-count, or upload-size validation.
- Model download size, memory use, latency, and hardware requirements are not measured in this repository.
- The label mapping is maintained manually rather than read from model metadata.
- Requests have no authentication, rate limit, queue, timeout, health endpoint, or structured model-error handling.
- The score is reported as provided by the model and should not be treated as a calibrated probability without separate evaluation.
- There are no tests or evaluation artifacts proving accuracy on IEMOCAP or any other dataset in this repository.
- No license file is included, so reuse terms are not currently stated.

emotion2vec-speech-emotion-detection-api

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

emotion2vec-speech-emotion-detection-api

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README