N.A.V.R.A.A.H

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #llm #machine-learning #deep-learning #docker #awesome #vue #security #pytorch #mcp #open-source

(★ 16)

Navigation Assistant For Visually Restricted And Aided Humans (N.A.V.R.A.A.H.): An intelligent assistive framework integrating Raspberry Pi and Arduino to enable real-time object detection via TensorFlow and haptic-audio feedback for safer, independent navigation.

N.A.V.R.A.A.H Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# N.A.V.R.A.A.H
### Navigation Assistant Via Real-time Awareness and Audio Help

> A Python project that helps visually impaired people navigate the world around them using a camera and a speaker.

---

## 🧠 What Does This Project Do? (The Big Idea)

Imagine you are blind. You cannot see walls, people, doors, or objects in front of you. This project acts like a pair of **AI-powered eyes with a voice**.

It uses:
- 📷 **A Camera** — to see the world in real time
- 🧠 **An AI Model** — to figure out *what* is in the frame and *where* it is (left, ahead, or right)
- 🔊 **A Speaker** — to say things like *"I see a person on your left and a wall directly ahead"*
- 📳 **A Vibration Motor** — to buzz when something is dangerously close (this runs on a Raspberry Pi with physical hardware)
- 📡 **An Ultrasonic Sensor** — like the reverse sensor in a car, it measures how far away the nearest object is in centimetres

On a **Windows laptop**, you can run and test it using just the built-in camera and speakers — no special hardware needed.

---

## 📁 Project Structure — What Each File Does

```
N.A.V.R.A.A.H/
│
├── main.py              ← The entry point. Run this to start the assistant.
├── config.py            ← All settings in one place (thresholds, pins, flags).
├── test_mocks.py        ← Automated tests to verify everything works.
├── download_model.py    ← Downloads the AI model files automatically.
├── requirements.txt     ← List of Python libraries this project needs.
│
├── hardware/            ← The "organs" of the assistant (each is a Python class)
│   ├── hardware_base.py ← A base class that sensor/vision/feedback all inherit from.
│   ├── sensor.py        ← Reads distance from the ultrasonic sensor (or fakes it).
│   ├── vision.py        ← Reads the camera, runs the AI, returns what it sees.
│   └── feedback.py      ← Says things out loud and controls vibration.
│
└── models/              ← The AI brain files (downloaded, not written by hand)
    ├── yolov8n-oiv7.onnx   ← Main AI model — can recognise 600+ types of objects
    ├── labels_oiv7.txt     ← List of all 600+ object names the AI knows
    └── ...                 ← Backup models (Caffe, TFLite) used if YOLO is missing
```

---

## 🧩 How the Code Is Organised (OOP Explained Simply)

The project uses **classes** — think of each class as a worker with a specific job.

### `UltrasonicSensor` (in `hardware/sensor.py`)
- **Job:** Measure distance to the nearest object.
- **On a Raspberry Pi:** Sends a sound pulse and times how long it takes to come back (like a bat).
- **On Windows:** Returns a random number between 20–200 cm to simulate a sensor (this is called "mocking").

### `VisionSystem` (in `hardware/vision.py`)
- **Job:** Look at the camera feed, run the AI, and return a list like:
  `["person directly ahead", "wall on your left", "clothing on your right"]`
- Uses a pre-trained AI model (YOLOv8) that has already learned what thousands of objects look like.
- Applies several intelligence layers (explained below).

### `FeedbackSystem` (in `hardware/feedback.py`)
- **Job:** Turn the detected object list into spoken words, and trigger vibration.
- Takes `["person directly ahead", "door on your left"]` and says:
  *"I see a person directly ahead and a door on your left"*
- Speech runs in a **separate background thread** so vibration is never delayed by talking.

---

## 🤖 How the AI Vision Works (Step by Step)

When `vision.detect(frame)` is called with one camera image, here is exactly what happens inside:

### Step 1 — Run YOLOv8 (the main AI)
- The camera image is resized to 640×640 pixels and fed into the AI model.
- The AI outputs bounding boxes: rectangles around objects it found, with a confidence score (0.0 to 1.0).
- Only detections above **0.28 confidence** are kept. (Below this is considered unreliable noise.)
- Each detected object's raw AI label (like `"Top, T-shirt, Sweatshirt"`) is translated to a plain name (like `"clothing"`) using a dictionary called `LABEL_MAP`.

### Step 2 — Supplement with Face Detection (Haar Cascade)
- A simpler, faster algorithm specifically for detecting faces is also run.
- If the main AI missed a person, this catches them.
- **Only runs if "person" was not already found** — no double counting.

### Step 3 — Temporal Smoothing (Anti-Flicker)
- A single frame is not trusted immediately.
- The system keeps a history of the **last 5 frames**.
- An object must appear in a minimum number of frames to be confirmed as "really there":
  - **Walls, doors, stairs** → 2 out of 5 frames (they are always present, confirm fast)
  - **People, clothing** → 2 out of 5 frames (important, should not be delayed)
  - **Small items like glasses, coins** → 3 out of 5 frames (prevent false positives)
- This stops the AI from saying random objects that appeared for just one blurry frame.

### Step 4 — Directional Awareness (Left / Ahead / Right)
- Every detected object has a bounding box. The centre X coordinate of that box tells us where it is in the frame.
- The frame is divided into 3 zones:
  ```
  |--- Left ---|--- Ahead ---|--- Right ---|
  0           33%           66%          100%
  ```
- So `"person"` at 20% → `"person on your left"`

### Step 5 — Priority Sorting
- Not everything is equally important for a blind person.
- Objects are sorted by an `ACCESSIBILITY_PRIORITIES` score:
  - `stairs`, `traffic light`, `wall` → Priority 9–10 (spoken first)
  - `person`, `vehicle` → Priority 7–8
  - `clothing`, `glasses` → Priority 5–6
  - `book`, `pen` → Priority 3–4

### Step 6 — Scene Memory (Anti-Repetition)
- If an object was already announced, it is not announced again for a while (cooldown):
  - `wall` → 15 seconds (it is still there, no need to repeat)
  - `stairs`, `door` → 8 seconds
  - `person`, `clothing` → 5 seconds
- This prevents the assistant from being annoying by repeating itself every second.

---

## ⚙️ Settings You Can Change (`config.py`)

| Setting | What it does | Default |
|---|---|---|
| `TEST_MODE` | `True` = use fake sensor data (for laptop testing), `False` = use real hardware | `False` |
| `ENABLE_REAL_CAMERA` | `True` = use your laptop/Pi camera | `True` |
| `DEBUG_VISION` | `True` = draw coloured boxes on screen around detected objects | `True` |
| `DANGER_DISTANCE` | Distance in cm where vibration triggers immediately | `50` |
| `WARNING_DISTANCE` | Distance in cm for a voice warning only | `100` |
| `CONFIDENCE_THRESHOLD` | How certain the AI must be (0.0–1.0) — used by Caffe/TFLite fallback models | `0.5` |
| `LOOP_DELAY` | Seconds between each detection loop (lower = faster but more CPU) | `0.1` |

---

## 🚀 How to Run It (On Your Laptop)

### First Time Setup
```powershell
# 1. Create a virtual environment (an isolated Python box for this project)
python -m venv .venv

# 2. Activate it
.\.venv\Scripts\activate

# 3. Install all required libraries
pip install -r requirements.txt

# 4. Download the AI model
python download_model.py
```

### Run the Tests (Recommended First)
```powershell
python test_mocks.py
```
You should see `OK` at the end. The live camera test will open your webcam for ~8 seconds and print everything it detects.

### Run the Full Assistant
```powershell
python main.py
```
Point your laptop camera at the room — it will start announcing what it sees.

**Press `Ctrl + C`** to stop it cleanly.

---

## 🧪 What Do the Tests Check? (`test_mocks.py`)

| Test Name | What it checks |
|---|---|
| `test_sensor_mock` | Sensor returns a sensible distance number |
| `test_vision_pipeline` | Vision system returns a list even when camera is off |
| `test_mock_passthrough` | Fake object injection works (for simulating specific scenes) |
| `test_feedback_system` | TTS and vibration run without crashing |
| `test_live_camera_realtime` | Opens real camera for 8 seconds and prints everything seen |

---

## 🔌 On a Raspberry Pi (Real Hardware)

On the actual device worn by a visually impaired person:
- The **ultrasonic sensor** (HC-SR04) is wired to GPIO pins 23 (trigger) and 24 (echo).
- A **vibration motor** is on GPIO pin 25.
- A **buzzer** is on GPIO pin 18.
- The **Pi Camera** is connected via the ribbon cable port.
- Set `TEST_MODE = False` in `config.py` to enable all real hardware.

---

## 📦 AI Models Explained Simply

| File | What it is | Size |
|---|---|---|
| `yolov8n-oiv7.onnx` | **Main AI** — knows 600+ objects (OIV7 dataset). Used on Windows/Pi. | 14 MB |
| `mobilenet_ssd.caffemodel` | **Backup AI** — knows 20 objects. Used if YOLO fails. | 23 MB |
| `detect.tflite` | **Lightweight AI** — designed for Raspberry Pi. | 4 MB |
| `labels_oiv7.txt` | Text file listing all 600+ object names in order. | 6 KB |

The system tries them in order: **YOLO → TFLite → Caffe → Haar Cascade (face only)**.

---

## 💡 Glossary (Plain English Definitions)

| Term | Meaning |
|---|---|
| **AI Model** | A file containing millions of learned patterns from training on labelled images. |
| **ONNX** | A universal file format for AI models — works across frameworks. |
| **Bounding Box** | A rectangle the AI draws around a detected object. |
| **Confidence Score** | How sure the AI is (0 = not sure at all, 1.0 = 100% certain). |
| **Thread** | A second task running alongside the main program simultaneously. |
| **GPIO** | General Purpose Input/Output — the physical pins on a Raspberry Pi. |
| **Mock** | Fake data used in tests so you don't need real hardware to test logic. |
| **OIV7** | Open Images V7 — a dataset of 9 million labelled images used to train the AI. |
| **Haar Cascade** | An older, simpler face-detection algorithm built into OpenCV. |
| **Scene Memory** | A dictionary tracking when each object was last announced, to avoid repetition. |

---

*Built for accessibility. Made in Python.*

N.A.V.R.A.A.H

N.A.V.R.A.A.H Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

N.A.V.R.A.A.H

N.A.V.R.A.A.H Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README