Image-segmentation-and-annotation-with-SAM-3-model

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

Image-segmentation-and-annotation-with-SAM-3-model

★ 13 Open GitHub ↗

A Python package for object segmentation using Segment Anything Model (SAM-3) with configurable export and visualization options.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Image Segmentation and Annotation with SAM-3

**For a detailed explanation of the design and development of this code, see the related article on [*Data Science Collective*](https://medium.com/data-science-collective/how-i-built-a-tool-to-literally-segment-anything-in-images-and-automatically-prepare-labelled-13d88d2ad14f)**

A Python package for object segmentation using Segment Anything Model (SAM-3) model with configurable export and visualization options. Allows the creation of labeling metadata in COCO and LabelMe format. This data can be used as a pre-annotation data to quickly improve the annotations for training a model or finetuning SAM-3 for improved performance.

For instance, if you have a specific segmentation task, you can quickly create the segmentation masks with SAM-3 and later import the annotations to a labeling tool, such as CVAT or LabelMe, where you can manually adjust the annotations, wherever needed. This will significanlty save the time required for annotating thousands of images.

## Features

- **Modular Design** - Separate classes for segmentation, export, and visualization
- **Reusable** - Import and use in any Python project
- **Multiple Objects** - Segment any object type with text prompts
- **Multiple Export Formats** - COCO JSON, LabelMe JSON, overlay images

![An example of teeth segmentation. Input image (left) and overlay/segmented image (right) - image source: stock.adobe.com](https://raw.githubusercontent.com/umairalipathan1980/Image-segmentation-and-annotation-with-SAM-3-model/master/assets/sample_image.png)
An example of teeth segmentation. Input image (left) and overlay/segmented image (right) - image source: stock.adobe.com

## Installation

**Prerequisites**
- Python 3.12 or higher
- PyTorch 2.7 or higher
- CUDA-compatible GPU with CUDA 12.6 or higher (strongly recommended for speed)

**Install project requirements**
```
pip install -r requirements.txt
```

**Install PyTorch with CUDA support**
```
pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
```

**Install SAM-3**
```
git clone https://github.com/facebookresearch/sam3.git
cd sam3
pip install -e .
```

**Get a Hugging Face Token and Log In**

This project may require access to models or datasets on the Hugging Face Hub. To authenticate your machine and use the Hugging Face CLI, follow these steps:

1️⃣ Create a Hugging Face Account  
Go to https://huggingface.co and sign up or sign in to your account.

**Install the Hugging Face CLI**
```
pip install -U "huggingface_hub[cli]"
```

**Log In from the Command Prompt**
```
huggingface-cli login
```

This will prompt you to paste your token. Paste the token you copied from the Hugging Face settings and press Enter.

Then ensure this package directory is on your Python path or install it in editable mode if desired.

## Repository Structure

```
.
|-- README.md
|-- requirements.txt
|-- sam.py                      # Legacy wrapper using SAMSegmenter
|-- examples.py                 # Top-level examples using SAMSegmenter
|-- images/                     # Sample images and generated overlays
|   \-- segmented_images/       # Generated overlay outputs
\-- sam_segmentation/           # Core package
    |-- __init__.py             # Package exports
    |-- segmenter.py            # SAMSegmenter class and SegmentationResult
    |-- exporters.py            # COCOExporter, LabelMeExporter
    |-- visualizer.py           # OverlayVisualizer
    \-- utils.py                # Helper functions
```

## Quick Start

### Simple Batch Processing

```python
from sam_segmentation import SAMSegmenter

# Option 1: Set images_dir in constructor
segmenter = SAMSegmenter(
    text_prompt="car",
    labels=["car"],
    images_dir="./images"
)
results = segmenter.process_directory()  # Uses class's images_dir
print(f"Processed {len(results)} images")

# Option 2: Pass directory to process_directory()
segmenter = SAMSegmenter(text_prompt="car", labels=["car"])
results = segmenter.process_directory("./images")
print(f"Processed {len(results)} images")
```

### Process Single Image

```python
from sam_segmentation import SAMSegmenter

segmenter = SAMSegmenter(text_prompt="person", labels=["person"])
result = segmenter.process_image("image.jpg")
print(f"Detected {result.num_detections} objects")
```

### Custom Configuration

```python
from sam_segmentation import SAMSegmenter

segmenter = SAMSegmenter(
    text_prompt="Segment each car accurately",
    labels=["car"],
    mask_threshold=0.6,        # Higher threshold
    export_format="labelme",   # Only LabelMe format
    save_overlay=True,
    overlay_alpha=0.5,          # More opaque
    category_name="vehicle"
)

results = segmenter.process_directory("./data", output_dir="./output")
```

## API Reference

### SAMSegmenter Class

Main segmentation pipeline class.

**Constructor Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_path` | str | `"C:\Users\h02317\sam3"` | Path to SAM3 installation |
| `text_prompt` | str or List | `"object"` | Text prompt(s) for segmentation (e.g., "car" or ["bus", "car"]) |
| `labels` | List | `None` | Optional labels aligned with prompts for annotations/exports |
| `mask_threshold` | float | `0.5` | Threshold for binarizing masks |
| `enable_exif_transpose` | bool | `True` | Auto-correct image orientation |
| `export_format` | str or List | `"coco"` | Export format(s): "labelme", "coco", or ["labelme", "coco"] |
| `save_overlay` | bool | `True` | Save overlay visualizations |
| `images_dir` | Path | `None` | Default directory containing images |
| `output_dir` | Path | `None` | Base output directory |
| `overlay_subdir` | str | `"segmented_images"` | Overlay subdirectory name |
| `category_name` | str | `"object"` | Object category name |
| `dataset_name` | str | `"Segmentation Dataset"` | Dataset name for COCO |
| `overlay_alpha` | float | `0.4` | Overlay transparency (0-1) |
| `overlay_colors` | List | `None` | Custom color palette |
| `contour_thickness` | int | `2` | Contour line thickness |
| `polygon_tolerance` | float | `2.0` | Polygon approximation tolerance |
| `supported_extensions` | List | `None` | Image file extensions |

**Methods:**

```python
def process_image(image_path, output_dir=None) -> SegmentationResult:
    """Process a single image."""

def process_directory(images_dir=None, output_dir=None) -> List[SegmentationResult]:
    """Process all images in a directory. Uses class's images_dir if not provided."""

def export_results(results, output_dir) -> Dict[str, Path]:
    """Export results to configured formats."""
```

### SegmentationResult Class

Data container for segmentation results.

**Attributes:**
- `image_path`: Path - Source image path
- `image_size`: Tuple[int, int] - Image (width, height)
- `masks`: np.ndarray - Segmentation masks (N, H, W)
- `scores`: np.ndarray - Confidence scores (N,)
- `timestamp`: datetime - Processing timestamp

**Properties:**
- `num_detections`: int - Number of detected objects

### COCOExporter Class

Export to COCO JSON format.

```python
from sam_segmentation import COCOExporter

exporter = COCOExporter(
    category_name="car",
    dataset_name="My Dataset"
)
exporter.export(results, "annotations.json")
```

### LabelMeExporter Class

Export to LabelMe JSON format.

```python
from sam_segmentation import LabelMeExporter

exporter = LabelMeExporter(category_name="car")
exporter.export(result, "output.json")
```

### OverlayVisualizer Class

Create overlay visualizations.

```python
from sam_segmentation import OverlayVisualizer

visualizer = OverlayVisualizer(
    alpha=0.6,
    contour_thickness=3,
    draw_labels=True
)

overlay = visualizer.render(image, masks, labels=["car_1", "car_2"])
visualizer.save(result, "overlay.png")
```

## Usage Examples

See `examples.py` for 10 comprehensive examples demonstrating different use cases.

## Configuration Options Summary

### Export Formats
- **LabelMe JSON**: One file per image with polygon annotations
- **COCO JSON**: Single file with all annotations for the dataset
- **Overlay Images**: Visual representation with colored masks

### Customizable Settings
- Text prompts for different objects
- Mask confidence thresholds
- Export format selection
- Output directory structure
- Visualization appearance (colors, transparency, contours)
- Polygon precision
- EXIF orientation handling

## Dependencies

Same as original `sam.py`:
- numpy
- opencv-python (cv2)
- torch
- Pillow (PIL)
- SAM3 model (from specified path)

## Notes

- Model is loaded lazily on first use (not at initialization)
- All output paths are automatically created if they don't exist
- EXIF orientation is handled automatically by default
- Supports: .jpg, .jpeg, .png, .bmp, .tiff, .tif, .webp

Image-segmentation-and-annotation-with-SAM-3-model

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README