Smoking Detection Pipeline

A robust, multi-stage computer vision pipeline designed to detect smoking events in video files or real-time RTSP streams. The system uses a cascade of YOLO models to ensure high precision and low false positives.

🏗 Architecture

The pipeline operates in a "Fail-Fast" cascade to optimize performance:

1. Broad Phase (Person Detection):

   • Detects humans in the frame.
   • Optimization: If no people are found, processing stops for that frame.

2. Narrow Phase (Cigarette Candidate Detection):

   • Crops the detected person (with a 20% margin for context).
   • Runs a specialized YOLO detector to find small objects (cigarettes/lighters).
   • Filter: Rejects candidates smaller than 10x10 pixels.

3. Verification Phase (Classification):

   • Crops the candidate object.
   • Runs a dedicated Binary Classifier to verify "Smoking" vs "Not Smoking".
   • Output: Only high-confidence events are flagged as alerts.

Key Features

• GPU Acceleration: Auto-detects CUDA and runs inference on GPU when available.
• Async Inference: Detection runs in a background thread; video output never lags.
• Adaptive Sampling:
  • Idle Mode (2 FPS): When no people are detected, saves compute.
  • Active Mode (Max FPS): Switches to full speed when activity is detected.
  • Cooldown: Stays active for 2 seconds after people leave frame.
• Non-Maximum Suppression (NMS): Cleans up overlapping boxes per-class.
• Smart Event Recording: Captures 2s before + 3s after each smoking event as separate clips.

🚀 Usage

Prerequisites

• Python 3.13+
• uv package manager (recommended)
• CUDA 13.0+ (optional, for GPU acceleration)

Installation

# Clone and enter directory
cd gp

# Install dependencies with uv
uv sync

Running the Pipeline

# Run on a local video file
uv run main.py --source videos/test_footage.mp4

# Run on an RTSP stream
uv run main.py --source rtsp://admin:[email protected]:554/stream --rtsp

# Enable debug logging (see frame-by-frame details)
uv run main.py --source videos/test.mp4 --debug

CLI Arguments

Argument	Default	Description
--source	Required	Path to video file or RTSP URL.
--rtsp	False	Set this flag if the source is an RTSP stream (enables threaded reading).
--output	output.mp4	Path for the annotated output video.
--person-model	models/human-det-best.pt	Path to the Person Detection YOLO model.
--cig-model	models/cig-det-best.pt	Path to the Cigarette Detection YOLO model.
--verify-model	models/cig-cls-best.pt	Path to the Verification/Classifier model.
--debug	False	Enable verbose logging.

📊 Outputs

The system generates four types of artifacts during execution:

1. Visual Debug (output.mp4):

   • A video file with bounding boxes drawn for all stages.
   • Blue: Detected Person.
   • Yellow: Cigarette Candidate.
   • Red: Confirmed Smoking Event.

2. Frame Metadata (output.jsonl):

   • A synchronized JSON Lines file containing raw detection data for every frame.
   • Useful for debugging model performance or retraining.

3. Alert Log (alerts.jsonl):

   • A structured log file containing only confirmed smoking events.
   • Use this file to trigger external notifications or audit logic.

4. Event Clips (events/event_<timestamp>.mp4):

   • Smart clips containing only the smoking incidents.
   • Includes 2 seconds of pre-event context and 3 seconds post-event.
   • Automatically extends if smoking continues.

5. Application Logs (logs/pipeline.log):

   • Rotating log files with detailed debug information.
   • Includes thread names, function calls, and timing data.

📂 Project Structure

gp/
├── main.py                  # CLI Entry point
├── pyproject.toml           # Project config (uv/pip)
├── models/                  # Directory for .pt model weights
├── logs/                    # Rotating application logs
├── events/                  # Smart event clips (auto-created)
├── pipeline/
│   ├── core.py              # Main logic loop (Pipeline class, Async Inference)
│   ├── models.py            # Ultralytics model wrappers (GPU-enabled)
│   ├── sources.py           # Stream inputs (File, ThreadedRTSP)
│   ├── outputs.py           # Sinks (VideoFileStore, SmartEventSink, FileAlertStore)
│   ├── protocols.py         # Interfaces (Model, Verifier, Sink, StreamSource)
│   └── types.py             # Data classes (Frame, Detection, Ticket, NMS)

⚙️ Performance Tuning

The pipeline is designed for real-time performance. Key parameters in main.py:

pipeline = SmokingDetectionPipeline(
    ...
    inference_interval=0.5,  # Idle: 2 FPS (saves GPU when empty)
    active_interval=0.0,     # Active: Process every frame
    cooldown_frames=60,      # Stay active 2s after person leaves
)

For RTSP streams, the ThreadedRTSPSource uses a background thread to prevent frame buildup and handles reconnection automatically.