ShyldAI — Operating Room Activity Monitor
Built a production medical AI system deployed on NVIDIA Jetson Orin Nano in hospitals — a 6-node distributed ROS2 system for real-time OR activity monitoring.
The Challenge
Hospital operating rooms need continuous monitoring for safety risks, equipment issues, and activity logging. The system had to run on a Jetson Orin Nano with only 7GB of memory, processing both audio and video streams in real-time.
The Solution
Designed a 6-node distributed ROS2 architecture: Audio Recording, Image Recording, Whisper Transcription, OR Analysis (Qwen LLM), VLM Analysis (Qwen3-VL), and Unified Pipeline Orchestrator. Optimized all models to fit within 7GB memory constraints using TensorRT and careful model lifecycle management.
Tech Stack
Key Features
- Real-time audio transcription with safety risk extraction
- Visual scene understanding for patient/bed presence detection
- Continuous and on-demand processing modes
- Full model lifecycle management within 7GB memory
- Docker deployment on Jetson Orin Nano
Impact
Deployed in hospital operating rooms for 24/7 monitoring
Related Projects
Carter — Autonomous Object Tracking & Navigation
Built a complete autonomous object tracking and navigation system from scratch — a 4-service microservice Docker architecture for robot navigation using NVIDIA Isaac.
Accident Detector
Real-time accident detection from video using InternVideo2 + BERT.
Badminton Analysis (MatchFlow)
Rally detection, highlights generation, and Streamlit dashboard for badminton match analysis.