SafetyBuddy: A Multimodal LLM-Based Safety Intelligence Platform for Regulatory Compliance in Process Industries
TL;DR
The problems addressed in this paper are platform architecture, multimodal data fusion and LLM grounding mechanism in the Nigerian industrial system in addition to evaluation metrics, security control, human-in-the-loop validation of safety alerts and the limitations to real-world deployment.
Abstract
Process industries are a major source of occupational injuries and fatalities worldwide due to safety violations. Nigeria alone has recorded more than 412 fatalities in the oil and gas industry and the construction industry has a documented history of accidents associated with personal protective equipment (PPE) non-compliance. According to the World Risk Poll (2024), Sub-Saharan Africa is the highest-ranked region in terms of workplace injuries, where 21% of the existing workforce experiences serious harm. The currently used compliance monitoring methods are largely manual, ad hoc, and not linked with regulatory frameworks. This paper describes SafetyBuddy, a multimodal safety intelligence application designed to automate compliance monitoring. The system incorporates a YOLO26-nano real-time object detector, retrieval augmented generation (RAG) over OSHA regulatory documents, and reasoning on a continuous PPE compliance system powered by the open-weight Gemma 4 multimodal large language model (LLM), self-hosted on serverless GPU infrastructure. Its functionality spans four states of interaction, including safety advisory, incident root cause analysis, compliance auditing, and real-time video violation alerting. After 50 training epochs, the YOLO26 model obtained mean average precision (mAP@50) of 0.691 and precision of 0.775 on a 10-class dataset, with NMS-free inference at 25-40 frames per second on CPU hardware. All recommendations are enhanced with OSHA regulations traceability by an independent compliance mapping module. Running on scale-to-zero serverless infrastructure and costing around USD 10-15 each month, the platform proves that a resource constrained industrial setting can have perception, regulatory knowledge and reasoning that are united to enforce safety compliance. The problems addressed in this paper are platform architecture, multimodal data fusion and LLM grounding mechanism in the Nigerian industrial system in addition to evaluation metrics, security control, human-in-the-loop validation of safety alerts and the limitations to real-world deployment.