A production-ready AI safety platform combining semantic guardrails, specialized classifiers, and human-in-the-loop policy enforcement for datasets, classifier and live LLM traffic.
The Core Problem: Invisible Bias in AI Systems
Machine learning models inherently learn patterns from historical data, which often contains unbalanced representations, stereotypes, and historical discrimination. When deployed in high-stakes environments, these models can amplify existing biases, leading to unfair decision-making at scale.
Most safety solutions fail because they react to bias too late, rely on static regex rules, or ignore real-time interactions. Aegis AI was engineered to solve this by providing end-to-end lifecycle protection, bridging the gap between pre-training data sanitization and live production guardrails.
System Design & Infrastructure
Every incoming prompt passes through a lightweight semantic routing layer before reaching the production model. Depending on the detected intent and risk profile, the request is either forwarded directly, evaluated by specialized classifiers, or blocked with an explainable policy response. Background audit pipelines continuously analyze historical traffic to recommend new guardrails without affecting real-time inference.
To achieve high-performance inline prompt evaluation while supporting asynchronous rule generation, the architecture was split into a hybrid infrastructure. We explicitly avoided the "one-LLM-to-rule-them-all" anti-pattern, instead routing workloads to specialized models based on task requirements.
- Frontend & Orchestration: The Next.js dashboard streams evaluation results while long-running auditing jobs execute asynchronously, preventing UI blocking during large dataset analysis.
- Inference Backend: Built using FastAPI to isolate model execution. Python's native ML ecosystem allows us to load ONNX-quantized models directly into memory.
- Data & Telemetry: PostgreSQL is utilized as the primary database for managing policies, storing telemetry, and logging historical decisions.
Specialized AI Pipeline:
- MiniLM + FAISS perform semantic retrieval and policy routing.
- DeBERTa & RoBERTa evaluate prompts for bias, toxicity, and policy violations.
- Gemini (Vertex AI) performs asynchronous reasoning, audit generation, and rule synthesis.
Engineering Decisions
1. Real-Time Semantic Guardrails (The FAISS Decision) For AI models already in production, Aegis AI acts as an inline API proxy. Every user prompt is intercepted and passed through our semantic router.
- Engineering Tradeoff: Instead of using a managed vector database (like Qdrant or Pinecone), we utilized FAISS running locally within the inference memory space. Because every request passes through the guardrail pipeline, eliminating network latency was critical. Running FAISS in-memory allowed semantic routing without an external vector database hop.
- If a prompt triggers a policy boundary, it is routed to the specific classifier for a final block/allow verdict.
2. Natural Language Rule Builder Engineers and compliance officers can author strict semantic boundaries using plain English.
- How it works: When a user submits a rule (e.g., "Block requests that specify geographic boundaries"), a background LLM translates this intent into a multi-faceted policy object. It generates semantic embeddings and classifier rules that capture intent rather than relying on brittle keyword matching. This eliminates the brittleness of regex-based keyword blocking.
3. Asynchronous Self-Healing & Human-in-the-Loop Static rules decay over time as users find new ways to bypass them or as new bias vectors emerge. We engineered an automated feedback loop—strictly bounded by a human approval process.
- Live telemetry (prompts, latency, rule scores) is continuously logged.
- An asynchronous auditor reviews this telemetry in the background to identify false positives, false negatives, and emerging threat patterns.
- Safety Boundary: The engine does not automatically modify production guardrails. Instead, it performs a gap analysis and pushes new rule configurations to a "Rule Suggestion Queue." A Security Admin must review the AI-generated fix, approve it, and manually trigger the deployment to update the active index and classifiers.
4. Batch Data Auditing (CSV Analysis) Before a model is even trained, the data must be sanitized. Aegis AI features a Bulk CSV Engine designed to process historical datasets.
- The Bias Audit Engine scans structured data to detect unbalanced representation and compliance violations, ensuring the AI model does not absorb foundational biases before deployment.
Deployment & Developer Experience
To ensure complete isolation of memory-intensive model execution, the system was deployed and tested on a dedicated Virtual Machine (VM). Background auditing jobs execute asynchronously within this environment, allowing real-time requests to remain unaffected by long-running analysis.
For developers, the deployed platform exposes a secure API for seamless integration into existing AI pipelines, complemented by an Interactive Playground. This allows engineering teams to safely test payloads, evaluate semantic rule triggers, and visualize exact latency breakdowns in real-time before pushing policies to production.
Engineering Outcomes
The resulting architecture delivers:
- Sub-second runtime inference overhead.
- Explainable block decisions tied to specific semantic triggers.
- Policy versioning for reliable state management and rollbacks.
- Human approval gating before any active deployment of self-healed rules.
- Unified monitoring for both batch data and live runtime traffic.
Project Snapshot
- Project Type: AI Safety & Compliance Platform
- Core Stack: Next.js • Python • PostgreSQL • Vertex AI • ONNX • FAISS • Hugging Face
- Engineering Highlight: Multi-model inference pipeline combining semantic routing, specialized classifiers, and LLM-assisted rule generation for low-latency AI safety.