The platform

Three modules, in detail.

Private AI, Vision and Cyber defense share one software stack and one rule: the models, the data and the audit trail live on hardware the customer owns.

Module 01 · Prototype running

Private AI: the software stack.

A chat workspace that answers questions about the organization's own documents, with every part of it running in the building. What runs today, and the production path for each layer:

Model servingOllama today. NVIDIA NIM on TensorRT LLM in production.
Chat workspaceOpen WebUI with single sign on, roles and an audit log.
ModelsNemotron, Qwen and Gemma, picked per system size.
RetrievalA local document index today. NeMo Retriever for embeddings and reranking.
SafetyNeMo Guardrails for topic, PII and jailbreak rails.
DataEncrypted disk, per user access, no outbound traffic.

Reference systems

Three sizes, set by headcount.

The software is identical on all three. Only the NVIDIA hardware underneath changes.

Compact desktop AI system

CORE

5 to 10 people

NVIDIA DGX Spark with 128 GB unified memory and a 20 core Arm CPU. Runs models up to about 120B parameters (4 bit mixture of experts) plus vision. 10GbE with a capture port.

Tower workstation

TEAM

About 25 people

RTX PRO 6000 Blackwell Max-Q with 96 GB GDDR7 ECC, Threadripper 9960X and 256 GB RAM. Runs a 70B dense or 120B mixture of experts model with vision. Dual 10GbE with a capture port.

Rack mounted server

ENTERPRISE

75 plus people and campuses

Two RTX PRO 6000 Blackwell Server GPUs with 192 GB GDDR7 ECC, expandable to four, EPYC 9355 and 512 GB RAM. Several large models at once. Dual 25GbE, capture port and BMC.

Growth path: Core links a second DGX Spark, Team adds a second GPU, Enterprise grows to four GPUs and adds Jetson edge units.

Module 02 · Vision

Detect the events that matter. Then prove they are real.

Detection classes

Firearm, aggressive behavior, falls, loitering, and fire and smoke. One model family, one alert pipeline.

Training and serving

Fine tuned with NVIDIA TAO, optimized with TensorRT, served through Triton and streamed through DeepStream.

Metrics we will publish

Precision, recall, mean average precision, and false alarms per camera per day, all on a held out real test set.

Office lobby with detection boxes drawn around two people

Training data

Synthetic first. Real data to validate.

Real gun footage is rare and legally hard to collect. So we generate it: labeled scenes with varied lighting, angles, occlusion and camera types.

1 · Scene designOmniverse3D sites such as lobbies, parking lots, hallways and school entrances
→
2 · GenerateReplicator + CosmosRandomized lighting, angles, props, crowds and weather, plus realistic video variation
→
3 · Auto labelFrom the simulatorBoxes, masks and class tags with no manual labeling
→
4 · TrainTAOA detector, then a ViT and multimodal check for hard cases
→
5 · DeployTensorRTEngine in Triton and DeepStream on the customer GPU

Validation rule

Every model is tested on real footage it never trained on. Synthetic data widens coverage. Real data decides whether we ship.

Why it matters

Labeled, privacy safe training data for rare events is the scarce asset. Each new site adds scenes without exposing a single real video.

Architecture

Detect at the edge. Reason in the core.

EdgeJetson at the cameraDeepStream plus Metropolis microservices run the detector on the stream. Only events and short clips move.
→
SiteAirman Vision serverCore, Team or Enterprise system: VLM second check, footage search, analytics, agents.
→
Core, optionalDGX or private cloudHeavy VLM agent, fleet analytics and retraining in the customer's own cloud account.

Data stays put

Raw video stays on the edge or site box. The upstream link carries alerts, metadata and clips the customer approves.

Planned

Open vocabulary detection

A VLM lets operators describe a new threat in plain words and get a detector candidate without retraining first. A demo with open source VLMs is planned.

The second check

Every alert is re read by a vision language model before a human sees it.

1Detector firesA fast YOLO class model flags a clip
→
2VLM reviewFrames plus a prompt: is a person holding a firearm? Explain.
→
3DecisionConfirm, downgrade or suppress, with the reason stored
→
4HumanA verified alert and the explanation

Module 03 · Planned

Watch the LAN. Map findings to the controls auditors ask about.

Network defense agent

NVIDIA Morpheus reads mirrored traffic on the capture port. Phishing detection and digital fingerprinting flag users and devices that behave out of pattern.

CMMC compliance agent

Findings and logs mapped to CMMC practice families: access control, audit and accountability, incident response, and system and communications protection.

Evidence on demand

The agent assembles a dated evidence pack per control: what was monitored, what fired, what was done.

Honest scope

The software collects and organizes evidence. It does not certify anyone. The assessor does.

Close up of glowing network hardware

Inside Private AI

A local agent that can act, with a human holding the keys.

Prototype running

Prototype today

Ollama and Open WebUI with MCP tools for browser and PC control, running on an RTX 5090 system.

Planned

Production design

The agent runs on NIM with NeMo Guardrails: allowed tools, allowed sites, PII rails, and refusal of off policy actions.

Built in agent
→
Human approval
→
Send, delete, submit, purchase

Human approval

Any action that changes something pauses for a click from the operator.

Full logging

Every prompt, tool call and result is written to an append only log the customer owns.

All planned

Analytics on GPU, plus two demos that prove the pattern.

Crime Locater

Map and trend analysis of incident data accelerated with RAPIDS cuDF. We will publish GPU versus CPU speedups on the same dataset.

Open source VLM demo

Edge and cloud designs with open vocabulary detection, shown as a public demo of the second check idea.

VLM agent on DGX

A larger VLM agent on a DGX that reviews flagged events and answers investigator questions.