02Vinod Bijlani · The AI Stack

The AI Stack

From AI code in 2008 to the data hall today. I design AI factories layer by layer, from the agents people use down to the GPUs, fabrics, cooling and power underneath, at any scale from 12 kW to 12 MW.

Since 2008building AI solutions
12 kW → 12 MWAI factory designs
Building blocks of an agentic AI factory
AI agents
AI security and governance
AI and LLM tooling
Data and integration
Model operations
Foundation models
Software platform
AI-native hybrid infrastructure

Tap a bracket to see what each service includes

→

From 12 kW to 12 MW

The same principles at every size. What changes is power density, cooling, the fabric and how the factory is governed.

12 kW

One GPU server

Eight GPUs in a single node. A lab, a proof of concept or a first private model.

  • Air or direct liquid cooling
  • Standard rack power
  • Local NVMe and a shared file system
  • A single 400G uplink

130 kW

One rack-scale system

A rack that behaves like one giant GPU. Density rules out air cooling.

  • Direct liquid cooling to the chip
  • Coolant distribution units
  • Busbar power at 100 kW+ per rack
  • Scale-up GPU interconnect inside the rack

1 MW

A scalable unit

A pod of racks sized as a repeatable building block, often delivered as a modular data centre.

  • Rail-optimised 800G east-west fabric
  • Separate 400G north-south and storage networks
  • Parallel storage with GPU-direct access
  • Multi-tenant Kubernetes and a token gateway

12 MW

An AI factory

Many scalable units in a purpose-built hall. Power, heat and governance now decide the design.

  • Medium-voltage power and on-site distribution
  • Facility water loops and heat rejection
  • Spine-level fabric across pods
  • Sovereign operations
  • metering and chargeback
→

Case study. Inside a sovereign token factory

A national telecom operator in Asia. A sovereign AI platform that serves models to the operator's own applications and enterprise customers as a governed, metered token service, running on a dedicated GPU cluster inside the country.

What I designed

  • Designed the path from internet edge to GPU, with firewalls, load balancers and a 400G border gateway in front of the platform.
  • Built a multi-tenant token service with an OpenAI-compatible gateway, API keys, quotas, metering and inline prompt and response guardrails.
  • Ran inference on bare-metal Kubernetes GPU workers behind a highly available control plane, with several model runtimes side by side.
  • Separated three fabrics, an 800G InfiniBand east-west GPU fabric, a 400G converged north-south network and out-of-band management, plus a parallel file system with GPU-direct storage.
  • Integrated the operator's own identity, PKI, secrets, SIEM and scanning services so the platform fits existing security operations.

Product and customer names are left out. Speeds and topology are as deployed.

Edge and security

End usersApps · APIs
Operator networkInternet
FirewallHA pair
Load balancerHA pair
Web app firewallPlanned
Border gateway400G
Admin accessHTTPS console · 1G out-of-band

AI platform · token factory

Platform hosts · HAThree virtualisation hosts for token-factory controllers, guardrail engine, observability, registry, jump host and management
Data plane · token factory pods on the GPU cluster
AI gateway OpenAI-compatible API · keys · quotas · metering
Guardrails inline prompt and response checks
Model runtimesInference microservicesDistributed inferenceOpen-source LLM serving
Kubernetes control planeThree masters, highly available
Fabric and storage managementInfiniBand fabric manager · storage and cluster management
GPU compute · bare-metal Kubernetes workers
8 GPUs per node800G east-west per GPU2 × 400G north-south DPULocal NVMeGPU partitioningStorage client on every node
Parallel file system · all-flash NVMe
POSIXS3NFSGPU-direct storage2 × 400G per node
Kubernetes backupPlanned

Network fabric

East-west GPU back-end800G InfiniBand · rail-optimised leaf–spine
North-south converged400G Ethernet for storage, management and API · leaf–spine
Out-of-band management1G management and BMC for every server

Operator-provided services

Identity · SSO · MFADNSNTPPKISecrets vaultSIEMSMTP relayVulnerability scanningImage scanning
Follow a requestWatch a request move through the factory.

Invite Vinod to speak, write or advise.

Keynotes, panels and executive briefings on AI factories, sovereign AI and AI economics. Send a message on LinkedIn.