When OpenAI revealed at DevDay that ChatGPT had surpassed 1.2 billion weekly active users—approximately 15% of the global population—the engineering community focused on model context windows, compute clusters, and distributed GPU inference. Yet beneath every chat session, API call, and model invocation lies an equally demanding systems problem: Identity and Access Management (IAM) operating at internet scale.

Before OpenAI's backend delivers a token from an LLM, it must authenticate the request, validate OAuth2 access tokens, check fine-grained permissions, and preserve secure session contexts across the globe. To handle this enormous transaction volume without creating latency bottlenecks, OpenAI partnered with Ory, leveraging its lightweight, decoupled authentication engine and OAuth2/OIDC infrastructure.

The scale problem: human traffic vs. agentic workloads

Traditional IAM platforms struggle at extreme scale because user authentication patterns shift fundamentally as systems evolve from standard web applications to agentic AI platforms. When an enterprise transitions from human-initiated UI clicks to autonomous AI agents—such as OpenAI's Work and Codex suites—the ratio of token validations to human interactions scales non-linearly:

  1. Transaction velocity: A single user request to an AI agent can spawn dozens of downstream sub-tasks, database queries, and third-party API tool calls. Each step demands session verification and token introspection.
  2. Token introspection overhead: Legacy central identity providers typically validate access tokens through centralized RPCs or database lookups. At billions of requests per minute, centralized introspection creates severe network hops and database lock contention.
  3. Stateless edge verification: To prevent authorization from becoming a request-path bottleneck, session validation must occur synchronously at the edge using lightweight cryptographic verification (such as JSON Web Key Sets / JWKS caching) rather than stateful network calls.

Architectural comparison: modern composable IAM vs. legacy monolithic SaaS

Engineering teams evaluating identity platforms for ultra-high throughput frequently encounter performance and cost barriers with traditional multi-tenant identity SaaS offerings (Auth0, Okta, Ping Identity):

  • Legacy approach: Centralized IdP round-trips or heavy server-side SDKs; user store hosted in vendor-managed SaaS clouds; proprietary SaaS database clusters subject to vendor rate limits; per Monthly Active User (MAU) or per-API request execution fees; restricted to vendor webhooks, rules, or custom code sandboxes
  • Modern approach: Edge-native stateless JWT validation & local session caching; complete data sovereignty (runs directly in your VPC / multi-cloud k8s); highly available distributed SQL layers (CockroachDB, Spanner, Postgres); infrastructure-based scaling with cost correlating to compute/memory footprint; Go-native microservices with clean API boundaries and headless UI integration

OpenAI wanted a partner that could help enable our vision for owning our identity processes, data, and success. We have a lot of partners, and Ory is one of our best.

Benjamin Billings, Engineering Manager, OpenAI

Key infrastructure requirements for billion-user identity systems

Securing identity at scale demands clear separation of concerns across the auth stack. High-throughput deployments like OpenAI's depend on four core principles:

1. Separation of identity state from business logic

Monolithic frameworks often bind user profiles, authentication state, and business roles together in single relational databases. High-throughput architectures decouple these responsibilities into distinct microservices:

  • Identity storage & auth engines: Dedicated microservices handling identity lifecycle, credential hashing, and session management (e.g., Ory Kratos)
  • OAuth2 / OIDC token issuance: High-performance OAuth2 server implementations capable of issuing signed access tokens and managing grant flows with minimal latency (e.g., Ory Hydra)
  • Fine-grained access control (FGAC): Relationship-based access control engines based on Google's Zanzibar model (e.g., Ory Keto) to evaluate tuple relationships (ObjectRelationSubject) in sub-millisecond execution times

2. Distributed persistence across cloud regions

To support global edge routing, the underlying storage layer must enable geo-distributed writes and low-latency local reads. Combining a self-hosted identity engine with distributed SQL engines (such as CockroachDB or cloud-native multi-region databases) ensures token-verification state replicates globally without split-brain risks or cross-continental network latency.

3. Agent security: transitioning from "enforce first" to "observe mode"

As enterprises deploy autonomous agents that execute commands on users' behalf, writing restrictive IAM policies immediately can break workflows or cause silent failures. Modern security practices emphasize runtime observability before strict policy enforcement:

  1. Agent fingerprinting: Map every machine-to-machine (M2M) credential and agent service account operating across internal networks
  2. Execution tracing: Audit tool usage, API invocations, and database queries initiated by autonomous runtimes
  3. Identity delegation: Implement scoped OAuth2 Token Exchange (RFC 8693) to ensure agents operate under strict downscoped permissions tied directly to the delegating human user

For a limited time, you can create a complimentary Ory account and turn on observe mode to learn how best to build enforcement.

Identity as a distributed systems problem

Scaling authentication to 1.2 billion weekly active users demonstrates that identity is no longer just a security checkbox; it is a core distributed systems challenge. As software shifts from human-driven UI interactions to high-velocity AI agent execution, engineering teams must evaluate whether their identity stack can handle exponential API volume. Decoupled, stateless-first identity engines offer the low latency, data sovereignty, and predictability modern platforms need to scale into the agentic era.