Reserved GPU capacity on AWS saw price jumps of roughly 15% in January and 20% in July, totaling approximately 38% for H200 instances that absorbed both increases. Consider a platform team tracking a training job's expenses as it transfers several hundred terabytes of sensor data across regions simply to access available GPUs—only to discover later that the data movement was unnecessary. For AI systems, the placement question boils down to a single choice: send compute to existing data, or move data to where compute lives. Data gravity typically settles the matter.
Data gravity exerts stronger pull on AI than on analytics
Data gravity describes the economic force pulling compute, applications, and services toward concentrations of data, since transferring data often costs more than relocating the logic that processes it. As datasets expand, each copy becomes progressively more expensive, and systems requiring that data naturally cluster nearby.
Business intelligence queries can work against sampled or aggregated snapshots on a schedule. AI training at scale demands repeated full-dataset reads, while agent-based systems continuously retrieve fresh context throughout execution. Cross-region or cross-environment reads incur transfer fees and latency penalties. Each agent output generates new data feeding the next retrieval cycle, intensifying the gravitational pull over time.
A January 2026 analysis published in BigDATAwire concluded that AI workloads eliminated the tolerance enterprises had maintained for ignoring data gravity over the previous decade. Recent pricing adjustments reinforce that message from a financial angle. The cumulative effect of both AWS increases reaches roughly 38% for H200 instances experiencing both hikes.
Centralizing compute fails at AI scale
Concentrating compute resources often requires copying data into that environment before execution begins, and such transfers frequently incur charges. For a concrete example: assume a transfer rate of $0.02 per GB and a 300 TB dataset totaling 300,000 GB. Transfer costs alone would reach $6,000 per copy. This is illustrative only, not an AWS price; actual charges vary by source, destination, routing, service, and contract terms. Cross-region and cloud-to-on-premises transfers require explicit cost analysis. Remote reads also introduce network latency, a critical factor when workloads have strict response-time requirements.
Beyond GPU rates, two additional expenses often escape cost calculations:
- Interoperability overhead: pipelines spanning environments need translation layers when engines, query languages, connectors, and software development kits (SDKs) diverge.
- Lock-in: transfer charges and engineering labor required to migrate data and pipelines can make switching environments prohibitively expensive.
Transfer bills, duplicate pipelines and the engineering hours spent maintaining them belong in the placement calculation alongside compute charges.
Bringing compute to data instead
A scheduling and placement layer can route jobs to environments already holding their datasets, eliminating repeated full-dataset transfers. The job still requires compatible accelerators, sufficient memory, a supported runtime, and adequate storage throughput. Model weights, container images, and checkpoints may still need transfer.
Sound placement decisions integrate four inputs at each scheduling event:
- Cost: GPU rates, transfer fees, storage, and queueing time
- Data gravity: where the dataset resides
- Residency: legal or contractual location requirements
- Policy: access rules and approved environments
A placement layer first eliminates environments violating residency or access rules, then ranks feasible options by capacity, performance, and total cost. It dispatches the job with its runtime policies and reassesses placement if those conditions shift.
Placement depends on workload characteristics
No single rule applies universally. Each workload demands its own evaluation.
- Bursty training: public cloud suits elastic demand if suitable GPU capacity exists and transfer costs don't eliminate savings.
- Residency-restricted work: applicable law or contract dictates permitted jurisdiction and environment. An approved in-region cloud may qualify where an out-of-region on-premises cluster cannot. Large data volume adds economic constraint, not legal obligation.
- Transaction-near inference: place execution close enough to the transaction to meet latency targets, whether in cloud, on-premises, or at the edge.
Return to the platform team scenario. Their training job reads a large dataset from an on-premises store, and policy restricts processing outside an approved region. A local cluster provides required GPU memory and runtime. An approved cloud region offers suitable GPUs but requires dataset copying. They compare completion time and total cost, including transfer, storage, and queueing. If local execution meets the deadline at lower total cost, the job runs there with identical access controls. If not, they evaluate the approved cloud option.
If neither environment satisfies policy and capacity constraints, the job gets queued or rejected rather than sent to an unapproved region with restricted data. They document the choice and its assumptions so that price changes, dataset growth, or missed deadlines trigger re-evaluation.
Formal governance on this topic remains inconsistent. BARC's State of Data and AI Governance study, sponsored by Acceldata, found that 56% of North American respondents maintain formal sovereignty policies for AI processing and storage, compared to 30% in Europe, the Middle East and Africa (EMEA). This contradicts expectations, given Europe's reputation for strict data regulations. EMEA respondents also cite siloed data as a top governance obstacle more than twice as frequently as North American counterparts—35% versus 17%.
Hybrid architecture outlasts cloud-only transitions
Hybrid can represent a permanent architecture, not merely a waypoint toward cloud consolidation. Data-intensive, latency-sensitive, or sovereignty-bound workloads can remain on-premises, while cloud handles elastic demand where its cost justifies the expense.
This requires consistent operational controls spanning both environments: identity, runtime compatibility, observability, and policy enforcement. An all-cloud infrastructure incurs transfer and lock-in costs when data must leave a region or provider. An all-on-premises infrastructure leaves bursty workloads waiting for fixed capacity.
Hybrid can be a lasting architecture, not merely a stop on the way to the cloud.
Job portability works only where target environments support required engines, dependencies, and hardware. Open table formats such as Apache Iceberg, Delta Lake, and Apache Hudi help here, because multiple engines can read the same tables in place. Moving a job no longer requires converting or copying data first.
This challenge motivated Acceldata to build its xLake platform: keep data where it resides in open formats, and bring governed compute to it across cloud and on-premises environments.
Sovereign AI demands more than GPU inventory
Sovereign AI discussions typically begin with GPU capacity, but GPU compute alone forms an incomplete architecture. Owning or reserving GPU capacity says nothing about whether the data feeding those GPUs, or the retrieval and orchestration layers surrounding them, stays within infrastructure the organization controls. Sovereignty encompasses the entire data and AI stack. Compute represents only one layer.
For strict customer-controlled boundaries, Acceldata's May 2026 analysis of sovereign GPU infrastructure identifies four required controls:
- Route tool calls through internally hosted endpoints so operational data, prompts, and workflow context never exit the organization's network.
- Retrieval infrastructure (embedding indexes and vector search) running on infrastructure the organization controls rather than a provider-managed vector service.
- Orchestration of prompts, memory, and intermediate agent state inside the organization's own environment.
- Access policies for permissions and retrieval scope enforced within that same governed boundary.
A workload can run on the most expensive reserved GPUs available and still fail a sovereignty review if its prompts, embeddings, or agent state cross a boundary they were never cleared to cross.
Placement decisions must account for all four controls. A workload running on premium reserved GPUs can still fail sovereignty requirements if prompts, embeddings, or agent state cross unapproved boundaries.
Send workloads to existing data
Compute price changes warrant placement review, but they do not prove one environment always costs less. Transfer charges, storage, engineering effort, and completion time belong in the same comparison. The expense of reaching data must become a primary input to placement decisions, not a secondary note appended to GPU pricing.
Retain assumptions alongside each decision: dataset location, permitted destinations, compatible capacity, expected completion time, and total cost. When any assumption changes, reassess the affected workload. Where running beside the data proves advantageous, move the job instead of the dataset.
Teams that reconsider placement now, while datasets remain manageable, will expend less effort fighting data gravity later than teams that continue copying data to wherever GPUs happen to be available.
That lesson came hard to the platform team at the beginning of this scenario. The data never needed to move. The job did.