Microsoft Unveils Windows ML 2.0 to Unify AI Development Across PC Hardware

Microsoft's latest push to democratize AI development centers on Windows ML 2.0, a core runtime designed to function as the AI equivalent of DirectX—the technology that transformed Windows 95 into a gaming platform by abstracting hardware complexity. The new framework, unveiled at Microsoft's Build developer conference, allows machine learning models to run seamlessly across diverse PC hardware configurations without requiring developers to manage compatibility headaches.

The problem Windows ML 2.0 addresses mirrors challenges from PC gaming's early days. Just as games in the MS-DOS era required specific sound cards and drivers to function, AI models today struggle with fragmented hardware ecosystems. DirectX solved this by creating a unified interface that let games run on any hardware. Microsoft now aims to replicate that success in the AI space, where Windows PCs ship with processors from Intel, AMD and NVIDIA, along with neural processing units from Qualcomm and other vendors.

Built on the ONNX runtime, Windows ML 2.0 functions as a wrapper enabling developers to deploy their own AI models and build applications without worrying about underlying hardware specifics. The runtime can compile reasonably sized models within minutes, automating the detection and support of available hardware while extracting maximum performance from AI chips.

Learning from DirectML's Limitations

Microsoft previously relied on DirectML, a GPU-focused descendant of DirectX, to handle AI workloads. However, the company encountered performance bottlenecks while developing "Click to Do," a feature that identifies and acts on text and images displayed on screen.

We've come to the realization that we need something faster

Ryan Demopoulos, principal product manager at Microsoft

Windows ML 2.0 expands support beyond GPUs to include CPUs and neural-processing units, ensuring AI applications automatically leverage whatever acceleration hardware is available on a given device.

Hardware Abstraction and Dependency Management

The runtime handles identification of available hardware and automates support for AI models across the diverse ecosystem. It also manages dependencies, procurement and updates, bundling these typically manual tasks into installers—a significant simplification for fragmented environments.

An experimental version is now available for developer testing, though Microsoft cautions against production use. According to Demopoulos, "It's not yet ready or meant for production apps. Please don't use it in your production apps."

Industry Perspective: The 'Holy Grail'

Reincubate, the company behind the popular Camo webcam application, participated in early testing of Windows ML 2.0. The firm sees the technology as addressing a long-standing challenge in AI deployment.

The holy grail is being able to take a single high-precision model and have it JIT — or 'just work' — seamlessly across Windows silicon with different drivers, different capabilities and different precision

Aidan Fitzpatrick, CEO of Reincubate

Fitzpatrick emphasized that having Windows manage framework and model versioning dynamically reduces friction. Reincubate is developing a real-time retouching feature for Camo that adjusts user appearance during meetings and streams, leveraging Windows ML alongside feature and landmark detection models.

Windows ML has a lot of existing, robust components behind it such as ORT (ONNX runtime), and that's made it a lot more straightforward than it otherwise might have been — in adopting it, we've not had to blow things up or start over

Fitzpatrick

Windows ML should be a powerful tool in … helping us to move at the speed of silicon innovation

Fitzpatrick

How the Runtime Operates

Developers import AI models through Windows ML public APIs, and the runtime automatically identifies hardware and manages dependencies. An "execution provider" layer bridges models to silicon, identifying hardware capabilities and scaling performance accordingly. This eliminates the need to create separate executables for different configurations, while Microsoft handles runtime updates so teams can focus on model development.

Once your app is installed and initializes Windows ML, then we will scan the current hardware and download any execution providers applicable for this device

Xiaoxi Han, senior software engineer at Microsoft

Practical Implementation Walkthrough

During a Build demonstration, Han showed how developers can get AI models running on Windows 11 PCs using the VS Code extension toolkit. The example converted a ResNet model to evaluate an image of a puppy.

The workflow begins with converting models to the open-source ONNX format, optimizing them and applying quantization. Models from Hugging Face can be converted directly to ONNX. A small ResNet model took approximately 30 seconds to convert.

If you have a PyTorch model, if you've trained one or if you've obtained one, you can convert it into ONNX format and run it with Windows ML to run your on-device workloads

Demopoulos

The conversion feature optimizes models for NPUs from Intel, Qualcomm and AMD. Microsoft plans to eventually eliminate this step as conversion support expands to all chips.

Creating the application in Visual Studio involves starting a console project, configuring .NET version and Windows OS target in project properties, and installing the Microsoft.AI.Windows.MachineLearning NuGet package, which includes ONNX runtime components and ML Layer APIs for execution providers. NuGet automatically resolves dependencies between application code and the Windows ML runtime.

The developer then initializes an ONNX runtime environment and Windows ML, creating an infrastructure object that scans hardware and downloads relevant execution provider packages. One example is QNN, which enables AI models to leverage NPUs in laptops with Qualcomm's Snapdragon chip.

Standard AI coding follows: setting file paths for the model, labels and images, configuring an ONNX session for inference, loading images and establishing policies based on chip type or power consumption. Running inference processes the image tensor through the ONNX session, returning raw prediction scores that are converted to probabilities and human-readable format.

Developers can specify performance preferences through configuration options. "MAX_PERFORMANCE" prioritizes speed, while "PREFER_CPU," "PREFER_NPU" or "PREFER_GPU" suit background workloads. Additional settings enable minimal-speed operation to conserve battery life.

In the not-too-distant future, we also want to add … 'workload splitting.' You can have a single AI workload that is split across multiple different types of processors to get even greater performance

Demopoulos

The complete demonstration codebase is available on GitHub.

API Architecture

The Windows ML layer comprises initialization APIs under Microsoft.Windows.AI.MachineLearning that maintain runtime currency and download necessary components for model-to-hardware communication. A separate generative AI API layer, Microsoft.ML.OnnxRuntimeGenAI.WinML, supports large language model workflows. A runtime API layer grants developers fine-grained control over model execution.

While these layers are exposed through WinRT, Microsoft also provides flat C wrappers with managed projections, allowing developers to avoid learning WinRT directly.

Source: The New Stack