French Startup ZML Releases Free LLMD Inference Server Across Nvidia AMD Google TPU Apple Intel Chips

French AI infrastructure startup ZML released ZML/LLMD, a free inference-performance tool meant to accelerate open-source large language models across chips including Nvidia’s, AMD’s, Google’s TPU, Apple Metal, and Intel Arc.

“The French startup ZML aims to reduce companies' dependence on a single AI chip supplier”

KultureGeek

TechCrunch reported that ZML/LLMD is an LLM inference server whose ambition is to break existing silos and make different chips available for AI use cases at their maximum available speed, and sometimes faster, according to ZML founder Steeve Morin.

KultureGeek

The Tech Buzz described ZML/LLMD as chip-agnostic software designed to accelerate AI inference across diverse chip architectures while slashing operational costs, positioning it to sit at the center of the AI infrastructure stack.

The Tech Buzz also said the release is backed by Turing Award winner Yann LeCun, and that ZML is betting that democratizing inference optimization will help it become essential infrastructure as companies race to deploy AI applications.

Morin told TechCrunch that “The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated,” framing the release as a way to let enterprises and clouds choose a mix of chips.

TechCrunch also quoted Morin saying optimizing inference has been outpacing model training in importance, but often feels patchy behind the scenes due to software and architecture barriers that lead to vendor lock-in.

Let's Data Science

The Tech Buzz described LeCun’s endorsement as adding “serious weight,” saying the endorsement signals ZML’s technical approach has merit in a field crowded with startups making bold claims.

In the same Tech Buzz account, ZML’s free release is presented as a strategy to position the company as essential infrastructure, similar to how Meta open-sourced Llama to drive adoption.

TechCrunch said ZML faces competition including Baseten, recently valued at $13 billion, Inferact from the creators of open source project vLLM, and RadixArk, the commercial company behind SGLang.

“The days of Nvidia’s unparalleled market dominance aren’t over, but challengers and choices are arising from all directions”

TechCrunch

TechCrunch reported Morin credited ZML’s lean team of 20 people as the reason the Paris-based startup has been able to move fast, with more releases in the plans.

TechCrunch also said Morin raised $20 million from venture firms including 20VC, >commit, AALVC, Drysdale Ventures, Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures, and that ZML/LLMD is not open source but is launching as a free product with the goal of learning about usage.

The Tech Buzz added that major cloud providers like Amazon Web Services, Google Cloud, and Microsoft Azure are building their own inference optimization tools, creating distribution advantages that a startup cannot match easily.