source: hugging face blog: hugging face models on foundry managed compute
level: technical
microsoft foundry is a platform for building agentic ai applications with a wide model selection from various providers, all accessible through a single endpoint and sdks. it includes the foundry agent service for multi-agent orchestration with memory, knowledge grounding, and tool connections. the platform provides end-to-end tracing, monitoring, evaluations, and a prompt optimizer. developers also get content safety filters, guardrails, an ai red teaming agent, unified rbac, private networking, and azure policy integration. foundry managed compute is a managed gpu platform-as-a-service for open-source and custom models, handling gpu topology automatically. microsoft manages container updates, runtime upgrades, and security patches for supported runtimes like vllm, sglang, tensorrt-llm, nim, tei, and llama.cpp without redeploying models.
hugging face models on foundry brings a curated subset of models into the foundry model catalog, refreshed weekly and covering text, vision, audio, and multimodal modalities. models are security-screened, using safetensors format with no untrusted code unless rigorously reviewed. the right runtime is selected for each model, such as vllm and sglang for llms, tei for embeddings, and llama.cpp for cpu. a multi-stage curation pipeline identifies trending models, screens for compliance and security, builds and scans runtimes, uploads weights to azure storage, and validates performance before publishing. weights are pre-staged in azure, so deployments can run inside private networks without outbound access to hugging face hub.
deploying a model involves browsing the catalog, choosing a deployment template that pins the runtime, accelerator, context length, and tuning, configuring instance count, and deploying via portal, cli, sdk, or rest. scoring uses the unified foundry endpoint with the openai sdk. models can also be used in foundry agents. available now in preview are thousands of models deployable onto nvidia a100, h100, or amd mi300x accelerators in global and data zone scopes.
why it matters: this integration simplifies deploying and managing open-weight models at scale with enterprise security and governance, reducing operational overhead for ai teams.
source: hugging face blog: hugging face models on foundry managed compute