How it works

How Azure AI Works

A layered platform: AI supercomputers, 1,900+ models, and Microsoft Foundry on top — where you build, ground, evaluate and ship AI apps and agents.

The Azure AI stack — bottom to top
Each layer builds on the one below. You work at the top three; Azure runs everything beneath.
6 · ApplicationsCopilots, custom apps, Copilot Studio (low-code), embedded in Microsoft 365, Teams & Fabricwhat users see
5 · AgentsFoundry Agent Service — tools (1,400+), memory, multi-agent orchestration, agent-to-agent (A2A)act & reason
4 · Grounding (RAG)Azure AI Search (vector + keyword + semantic) & Foundry IQ — ground answers in your data with citationsyour knowledge
3 · PlatformMicrosoft Foundry — portal, projects, SDK, Responses API, evaluations, observability, content safety, RBACbuild & govern
2 · ModelsFoundry Models catalog — Azure OpenAI (GPT-5.5/5.4), Claude, Llama, Mistral, DeepSeek, Grok, Phithe brains
1 · InfrastructureAzure global datacenters & AI supercomputers — NVIDIA GPU clusters + Microsoft Maia accelerators, 70+ regionsthe compute
The building blocks
The services that make up Azure AI and what each is for.
Platform

Microsoft Foundry

The unified AI platform (was Azure AI Foundry / AI Studio). One resource with projects, model deployments, playgrounds, a single SDK & Responses API, plus tracing, evaluations and policy.

Models

Azure OpenAI & Foundry Models

GPT-5.5, GPT-5.4 and o-series from OpenAI plus Claude, Llama, Mistral, DeepSeek, Grok and Phi — 1,900+ models — hosted in your tenant with enterprise SLAs and fine-tuning.

Grounding

Azure AI Search + RAG

Index your documents as vectors, retrieve the most relevant passages at query time, and feed them to the model so answers are grounded in your data with citations.

Agents

Foundry Agent Service

Turn a model into an agent that uses tools, remembers context, calls other agents (A2A), runs multi-step routines, and publishes to Teams/M365 — with governance built in.

Task AI

Azure AI Services

Prebuilt APIs for specific tasks: Vision (OCR), Speech (STT/TTS), Language (sentiment, summarization), Document Intelligence and Translator.

Trust

Responsible AI & Security

Content Safety filters harmful input/output; evaluations score quality & safety; Entra ID, Key Vault, private networking and Azure Policy keep data in your tenant.

Models & modalities — the full model suite
Microsoft Foundry gives you 1,900+ models from many providers under one Responses API. Here are the families, what they cover by modality, and how Azure serves them.
Model families (by provider)
OpenAI · Azure OpenAI

GPT-5.5 · GPT-5.4 · o-series

Flagship reasoning & multimodal (GPT-5.5), best balance for production (GPT-4.1 / 4.1 mini), deep step-by-step reasoning (o-series). Plus GPT-4o (omni), DALL·E (images), Sora (video), Whisper (speech), text-embedding-3 (vectors).

Anthropic

Claude — Opus · Sonnet · Haiku

Strong reasoning, code generation and multimodal with long context; tiered for capability vs speed/cost.

Meta

Llama 4

Open-weight models for customization and fine-tuning — full control, on-prem / edge options.

Mistral

Mistral

Efficient code generation, multilingual and general-purpose chat.

DeepSeek

DeepSeek V4

Open-weight reasoning at scale — strong math & code reasoning.

xAI

Grok 4.20

Reasoning, coding and data extraction.

Microsoft

Phi

Small language models (SLMs) for on-device, resource-constrained and low-latency scenarios.

Hugging Face & more

Open catalog (Cohere, AI21, NVIDIA…)

Thousands of open models deployable to managed endpoints — pick the right one per task.

By modality — what they can do
CapabilityUse it forExample models
Chat & reasoningQ&A, agents, copilots, complex multi-step tasksGPT-5.5, Claude Opus 4.8, o-series, DeepSeek V4
MultimodalUnderstand text + images (+ audio/video)GPT-5.5, GPT-4o, Claude Opus 4.8
EmbeddingsVector search & RAG groundingtext-embedding-3
Image generationCreate & edit imagesDALL·E, Stable Diffusion
Video generationGenerate video from promptsSora
SpeechSpeech-to-text, text-to-speech, realtime audioWhisper, GPT-Realtime
Small / edge (SLM)On-device, low-cost, low-latencyPhi-4
How Azure delivers & supports them
Access

One API, any model

The Responses API + model router call any model by name; "instant access" lets you skip creating a deployment first.

Deploy

Pay-as-you-go or provisioned

Standard (per-token) for elasticity, or Provisioned Throughput (PTU) for predictable, reserved capacity.

Customize

Fine-tuning & grounding

Fine-tune on your data, or ground with RAG (AI Search / Foundry IQ) — no model training required.

Govern

Safety, security & regions

Content filtering, evaluations, Entra ID / private networking and data-residency control — models run in your tenant.

How a single request flows
A typical grounded (RAG) agent request, end to end.
1

User prompt

A question arrives from an app, copilot or agent endpoint.

2

Retrieve

AI Search finds the most relevant passages from your indexed data.

3

Ground

Retrieved context is added to the prompt so the model answers from your knowledge.

4

Reason

The model (Responses API) generates an answer; an agent may call tools.

5

Safety

Content Safety checks input & output; policies are applied.

6

Respond + trace

The cited answer returns; observability logs tokens, latency & quality.

Key concepts
The vocabulary that shows up across Azure AI.
TermWhat it means
Foundry projectThe workspace grouping your models, agents, tools, data connections and access for one app.
DeploymentA specific model made callable at an endpoint (this is what you pay for, per token).
Responses APIThe unified, stable API to call any model or agent — one contract across providers.
RAGRetrieval-Augmented Generation — ground a model in your data instead of only its training.
AgentA model plus tools, memory and instructions that can take multi-step actions toward a goal.
Foundry IQKnowledge layer grounding agents in enterprise or web content with citation-backed answers.
Content SafetyService that detects and filters harmful content in prompts and responses.
EvaluationScoring model/agent output for quality, groundedness and safety — before and after shipping.

Sources: Microsoft Foundry overview · Azure AI Services · Azure AI Search. Reflects Azure AI as of 2026.