How Azure AI Works
A layered platform: AI supercomputers, 1,900+ models, and Microsoft Foundry on top — where you build, ground, evaluate and ship AI apps and agents.
Microsoft Foundry
The unified AI platform (was Azure AI Foundry / AI Studio). One resource with projects, model deployments, playgrounds, a single SDK & Responses API, plus tracing, evaluations and policy.
Azure OpenAI & Foundry Models
GPT-5.5, GPT-5.4 and o-series from OpenAI plus Claude, Llama, Mistral, DeepSeek, Grok and Phi — 1,900+ models — hosted in your tenant with enterprise SLAs and fine-tuning.
Azure AI Search + RAG
Index your documents as vectors, retrieve the most relevant passages at query time, and feed them to the model so answers are grounded in your data with citations.
Foundry Agent Service
Turn a model into an agent that uses tools, remembers context, calls other agents (A2A), runs multi-step routines, and publishes to Teams/M365 — with governance built in.
Azure AI Services
Prebuilt APIs for specific tasks: Vision (OCR), Speech (STT/TTS), Language (sentiment, summarization), Document Intelligence and Translator.
Responsible AI & Security
Content Safety filters harmful input/output; evaluations score quality & safety; Entra ID, Key Vault, private networking and Azure Policy keep data in your tenant.
GPT-5.5 · GPT-5.4 · o-series
Flagship reasoning & multimodal (GPT-5.5), best balance for production (GPT-4.1 / 4.1 mini), deep step-by-step reasoning (o-series). Plus GPT-4o (omni), DALL·E (images), Sora (video), Whisper (speech), text-embedding-3 (vectors).
Claude — Opus · Sonnet · Haiku
Strong reasoning, code generation and multimodal with long context; tiered for capability vs speed/cost.
Llama 4
Open-weight models for customization and fine-tuning — full control, on-prem / edge options.
Mistral
Efficient code generation, multilingual and general-purpose chat.
DeepSeek V4
Open-weight reasoning at scale — strong math & code reasoning.
Grok 4.20
Reasoning, coding and data extraction.
Phi
Small language models (SLMs) for on-device, resource-constrained and low-latency scenarios.
Open catalog (Cohere, AI21, NVIDIA…)
Thousands of open models deployable to managed endpoints — pick the right one per task.
| Capability | Use it for | Example models |
|---|---|---|
| Chat & reasoning | Q&A, agents, copilots, complex multi-step tasks | GPT-5.5, Claude Opus 4.8, o-series, DeepSeek V4 |
| Multimodal | Understand text + images (+ audio/video) | GPT-5.5, GPT-4o, Claude Opus 4.8 |
| Embeddings | Vector search & RAG grounding | text-embedding-3 |
| Image generation | Create & edit images | DALL·E, Stable Diffusion |
| Video generation | Generate video from prompts | Sora |
| Speech | Speech-to-text, text-to-speech, realtime audio | Whisper, GPT-Realtime |
| Small / edge (SLM) | On-device, low-cost, low-latency | Phi-4 |
One API, any model
The Responses API + model router call any model by name; "instant access" lets you skip creating a deployment first.
Pay-as-you-go or provisioned
Standard (per-token) for elasticity, or Provisioned Throughput (PTU) for predictable, reserved capacity.
Fine-tuning & grounding
Fine-tune on your data, or ground with RAG (AI Search / Foundry IQ) — no model training required.
Safety, security & regions
Content filtering, evaluations, Entra ID / private networking and data-residency control — models run in your tenant.
User prompt
A question arrives from an app, copilot or agent endpoint.
Retrieve
AI Search finds the most relevant passages from your indexed data.
Ground
Retrieved context is added to the prompt so the model answers from your knowledge.
Reason
The model (Responses API) generates an answer; an agent may call tools.
Safety
Content Safety checks input & output; policies are applied.
Respond + trace
The cited answer returns; observability logs tokens, latency & quality.
| Term | What it means |
|---|---|
| Foundry project | The workspace grouping your models, agents, tools, data connections and access for one app. |
| Deployment | A specific model made callable at an endpoint (this is what you pay for, per token). |
| Responses API | The unified, stable API to call any model or agent — one contract across providers. |
| RAG | Retrieval-Augmented Generation — ground a model in your data instead of only its training. |
| Agent | A model plus tools, memory and instructions that can take multi-step actions toward a goal. |
| Foundry IQ | Knowledge layer grounding agents in enterprise or web content with citation-backed answers. |
| Content Safety | Service that detects and filters harmful content in prompts and responses. |
| Evaluation | Scoring model/agent output for quality, groundedness and safety — before and after shipping. |
Sources: Microsoft Foundry overview · Azure AI Services · Azure AI Search. Reflects Azure AI as of 2026.