Building the Foundations for AI on Azure

Introduction

Build Azure AI workloads that engineers can support: establish the operating boundaries before model selection turns into deployment work.

Picking a model can be easy. Explaining where its inference can run, who carries the spend, and how the application reaches protected services is the architecture work.

An Azure AI request becomes an engineering workload when those boundaries have names, owners, and a place in the platform. Model selection belongs after that conversation, not ahead of it.

This post sets the minimum foundation for engineers and architects working with Microsoft Foundry. It stays out of code, agents, and deployment mechanics. The aim is a workload that can be supported when a model deployment becomes the next decision.

Give the workload an operating home

Before discussing a model, establish the home in which the workload will be operated. These five questions make the early operating concerns visible across the people and platform boundaries that must carry the workload:

• Who owns the workload and can answer for its decisions?
• Which subscription carries its billing and relevant quota?
• Which resources share a lifecycle and should be managed together?
• Where may model-request processing happen for this workload?
• How does the application reach its dependencies safely while people can see its cost?

The Azure management hierarchy gives these decisions durable places. Management groups shape broader governance, subscriptions carry billing and common quota administration, resource groups group shared lifecycles, and resources provide the capability.

The value comes from how those controls work together. A private endpoint without a usable access pattern will be worked around and a budget alert without attribution tells you money was spent but not who needs to act. An API gateway without resource governance does not stop inconsistent resources appearing behind it.

A landing zone should resolve those platform decisions once and package the result into something teams can consume repeatedly. Product teams can then spend their time on application behaviour, prompts, evaluations, data and user experience instead of rebuilding the surrounding Azure platform.

Microsoft’s Azure AI Landing Zones design checklist covers identity, networking, governance, cost, monitoring, reliability and model deployment, so it is a useful starting point rather than inventing the whole pattern yourself.

Use the management hierarchy in the right manner

The Azure hierarchy is management group, subscription, resource group, and resource. It is useful because it separates broad governance from workload operation instead of presenting a single container as the answer to every concern.

Management group

A management group sits above subscriptions and provides a scope for governance across several subscriptions. Platform teams can apply policy and access patterns there when those controls need consistent application across the estate.

Subscription

A subscription is a billing boundary and a common administrative scope for service quotas. Quotas can vary by service, region, model, and deployment type, so a subscription identifies where the capacity discussion belongs without promising that capacity exists.

Resource group

A resource group is a lifecycle container for resources created, managed, and retired together. A Foundry resource, Key Vault, private endpoints, and monitoring resources can belong together when their operational lifecycle aligns.

A resource group does not create a hard security, billing, or network-isolation boundary. Those outcomes come from the specific identity, authorization, network, and cost controls that govern the surrounding resources over their operating life.

Resources and ownership

Resources are the individual Azure services and components that deliver the workload. Clear purpose-oriented names make changes, invoices, incidents, and architecture reviews easier for engineers and platform teams to interpret consistently.

Ownership gives this hierarchy practical meaning. Application engineering can state feature and operating needs; platform engineering can assess subscription and network feasibility; security or data governance can assess applicable requirements. Name a workload owner and a budget owner so a change request or cost alert has a destination.

Put deployment terms where they belong in the Foundry

Microsoft Foundry Models offers several ways to run models. Instant access (preview) lets supported models be called by name without a deployment. Managed compute (preview) serves open-source and custom models and does not use the Serverless API deployment-type taxonomy.

The terms in this section apply to Serverless API deployments. Standard, Provisioned Throughput, Global, Data Zone, and single-region processing describe that taxonomy. Batch is a separate asynchronous serving category, so it remains outside this focused introduction to online inference deployment choices. Availability for a Serverless API deployment depends on the selected model, version, region, and deployment type.

Two dimensions

For supported Serverless API models, deployment selection has three related characteristics: capacity and commercial model, processing location, and performance characteristics such as latency variance and throughput limits. The first two form a clear two-axis mental model. The performance characteristics still matter because they shape what a selected deployment can deliver.

Microsoft’s deployment-types documentation describes supported combinations, including types such as Global Standard and Data Zone Provisioned.

Capacity and commercial model

This dimension describes how online inference capacity is supplied and billed. It does not settle the processing location.

Standard uses shared inference capacity and pay-per-token billing.

Provisioned Throughput provides fixed dedicated model-processing capacity exclusively for a deployment. Billing is per deployed PTU per hour or through Azure reservations.

Processing location

This dimension describes where inference processing may occur for supported Serverless API deployments. It is distinct from the capacity and commercial model.

Global deployments may process inference in any Azure region supported by the deployment type.

Data Zone deployments may process inference only within the Microsoft-defined US, EU, or APAC zone supported by the deployment type.

Single-region processing keeps inference in the deployment region supported by the deployment type.

For Models sold by Azure, data at rest remains in the customer-designated Azure geography for Global and Data Zone deployments while permitted inference processing can have a broader scope. Marketplace, partner, and community models have their own terms, so their selected offering documentation needs its own review.

Availability is a separate fact

Quota is a policy limit, while deployable capacity is a separate model-, version-, region-, and deployment-type-specific fact. PTU quota does not reserve or guarantee deployable capacity for a model version. Performance characteristics also vary by selected deployment. Microsoft’s deployment types, Azure OpenAI quota, privacy, provisioned billing, and model availability document the current boundaries.

Make the service paths work

After the operating home and inference vocabulary are clear, the workload needs usable paths to its dependencies. Identity obtains a token, authorization grants actions, network design supplies an accepted route, and secret handling protects credentials that remain necessary.

Microsoft Entra ID and authorization

Microsoft Entra ID provides the identity system behind Azure access decisions. Azure role-based access control, or Azure RBAC, combines a security principal, role definition, and scope to define allowed actions. The Azure RBAC overview explains those assignments and scopes.

For people, use the smallest scope that matches the responsibility and give elevated assignments a named reason and an expiry or lifecycle choice. Platform staff can require broader governance access, while application engineers can have workload-resource-group or resource-level access.

For a running application, a managed identity provides an Entra token path without a credential in application configuration. Authentication does not authorize an action. Each target still needs Azure RBAC or another supported data-plane authorization mechanism, as the managed identities documentation explains.

Network

Network design follows each dependency because Azure services expose different connectivity options and constraints. Each dependency needs a defined design for public access, virtual network rules or service endpoints where supported, or private endpoints for private connectivity.

Private connectivity also needs DNS that resolves from the client networks that use the service. Test the application, operational, and dependent-service paths from those networks, using the Azure AI services virtual network documentation for service-specific guidance.

Key Vault

Key Vault provides a controlled home for secrets, keys, and certificates. Keep secret values out of source control, tags, and plaintext configuration, while retaining non-secret vault URIs or secret-name references where configuration requires them.

Name the vault authorization model, rotation and expiry expectations, and recovery safeguards. Key Vault soft delete and purge protection help recover from accidental deletion or destructive changes. Key Vault protects secret material; workload isolation still depends on identity, authorization, network design, and each dependent service.

Record operational accountability

The foundation now has a home, a bounded deployment vocabulary, and service paths. The remaining job is to retain the small amount of shared operational context that makes ownership, spend, and changes understandable over time.

Tags

Tags make resources easier to find, allocate, and explain when teams use a short shared taxonomy. Apply required tags to cost-accruing resources that support tags, and use Azure Policy tag policies where enforcement is needed. The Azure tag guidance explains tag behavior and limits.

Resource-group tags do not automatically propagate to resources, so resources that support tags need direct attention. A concise set can include workload name, environment, owner, cost center where appropriate, and lifecycle context.

Tags are plaintext operational metadata, not a security or data-classification enforcement mechanism; never place secrets, personal data, or regulated content in tag names or values.

Budgets and preventive controls

Budgets in Azure Cost Management monitor costs and send delayed alerts at configured actual or forecasted thresholds. The Azure budget guidance explains the alert model and threshold choices.

Budgets do not stop resource consumption. One preventive control can use Azure Policy to allow only approved deployment SKUs and regions, while alert recipients and response responsibilities are agreed before the first delayed signal appears.

A shared decision record

Keep a short shared record that names the workload owner, budget owner, subscription, resource group, and lifecycle-aligned resources. Add the permitted processing location, selected model and version, region, deployment type, identity and authorization path, network and DNS paths, Key Vault protections, required tags, and the destination for budget alerts and operational logs.

The aim is continuity across later implementation work. It gives engineers one place to understand the architectural decisions already made and the people responsible for maintaining them.

What comes next

This foundation prepares the next series step: service and model choices, followed by deployment and application implementation with current documentation. The sequence keeps this post focused on the platform conditions that make those later choices operable.

About the author

Z, Mehdi (08/09/2026) Building the Foundations for AI on Azure