Azure DevOps Pipelines

This guide explains the Azure DevOps starter pipeline generated by haute init. It runs validation and deployment commands. Its production deployment runs under an Azure DevOps environment, whose approval checks decide who may release (Who can start production). Databricks is the end-to-end service-update adapter; generic container builds/pushes only, and Azure Container Apps, AWS ECS, and GCP Cloud Run build and push the image and leave the service update to you.

New to Git, pull requests, or CI/CD?

This page assumes you understand basic Git concepts like branches, commits, and pull requests. If these are unfamiliar, read Before You Start first - it explains everything in plain English.

Not sure if this is the right page?

If you access your project on dev.azure.com, this is the right page. If it's on github.com, go to GitHub Actions. If it's on gitlab.com (or a company GitLab server), go to GitLab CI/CD. If you're not sure, ask your IT team or tech lead: "Where is our code repository hosted?" They'll tell you GitHub, GitLab, or Azure DevOps.

What is Azure DevOps?

Azure DevOps is Microsoft's platform for managing code, builds, and releases. Pipelines is the CI/CD feature - it runs scripts automatically when you push code or open a pull request. If your organisation uses Azure DevOps for source control (instead of GitHub or GitLab), this is your CI/CD provider.

Who does what on this page?

This page has two parts. As a pricing analyst, you only need to do Step 4 (test it). Steps 1-3 (creating the variable group, environment, and pipeline) are usually done by your IT team or tech lead - they have the credentials and know the Azure DevOps UI. If you're setting this up yourself, the steps walk you through it, but it's perfectly fine to hand it to IT and say: "Can you set up CI/CD for our Haute project?", with the credential names haute init printed (Step 1 lists them by target).

---

What Haute generates

When you run haute init --ci azure-devops, Haute creates an azure-pipelines.yml file in the root of your repository. It defines five stages with ordering dependencies (three for Azure Container Apps, AWS ECS and GCP Cloud Run, where DeployProduction follows DeployStaging directly); whether a stage can create or update a live endpoint depends on the selected target:

Stage	When it runs	What it does
Validate	On PRs and pushes to main	Lints, type-checks, tests, validates pipeline, dry-run deploy
DeployStaging	On push to main only	Runs the target-dependent staging deploy command
SmokeTest	After a successful staging deploy	Scores test quotes against an existing live staging endpoint
ImpactAnalysis	After smoke succeeds	Compares existing staging and production endpoints
DeployProduction	After ImpactAnalysis (after DeployStaging for the platform-container targets)	Runs production deployment. It runs automatically unless the production environment has an approval check (Step 2)

This gives you the same command sequence as GitHub Actions and GitLab; a live release still depends on the selected target and hosting process:

Pull request → Validation → Merge → Staging → Smoke test → Impact analysis → [Approval] → Production

Workbench projects

A workbench project hosts no API: what it releases is the underwriter app. Its azure-pipelines.yml has three stages instead: Validate, which lints, type-checks and runs tests, then builds the app's release, which checks your pipeline and schema and scores your test quotes, on every pull request and on main; Publish, on main only, a deployment to the staging environment that builds the app's image once, pushes it to the registry [deploy.container] names, rolls the staging app out to it and compares staging with production on saved quotes, publishing impact_report.md as the impact-report artifact; and DeployProduction, a deployment to the production environment that you run by hand, which rolls the production app out to the release staging runs. Each rollout checks the app and puts the previous release back when the new one fails (The underwriter app). Add an Exclusive lock check to both environments so one rollout of each runs at a time. The stages read your host's variables from the haute-credentials group, AZURE_SUBSCRIPTION_ID, AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET.

---

How it works, step by step

1. You make a change and open a pull request

You edit your pipeline, update a model, or change a configuration. You push your changes to a branch and open a pull request (PR) in Azure DevOps - which is a way of asking your teammate to review your changes before they go live.

2. Validation runs automatically

The Validate stage runs immediately. It has four parallel jobs:

• Lint & Format - checks your code style with ruff
• Type Check - runs mypy to catch type errors
• Test - runs pytest
• Pipeline Validation - runs haute lint and haute deploy --dry-run

If any of these fail, the PR gets a red ❌ and you can see what went wrong in the pipeline logs.

3. You merge to main

Once validation passes and your teammate has reviewed the PR, you merge it - which applies your changes to the main version of the project. This triggers the remaining stages.

4. Staging deployment (automatic)

The DeployStaging stage runs haute deploy --endpoint-suffix "-staging". On Databricks it waits until the staging endpoint serves the new version, and fails with Databricks' build log if the model cannot be served; the generic container target does not host an endpoint; platform-container targets push the image and leave the service update to you, so their pipeline has no smoke or impact stage and DeployProduction follows DeployStaging directly.

5. Smoke test (automatic)

If deploy succeeds, the SmokeTest stage scores configured test quotes against an existing live staging endpoint. On Databricks the endpoint is already serving when it starts, since the deploy waited for it. On the generic container target it calls the staging address in haute.toml.

6. Impact analysis (automatic)

If smoke succeeds, the ImpactAnalysis stage compares existing staging and production endpoints on the portfolio sample in the repository, or on your test quotes when no sample is named. The report is uploaded as a pipeline artifact only when this command runs.

7. You review and approve

The DeployProduction stage targets an Environment called production. Azure DevOps environments can enforce approval checks when you configure them; Haute's min_approvers configuration does not create or enforce those checks. Without an approval check on that environment, production deploys straight after the impact analysis, on every merge to main.

The impact report shows exactly what changed, for each output column:

  Output: technical_price
  ────────────────────────────────────────────────────────────
  Staging mean:          548.57     Production mean: 536.12
  Rows changed:           8,420 / 10,000 (84.2%)
  Mean change:            +2.3%
  Median change:          +1.9%
  Premium impact:         +2.3%
  Max increase:          +18.7%
  Max decrease:           -4.2%

  Distribution:  P5=-0.8%  P25=+0.6%  P50=+1.9%  P75=+3.4%  P95=+7.1%

Segment tables follow, one for each text column of the portfolio sample with 2 to 50 values, listing the segments whose prices move most. The report is advisory: it never fails the stage.

8. Production deploy

Once approved, the pipeline runs haute deploy and then tags the release in Git. Databricks can update its serving endpoint; the generic container target only packages an image, and the Azure, AWS, and GCP platform-container targets build and push the image for a manual service update.

---

Setup

Step 1: Create a Variable Group for credentials

This step is usually done by IT

If you're not comfortable navigating Azure DevOps, ask your IT team or tech lead to do Steps 1-3. Give them the credential names haute init printed; the tables below list them by target.

Azure DevOps stores shared secrets in Variable Groups. Haute's generated pipeline expects a group called haute-credentials.

1. Go to your project page in Azure DevOps (e.g. dev.azure.com/yourcompany/motor-pricing)
2. In the left sidebar, click Pipelines → Library (the 📚 icon)
3. Click + Variable group
4. Name it haute-credentials
5. Add each credential as a variable:

For Databricks:

Name	Value	Secret?
DATABRICKS_MLFLOW_HOST	The workspace URL for MLflow and the model registry	✓ Lock
DATABRICKS_MLFLOW_TOKEN	A personal access token scoped for MLflow and the model registry	✓ Lock
DATABRICKS_RATING_HOST	Your workspace URL	✓ Lock
DATABRICKS_RATING_TOKEN	Your personal access token scoped for Model Serving	✓ Lock

For Docker/Container targets:

Name	Value	Secret?
DOCKER_USERNAME	Your registry username	✓ Lock
DOCKER_PASSWORD	Your registry password	✓ Lock

For AWS ECS:

Name	Value	Secret?
AWS_ACCESS_KEY_ID	Your AWS access key	✓ Lock
AWS_SECRET_ACCESS_KEY	Your AWS secret key	✓ Lock
AWS_DEFAULT_REGION	e.g. eu-west-1	-

For Azure Container Apps:

Name	Value	Secret?
AZURE_SUBSCRIPTION_ID	Your subscription ID	✓ Lock
AZURE_TENANT_ID	Your tenant ID	✓ Lock
AZURE_CLIENT_ID	Your service principal app ID	✓ Lock
AZURE_CLIENT_SECRET	Your service principal secret	✓ Lock

For GCP Cloud Run:

Name	Value	Secret?
GCP_PROJECT_ID	The Google Cloud project holding the registry and the service	-
GCP_SERVICE_ACCOUNT_KEY	The JSON key of the service account CI deploys as, on one line	✓ Lock

The deploy jobs log in to Artifact Registry with the service account's key.

Click Save when done.

Lock secrets

Click the 🔒 lock icon next to each secret value. This encrypts it and prevents it from appearing in pipeline logs.

Step 2: Create the production Environment

The production deploy stage runs under an Azure DevOps Environment: production, unless [ci.production] environment in haute.toml names another (run haute ci update after changing it). Add approval checks in Azure DevOps if you need them; this is not configured or enforced by Haute. Without one, every merge to main that passes the earlier stages deploys production.

1. Go to Pipelines → Environments
2. Click New environment
3. Name it production
4. Resource: None
5. Click Create

Now add an approval check:

1. Click the production environment you just created
2. Click the ⋮ menu (top right) → Approvals and checks
3. Click + Add check → Approvals
4. Add the people who should approve production deploys
5. Set Minimum number of approvers (e.g. 1)
6. Click Create

When the pipeline reaches the production stage, it will pause and send a notification to the approvers. They review the impact report, then approve or reject.

Step 3: Create the Pipeline

First commit azure-pipelines.yml with pyproject.toml and uv.lock: every stage installs with uv sync --frozen, which needs the lock file (uv add creates it; run uv sync once if it is missing).

1. Go to Pipelines → Pipelines
2. Click New pipeline
3. Select your repository source (Azure Repos Git, GitHub, etc.)
4. Select your repository
5. Choose Existing Azure Pipelines YAML file
6. Select /azure-pipelines.yml from the branch dropdown
7. Click Run (or Save if you want to run it later)

Two more settings, if your code is in Azure Repos Git:

• Pull request checks. Azure Repos ignores the file's pr: trigger, so the Validate stage runs on pull requests only once you add a Build validation branch policy on main that runs this pipeline (Repos → Branches → main → Branch policies).
• Release tags. The production stage pushes a Git tag, so allow the project's build service Contribute and Create tag on the repository, or the tag push fails.

Step 4: Test it

1. Create a branch, make a small change, and open a pull request
2. The validation stage should run automatically on the PR
3. If it passes, merge the PR
4. Watch the target-dependent pipeline; smoke and impact need a ready staging endpoint and do not run after a failed deploy
5. If the impact job ran, check its report artifact before approving production

---

The generated pipeline in detail

You don't need to read or understand the YAML below

The pipeline file was generated by haute init and works out of the box. The sections below are reference only - for troubleshooting or if your IT team wants to understand what’s happening.

azure-pipelines.yml structure

trigger:
  branches:
    include: [main]
  paths:
    include:
      - rating/
      - tests/quotes/
      - haute.toml
      - pyproject.toml
      - uv.lock

pr:
  branches:
    include: [main]

stages:
  - stage: Validate        # 4 parallel jobs
  - stage: DeployStaging    # depends on Validate, main branch only
  - stage: SmokeTest        # depends on DeployStaging
  - stage: ImpactAnalysis   # depends on SmokeTest
  - stage: DeployProduction # depends on ImpactAnalysis, uses 'production' environment

Key features

• Path filters - the pipeline only triggers when relevant files change (the rating/ pipeline package, config, data, models, test quotes, and the dependency files pyproject.toml and uv.lock; a workbench's files are in the pipeline package, so a change to them deploys too), not on README changes
• PR validation - the Validate stage runs on pull requests too (through a branch policy on Azure Repos), giving you feedback before merge
• Variable group - credentials come from the haute-credentials variable group, not hardcoded
• Environment approvals - configure approval checks on the production environment if you need manual approval before production deploy
• Deployment job - the production stage uses a deployment job type, which gives you deployment history and tracking in Azure DevOps
• Artifacts - the impact report is published as a pipeline artifact you can download from the build summary
• Git tagging - after production deploy, the pipeline tags the release: deploy/v<version> (the registered model version) for Databricks, deploy/<commit> for container targets, whose image tag carries the same commit. On Databricks the tag step reads the version with haute status, with the same variables as the deploy step, and fails rather than tagging a version it could not read. The production checkout keeps its credentials (persistCredentials: true) so the tag can be pushed; the pipeline's build service needs permission to create tags in the repository

Stage details

Stage	Jobs	Timeout
Validate	Lint, Type Check, Test, Pipeline Validation (parallel)	10 min each
DeployStaging	Deploy to staging	60 min on Databricks, which waits for its endpoint; 15 min on the container targets
SmokeTest	Score test quotes against staging	10 min
ImpactAnalysis	Compare staging vs production	10 min
DeployProduction	Deploy to production + tag	60 min on Databricks; 15 min on the container targets

---

Azure DevOps vs GitHub/GitLab - key differences

Concept	GitHub	GitLab	Azure DevOps
Config file	.github/workflows/*.yml	.gitlab-ci.yml	azure-pipelines.yml
Credentials	Repository Secrets	CI/CD Variables	Variable Groups
Manual approval	Separate workflow + button	when: manual + ▶ button	Environment approval checks
Artifacts	Upload/download actions	artifacts: keyword	publish: keyword
PR checks	Workflow on pull_request	Job with merge_request_event rule	pr: trigger, or a build validation branch policy on Azure Repos

The biggest difference is how manual approval works:

• GitHub: you manually run a separate workflow
• GitLab: you click a play button on a manual job
• Azure DevOps: once the production environment has an approval check, the pipeline pauses automatically and sends a notification to approvers - they approve directly from their email or the Azure DevOps UI

This makes Azure DevOps the most "enterprise-friendly" option for approval workflows.

---

Working solo?

If you're the only person on the team:

1. min_approvers in haute.toml is metadata, not an enforced Azure DevOps approval setting
2. In the production environment, either remove the approval check (then every merge to main deploys production) or set yourself as the sole approver
3. The generated stages still attempt validation → staging → smoke test → impact analysis; later stages do not run after a failed deployment
4. Approve your own production deploys after reviewing the impact report

---

Troubleshooting

Pipeline doesn't trigger

• On PR: Make sure the PR targets the main branch
• On merge: Check the trigger paths - if you only changed a markdown file, the pipeline won't trigger (by design)
• Check Pipelines: Go to Pipelines → Pipelines and look for recent runs

"Variable not found" or "Authentication failed"

Your variable group is missing or misconfigured:

1. Go to Pipelines → Library
2. Check the variable group is named exactly haute-credentials
3. Check variable names are correct (case-sensitive)
4. Make sure secrets are saved (not just typed - click Save)

Pipeline authorisation error on variable group

The first time a pipeline uses a variable group, Azure DevOps may ask you to authorise it:

1. Go to the failed pipeline run
2. Click View on the authorisation prompt
3. Click Permit

This only happens once per variable group per pipeline.

"Environment not found"

Make sure you created the environment the production stage runs under in Pipelines → Environments: production, or the name [ci.production] environment gives. The name must match exactly.

Impact report not appearing

The impact report is published as a pipeline artifact. To find it:

1. Go to the pipeline run → click the Summary tab
2. Under Published artifacts, look for impact-report
3. Click to download impact_report.md

Approval notification not received

Check that:

1. The production environment has an approval check configured
2. Your email notifications are enabled in Azure DevOps
3. You're listed as an approver on the environment

---

Keeping the pipeline file up to date

After upgrading Haute, or changing [ci.staging] endpoint_suffix or [ci.production] environment in haute.toml, run haute ci update and commit azure-pipelines.yml. It overwrites changes you made to the file by hand: see Keeping the generated files up to date for what it rewrites and removes.