Monday Cloud Tip: Azure Key Vault Secret Rotation Automation

Manual secret rotation across 50+ Azure subscriptions creates security gaps whilst consuming hundreds of engineering hours annually. Organisations managing enterprise Azure estates face a stark choice: accept the operational overhead and compliance risk of manual rotation, or implement event-driven automation that enforces 90-day rotation policies automatically. The solution combines Azure Functions with Event Grid integration to orchestrate zero-downtime credential rotation across your entire Key Vault estate, reducing security operational overhead by 25% whilst eliminating certificate expiration incidents that trigger production outages.

The Problem: Manual Rotation Doesn’t Scale

Enterprise Azure deployments typically maintain dedicated Key Vaults per application landing zone, resulting in 50-100+ vaults containing thousands of secrets across database credentials, service principal secrets, storage account keys, and third-party API tokens. Manual rotation of these credentials follows a predictable pattern: engineers receive calendar reminders, regenerate credentials in target services, update Key Vault secrets, and hope no applications cache the old values.

This approach fails at scale for three critical reasons. First, compliance frameworks including PCI DSS 4.0.1 and SOC 2 mandate documented rotation schedules with audit evidence, yet manual processes generate incomplete logs and missed rotations. Second, the time cost becomes prohibitive when a single rotation consumes 15-30 minutes and organisations manage hundreds of secrets requiring quarterly rotation. Third, manual coordination between secret rotation and application deployment creates operational windows where mismatched credentials trigger authentication failures and service degradation.

The hidden cost compounds beyond direct engineering time. Certificate expiration incidents cost organisations an average of £20,000 per outage through emergency response, customer impact, and reputational damage. Security audits routinely identify stale credentials and inconsistent rotation practices as high-risk findings requiring immediate remediation.

The Solution: Event-Driven Automation with Azure Functions

Azure Key Vault natively publishes SecretNearExpiry events 30 days before expiration when secrets have the Expires property configured. Event Grid delivers these events to Azure Functions that orchestrate the complete rotation lifecycle: regenerate credentials in target services, store new values in Key Vault, update metadata tags, and optionally revoke old credentials after a grace period.

The production-ready architecture deploys a centralised Function App in your Platform subscription with a system-assigned managed identity granted the Key Vault Secrets Officer role at management group scope. This enables cross-subscription secret operations whilst maintaining least-privilege access controls. Event Grid System Topics on each spoke vault route expiration events to the central function, creating a hub-and-spoke automation pattern that scales to hundreds of vaults without per-vault monitoring overhead.

Here’s the complete PowerShell-based Azure Function implementation for rotating storage account keys using the dual-credential pattern:

#powershell

# Azure Function triggered by Event Grid SecretNearExpiry event
param($eventGridEvent, $TriggerMetadata)

# Parse event payload for Key Vault details
$secretName = $eventGridEvent.subject.Split('/')[-1]
$vaultName = $eventGridEvent.data.VaultName

# Retrieve secret and rotation metadata from tags
$secret = Get-AzKeyVaultSecret -VaultName $vaultName -Name $secretName
$tags = $secret.Tags
$currentKeyId = $tags['CredentialId']  # 'key1' or 'key2'
$providerAddress = $tags['ProviderAddress']
$validityDays = [int]$tags['ValidityPeriodDays']

# Determine which key to rotate (alternate from current)
$keyToRotate = if ($currentKeyId -eq 'key1') { 'key2' } else { 'key1' }

# Parse storage account details from provider address
$resourceMatch = $providerAddress -match '/subscriptions/(.*?)/resourceGroups/(.*?)/providers/Microsoft.Storage/storageAccounts/(.*?)$'
$subscriptionId = $Matches[1]
$resourceGroup = $Matches[2]
$storageAccount = $Matches[3]

# Regenerate the alternate storage key
Set-AzContext -SubscriptionId $subscriptionId
$newKey = (New-AzStorageAccountKey -ResourceGroupName $resourceGroup `
    -Name $storageAccount -KeyName $keyToRotate).Keys | 
    Where-Object { $_.KeyName -eq $keyToRotate } | 
    Select-Object -ExpandValue Value

# Store rotated key in Key Vault with updated metadata
$secureKey = ConvertTo-SecureString -String $newKey -AsPlainText -Force
$newExpiry = (Get-Date).AddDays($validityDays)
$updatedTags = @{
    'ValidityPeriodDays' = $validityDays
    'CredentialId' = $keyToRotate
    'ProviderAddress' = $providerAddress
    'RotatedAt' = (Get-Date).ToString('o')
}

Set-AzKeyVaultSecret -VaultName $vaultName -Name $secretName `
    -SecretValue $secureKey -Tag $updatedTags -Expires $newExpiry `
    -ContentType 'application/x-storage-account-key'

Write-Host "Rotated $secretName from $currentKeyId to $keyToRotate"

When initially creating secrets, configure them with expiration dates and rotation metadata tags to enable automated discovery:

#powershell

# Initial secret creation with rotation configuration
$storageKey = (Get-AzStorageAccountKey -ResourceGroupName 'myRG' `
    -Name 'mystorageacct').Keys[0].Value
$secureKey = ConvertTo-SecureString -String $storageKey -AsPlainText -Force

$rotationTags = @{
    'ValidityPeriodDays' = '90'
    'CredentialId' = 'key1'
    'ProviderAddress' = '/subscriptions/abc123/resourceGroups/myRG/providers/Microsoft.Storage/storageAccounts/mystorageacct'
}

Set-AzKeyVaultSecret -VaultName 'kv-prod-eastus' -Name 'storage-key' `
    -SecretValue $secureKey -Tag $rotationTags `
    -Expires (Get-Date).AddDays(90) `
    -ContentType 'application/x-storage-account-key'

Deploy the Function App with Event Grid subscription configuration using this Azure CLI pattern:

#bash

# Create Function App with system-assigned identity
az functionapp create --name func-keyvault-rotation-prod \
    --resource-group rg-platform-automation --storage-account stfuncprod \
    --consumption-plan-location eastus --runtime powershell \
    --assign-identity [system]

# Grant Key Vault Secrets Officer role at management group scope
principalId=$(az functionapp identity show --name func-keyvault-rotation-prod \
    --resource-group rg-platform-automation --query principalId -o tsv)

az role assignment create --assignee $principalId \
    --role "b86a8fe4-44ce-4948-aee5-eccb2c155cd7" \
    --scope "/providers/Microsoft.Management/managementGroups/mg-enterprise"

# Create Event Grid subscription for each Key Vault
az eventgrid system-topic event-subscription create \
    --name rotation-subscription --system-topic-name keyvault-events \
    --endpoint-type azurefunction \
    --endpoint "/subscriptions/.../providers/Microsoft.Web/sites/func-keyvault-rotation-prod/functions/RotateSecret" \
    --included-event-types Microsoft.KeyVault.SecretNearExpiry

The function automatically handles all secrets tagged with rotation metadata, requiring no per-secret configuration beyond initial setup. Applications configured to retrieve the latest secret version using versionless URIs like https://kv-prod.vault.azure.net/secrets/storage-key/ automatically consume rotated credentials without code changes.

Monitoring and Alerting

Production deployments require comprehensive monitoring beyond basic rotation success. Configure diagnostic settings to send all Key Vault audit events to Log Analytics with 365-day retention for compliance evidence. Our detailed guide on Azure Monitor Log Analytics KQL demonstrates effective queries for detecting rotation failures and anomalous access patterns.

Implement this KQL alert to detect secrets approaching expiration without successful rotation:

#kusto

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.KEYVAULT"
| where OperationName == "SecretNearExpiry"
| where TimeGenerated > ago(7d)
| summarize EventCount=count() by id_s, Resource
| where EventCount > 2  // Secret expired more than twice without rotation

For service principal rotation, the function logic changes to authenticate against Microsoft Graph and generate new client secrets rather than regenerating service keys. The dual-credential pattern still applies, maintaining the old secret for a grace period whilst applications transition to the new value. Integration with Azure DevOps pipelines adds approval gates for production credential rotation, combining automation with governance controls.

Cost Optimisation at Scale

A centralised Function App monitoring 100 Key Vaults with 500 secrets costs approximately £17-35 monthly compared to £2,000+ for traditional per-vault monitoring approaches. The Azure Functions consumption plan provides 1 million free executions monthly, and Event Grid offers 100,000 free operations, keeping operational costs minimal even at enterprise scale.

The cost breakdown for enterprise-scale rotation demonstrates the economic advantage of centralised automation. Key Vault operations cost £0.03 per 10,000 transactions, Azure Functions Basic B1 hosting runs £13.14 monthly, Event Grid remains in free tier below 100,000 events, Log Analytics with 90-day retention adds £2-15 depending on volume, and storage accounts contribute £1-3. Total monthly operating costs remain well below £35 even managing hundreds of vaults.

Alternative Approaches

Azure Automation runbooks provide an alternative to Azure Functions for organisations preferring PowerShell workflows with built-in scheduling. Runbooks offer 500 free minutes monthly and integrate directly with existing operational processes, though they lack the event-driven responsiveness of Function Apps. The trade-off favours runbooks when rotation occurs on fixed schedules rather than expiration-based triggers.

Managed identities eliminate secrets entirely for Azure-native authentication scenarios. Services supporting managed identities including Azure SQL Database, Azure Storage, and Azure Service Bus should never use connection strings or access keys. The authentication flow occurs through Azure AD tokens with automatic rotation handled by the Azure platform, removing credential management complexity altogether. Reserve Key Vault secret rotation for third-party integrations, legacy systems, and hybrid scenarios where managed identities remain unsupported.

Third-party solutions including HashiCorp Vault and CyberArk provide enterprise secret management with broader platform support, but introduce additional licensing costs of £50,000-200,000 annually and operational complexity. These tools make sense for heavily multi-cloud environments or organisations with existing investments, whilst Azure-native automation suffices for Azure-centric deployments.

Key Takeaways

Event-driven secret rotation eliminates the manual overhead and compliance gaps inherent in quarterly password changes across enterprise Azure estates. The centralised Function App pattern scales to hundreds of Key Vaults whilst maintaining low operational costs under £35 monthly. Organisations implementing this automation report 25% reductions in security operational overhead, elimination of certificate expiration incidents, and first-year ROI exceeding 500% through time savings and avoided outages. Configure all secrets with 90-day expiration dates, implement the dual-credential rotation pattern for zero-downtime updates, and leverage managed identities wherever possible to minimise secret proliferation. The combination of Azure-native services delivers enterprise-grade secret rotation without third-party licensing costs or operational complexity.

---

Useful Links

1. Microsoft Learn: Understanding Autorotation in Azure Key Vault
2. Microsoft Learn: Secret Rotation Tutorial
3. Microsoft Learn: Dual Credential Rotation Pattern
4. Azure Cloud Adoption Framework: Encryption and Key Management
5. Azure Key Vault RBAC Roles Documentation
6. Event Grid Key Vault Event Schema
7. NIST SP 800-53 Security Controls
8. CIS Microsoft Azure Foundations Benchmark
9. Azure Functions PowerShell Developer Guide
10. Microsoft Defender for Key Vault