April 10, 2026

Aaron Mumm

Read Time: ~10 minutes

<aside> 📌

At Tilt, we seek individuals who challenge personal assumptions, value ownership and trust, and strive for excellence to inspire and empower their team. If this article connected with you, join our team!

Join Tilt.

</aside>

<aside> 📎

When 50+ Azure App Service instances restart simultaneously and all reach for the same Key Vault, things break. This post walks through how Tilt replaced its secrets management with TiltSecret — a lazy-loading, strongly-typed, pipeline-validated system that eliminated Key Vault throttling, prevented deployment failures, and gave engineers a frictionless local development experience. If you run .NET at scale on Azure, this one's for you.

</aside>

https://open.spotify.com/episode/1jDuZEvw2brH28FgKJ1f0a?si=416bf98b3af649b8

At Tilt, we follow the Modern Monolith Architecture pattern that we host on Azure, leveraging App Services to run our API and Webjobs with individual instances across these scaling to 100+ at a time. All of these instances need access to the same secrets stored in Azure Key Vault—API keys for third-party services, database connection strings, and other sensitive configuration. For years, we managed these secrets using Azure App Service's built-in Key Vault integration, but as we scaled, this approach began to show its cracks. This is the story of how we built TiltSecret to solve our secrets management challenges.

The Old Way: App Service Environment Settings

Before TiltSecret, our secrets management looked like this:

How It Worked

  1. Secrets lived in Azure Key Vault - We stored all our secrets in a single Key Vault per environment (development, staging, production, etc.)
  2. App Service settings referenced Key Vault - Each App Service had environment variables configured, sometimes engineers would add a default value to help note it's a secret but at no point was it a requirement:
{
  "name": "Socure:ApiKey",
  "value": "@Microsoft.KeyVault(VaultName=test-keyvault;SecretName=TestService--ApiKey)"
}
  1. Code read from IConfiguration - Our application code accessed secrets through the standard .NET configuration system:
public class SocureConfiguration
{
    public string ApiKey { get; set; }  // Populated from IConfiguration["Socure:ApiKey"]
}

This approach seemed reasonable at first. Azure handled the Key Vault integration automatically, and we could use the familiar IConfiguration pattern throughout our codebase.

The Modern Monolith Context

To understand why this became problematic, you need to understand our architecture: