Azure @ Enterprise - Time bound SAS for WebJob to dequeue ServiceBus Queue messages

Context

Enterprise in 'Azure @ Enterprise' series refers to companies or projects which have stringent security measures and process guidelines which a normal developer may not think of. Or are not expected in the self service cloud world. Often the security measures are taken to make sure that the responsibility is moved to some other party or vendor so that people at Enterprise are free from low level details of those concerns.

If we are in non enterprise projects, developing a queued back end processing in Azure is super cool with Azure Queue and WebJobs. But in enterprise that is not the case, we have to make sure the service supports virtual network or not. Will that support encryption at rest as well as in transit. If it encrypt can the enterprise provide key etc...Basically enterprise don't want to fully trust the cloud vendor though in reality vendor own all and have full control.

Problem

Enterprise may evaluate Azure ServiceBus Queue is better than the Azure Storage Queue as per current feature sets and the standards what it used to evaluate. It may come out viceversa. If it demand key rotation for ServiceBus connection string with expiry, it is little difficult, if we had used only configuration(web.config or app.config) based connection string.

Even if it is not enterprise project, it is good practice to rotate the keys with expiry as long as the ServiceBus don't support hosting it inside the vNet. If it support hosting inside vNet, the attack surface could be less. Currently any Tom,Dick or Harry can launch brute force attack against the service bus end points.

Key rotation will not remove the attack surface but reduce the possibility of a successful attack.

https://feedback.azure.com/forums/216926-service-bus/suggestions/15619302-add-service-bus-to-vnet

Solution

It is possible to have key rotation even with the attribute based WebJob functions. We can have expiry to the keys as well. Lets see step by step.

Generating time based SAS (Shared Access Signature)

It is supported in ServiceBus. In order to generate one we need one Shared Access Policy. Normally when we create the SB instance, there will be 'RootManagerSharedAccessKey' with primary and secondary keys.

Ideally this is supposed to be done outside of the main application such as Azure Automation Code and the generated new time based SAS has to be in KeyVault. This will help to have only automation account knowing the high privileged Shared Access Policy key and WebJobs know only KeyVault secret name where the the time based Shared Access Signature is stored by Automation Runboook. If application has access to the KV, it can retrieve the SAS. 

Since the code to do SAS rotation using Azure Automation Runbook is not under the scope of this post, omitting that code. If anyone request through comments section, the code will be provided.

WebJob to read SAS and dequeue

Now lets come to WebJob code side. Before we start the JobHost to listen using RunAndBlock() on the thread, there is option to override to specify the connection string to the ServiceBus which uses the time based SAS. Below goes the code.

---

private static void RunAndBlock()
{
    var config = new JobHostConfiguration();
    config.UseServiceBus(ServiceBusConfigurationFactory.Get());

    if (config.IsDevelopment)
    {
        config.UseDevelopmentSettings();
    }
    var host = new JobHost(config);            
    host.RunAndBlock();
}

---

static class ServiceBusConfigurationFactory
{

    /// <summary>
    /// Returns the SB configuration
    /// </summary>
    /// <returns></returns>
    /// <remarks>Change this to read from Azure KV</remarks>
    internal static ServiceBusConfiguration Get()
    {
        return new ServiceBusConfiguration() {
            ConnectionString = BuildsSBConnectionString(GetITimeSensitiveSASTokenProvider().Get()),
        };
    }

    private static string BuildsSBConnectionString(string sharedAccessSignatureToken)
    {
        ServiceBusConnectionStringBuilder builder = new ServiceBusConnectionStringBuilder();
        builder.SharedAccessSignature = sharedAccessSignatureToken;
        builder.Endpoints.Add(new Uri("https://<name of SB instance>.servicebus.windows.net/"));
        string finalCon = builder.ToString();
        return finalCon;
    }

    private static ITimeSensitiveSASTokenProvider GetITimeSensitiveSASTokenProvider()
    {
        return new KVBasedimeSensitiveSASTokenProvider();
    }
}

---

The code is mostly self explanatory. When the first time main() of WebJob is started, it will get the ServiceBusConfiguration from the factory class. Factory uses provider class which knows how to talk to Azure KV or some other store where the SAS is present. Once the SAS is present, the connection string can be built from it.

One thing to remember is that the SAS is not connection string. Connecton string has to be 

Things to remember when working with ServiceBus

API Collision

There are 2 dependencies when we work with Azure ServiceBus. Microsoft.Azure.ServiceBus.dll & Microsoft.ServiceBus.dll and they both has classes with same name. Sometimes its very difficult to get the code compiled if downloaded as snippet.

What if the token expires after RunAndBlock() is invoked?

At least in testing there were no issues to the dequeued messages. All new instances of WebJob process, it takes the new connection string.