Azure @ Enterprise - Resubmitting messages from Service Bus dead-letter queue using various .Net SDKs

When we start using the Azure service bus for integration one of the operations job is to monitor and handle the messages in the dead-letter queue DLQ hereafter. As many of us already know, the DLQ is automatically created as a sub-queue with a naming convention. Messages will end up in DLQ when they are not deliverable or not processable. A non-processable situation occurs when the processors try to process but end up in exceptions and retry attempts exceed the Max delivery count. 

Problem

When we think of operations handling the DLQ, we think of manual troubleshooting of why messages failed and a PowerShell script that puts those messages back after fixing the issue. Obviously, some messages might be in the state to be reattempted. The issue here is that there is no built-in support for dealing with the data plane of Service Bus in the official library. I had written a previous post about how we can deal with the Service Bus data plane with PowerShell by writing custom snippets that interact with raw HTTP API.

Readers may be thinking why write PowerShell for operations, why can't operations just go to Azure Portal select all messages in DLQ, and hit the re-queue button. Unfortunately, there is no such feature available in Azure Portal

Solution

One way is to use already available Service Bus explorers provided by third parties other than in the label of Azure or Microsoft. There are some but not much user-friendly in my research. Hence this post about resubmitting the messages from DLQ using C#.

We can do this using PowerShell also but as I mentioned in the previous post we need to write a script to prepare raw HTTP messages. There are standard libraries in.Net that help us to write less code. 

High-level flow

This post is not going to have full code as the code is available in the GitHub repo. The high-level flow is as follows

• Monitor and receive a message from DLQ using the peek and lock method.
• Clone the message and send it to the queue again.
• Complete the received message from DLQ. This will remove the message from DLQ

There is no operation on the message just to toggle it to be a normal message from DLQ.

Selecting the SDK

There are 3 official .Net SDKs to interact with Azure Service Bus Queues. There is already a post in this blog about the Dilemma of choosing the SDK.  The general guideline is to use the latest SDK which is Azure.Messaging.ServiceBus.

Sample location

There are already samples to resubmit messages from DLQ using the earliest SDK WindowsAzure.ServiceBus, its successor Microsoft.Azure.ServiceBus and the latest Azure.Messaging.ServiceBus. So there will be questions, why one more sample? 

The reason for putting together this sample was that none of the samples were fully usable when we encounter a situation of resubmitting 100+ messages from DLQ. One does support only AMQP which is blocked by the enterprise network etc... 

There are 3 samples in the below repo that shows how to resubmit using all the 3 SDKs.

https://github.com/ms-azure-demos/service-bus-queue-dlq-manager

The samples are targetting the legacy .Net Framework, not .Net Core. The Azure.Messaging sample can be easily targetted to .Net Core and higher versions.

References

https://dzone.com/articles/azure-service-bus-dead-letter-queues-1

https://docs.microsoft.com/en-us/samples/azure/azure-sdk-for-net/dead-letter-queues/