Tuesday, September 21, 2021

Azure @ Enterprise - PowerShell to send message to Azure Service Bus Queue

We all know that Azure and PowerShell are friends. This post discusses how can we use PowerShell to work with the data place of Azure Service Bus. The example here is about queueing a message into the Service Bus queue.

Problem

We did implement a messaging architecture using Azure Service Bus Queue. The consumer is a .Net app that connects to the Service Bus Queue using Service Principal + Certificate. It works fine in lower environments such as dev, QA, etc... but stopped working in higher environments. Below was the message.

"An existing connection was forcibly closed by the remote host ErrorCode: ConnectionReset (ServiceCommunicationProblem);"

As it is a higher environment, there is no way we can install troubleshooting tools or even don't have access to the Azure portal.

Troubleshooting

Since there is no portal access and cannot install tools we decided to simulate how the application consumes Service Bus using PowerShell.

The first step was easy to connect to Azure using cmdlets using the Service Principal+Certificate and generate JWT. But the next step was not easy.

By the way, this requires the installation of Az.Accounts PowerShell module in the production machine. :)

Since PowerShell is considered an admin thing than a dev tool, admins normally allow it.

Challenge

The problem is that there is no official PowerShell package from Microsoft to work with the data place of ServiceBus. The official PowerShell module supports control plane functions such as create service bus namespace(New-ServiceBusNamespace), create queues (New-AzServiceBusQueue), etc. Even interacting with the data plane is not there in Azure CLI. We were not able to find any community-built PowerShell cmdlets too except for some tutorials on how to send messages.  

Solution

Finally, we had to get our hands dirty by making HTTP calls from PowerShell. The REST endpoints are properly documented to easily craft the requests. The PowerShell source code to send messages is available on GitHub.

https://github.com/ms-azure-demos/servicebus-queues-powershell

The root cause of the problem

When we run the PowerShell script from the same machine the app is running, it worked!!!. That eliminated the questions of what ports are open etc. The remaining difference between the .Net application sending messages v/s PowerShell was the AMQP v/s simple HTTP. .Net application uses AMQP by default and we didn't put an option in the application to fall back to HTTP. We checked whether the ports are open and they are. Hence concluded the higher environment network is not allowing AMQP traffic. Some network devices are on the way that doesn't like the AMQP traffic. 

Tuesday, September 14, 2021

Azure @ Enterprise - PowerShell log in as service principal + certificate and generate JWT access token

The enterprise always loves to increase the security posture. One authentication approach enterprise takes in Azure is App registration with a service principal. The advantage here is that the service principal can use certificates to authenticate instead of passwords. Certificates are secure than passwords as those can be centrally managed.

Problem

If our application uses the service principal + certificate and it is working fine, there is no issue. But the problem starts when something goes wrong. If we log in to the portal using our credentials and try scenarios that are failing, we may see everything works fine. But things go wrong when the application logs in using service principal. It may be a permission issue, expired certificates or passwords, etc...

What if the problems appeared first in production where enterprises don't allow any changes to the environment. ie no debugging tools are allowed to install etc...

Solution

The solution is to troubleshoot the scenario as close as how the application works. We have to log in as the service principal and try the application scenarios. 

As always in enterprise the best method to troubleshoot in production environments is PowerShell. Below goes the code to log into Azure from PowerShell using service principal and generating a JWT access token.

The Connect-AzAccount cmdlet provides different ways to log in. Using the service principal is one of the methods.
Please note that the Az.Accounts need a minimum of Windows PowerShell 5.1 or PowerShell 7 version.

Once the PowerShell session is authenticated, we can perform various operations as the service principal. 

In most cases, the access token is not really required. There are PowerShell SDKs for most of the Azure services. We can directly use them. But in some cases like interacting with the data plane of Service Bus, we may need to use the access token and embed it in the Authorization header of the HTTP request.

Limitations

It reads the certificate from the personal store unless loaded from a file. We cannot pass an X509Certificate object to the Connect-AzAccount cmdlet. There is already an issue in GitHub to track it.

Tuesday, August 31, 2021

Install .Net Runtime and run .Net apps in Raspberry Pi 4

Here we are continuing the experiments with Raspberry Pi 4. As a .Net developer, what is the meaning if we cannot install .Net into RasPi and run one program?

Please note this post is aiming at installing the .Net runtime, not the SDK. Development and compilation will be done outside of RasPi. Also, this is not aiming to run ASP.Net, just simple .Net console apps only.

Do we really need to install .Net runtime to RasPi?

It depends. If we want to run a self-contained application, there is no need to install the .Net runtime. Follow the below steps as mentioned in Microsoft docs.
  • Publish with the 'linux-arm' option.
  • Copy the folder to RasPi
  • Give execute permission
  • Run the app

It worked like a breeze. But the self-contained publish model includes the runtime and is really big. The above screenshot shows the size is around 80MB for a simple console application. To avoid that, we can install the runtime.

Attempt 1 - Install the apt-get way

When we google, we get the answers that we have to execute some scripts to install .Net runtime. The sudo apt-get is nowhere available. But I decided to give it a try using the Microsoft tutorial targeted to Ubuntu 21.04.
It failed gracefully with a series of messages

 "E: Unable to locate package dotnet-runtime-5.0
E: Couldn't find any package by glob 'dotnet-runtime-5.0'
E: Couldn't find any package by regex 'dotnet-runtime-5.0'"

The screenshot is given below

Let us do the next attempt.

Attempt 2 - Install using the scripts 

This time let's follow the Microsoft link as is https://docs.microsoft.com/en-us/dotnet/iot/deployment. I run the first command and below goes the output.

Wow. It installed the SDK. The above link is not at all talking anything about SDK. It's just talking about deploying the framework-dependent applications only. Let us check what is installed

Microsoft is great. We just asked .Net runtime. But we got SDK and ASPNetCore runtime as well. Now our RasPi is a full-fledged web server.

Once we have the installation ready. Get the published application. Make sure published as Framework-dependent and Target runtime set to Portable. Then run the normal command as below

dotnet "app.dll"

Closing thoughts

It is better to publish as a self-contained app to avoid the installation of .Net. The footprint will be big but it is easy to clean up. There will be new .Net versions releasing every year and their patches. If we keep with those releases, the .Net installation itself may be a big footprint later. (I am yet to check what it means .Net 5 and .Net 6 installed side by side)

Happy IoTing.