Azure @ Enterprise - dealing with retired services is tough

Though there are so many advantages with cloud, there seems some problems due to vendor lock in.  Especially if we invest on vendors specific technologies. Below is one such scenario occurred in my day job related to Azure.

Story of Azure ChatBot

One of our clients who is in highly regulated industry was very much skeptical about cloud. After long time periods they were convinced into Azure. One reason could be the long term relationship with Microsoft. They started to use Azure with all the enterprise gates. In fact the Azure @ Enterprise series in this blog actually originated from dealing with those gates. 

Beginning of last year, we started to develop a chat bot demo. The idea was to integrate the chat bot into one of the big applications as a replacement to FAQ. Users can ask questions to bot thus avoiding obvious support tickets in the future.

Things went well. We got appreciation on the demo and started moving to production. About half way, things started turning south. The demo chat bot application used Bot SDK V3. It had voice recognition enabled which allow users to talk to it and get the response back in voice. During the demo we used Azure Bing Speech API. But later before the production, we got the notice that the service is obsolete and will be retired mid 2019. Another surprise was the introduction of Bot SDK V4 which is entirely different that Bot SDK V3. Something like AngularJS v/s Angular.

Retirement of Bing Speech API

As per the announcement, the service will no longer work after 15 Oct 2019!! We need to migrate to Speech Service.

https://docs.microsoft.com/en-us/azure/cognitive-services/speech/home

They already given the details on how to migrate.

https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-migrate-from-bing-speech

The sad part is that, soon after the announcement, we are no longer able to create Bing Speech API resource in the Azure portal. We were just started to test the application in development subscription.

Another problem is the out of the box incompatibility of new Speech Service with Bot SDK V3. Bing Speech API and Bot SDK V3 are compatible and easy to integrate.

Impacts

It was easy from Microsoft, but developers gets into trouble. Many internal & client meetings happened. Lot of time spend to do assessment on migration. Finally decided to contact Microsoft to get exception. 

Then the next phase of efforts started. Raising support tickets, meetings with Microsoft and luckily since our client is big for Microsoft, we got exception to create the Bing Speech API resource in new subscriptions.

If it was a startup, the decision might have been taken quick but might not get the Microsoft support as what we got. But for bigger enterprises, its a tedious job to get decision. Wasting time which was reserved for achieving features. The budget changes, delays etc...

Bot SDK V4

The problem is not ended there with Bing Speech API. Even if we decide to go with the proposed Speech Service which replace Bing Speech API, the Bot SDK V3 is not compatible out of the bot. So need to upgrade the Bot SDK to V4 as well.

With SDK V4, Microsoft changed the programming model of bots. Its almost rewrite of V3 code when upgrade to V4. Also Bot SDK V4 is only available in .Net Core. What if an enterprise is not ready to take .Net Core? Yes it can happen and happening for us. Again so much efforts.

Another problem seems the Bot SDK V4 is not compatible with Bing Speech API. Even with our exception, we cannot migrate to SDK V4. Both the upgrades has to happen together to make sure that the application is not breaking in the process. 

I am always a proponent of CI/CD pipeline. But here, since the application is having the pipeline with 'build on check in '  & 'deploy on success' enabled, there could be disturbances.

Further thoughts

We are not alone in the planet who were hit with this issue. But still need to think about future on how to handle the scenario.

What if it happens during the maintenance phase of application?

Now there is a active development team for the project. There could be time when the application is complete and expected to run for long. In that situation, if changes like this comes which cause application to fail, what is the guarantee Azure is giving?

Budget for future

Normally when the application is built, it goes to maintenance mode with less budget. What if there is not enough budget at later time to rewrite the application on these breaking changes? Seems we need to let the app into its fate similar to how we abandon the satellites in space.

Summary

We survived because the client is a big shot for Microsoft. If you are big enough to influence Microsoft, go to Azure proprietary PaaS. Else think twice and thrice.

Simple solution is to use vendor independent technologies. For example containers for web apps and APIs, Scala for analytics. That will help to host application in our own data center, if clouds became expensive or vanished in future.

PowerShell to validate certificate

Scenario

There is a WPF based enterprise application working based on certificate issued into the system by the certificate authority. When the application starts it make sure there is a certificate in the store which is issued by the enterprise. For remote service calls, certificate chain based trust is used to ensure the WCF communication is secured. The servers will accept the client request which comes with certificate issued by the same certificate authority of what is in server.

Problem

One day accidentally someone issued one more cert to the client machine in the same name. Then the poor application stopped working as it doesn't know what to do when there are more than one certificate.

Panic situation started. Some are sure that in such situation one of the certificate should be revoked. Some tells application should have intelligence to choose the cert with longest expiry, etc... 

How to understand what is really happened? Need to look into one of the client machine to which developers barely have access. So don't even think about installing or giving an test application. 

Modifying the application is not a big deal but without understanding what is happening is waste of time. The problem here is understanding what is happening in client machine. This might seem very silly to a fresher or someone working for public products. Welcome to Enterprise, it doesn't work that way in Enterprise.

PowerShell to rescue

The PowerShell is really a revolution where it helps developers to run code in restricted environments otherwise they cannot do anything via installer or utility exe. .bat files were there, but that won't let developers run C# code as is in a machine without compilation.

Lets see the snippet which will help us to check whether the certificates are revoked or not.

Get-childitem Cert:\CurrentUser\My -recurse | %{ write-host $_.Subject ; Test-Certificate -cert $_ }

The main API is the Test-Certificate command let. The initial code fragments are used to iterate the certificate store.

Happy scripting...

Searching for users in Active Directory

As developers for windows based systems, there are situations where we need to deal with Active Directory. This post is about searching user in Active Directory from C#. It can run from desktop as well as web applications as long as the user running application has the permission to AD and AD is reachable via network.

Prerequisites

Knowledge about Active Directory and terms such as Forest, Global Catalog etc... are preferred to understand this post

Searching in particular AD Forest

Here the input is the search string and the name of the AD Forest

---

public static IEnumerable<ADMember> GetUsersByADForestName(string searchString, string nameOFADForest)
{

    DirectoryContext rootDomainContext = new DirectoryContext(DirectoryContextType.Forest, nameOFADForest);
    Forest forest = Forest.GetForest(rootDomainContext);

    return GetUsersFromForest(searchString, forest);
}

---

As seen above, the Forest object is created using DirectoryContext which is created using the name of AD Forest.

The GetUsersFromForest() method is given below which does the search

---

private static IEnumerable<ADMember> GetUsersFromForest(string searchString, Forest forest)
{
    GlobalCatalog catalog = forest.FindGlobalCatalog();
    using (DirectorySearcher directorySearcher = catalog.GetDirectorySearcher())
    {
        //Use different options based on need. Below one is for searchnig names.
        directorySearcher.Filter = $"(&(objectCategory=person)(objectClass=user)(|(sn={searchString}*)(givenName={searchString}*)(samAccountName={searchString}*)))";

        SearchResultCollection src = directorySearcher.FindAll();

        foreach (SearchResult sr in src)
        {
            yield return new ADMember
            {
                Title = GetPropertyValueFromSearchResult(sr, "title"),
                FirstName = GetPropertyValueFromSearchResult(sr, "givenName"),
                MiddleName = GetPropertyValueFromSearchResult(sr, "middleName"),
                LastName = GetPropertyValueFromSearchResult(sr, "sn"),
                Phone = GetPropertyValueFromSearchResult(sr, "phone"),
                Email = GetPropertyValueFromSearchResult(sr, "mail"),
                DisplayName = GetPropertyValueFromSearchResult(sr, "name"),
                UserName = GetPropertyValueFromSearchResult(sr, "samAccountName")
            };
            
        }
    }
}

---

Hope the above code snippet is self explanatory. The ADMember is just a class with required properties. Below goes the GetPropertyValueFromSearchResult method.

---

private static string GetPropertyValueFromSearchResult(SearchResult searchResult, string property)
{
    return searchResult.Properties[property].Count > 0 ? searchResult.Properties[property][0].ToString() : string.Empty;
}

---

The assembly references needed are given below

1. System.DirectoryServices.dll

2. System.DirectoryServices.AccountManagement.dll

Searching in current AD Forest and its trusted Forests

There could be scenarios where we need to search the current AD Forest and any other forests trusted to it. Below goes the code for it.

---

private static IEnumerable<ADMember> GetUsersByADForestNameAndItsTrustedForests(string searchString)
{
    searchString = EscapeForSearchFilter(searchString);
    List<ADMember> userListFromActiveDirectory = new List<ADMember>();

    var currentForest = Forest.GetCurrentForest();
    userListFromActiveDirectory.AddRange(GetUsersFromForest(searchString, currentForest));

    IEnumerable<ADMember> userListFromGlobalCatalog = GetUsersFromTrustedForests(searchString);
    userListFromActiveDirectory.AddRange(userListFromGlobalCatalog);

    return userListFromActiveDirectory;
}

private static IEnumerable<ADMember> GetUsersFromTrustedForests(string searchString)
{
    var forest = Forest.GetCurrentForest();
    List<ADMember> userInfo = new List<ADMember>();

    var relations = forest.GetAllTrustRelationships().Cast<TrustRelationshipInformation>();
    var filteredRelations = relations.Where(IsTheTrustValid);
    Parallel.ForEach(filteredRelations, (TrustRelationshipInformation trust) =>
    {
        Trace($"TrustedRelation. Source {trust.SourceName}, TargetName {trust.TargetName},{trust.TrustDirection},{trust.TrustType}");
        try
        {
            DirectoryContext rootDomainContext = new DirectoryContext(DirectoryContextType.Forest, trust.TargetName);
            Forest trustedForest = Forest.GetForest(rootDomainContext);
            var userDetails = GetUsersFromForest(searchString, trustedForest);
            if (userDetails.Any())
            {
                userInfo.AddRange(userDetails);
            }
        }
        catch (Exception ex)
        {
            Trace($" Searching exception {ex.Message} for TrustedRelation. Source {trust.SourceName}, Destination {trust.TargetName}.  Continuing...");
        }
    });
    return userInfo;
}
private static bool IsTheTrustValid(TrustRelationshipInformation trust)
{
    return (trust.TrustDirection == TrustDirection.Bidirectional || trust.TrustDirection == TrustDirection.Outbound)
        && trust.TrustType == TrustType.Forest;
}

---

All the 3 methods are present in the above snippet which helps to filter the trust relation and to create object of Forest from TrustRelation object. 

Please note the code snippet is for demonstration purpose only. It has to be tweaked for production especially when we are dealing with parallelism.