First experience with Azure Service Fabric

As a programmer, we all will get excited when we first hear about Azure Service Fabric. Because it provides many thing out of the box which will take considerable amount of time, if we code. Hereafter lets refer it to ASF. Some features are

• Automatic fail over - If one node fails the requests (even web requests), the service fabric run time will copy our application to another node and traffic will be routed there.
• Load balancer can never copy our apps
• If we choose to store our application state with ASF, it will make sure the state if replicated in multiple nodes, if one node fails it moves our data to another node and resume running.
• During this switches our application will never see failures. (It will retry if one request fails with other node).
• ASF conducts leader election among nodes to find who can be next master. It depends up the load on different nodes.
• We can simulate node failure and all in our dev machine itself.

And many...

It was not different with me I jumped into the ASF preview.

Setting up Azure Service Fabric Preview

It was very simple. Get a Windows 8 machine with VS 2015. Install ASF SDK. More details in below link.

https://azure.microsoft.com/en-us/documentation/articles/service-fabric-get-started/

There are many samples for ASF out in Github. That is the best place to start ASF. I too started there. As usual hit the first issue

Issue 1 - System.Fabric.FabricConnectionDeniedException

It compiled nicely. The run time exception was as follows.

---

An unhandled exception of type 'System.Fabric.FabricConnectionDeniedException' occurred in System.Fabric.dll

Additional information: Not authorized to connect

   at System.Fabric.FabricRuntime.Create(Action fabricExitCallback)
   at Microsoft.ServiceFabric.Actors.Samples.Program.Main() in d:\guthub\servicefabric-samples\samples\Actors\VS2013\HelloWorld\HelloWorldActorService\Program.cs:line 16
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

Inner exception
{"Exception from HRESULT: 0x80071C43"}
   at System.Fabric.Interop.NativeRuntime.FabricEndCreateRuntime(IFabricAsyncOperationContext context)
   at System.Fabric.FabricRuntime.NativeFabricRuntimeFactory.<>c__DisplayClass1b.<InitializeFabricRuntimeAsyncHelper>b__1a(IFabricAsyncOperationContext callback)
   at System.Fabric.Interop.AsyncCallOutAdapter2`1.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously)

---

The exception says I have serious connectivity issues and I turned towards the connectivity side of the story. I thought some ports are not opened for the development Service Fabric run time on my machine as its company machine.

Solution

But after 1 hour, when I looked at the error message I got a click. I am opening VS 2013 sample project in VS 2015 and let me correct that.

Opened correct project in VS 2015 and the issue resolved. This was purely my problem. But Microsoft should have given a meaningful exception something like version mismatch.

Issue 2 - Deployment issue

It compiled well in VS 2015 but it is not able to deploy and run. The exception in the output windows goes as follows

Microsoft.VisualStudio.Azure.Fabric.ApplicationProject.Scripts.ApplicationScriptException

---

1>------ Build started: Project: HelloWorldApplication, Configuration: Debug x64 ------
2>------ Deploy started: Project: HelloWorldApplication, Configuration: Debug x64 ------
-------- Package started: Project: HelloWorldApplication, Configuration: Debug x64 ------
  HelloWorldApplication -> D:\guthub\servicefabric-samples\samples\Actors\VS2015\HelloWorld\HelloWorldApplication\pkg\Debug
-------- Package: Project: HelloWorldApplication succeeded, Time elapsed: 00:00:01.4545432 --------
2>Started executing script 'Deploy-FabricApplication.ps1'.
2>'D:\guthub\servicefabric-samples\samples\Actors\VS2015\HelloWorld\HelloWorldApplication\Scripts\Deploy-FabricApplication.ps1' -ApplicationPackagePath 'D:\guthub\servicefabric-samples\samples\Actors\VS2015\HelloWorld\HelloWorldApplication\pkg\Debug' -ErrorAction Stop -DoNotCreateApplication
2>Deploying application...
2>Connecting to the cluster...
2>Get-ServiceFabricDeployedReplica : Node Node.5 is not Up
2>At D:\guthub\servicefabric-samples\samples\Actors\VS2015\HelloWorld\HelloWorldApplication\Scripts\Deploy-FabricApplicat
2>ion.ps1:106 char:12
2>+     [void](Get-ServiceFabricDeployedReplica -NodeName $node.NodeName -Applicatio ...
2>+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2>    + CategoryInfo          : NotSpecified: (Microsoft.Servi...usterConnection:ClusterConnection) [Get-ServiceFabricDe 
2>   ployedReplica], AggregateException
2>    + FullyQualifiedErrorId : GetDeployedReplicaErrorId,Microsoft.ServiceFabric.Powershell.GetDeployedReplica
2> 
2>Unregistering application type...
2>Unregister application type succeeded
2>Copying application package...
2>Copy application package succeeded
2>Registering application type...
2>Register application type succeeded
2>Finished executing script 'Deploy-FabricApplication.ps1'.
2>Time elapsed: 00:00:15.5324363
2>Exception of type 'Microsoft.VisualStudio.Azure.Fabric.ApplicationProject.Scripts.ApplicationScriptException' was thrown.
========== Build: 1 succeeded, 0 failed, 2 up-to-date, 0 skipped ==========
========== Deploy: 0 succeeded, 1 failed, 0 skipped ==========

---

Spent sometime on the last executed Deploy-FabricApplication.ps1 file. But no chance. Took break. Still no chance. Then decided to learn some theory about Service Fabric and stack overflow. Some days back I have asked in SO about load balancing stateless services. Fortunately there was a reply and it says, we can deploy stateless services to more than one node and load balance the same. There is a config as well to tell how many nodes the application needs to be installed. 

Solution

I am playing around HellWorld Stateless ASF service. When I first installed ASF local cluster, I had stopped one node just to increase performance on the assumption that Why should I test with 5 nodes ? Isn't 4 enough. May be it is trying to deploy to all 5 nodes and it can't do because one is stopped.

Started node 5 and the deploy worked!!!

Issue 3 - Error when debugging...

I run the ASF application from Visual Studio to debug and see the behavior. Visual Studio deploy to the local cluster and just hung for some time then crashes with the below error message.

---

------------
A 64-bit debugging operation is taking longer than expected.This may be caused by incompatibilities with 3rd party networking software
---------------
Eventually gets into below error
---------------------------
Microsoft Visual Studio
---------------------------
Unable to start debugging 'C:\SfDevCluster\Data\Node.1\Fabric\work\Applications\HelloWorldApplication_App1\HelloWorldPkg.Code.1.0.0.0\helloworld.exe'.

The process has been terminated.

Refresh the process list before attempting another attach.
---------------------------

---

Tried sometime figuring out what is going on. But less luck. But I was able to run the client application and invoke the service. Yes kind of I am learn it so finally settled without debugging.

Of course I will spend sometime to identify why the debugging it not working. I strongly think that the 4 nodes may be overloading my system and debugger is not able to connect within configured timeout.

References

https://social.msdn.microsoft.com/Forums/azure/en-US/f5d05143-15ce-4685-9899-6b659ffb77a0/debugging-fails-in-vs2015-rc-systemfabricfabricconnectiondeniedexception?forum=AzureServiceFabric&prof=required