Convert traditional visual studio project to multi targeting SDK-Style library project

Background

To directly jump to the problem this post is dealing with by skipping the story, directly jump to the last paragraph in this section.

When we were developing .Net libraries or third-party components in the early days we had to keep one project per framework version. That is because there would be more than one version in the mainstream at any time. Though Visual Studio 2008 onwards there is some sort of multi-targeting which matured with VS 2010, it was not easy to publish libraries from one project targeting different framework versions.

The need to target multiple versions became intense with the introduction of .Net Core. Both Full .Net Framework, as well as .Net Core, keep getting new versions.

But it changed totally when Microsoft introduced multi-targeting library projects in Visual Studio 2017. For simplicity skipping the PCL(Portable Class Libraries) which was introduced in between to deal with common code base for Silverlight and .Net Framework. At that time, there was real confusion about what library project to choose when we create new. Later Microsoft introduced JSON based project file for .Net core which dropped with a migration path.

The new project format really simplified the .csproj files. Initially, it was more related to the .Net Standard but later it is used for different purposes as well even for console apps. Let us not get into .Net Standard as differentiating it is another post by itself. Finally, the new project format got the name SDK-Style project which we can identify by the below tag in the project file (.csproj)

<Project Sdk="Microsoft.NET.Sdk">

One of the earliest blog posts explains what is the vision when they revamped the project file format. No listing of files inside the project file, can produce nuget directly without nuspec, easy modification without IDE, etc... were some. Until then the csproj was considered as only software editable file format though it is XML. But with the SDK-Style project format, though it is still XML, humans are also allowed to edit manually. More details can be found here.

Now the csproj is more like the NodeJS development where the packge.json is human editable. Really dreaming for a day when Microsoft says "develop with NodeJS or something else and host in Azure". Just a dream.

Let us come back to the purpose of this post. Here we are seeing how to convert existing old nuget library projects to modern SDK-Style library projects which can target multiple frameworks and versions including from .Net 4.5 to .Net Core 3.1.

Existing approaches

The easy way most of the internet says on converting existing projects to a multi-targeting SDK-Style library project is to create a new project. Then just copy-paste the files from old project location to new. Since the new project format doesn't require all the files to be added to the project file its easy as just copy. Links are given in the references section.

Just compare with NodeJS development where package.json doesn't need to know all the files. 

There is another way to convert the projects to SDK-Style. First, convert the nuget package references from the packages.config to <packagereference> tag in csproj. Then use a tool to migrate the project file. There seem multiple tools listed below though they point to the same GitHub repo. Maybe one is a wrapper over other when VS 2019 released.

https://www.nuget.org/packages/Project2015To2017.Migrate2019.Tool/

https://www.nuget.org/packages/Project2015To2017.Cli/

Manual approach

However, it seems easy to convert by hand, if our project is not that much complicated and confirms the compatibility. The only problem is that we need to clean the project which is not required if we start a new project and copy files.

Manually porting project to SDK-Style library project

Below goes a diff of manual porting of one project. If the iFrame is not loading directly go to the GitHub URL. Another difference is that the SDK-Style doesn't really require the Assemblyinfo.cs

Manually porting test project to SDK-Style

The same can be applied to test projects as well. The link to the comparison is here.

Please note that it worked smoothly as this project is simple and only one person handles all at this point. When working in Enterprise environments or a large team environment, it would not be efficient as following the steps such as evaluate compatibility, refactoring projects to smaller pieces before doing anything.

References

http://hermit.no/moving-to-sdk-style-projects-and-package-references-in-visual-studio-part-1/

http://hermit.no/moving-to-sdk-style-projects-and-package-references-in-visual-studio-part-2/

https://weblog.west-wind.com/posts/2017/jun/22/multitargeting-and-porting-a-net-library-to-net-core-20
https://www.michaeltaylorp3.net/porting-a-net-framework-library-to-net-standard/

Published .Net Orchestration library to nuget via AppVeyor

Introduction

There is no need to explain about the importance of nuget package ecosystem to .Net development community. Now a days its very difficult to consider someone as .Net if he don't have a nuget package. Of course now I can tell because I just published my first nuget package :)

Its the same Orchestration library I have hosted some months back.

Standard way

How to package our .Net library(.dll or scripts) as nuget package and publish to www.nuget.org is documented very clearly in nuget web site. Link below.

http://docs.nuget.org/create/creating-and-publishing-a-package

So I followed the same guidelines and published it. All manual and using Visual Studio and command line tool nuget.exe. Its available in below location.

https://www.nuget.org/packages/Orchestration

CI & CD way

Manually publishing every time from our development machine is really boring task. Its highly repetitive. Also how to ensure that the the library actually works after each change? If we cannot solve our this problem, how can we solve other's problem.

As everybody knows, the answer to this problem is continuous integration. If that is hosted its too easy. If its free too, don't ask any more questions, just do it.

As we have seen in some of previous posts, AppVeyor is the best place for doing free hosted CI & CD for .Net projects at this time. I will highly recommend than Microsoft's Visual Studio Online service which gives some free build minutes. Infact Microsoft itself using AppVeyor to do their integration.

Below are the official AppVeyor documentation about how to setup CI for nuget projects.

http://www.appveyor.com/docs/nuget#automatic-publishing-of-nuget-projects
http://blog.appveyor.com/blog/2014/02/21/nuget-support-in-appveyor-ci

Compilation of the project and running unit tests are trivial. There won't be usually any issues at all. But we want to automatically create the nuget package and increment it's version, we may need to think twice.

There are basically 2 options. The first option is kind of obsolete now.

1. Explicitly run our own script to auto increment the nupkg file version and fire the nuget pack command to create the .nupkg file
1. http://www.codeproject.com/Tips/806257/Automating-NuGet-Package-Creation-using-AppVeyor
2. https://code.msdn.microsoft.com/Create-and-Push-NuGet-c6072402
2. Use AppVeyor's out of the box settings to auto increment the version.
1. Enable Assembly patching in AppVeyor
2. Use $version$ token in version property inside .nuspec file
3. Enable the nuget build in AppVeyor to automatically package the file.

We can take the second approach which is out of the box which don't.require any coding. Remember less code less defects!!!. Once this is working file we can see the .nupkg file in the build artifacts. From there we can push via AppVeyor itself. We need to setup our nuget API key either in appveyor.yml (encrypted) or in the web interface.

---

I was in little trouble in the beginning to do things without custom scripts and contacted their support. They helped immediately. I really appreciate it.
http://help.appveyor.com/discussions/problems/525-nupkg-getting-created-at-same-version-every-time

Orchestration library for .Net 4.0

Introduction

It was started with queued processing in one of the projects. We have a queue framework mechanism for long running processes. If anyone needs to use that queue, they should host a web service which will be called from queue framework after de-queue. The queue type and service url needs to follow some convention to get the connection.

Problem

If it is a technology shop where developers follow principles such as SOLID this problem will never occur. Here in the delivery shop, the problem was in the service method which does the long running operation people started writing 2000+ lines.

May be the developers were securing their jobs. Soon the code base became monopoly to corresponding devs. No other developers were able to touch it.

It went without noticed for couple of years. It became problem soon after people left the team and started getting failures on really long running (more than 1 hour) queue processes. It was supposed to resume but none of those resumed from the failure point on requeue. It became nightmare to makre it resumable. 

Why you developed it in sequential way? The developers answered this is the way we know. We need to do sequence of operation of the de-queued message. Output of one operation may affect the subsequent operation sometimes they are independent. If required we can separate to functions.

SRP

It is clear that developers were not following the Single Responsibility Principle. Separating steps to methods doesn't help here because some long running operations share steps. So its the time to enforce SRP.

Another problem is the way queue framework relies the control to actual long running program. It was a simple service call based on convention. It helped the developers to write a service and write the logic inside the implementation method.

The solution is to accept step objects by the queue framework and call those one by one instead of calling a service method. So after each step Orchestrator can store the state so that it can restart from the last step. The steps can be unit tested easily.

The Orchestrator born here.

Mutability - Pipelined operations on same object context

Another concern was to manage the state. These operations were sharing same state and some time one required output of previous. Not necessarily the immediate previous step.

Everywhere its evil to have mutability and people strive for immutable systems. But here it was little difficult to achieve the same. So continued with mutability.

But how to control the flow better?

Stateless steps

Why should a step be ever stateful if Execute(context) method gets the context via parameter? If the  Orchestration engine can pass the same context object to every step and step can modify the state and there is no parallel step execution we are good. Isn't it?

Yes we are good without parallelism. 

Concurrency & Parallel step execution

Parallel processing is always difficult to get right. One first factor is to decide who defines the what to run in parallel. Whether its the Orchestrator framework or the step authors? Its a debatable topic and most probably we will end up with step developers to tell what to run in parallel. What will happen if developers, who don't know SRP defines the unit of parallelism?

Most of the time stability is better than speed. So no concurrency as of now.

Source

Being an architect I can suggest the solution. But cannot enforce it as long as I don't have the rights to hold a release. But there is no problem in developing such a framework and open sourcing the same.

https://github.com/joymon/Orchestration

Note: This is for .Net 4.0 users. If the target is .Net 4.5, there is better library called Banzai.