Travis-CI v/s Windows development environment

We are in the world of DevOps where we do CI & CD activities automatically for even individual code commits. Don't think DevOps means just doing CI & CD. One of the practice in DevOps is CI&CD. Though there are so many CI&CD tools, the leading 'free for open source hosted SaaS' tools are AppVeyor & Travis-CI. Azure Pipeline is new player in the market giving free service.

AppVeyor is my default choice. But in order to get a feel of another CI&CD mechanism, but I used Travis-CI in one of my personal projects.

Problem

It was going good till the builds started failing recently. After inspecting logs, I could understand that there are new NPM module versions available and conflicting. They were used because I used ^ in versions which is mentioned in packages.json. So decided to fix.

It started working fine in my local machine but when I pushed into GitHub, the CI&CD pipeline again failed.

This time its was on the tests. Some files were not served during tests. The http error code is 404-Not found. Initially I thought, since the file names have unicode characters in it, Webpack dev server is not able to serve the files. But it worked in my machine. The classic case of 'it works in my machine'.  But here it is my personal project and I am the only one to fix.

Cause

After hours of debugging via log statements, I was able to figure the silly issue. It is the difference in Linux v/s Windows. Windows is not case sensitive for file paths but Linux is. The files are accessible in Windows but due to case sensitivity the files are not accessible in Linux. Travis-CI uses Linux in their build environment.

Seems others too faced the same issue and asked Travis-CI to add documentation.
https://github.com/travis-ci/docs-travis-ci-com/issues/209

But Travis-CI is smart and had the documentation already. It was I who is now into the lazy to read group.
https://docs.travis-ci.com/user/multi-os/#operating-system-differences

Moral of the story

Better be consistent with Dev, QA, Stg and Prod environments. 10th item in the 12-Factor App methodology.

https://12factor.net/dev-prod-parity

Updates

15Oct2018

More best practices on TravisCI - https://eng.localytics.com/best-practices-and-common-mistakes-with-travis-ci/

Architecture Decision Record (ADR)

Basics

Software Engineering is a relatively new field of engineering. There are still debates whether it is engineering or art. Regardless it needs an Architecture similar to other engineering disciplines. Unlike the other fields, the main challenge software architects faces is to make sure the delivered code is inline with the Architecture. With the advent of Agile which is very difficult to practice in other field, finalizing Architecture in software is really challenging. If we religiously finish architecture before coding, some other competition might have taken the market over.

But still we need to document the architecture even it is after the release. Don't laugh. It is needed to refer in future at least. Software unlike the other fields, is change friendly. It evolves really fast. There are different ways to document architecture. We can use UML diagrams, new trend of C4 architecture model etc...Even if we create beautiful architecture diagrams of delivered software, the problem is that it is very difficult to document why we took the decision. If we get a software in Silverlight, we should understand in the first place why  that technology is selected? Why there are WCF web service calls instead of ReST services? etc... Nothing happens without a reason in software development. So it is good if that reason can be recorded for future developers.

We can sit and write a beautiful document around diagrams and add the decisions. But its really boring and it becomes obsolete immediately as the software evolves. So what another approach.

ADR ie Architecture Decision Records in its simple form can be interpreted as the adoption of agile into documentation. Below is one good article about that practice.

http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions

Recently ThoughtWorks brought it to main attention. They call it as Light weight ADR. Yes in teh world of agile everything has to be light weight or at least in the name. As per the past history they promote after they had tried it in the field. There are more references about ADR which are included in the References section of this post.

Contents of ADR

When we adopt the ADR into project, the first thing to decide is what contents needed in ADR. The main problem is to make it light weight. If we add all the diagrams, meeting minutes etc... it will be another documentation nightmare. So we have to choose what fields to be included. Below link summarize many formats.

https://github.com/joelparkerhenderson/architecture_decision_record

Format of ADR

Now a days developers even write official letters in markdown. It got that much attention due to its support in opensource communities such as GitHub. So without any confusion the ADR can use markdown.

Where to keep the ADR

Another question is where to keep the ADR. Since it is small textual representation, it can be inside a shared folder, SharePoint or in email. If we keep the ADR in a place other than source code, it may not help us in future. If the ADR is with source, where ever the source goes the ADR too goes. Today it can be TFS tomorrow it can be in Git. Sometimes companies opensource via GitHub.

Since ADR don't have any relevance without source code, the better places is with code.

Open source

Now a days GitHub is the synonym of open source. They support markdown in wiki as well as in source. Lets see the differences in keeping ADR in wiki v/s source

ADR in wiki

Wiki is independent of code. The main problem is that when we branch to develop a new feature we cannot have ADRs inside branch which are needed for that feature development. We can workaround this by many means but still little difficult. The advantage is easy editing. No need to check out, commit and push to get some changes done.

Below is one example for keeping ADR in wiki.

https://github.com/getgauge/gauge/wiki/0005-multi-threaded-parallel-execution

ADR in Source

The opposite way helps us to keep the ADR with source. When we branch the ADR comes with us. If we are doing any overriding of Architecture, we can document there. The pull request can include the same which gives reviewer that there is something fundamentally happened due to this feature.

Naming

The main purpose of naming is to distinguish the ADRs. When we keep the ADR record files, either we can keep then inside a folder called ADR or prefix the files. Similarly we can sequentially number them and keep that number in the file name or inside the contents. Right now there doesn't seems to be a standard. Hopefully something will evolve soon similar to Swagger for APIs.

Some real world usage

Below is one real world usage. The ADRs are kept in below location.

https://github.com/eclipse/winery/blob/master/docs/adr/

Rendered as below in the documentation.

http://eclipse.github.io/winery/adr/

How I implemented

My open source projects started adopting the ADR. Below is one example ADR

https://github.com/KarelRobot/karel-web/blob/master/docs/adr/

Rendered as
https://karelrobot.github.io/karel-web/adr/0001-use-webpack

There are only 6 fields used to make it or call it lightweight.

References

https://adr.github.io/

Functional Programming - Randomize IEnumerable

.Net has IEnumerable to represent a sequence. Though it is not advertised as a functional helper, we can use IEnumerable to get really clean functional programming in .Net. It has so many methods to manipulate and select elements but it really lacks a mechanism to take random numbers from the sequence. Below is one which gives us somewhat random elements from the IEnumerable sequence.

---

public static IEnumerable<TResult> Randomize<TResult>(this IEnumerable<TResult> source)
{
            return source.
                Select((sourceItem, index) => new
                {
                    Item = sourceItem,
                    Id = Guid.NewGuid()
                }).
            OrderBy(t1 => t1.Id).Select(t1 => t1.Item);
}

---

How to use the above?

---

IEnumerable<int> input = new List<int>() { 1, 2, 3, 4 };
int randomElement = input.Randomize().FirstOrDefault();

---

As seen in the source the randomization is depended on the GUID generation. If the GUIDs are generated in increasing order the randomization will not work.

The advantage of this method is to randomize as lazy collection.

Nuget support

The above is available as nuget. Below is the URL.

https://www.nuget.org/packages/DotNet.Helpers

PowerShell to get list of email addresses from company AD

Often we may need to send mail to everyone to the company as an announcement or requesting urgent help etc...Normally companies might have a group mail address to do such things. Even if there is none we can easily get the list of all emails

First to get the OU and DC details of your AD. If there are confusion on what is OU, DC refer the details here. Better search using your email id itself to get the OU and DC details.

---

Get-ADUSER -Filter 'EmailAddress -like "<your email address>"'

---

This will give the details in the DistinguishedName property. Now fill that information in below script and run.

---

$container = "OU=<your OU>,DC=<DC>,DC=<DC>"

get-ADUSER -Filter * -SearchBase $container | `
select -Property UserPrincipalName | `
Export-Csv -Path "<path>.csv"

---

This export the email addresses to the csv file mentioned. It is interesting to see that the UserPrincipalName has the email. If the email is kept separately, the code has to be modified to select proper attribute.

Happy Scripting...