Tuesday, November 13, 2018

Freemium - What is always free in clouds platforms

Background

There are any different pricing model in the software industry which doesn't require any money to be spend to use software. Trial, totally free, freemium, sand boxed trial account etc...They are different in their own way. Totally free and freemium are the models which really help us build something based on the offering. Freemium might not be a new word for a follower of this blog. There were posts which has freemium reference such as Serverless monitoring, online diagramming tools, tool box for open source more precisely developers who don't want to invest on tools.

Once again Freemium is a model where we can use systems in free but for advanced usage we need to pay. Event if we don't want advanced features the basic features are free for ever. Refer wiki for the Freemium

Freemium cloud features

Lets see what is there from Cloud providers in freemium tier. ie what is there always free which can help us to develop production prototypes and start own company without investing any money other than domain if we want.

Please note that, these are not the free account or free trial which we cannot depend on to put something into production.

https://cloud.google.com/free/docs/always-free-usage-limits#container_name
https://azure.microsoft.com/en-in/free/free-account-faq/
https://firebase.google.com/pricing/
https://aws.amazon.com/free/?awsf.Free%20Tier%20Types=categories%23alwaysfree
https://www.heroku.com/pricing

Since the cloud providers may add more services as freemium, only links are provided not another copy paste which needs more time to make it up to date.

Freemium doesn't necessarily guarantee that it its free for lifetime. The vendors have their rights to change the pricing.

More links

https://en.wikipedia.org/wiki/Cloud-computing_comparison
https://www.infoworld.com/article/3179785/cloud-computing/aws-vs-azure-vs-google-cloud-which-free-tier-is-best.html
https://www.redpixie.com/blog/aws-vs-azure-vs-google-cloud

Tuesday, November 6, 2018

PowerShell to find the listening port and its process - eg SQL Server Analysis Service

Problem

Enterprise environment always enforce security in different levels. One such level is network ports. There will be less questions when we want to open port 443 for http(s) communication. Also it is fine if the ports are familiar such as 1433 for SQL Server. But when it comes to some unfamiliar ports its difficult. Also they wants to open very minimal ports. So finding and documenting exactly what ports are required is a considerable job for an Architect. This is obviously not applicable to Architects who are there when solutions or define the solution themselves. But its trouble for Architects who reverse engineer what is in the development / testing / staging environment and prepare document which is essential to move the application to the higher environment.

One such thing is SQL Server Analysis Service. For SQL Server OLTP engine its easy to find out. But for analysis service, its difficult if the installation is done using custom port.

What Google says

This is not the first time people encounter the problem and there are lot of solutions to it. Some from google are below.

To find ports and processes in general

http://blogs.microsoft.co.il/scriptfanatic/2011/02/10/how-to-find-running-processes-and-their-port-number/
https://stackoverflow.com/questions/42857466/find-process-that-is-blocking-a-port-on-windows

https://superuser.com/questions/1215093/powershell-one-liner-to-show-process-on-same-line-as-port-using-netstat-issue

Find SSAS Ports

https://support.microsoft.com/en-us/help/2466860/how-to-determine-and-change-the-port-of-an-ssas-instance
https://www.cisco.com/c/en/us/support/docs/conferencing/telepresence-management-suite-analytics-extension/112506-pqa-112506-00.html#q1

The general scripts are either based on long custom PS command or text analytics based. The SSAS port findings required 2 steps to figure port out.

Easy solution using PowerShell

Below is another one which does the things in singe PS command. Yes obviously there is piping and we can argue whether its multi-line script or not.

Finding ports and processes


Get-NetTCPConnection | `
where {$_.State -eq 'Listen'} | `
select -Property LocalAddress, `
                        @{name="Process";Expression={$psid = $_.OwningProcess; (Get-Process | where {$_.Id -eq $psid }).Name + ', ' + $_.OwningProcess}}, `
                        LocalPort ` |
sort -property Process

Pretty much straight forward. It relies on the Get-NetTCPConnection commandlet. Then does some filtering and selection to get the list back.

Finding port of SSAS Service

We can apply the above technique easily to filter the SSAS process

get-nettcpconnection | `
where {$_.State -eq 'Listen'} | `
where {$_.OwningProcess -in (Get-Process | Where {$_.Name -eq 'msmdsrv'}).Id} | `
Select-Object -Property LocalAddress, `
                        @{name="Process";Expression={$psid = $_.OwningProcess; (Get-Process | where {$_.Id -eq $psid }).Name + ', ' + $_.OwningProcess}}, `
                        LocalPort `


It takes a big assumption that the process name of SSAS is 'msmdsrv'. In future Microsoft change the name the script will not work.

References

Tuesday, October 30, 2018

Travis-CI - Continuous deployment to staging but production deployment only on tag commit

Background

Travis-CI is a hosted Continuous Integration & Continuous Delivery tool. Based on configurations, users can have their application deployed to production. They provide free service to open source projects. The CI&CD environment currently using Linux.

Aim

  • Have the CI & CD pipeline trigger for each and every commit going into GitHub. Deploy only to staging or testing area.
  • Testing / staging area is GitHub pages hosted in the same repo. Deploy to staging should not trigger another build which ends up as infinite loop. Not into gh-pages branch but to \docs folder.
  • When there is tagged commit deploy to production.

How its done

Avoid build on commit from Travis-CI bot

The word bot is used to indicate the CI&CD account used by Travis-CI. We can see that account on the commits from Travis-CI. When the bot commits, it uses message as follows.

""Deploy <github userid>/<project> to github.com/<github userid>/<project>.git:master""

One such example given below.

"Deploy joymon/prayerbook to github.com/joymon/prayerbook.git:master"

We need to write build conditions at the top of .travis.yml file to bye pass the build if the commit message is matching.

Build and deploy on commit tag

This is another condition we need to put into condition. If we don't put this Travis-CI will not build on tag commit. Below is one example.

if: tag IS present OR commit_message != "Deploy joymon/prayerbook to github.com/joymon/prayerbook.git:master" OR tag = true

If we use tag = true alone it may not work. Needs more analysis on the Travis-CI yml schema.

A sample travis.yml file can be found below which works based on the above conditions.

https://github.com/joymon/prayerbook/blob/master/.travis.yml

References

https://stackoverflow.com/questions/18216991/create-a-tag-in-github-repository
https://docs.travis-ci.com/user/conditions-v1
https://github.com/travis-ci/travis-ci/issues/8518

Disclaimer

The author is not working for Travis-CI or any of their subsidiaries or partners. There is no guarantee that the technique mentioned in this post will work forever. Travis-CI may change their system the way they want it to be. 
This is not at a sponsored post.

Tuesday, October 23, 2018

Explaining youtube down to 4 years old kid

Though I had written about explaining technical things to kids, this is the first time I had to try it in production. At the time of writing this post 16oct2018:10PM EST, Google's well established youtube.com is down thus I got computer from my 4 years old son.

https://twitter.com/TeamYouTube/status/1052373937839980544
https://www.cnbc.com/2018/10/17/googles-youtube-outage-affected-users-in-us-australia-asia-europe.html

He was angry and slowly moved to crying mode when he understood there issue which his dad cannot fix. Below were our attempts to make him understand the situation.

Attempt 1 - Sympathy

This is my attempt with over confidence that I had writing experience of how to explain tech things to kids. Though the words were not exactly same as we speak mostly in our mother tongue Malayalam at home along with English .

I - Son... you know the youtube is a website which is built and maintained by engineers like us. Your mom also an engineer though not working now. There are some troubles in their computers and those poor guys now will be working hard like me. We have to wait till they fix the issue. 

Son - What? youtube is not working. Fix it...

Attempt 2 - Tech explanation

My wife started laughing already. But I did one more attempt.

I - You know...the videos you are seeing in our computer is not actually in our computer. Someone else uploaded those videos into youtube's computer sitting far away. Those youtube's computers are down due to some issues. We cannot do any thing from here, they have to fix it. 

Son (with no hope on me) - Can you fix it?

Attempt 3 - Mom's version

Mom - Son we usually call your dad when he is in office. right?
Son - yes
Mom - What if his phone is not working? We keep calling but we cannot speak with him.
Son - yes
Mom - Our phone is good but dad's phone is not working. Think youtube is dad's phone and is not working.
Son (sad) - ok. Lets sleep.

Tuesday, October 16, 2018

Progressive Web Apps - Index

Intro

Progressive Web Apps or PWA technology is expected to end the nightmare of developing platform  specific apps for achieving one purpose. Currently this is tough for mobile developers as they have to develop for Android and iOS. Thanks to Microsoft for ending Windows phones. But still there is Windows app store where applications for Windows 10 can be downloaded. Still its nightmare to have one app working for all the platforms though there are many technologies are there to address the same such as Xamarin. Yes that is a big prediction that one day PWA will make Xamarin like platforms obsolete similar to how HTML5 made Flash & Silverlight into history.

https://en.wikipedia.org/wiki/Progressive_Web_Apps

The purpose of this post is to index all the links related to PWA in one place at least for my personal use. It will be including links my future PWA related posts as well. I am not the person doing it and I tried to include similar indexes at the end of this post.

Basics

Feature detection

Not all browsers support PWA also not all support all features. Since PWA included bunch of features, better do feature detection than checking the browser version.

Features

Below are the features of PWA. Not a complete list as its still evolving.

Navigation Preload

Offline Storage

This is not really PWA feature but can be leveraged to build offline application.

Service Worker

Highlight of PWA which help us to develop offline applications

This will take the web apps further towards native feel. Also help us to reengage users. 

Installation

Another highlight. This feature is going to help us to get our web apps installed as native apps.

Publishing to AppStores

The app stores are getting PWA apps along with native apps. One day there will be more PWA apps than native. The day where all the device capabilities/APIs are exposed via PWA specs.

Developing PWA

What developers needs to know about PWA.

Debugging and experiences

Seems little difficult to debug at this point. But will definitely get improvements over time.
https://developers.google.com/web/fundamentals/codelabs/debugging-service-workers/

Testing PWA

In the world of DevOps we need to have coded test cases to have continuous delivery. This space needs more innovations.

Platform specific

Not all platforms started supporting same way. Below goes platform specific details. Hope some day I can make this section obsolete.

Windows

Sample apps

We are not alone. There are apps out there which use PWA. Its reality.
https://pwa.rocks/

For quick demos

https://expensemanager.demo.vaadin.com/ - Keeps data in local cache, Installable

Source code

Some repos to browse for PWA.
https://github.com/GoogleChromeLabs/airhorn - This demonstrates the installation scenario

Other indexes like this

https://github.com/hemanth/awesome-pwa

Tuesday, October 2, 2018

Online diagramming tools

We are living in the age of easy software development and business friendly environment. Lot of things are free if we want to develop software or start a company than 10 years back. Diagramming was difficult earlier but now there are lot of tools to create good stunning diagrams. Below is a list of online browser based diagramming tools for software developers and architects. All are either free, or has freemium pricing model.

General

Below has general diagramming support as well as software engineering diagrams.

www.draw.io
https://www.lucidchart.com
https://cacoo.com

https://cloudcraft.co - AWS specific, 3D diagrams

Architecture as Code

https://structurizr.com/ - Used to draw C4 arch diagrams using code!


This list demands periodic update as new tools will appear and some fade away or move to paid model. Hopefully updates can be done at least once an year.

Tuesday, September 25, 2018

Travis-CI v/s Windows development environment

We are in the world of DevOps where we do CI & CD activities automatically for even individual code commits. Don't think DevOps means just doing CI & CD. One of the practice in DevOps is CI&CD. Though there are so many CI&CD tools, the leading 'free for open source hosted SaaS' tools are AppVeyor & Travis-CI. Azure Pipeline is new player in the market giving free service.

AppVeyor is my default choice. But in order to get a feel of another CI&CD mechanism, but I used Travis-CI in one of my personal projects.

Problem

It was going good till the builds started failing recently. After inspecting logs, I could understand that there are new NPM module versions available and conflicting. They were used because I used ^ in versions which is mentioned in packages.json. So decided to fix.

It started working fine in my local machine but when I pushed into GitHub, the CI&CD pipeline again failed.

This time its was on the tests. Some files were not served during tests. The http error code is 404-Not found. Initially I thought, since the file names have unicode characters in it, Webpack dev server is not able to serve the files. But it worked in my machine. The classic case of 'it works in my machine'.  But here it is my personal project and I am the only one to fix.

Cause

After hours of debugging via log statements, I was able to figure the silly issue. It is the difference in Linux v/s Windows. Windows is not case sensitive for file paths but Linux is. The files are accessible in Windows but due to case sensitivity the files are not accessible in Linux. Travis-CI uses Linux in their build environment.

Seems others too faced the same issue and asked Travis-CI to add documentation.
https://github.com/travis-ci/docs-travis-ci-com/issues/209

But Travis-CI is smart and had the documentation already. It was I who is now into the lazy to read group.
https://docs.travis-ci.com/user/multi-os/#operating-system-differences

Moral of the story

Better be consistent with Dev, QA, Stg and Prod environments. 10th item in the 12-Factor App methodology.
https://12factor.net/dev-prod-parity

Updates

15Oct2018

More best practices on TravisCI - https://eng.localytics.com/best-practices-and-common-mistakes-with-travis-ci/