Tuesday, March 28, 2017

My JavaScript Module experiments - CSS

This post is continuation of my JavaScript module experiments series with Webpack. Earlier posts below.

1 My JavaScript Module experiments
2 My JavaScript Module experiments - CI & CD
3 My JavaScript Module experiments - CSS

This post assumes the basic knowledge about NodeJS, NPM, Webpack such as its config file and require mechanism.

Should we bundle css?

It really depends. 

Traditional CSS development

If we take the scenario of simple web site development, we may not need bundling provided entire CSS is in single file. But that is not at all the scenario right now. We need the sites responsive and for that we may have to add dedicated styles for different screen resolutions. One way is to write everything in the same css or separated files loaded on demand. But keeping in different files and downloading all to client and deciding which to use will slow down the performance. Because it needs more http file requests. Keeping in separate files is easy in terms of maintaining. Normally the decision to what style needs to be applied is taken at client side based on the screen size. It will be overhead to request the css for different sizes when the user resize the browser window. So it is better to bundle the css for different sizes at server and download as one file. 

Modern CSS

Modern CSS means writing CSS with the help higher level languages such as LESS or SASS. They help us to write less reusable code and get things done. Since the poor browser cannot understand this LESS or SASS languages, we have to convert to CSS. That process is called transpilation. Here there are chances that some styles will be in pure CSS, some in LESS and some in SASS. As we have to convert to CSS and refer into the application, it is better to bundle the outputs to one css file and refer same.

HTML component

Another scenario is development of web components or on-demand view development. In those scenarios we don't have to keep entire CSS in single place and refer from the beginning. We have to bundle the css along with the JavaScript component which only loaded when needed.

Webpack and CSS

Now lets see how the Webpack and CSS works together. 3 different scenarios are listed below to understand at the simplest level. To keep it simple no other things like ECMA 6 or TypeScript is used.

Simple scenario - webpack and css

Here we are simply going to require .css files from JavaScript to get it available in the html. Below are the steps
  • Import the npm packages named css-loader & style-loader. These are the Webpack specific NPM packages to process .css . The first one is to recognize the .css file and other is to load the .css as <style> tag.
  • Modify the webpack.config.js file to have loader. For syntax refer to the sample given below
  • Add require statements in to the JavaScript file
    • require('./stylesheet.css');
Below is the link to sample.


To run the sample. 
  • Make sure there is NodeJS and NPM
  • Clone the repo to the local machine
  • Open NodeJS command prompt and navigate to the \ECMA-CSS folder
  • run 'npm install' command to download dependencies.
  • run 'webpack' command.
    • If the webpack is not available at system level, '.\node_modules\.bin\webpack' is required
  • This will produce bundle.js file
  • Just run the index.html file from file system. Make sure there are permissions to run JavaScript from local file
When running we can see that there are <style> tags added to the <head> which are not present in index.html file. Those are added by the style loader at runtime.

Combined css

Injecting <style> at runtime is not a good solution. Better is to have a final combined css file. The below sample shows that mechanism using one more npm package called 'extract-text-webpack-plugin'. Since the sample uses webpack 1, the version of 'extract-text-webpack-plugin' should be 1.0.1.


Here the index.html is referring to a style file called bundle.css which is not there in repository. When this sample is run using the same steps mentioned for above previous scenario, the bundle.css will be created by Webpack. Once that file is present, we can run the index.html as usual.

The bundling of multiple css files to one is done by using ExtractTextPlugin configured in webpack.config.js file and here we don't need the style-loader.

CSS & LESS together

This is the next level where we mix plain .css with advanced .less file and finally produce combined .css file. Sample is available in the below location.


In this sample, instead of earlier childstylesheet.css, childstylesheet.LESS is used. The npm packages used are less, and less-loader for compiling .less files and to load the same respectively. The webpack.config.js also modified to tell how to handle the .less files and to combine using ExtractTextPlugin.

During the compilation the .LESS file will be compiled to .css and then combined with other .css file to produce bundle.css file.

Further reading

https://webpack.github.io/docs/stylesheets.html
http://jamesknelson.com/writing-happy-stylesheets-with-webpack/
https://css-tricks.com/css-modules-part-2-getting-started/
http://stackoverflow.com/questions/39015521/import-css-and-js-files-using-webpack
https://github.com/webpack/file-loader/issues/32

Tuesday, March 21, 2017

GoDaddy to use Github pages

Recently I setup my personal site www.joymononline.in to point to GitHub pages. The domain registration is still with GoDaddy but the hosting space is no more required from GoDaddy. Infact that was eating good amount of cost involved in maintaining the site.

How to configure GoDaddy to use GitHub pages is already a solved problem. There are many sites which explains the same. Below are some.

http://captainwhippet.com/blog/2014/05/11/blog-setup-details.html
http://mycyberuniverse.com/web/configuring-a-godaddy-domain-name-with-github-pages.html


Why this post?

The IP addresess

One of my confusion was about the IP addresses mentioned in those posts. Are those still valid?Some posts are written in 2015. Fortunately those are still valid.

GoDaddy Records

Another place I was little confused was about setting up A records. There were some A records. Instead of deleting all, I just modified to have GitHub IPs.
Also some sites talk about 2 IP addresses to be added via 2 different A records but some talks about only 1. I had updated 2 records and the site got up with GitHub pages in couple of mins.


Tuesday, March 14, 2017

Architecture of JoymonOnline - Serverless

This can be considered as continuation of 3 old posts.


The first post, talks about the architecture of my personal site JoymonOnline.in. Other posts explains about new software architecture trend called Serverless and the conversion of my site towards Serverless architecture. 

This post explains the new architecture.

Architecture of Serverless JoymonOnline.in

Hosting

Hope the diagram is pretty much self explanatory. Its going to be pure client side single page application (SPA). In other words there is no page refresh when user goes to different view. 

The domain is going to be continued with Godaddy. But the hosting will be done inside Github pages, which is free. When we say Github hosting is free, it comes with limitations and quotas. But for my personal site those limits are not going to hit at least for some years :). It clearly tells that for a different person who is really famous the limits will come into play. So Github pages is not the silver bullet for hosting.

Flow

Once the client browser gets the static site(html,js & css) from Github, it becomes active by bootstrapping Angular components. It calls required services present in external systems for data. Most of these external systems are already mentioned in my previous post. After the data retrieval, the JavaScript updates the DOM with the help of AngularJS framework. The Angular JS version is 1.x used and there is a blog post explaining why 1.x is selected when Angular 2 is out.

Future candidates


  • Azure Function - For Github & Blogger API due to anonymous quota limits.

Pros

  • No cost to host web site. Just the domain registration. Most of the services are free.
  • No need to license development tools. For  ASP.Net, Visual Studio community edition don't have much features.

Cons

  • Multiple points of failure - Earlier, if JoymonOnline.in domain was accessible entire site works. But now all the domains of external systems should be accessible from the client machine which is not possible in all places.
  • More dependencies - Has to monitor what changes in external systems and has to maintain accordingly.
  • JavaScript restriction - Some browsers (inside some companies) might be still blocked to download JavaScript and execute.
Note - As of today (14Mar2017), the redirection for the site from Godaddy to Github pages to serve site has not done. But the site is up and running in Github pages joymon.github.io.

Tuesday, March 7, 2017

ConcurrentCounter - Reduce code with Functional programming

The fashion of functional programming

Now a days functional programming already made its come back and every language, companies and developers want to embrace functional programming. When we look at their works we can see it needs to travel more to reach the functional programming world. It was same for object oriented world as well. People used C++ but the code looked like C.

That is not the concern with this post. One of the reasons we here when people adopt FP is that it helps us to achieve things with less code. This is just a scenario where FP can reduce some amount of code.

Scenario - Thread safe manipulation of resources

We are in the world of threads. Knowingly or unknowingly we had to deal with threads. Some languages / technologies already has better ways to deal threading without developer write any explicit line. NodeJS can be an example. But for C# guys like me, has to write programs properly to make sure it works with threads. If all the tasks are ready to run in parallel,  there are no issues. But the reality is not that way. There will be so many critical sections we had to handle to achieve a good working software. We rely heavily on thread synchronization mechanisms to achieve the same. Whenever we deal with those, it is nightmare and there are so many chances to produce duplicate code. That is the place functional programming can help us.

Taking one situation. We have a counter inside a web service.That counter can be considered as a resource just like file or other resources. Counter variable is accessed by multiple service methods, some reads some writes. So the first way comes to our mind would be C# lock (VB.Net SyncLock) keyword. When we implement it there are chances that we may write lock statement in multiple places. Below is a mechanism to avoid that by using functional techniques of C#.
Please note that in real world no one should use this mechanism for integers as far as Interlocked class is there in .Net framework. This is just a sample for understanding purpose.

Wrap the lock statement in fuction and pass the actions into it

The technique here is simple as anyone might have guessed. Just have a wrapper around the lock and inside the locked section execute the real manipulation code which is passed into it as Action<T>. Lets see the code snippet below.
internal class ThreadSafeCounter
{
    private int internalValue;
    private object lockObject = new object();
    public int Value
    {
        get
        {
            return ExecuteWithLock(() => { return internalValue; });
        }
    }
    public ThreadSafeCounter(int initialValue)
    {
        this.internalValue = initialValue;
    }
    public void Increment()
    {
        ExecuteWithLock(() => { internalValue++; });
    }
    public void Decrement()
    {
        ExecuteWithLock(() => { internalValue--; });
    }
    public void Reset()
    {
        ExecuteWithLock(() => { internalValue=0; });
    }
    #region Helpers
    private void ExecuteWithLock(Action action)
    {
        lock (lockObject)
        {
            action();
        }
    }
    private int ExecuteWithLock(Func<int> fun)
    {
        int result = 0;
        ExecuteWithLock(() => result = fun());
        return result;
    }
    #endregion
}
The main functions to notice here are as below.

ExecuteWithLock(Action a)
ExecuteWithLock(Func)

The lock(){} is there only in one function named ExecuteWithLock() and all other functions those wants to execute critical section of code has to call ExecuteWithLock and pass the code as parameter. That code passing as parameter is obtained by the well known Action<T>. Hope there is no need of further explanation. 

Tuesday, February 28, 2017

Puzzle - Pricing engine done for BarclayCard

Working as Software consultant than a full time employment is fun and make us up to date. We can see different architectures, coding practices, deployments and other software practices which will definitely help in case we have plans to start own company in future. But the down side is we had to attend interviews more often and there is no guarantee that when our assignment is over the market will be in good condition. Sometimes we may not get the job in the technology we want.

Recently I was attending many interviews. One of those was BarClay card. After 3-4 rounds of interview, I was asked to code a problem in production quality. It assumes 100% test coverage. Below is the problem.

Problem

An online retail company conducts market research to competitively price their products.Surveyed data contains Product code, Competitor and Price.

The retail company uses a Pricing engine which recommends most frequently occurring price. If multiple prices occur frequently, the least amongst them is chosen.

Products are classified based on parameters like Supply, Demand. Possible values are Low (L), High (H)

If Supply is High and Demand is High, Product is sold at same price as chosen price.
If Supply is Low and Demand is Low, Product is sold at 10 % more than chosen price.
If Supply is Low and Demand is High, Product is sold at 5 % more than chosen price.
If Supply is High and Demand is Low, Product is sold at 5 % less than chosen price.

Prices less than 50% of average price are treated as promotion and not considered.
Prices more than 50% of average price are treated as data errors and not considered.

Input consists of number of products, followed by each Product's supply and demand parameters.

followed by number of surveyed prices, followed by competitor prices.

Output must be recommended price for each product.

Sample inputs and outputs

Input 1:
2
flashdrive H H
ssd L H
5
flashdrive X 1.0
ssd X 10.0
flashdrive Y 0.9
flashdrive Z 1.1
ssd Y 12.5

Output 1:
A 0.9
B 10.5

Input 2:
2
mp3player H H
ssd L L
8
ssd W 11.0
ssd X 12.0
mp3player X 60.0
mp3player Y 20.0
mp3player Z 50.0
ssd V 10.0
ssd Y 11.0
ssd Z 12.0

Output 2:

A 50.0
B 12.1

This has to be completed within 2 days.

Solution

There can be different solutions. Below are some point about the one I did.  

  • For achieving pricing used decorator pattern
  • Coded in C#
  • Tests done using mUnit.
  • Mocks done using Moq library
  • Test coverage using ncover
Code is available in the puzzles repo. It took around 6-8 Hrs for me to complete. Please note that they didn't get back to me after I sent this code. So don't just copy paste the code and use for the same puzzle. May be there is something missing.

Tuesday, February 14, 2017

C# Overloading Action & Func and why {} matters

Basics - Overloading

Let us start with basics of overloading. Will the below code compile?
class SomeClass
{
    void Execute()
    {
 
    }
    int Execute()
    {
        return 10;
    }
}
Certainly not. We know that overloading applies to the arguments. Return type is not considered.

Now look at below code. Will it compile?
internal class SomeClass
{
    void Execute(Action handler)
    {
    }
    int Execute(Func<int> worker)
    {
        return worker() ;
    }
}

Don't spend more time. It will compile. For sure. Will it work as expected? That is tricky question. As of our understanding, we are using strong typed languages to avoid many runtime type related issues. So that we can concentrate on our logical issues more.

Consider the below code and guess what happens
internal class SomeClass
{
    internal void Test()
    {
        this.Execute(() =>  Console.WriteLine("Hi"));
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
    }
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        return worker() ;
    }
}
SomeClass cls = new SomeClass();
cls.Test();
The output will be
Inside Execute(Action)
hi

The problem

If we call Execute as shown below which function will execute?
internal class SomeClass
{
    int num = 0;
    internal void Test()
    {
        this.Execute(() =>  num++);
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
    }
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        return worker() ;
    }
}
SomeClass cls = new SomeClass();
cls.Test();
The output will be 
Inside Execute(Func)

Why did this happen? Since the C# language support lambda expressions without explicit return value which is just there it assumed that the intention is to return the incremented value. Sometimes the original intention of developer might be just to increment the num variable. Some conflict happened between the developer and the language.

This is kind of OK scenario. Nothing breaks. Lets see little more complex scenario where it cause trouble.
internal class SomeClas
{
    int value = 0;
    internal void Test()
    {
        this.Execute(() => value++ );
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
        CleanUp();
    }
 
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        int result = 0;
        Execute(() => result = worker());
        return result;
 
    }
    private void CleanUp()
    {
        Console.WriteLine("CleanUp");
    }
}
The developer wants to do some cleanup after Execute. So he added the CleanUp() inside action and he wanted to avoid CleanUp() in all the other functions. Other functions pass action to so that the CleanUp() will be performed for all the Execute() calls. 

The expected output event with call to Execute(<Func>) is
Inside Execute(Func)
InsideExecute(Action)
Hi

But what happens here is infinite loop. Some clever language geeks might have foreseen thie infinite loop. But normal developer will not.

Fix - Use curly brackets / braces / {}

People think that putting {} is time waste and they avoid it where ever possible. But here it would have saved some time. See the fix below.
internal class SomeClas
{
    int value = 0;
    internal void Test()
    {
        this.Execute(() => value++ );
    }
    void Execute(Action handler)
    {
        Console.WriteLine("Inside Execute(Action)");
        handler();
        CleanUp();
    }
 
    int Execute(Func<int> worker)
    {
        Console.WriteLine("Inside Execute(Func)");
        int result = 0;
        Execute(() => { result = worker(); });
        return result;
 
    }
    private void CleanUp()
    {
        Console.WriteLine("CleanUp");
    }
}

Just adding {} into lambda expressions helps us a lot. If Test() used {} it would have avoided one more function call and could have reached to Execute(Action) directly.

Moral of the story is don't hate braces. Omitting those is not going to save lot of time or decrease size of program.

Considering the API

But please be aware that if we are giving these Execute() methods as APIs there is no guarantee that our customers will use {} always. So we have to overload the functions properly.

Tuesday, January 31, 2017

Serverless JoymonOnline.in - Intermediate update & experience

The Serverless is the new buzzword and it is getting really good attention. It is nothing but an architectural style where developers don't need to worry about servers for deployment, scaling etc... Just write and push the code. It will scale automatically.

In the previous article about Serveless, I had mentioned I am in the process of converting my personal site to Serverless. The decision was purely financial. Why should I give money to GoDaddy for hosting some pages which can be served by GitHub pages for free?

This is just an intermediate update about that effort.

Steps

Below are the steps taken during this effort. It may vary from project to project based on complexity. JoymonOnline.in is just a profile site which don't use any custom modules or handlers.
  1. Avoid server side data rendering
    1. Replace .Net third parties with equivalent JavaScript API
    2. Convert each .aspx page to contain its own Angular application.
    3. Avoid ASP.Net specific skin files and use pure CSS
  2. Client side integration
    1. Replace ASP.Net master page with Angular routing.
    2. Bundling, Minification, CI & CD
  3. Backward compatibility
    1. Links
    2. SEO
Please note that these steps are relevant if the decision is 
  • Not to start a fresh app
  • To ensure that the code base is ready to delivery at any point in time.
If business can keep quite during the migration or the development can be done is separate branch ad later merge, these steps are not required. Just start a new app.

Avoid server side HTML rendering

The current www.JoymonOnline.in site uses ASP.Net web forms technology. In other words it uses server side HTML generation and sends it to the browser. There is no rule that Serverless should not use server side rendering. It can use, provided the server side rendering can scale automatically without doing any extra task. For example a WebAPI service end point can return HTML and it can be rendered at browser with the help of JavaScript. That introduce some kind of client side HTML manipulation. Once we take that decision, there is not much differences between rendering full contents at client side v/s rendering some contents at server and inject it to client side views. So in short Serverless goes hand in hand with client side web apps which we generally call SPA (Single Page application). It can't be done in one step. So sub tasks are as follows.

Replace .Net third party with JS API

If we want to replace the third party .Net API with JS equivalent there are 2 options
  1. Use equivalent JS library
    1. Has to make sure the critical API keys are not exposed. Can do client site authentication or be happy with free quota.
  2. Write a FaaS which uses same .Net API and let it return required data to client.
    1. We have to make sure it is secured and key present only at service and service can only be called from our app.
Coming back to JoymonOnline.in, it uses some third party APIs at server side to get data and render  via ASP.Net. GitHub API, Blogger API are the main third parties. So these calls needs to be done from client machine. For GitHub, it was easy as there is straight JS library. But for Blogger it was little difficult as the Google feed API is discontinued. Finally it worked with blogspot feed url.

But for GitHub, there is a limitation due to the quota for unauthenticated API calls. Since the developer key cannot be exposed to client, it allows only 60 requests / hour / IP address. Since JoymonOnline.in is expecting more than 60 calls from single API it is ok to go with quota. 

Convert each .aspx page to contain its own Angular application

Next step is to convert page by page to Angular. This is to make sure that the application is always in functional condition than like a demolished house before renovation. Convert individual pages to its own angular app and render that app inside the ASPX pages. Let that page be just an angular app host. Noting else. 
In this step we have to identify equivalent third party ASP.Net UI controls, if applicable. For example how to replace Telerik web forms controls with KendoUI.

In case the plan is to first get to ng1 and later to ng2, I would strongly recommend to go with components rather than directives. TypeScript is becoming defacto standard for angular. Use that to avoid debugging time. After this step, we will have ASP.Net application but the functionality working from browser via angular.

Avoid ASP.Net specific skin files and use pure CSS

Next thing to tackle are the styles. If the application is using skin files, we have to get equivalent CSS files. Sometimes we may need to do this along with the previous step. Depends on nature of third parties and our code.

Client side integration

The next big step is to integrate things at client side.

Replace ASP.Net master page with Angular routing.

The main pending thing is to have angular master page and routing. We already have separate components in different apps. Just create new app in index.html and have the routing to load proper components. This steps sound simple but it has many things to do.

Bundling, minification, CI, CD etc...

Now its the time to optimize the site and do integration to build pipeline. Till this point, the expectation is that we are able to use existing build system. Recommended is Webpack for client side activities such as compilation and bundling. CD is always confusing with Continuous Delivery and Continuous Deployment. At least make sure it is Continuous Delivery.

Current, JoymonOnline.in using AppVeyor for CI & CD. Though AppVeyor is mainly intended for Windows development, it supports NodeJS as well. So decided to continue in AppVeyor. At this point the build output will be pushed to \docs folder as staging. GitHub pages recently started supporting serving web pages from \docs folder instead of gh-pages branch.

It is not fun to develop JS based client side app using Visual Studio solution file. After this step we don't need .sln file to work with. Node rules the client side web development world. At some point we have to introduce Node. So why not from the start if it is possible to live with only Node? When it is said 'start', it means the starting of the migrated application. If anyone feels they can live with .sln file without Node, let them continue. But better don't use mix. Comparing Node with .sln is not the right comparison. But the intention is to use Node and its ecosystem such as NPM & package,json etc... 

Backward compatibility

These may be optional depends on scenarios.

Links

This will make sure that the bookmarks saved by users are still valid. The old ASP.Net application could have modified to redirect the .aspx pages to new URLs. But for JoymonOnline.in, this is not done.

SEO

The SEO efforts needs to be put if the application is public facing.

These steps itself needs explanations which will be having contents enough for separate post. At this point, the entire application has converted and available at joymon.github.io . The main pending thing is to change settings in GoDaddy to point JoymonOnline.in records to joymon.github.io.