Functional Programming - Finding valleys

If we were from ASP.Net Web Forms world and tasted the ASP.Net MVC at least once, it is very difficult to go back. It is similar when we go from imperative world to functional programming world. This post is a continuation of Functional Programming series. The advantages of FP has already discussed in earlier posts. 

The problems are mainly taken from HackerRank and trying to solve using FP methods. The main intention is to understand how to solve problems functionally which otherwise considered as 'solvable only imperatively'.

The Problem

Input is a sequence of D and U characters. D means one step down down and U means up. A person named Gary is hiking from sea level and end at sea level. We need to find out how many valleys he covered. If he is in a deep valley and had a small hill and again went down, it is counted as one valley. Detailed description can be found in original link

https://www.hackerrank.com/challenges/counting-valleys/problem

Traditional solution

The traditional (FP was there for long, but not in main stream) ie if someone comes from imperative programming world, they get it as state machine problem. Yes there are states where Gary reaches after each step. The state has to be mutated based on the rules.

In functional programming, mutation is not an appreciated word. So lets see how can we do this without mutating the state.

Functional way

Language used is JavaScript

---

function countingValleys(n, s) {
  
  var res = s.split('').reduce((context, value,index,arr) => {
    //console.log(context)

    if(value === 'D') {
      if(context.s === 0 ) { return {v:context.v+=1,s:context.s-=1}}
      else return {v:context.v, s:context.s-=1}
    }

    if(value ==='U'){
      return {v:context.v,s:context.s+=1}
    }
    else {
      //console.log('error');

    }
  }, {
    v: 0,
    s: 0
  });
  return res.v
}

---

Here n means number of steps and s is the input character sequence. If we enter this into HackerRank browser based editor, with their auto generated wiring code, this returns the number of valleys covered.

How it runs

It uses the Fold concept of FP which is implemented in JavaScript using reduce() function. Initial state is given to the reduce function, when the function progress through each element, it create new states (context variable hold the state) from existing state with the modification required. Yes as per FP, mutation is evil not creation of object from another. Also notice that -= and += is not mutating the state, instead its a transformation when assigning value during creation.

The commented console.log lines will help to under the flow it allowed those to run.

Happy functional coding...

ASP.Net bundling & minification via CDN Caching

It is normal for ASP.Net applications to use bundles to reduce http calls. It all works till the time CDN comes into the landscape. CDN (Content Delivery Network) uses a special approach to deliver contents from world wide distributed servers which we can envision as served from the edge of the network. That way so many requests get served from CDN server without coming to our web server.  It is nice theory about CDN. What is between ASP.Net bundling and CDN?

CDN normally cache static assets. Out of the box, they cache well known static assets such as html, css, js files. Yeah, image file formats too, if we configure. If we had bundled static assets using ASP.Net bundling mechanism which gets a different URL than just js files, those will not be seen by CDN as static assets. Hence no caching.

This is applicable if CDN is placed in between. Akamai provides such a service where we don't need to change our code when we introduce CDN. CDN servers cache when they get content from the backend web server. Normally when we introduce CDN, the domain has to change to serve from CDN. That has added benefit too since it increase the parallel request limit to same origin. If we want to integrate CDN via separate domain, better forget about ASP.Net bundling.

Here lets see how to make pass through CDNs work with ASP.Net bundling since they don't introduce new CDN URL.

How can we make CDN friend with ASP.Net bundles

Approach 1 - Don't use bundling

Sounds easy but if the ASP.Net application is designed as product which is hosted by individual customer, every consumer may not have CDN capability. So need bundling. For hosted applications ie there is only one instance of application in the world, yes we can try this. But still without http2, it would cause performance degradation as previous http versions make different connections for each file / resource request. If the application has angular or any other SPA frameworks used, the number of files may be large.

Approach 2 - Have CDN cache the ASP.Net bundle URL

When we bundle, ASP.Net provides a new URL instead of all the files inside the bundle. From browser we will see only one bundle URL request instead of individual resources in the bundle. That URL has a predefined format. If the CDN supports custom RegEx kind of way to specify the URL formats to cache, it works without much efforts.

The RegEx for recognizing ASP.Net bundle URL format goes as follows.

---

/\?v=[a-zA-Z0-9_-]{44}$/

---

This works only if there are no other API or dynamic resource URL ending with ?v={44 letter value}

Approach 3 - Embrace SPA with JS modules & forget server side rendering 

Instead of server side rendering done by ASP.Net, embrace the industry trend SPA (Singe Page Application) with JavaScript module system such as Webpack. Webpack can produce static bundles which can be referred in application as a normal file.

Recommended

If there is enough budget and time, embrace SPA else use the RegEx and stay with ASP.Net bundling and CDN.

---

I don't have any association with Akamai except using in projects. This post is not intended to promote any of their products.

Circular relationship diagram for Microservices and Serverless architectures

Microservices is the trend for splitting large systems to individual self contained subsystems. With the advancement of Serverless offerings more and more systems are created or moving towards Microservice approach.

Is Microservice same as Serverless?

Before we go further let us clarify the difference between Microservice and Serverless. There could be some readers who thinks both are same and Serverless is more advanced. To be frank, there are fundamental differences between these two.

Microservice - It is about splitting the systems into small sub systems which are deliverable by them selves. Microservices can be hosted either in Cloud or in onpremise data centers. Microservice has to manage its own data and that data should never be touched by other systems directly.
If the system consists of couple of services and both are maintained by same team, those services doesn't necessarily be separate.

Serverless - It is about freeing the developers from worrying about underlying infrastructure. They just have to write the code and give to the hosting environment. The environment takes care from there about scaling, fault tolerance, patching etc... to maintain a promised parameters such as  predictable throughput, OS level security etc...

Problem

The problem with Microservice is the integration of Microservices. If it is single big monolith system the integration is handled by the class relationship. When we move to Microservice the complexity is moved to the service integration level from the class level. The overall complexity of the system is not changing. It would be easy when we start Microservice based system from scratch or convert existing. But it gets difficult when the count of Microservices goes beyond 50 or 100. Especially to understand their relation to each other. Who calls who and what protocol ie ie dependency. 

The technical solution for this problem is to implement Service mesh or service discovery mechanism so that we have the connection details at central place.

But how do we create a diagram to represent those relationships. If we use traditional blocks and arrows model, it will take more time to understand than the time for coding the same. So lets take another approach.

Circular Relationship Diagram

It doesn't seems that there is a standard name for a diagram which puts the entities in the edge of a circle and shows connections by drawing lines between those through the circle. As programmers lets try to generate one by code than buying a software.

D3.JS for Circular Relationship Diagram

Most of us would be familiar with the D3.JS library for charting in the browser. That library have the support for creating circular relationship diagram. Below goes one link.

http://mbostock.github.io/d3/talk/20111116/bundle.html

The above is not intended for plotting the relationship between the Microservices. But as programmers we can easily change its data to provide our Microservice relationship. The above link doesn't need any server side coding. All it needs is web server to serve HTML pages. It can be IIS, Apache or anything.

Thanks to the D3 library and author of the sample.

Other references

https://github.com/nicgirault/circosJS
https://modeling-languages.com/javascript-drawing-libraries-diagrams/

Caution JavaScript Ahead - setTimeout() or setInterval() doen't make JavaScript multi threaded

If someone born into JavaScript, they know what is meant by JavaScript is single threaded. Developers coming from other languages also knows that JavaScript is single threaded. But some APIs in JavaScript force them to think that JavaScript is multi threaded. Below is such a situation

---

setTimeout(() => {
  console.log('Timeout hit and time is ' + new Date());
}, 1000);
console.log('setTimeout at ' + new Date());

---

There could be N number of reasons for someone wants to execute his code after specified time. It might be initialization delay, rendering time, session timeout handling etc... But is this going to solve the problem by executing the code exactly after 1 second (1000ms)?

If we consider any application there will be more code than these 2 lines. Consider someone else written below code which got executed after we register the handler via setTimeout()

---

setTimeout(() => {
  console.log('Timeout hit and time is ' + new Date());
}, 1000);
console.log('setTimeout at ' + new Date());
problemMaker()

function problemMaker() {
  //Any sync AJAX call or some code which execute for long time.
  var url = 'https://httpbin.org/delay/5';
  var request = new XMLHttpRequest();
  request.open('GET', url, false);
  request.send();
  document.writeln(request.responseText.length); 

}

---

Does this ensure that the function gets executed after 1 second? Native JavaScript developers can immediately identify the issue. Other may think it might work. Lets see a test run result in console.

---

setTimeout at Tue Oct 24 2017 19:46:14 GMT-0400 (Eastern Daylight Time)
Timeout hit and time is Tue Oct 24 2017 19:46:29 GMT-0400 (Eastern Daylight Time)

---

Yes JavaScript is single threaded whatever we do with setTimeout or setInterval functions. Better do no trust them on when they are going to execute. If we write like this it may work on development machine and may fail in higher environments such as business testing, staging or production. Highly inconsistent issue. Lets avoid saying "It works in my machine".

Sample code located at https://plnkr.co/edit/uexi2U

Caution JavaScript Ahead - Hoisting

What is variable hoisting

In the 'var' world of JavaScript, the variable is not bound to the boundary of block level curly braces {}. Even if we declare the variable inside an if loop, it gets moved to the function where if belongs. This is not the case in other languages such as C,C++, Java, C# etc...There the variable is limited to the boundary of block level braces but never in the case of JavaScript. That seems a developer friendly feature in JavaScript at first. But it can cause issues, if we write code without knowing what is variable hoisting.

Below is a good example

---

foo();

function foo() {
  if (new Date().getMinutes() % 1 === 0) {
    var time = new Date();
  }
  alert(time);
}

---

If we just look at the code we may conclude that the alert will always shows undefined. But if we know hoisting, we can easily identify the difference here. If the condition is true, the time variable is going to have value.

Even if the statements inside the condition didn't execute, we don't get any runtime exception in strict mode. It just shows undefined. Lets see below code.

---

foo();

function foo() {
  function innerFoo() {
    if (new Date().getMinutes() % 1 === 0) {
      var time = new Date();
    }
  }
  innerFoo();
  alert(time);
}

---

Here always we get exception if the code is in strict mode. This is because the time variable never gets hoisted to the outer function.

Variable hoisting and global variables

Below is another interesting scenario if global variable or variable in parent scope has same name as inner variable

---

time = new Date();
alert(time);
foo();

function foo() {
  alert(time);
  var time = new Date();
  alert(time);
}

---

When we run the above snippet. We could see that first alert shows the time. Second one shows undefined and third one shows time again. Why the second one shows undefined? Lets see how the variable hoisting worked here. Below code snippet shows the code after hoisting.

---

time = new Date();
alert(time);
foo();

function foo() {
  var time;
  alert(time);
  time = new Date();
  alert(time);
}

---

Of course we cannot see the hoisted code since JavaScript don't compile to output file. But this is what happens.

Does this mean global variable is not safe

The above may cause confusion that global variable is altered by the variable inside function. But it is not. Lets see the below code snippet.

---

time = new Date(1947,7,15);
alert(time);
foo();

function foo() {
  alert(time);
  var time = new Date();
  alert(time);
}

alert(time);

---

When we run we could see that the 4th alert is showing same date of first alert. That means the global variable is not altered by the function since it is different variable with same name. This is similar to the other languages. But this is true only to the variables out side of function. What if the outer variable is inside the function and inner is inside an if statement?

---

foo();

function foo() {

  var time = new Date(1947, 7, 15);

  alert(time);

  if (time.getYear() < 2000) {

    var time = new Date();

    alert(time);
}

}

  alert(time);

---

We could see that the 3rd alert shows current date not the old one initialized at the beginning of the function. This shows the hoisting is to the immediate function level and it is same variable through out the function.

Hoisting functions

In the above code snippets we can see that the foo() is called before it is defined. JavaScript has no problem with that. But lets see the below scenario. Guess what will happen

---

foo();

var foo =function () {

  var time = new Date();

  alert(time);

}

---

It complain that foo is not a function. Many of us might have spent good amount of time troubleshooting why this is not a function at first. The reason is hosting. When it does, it moves the variable foo declaration to the top of the function ie above the foo(); statement and when we call foo(); it is not yet a function as the function definition is assigned to that variable later.

If we move the calling foo(); below the assignment it starts working.

Enter let for block level scoping

The keyword 'let' is introduced into JavaScript ES6 / ES2015 to have block scope. ie the variable can be scoped to an if block than to function level. Lets have a look at below snippet.

---

foo();

function foo() {

  var time = new Date(1947, 7, 15);

  alert(time);

  if (time.getYear() < 2000) {

    let time = new Date();

    alert(time);
}

}

  alert(time);

---

We could see the above code shows old date. The only difference is the usage of let for declaring variable inside if block.

Does let hoist

If we use a variable before it is declared with let, it throws exception. But if we use a variable before it is declared using var it has undefined. Means hoisting is not happening as is in let.

---

foo();

function foo() {

  try {

    alert(oldTime);

    var oldTime = new Date(1947, 7, 15);

  } catch (ex) {    alert(ex)  }

  try {

    alert(time);

    let time = new Date();

  } catch (ex) {    alert(ex)  }

  try {

    alert(noTime);

  } catch (ex) {    alert(ex)  }

}

---

We could see in the above code, the oldTime gets hoisted with undefined value. The let throws exception similar to an real non declared variable noTime.

There are more in to 'let' such as temporal dead zone and restriction of redefinition etc... The new constructs such as class in JavaScript works the same way of let. Classes in JavaScript don't get hoisted.

If we think about different scenarios, things gets very much interesting. Eg: What if we declare same variable using var and let in same scope, different scopes, how they overlap etc...See the reference section for more such interesting scenarios.

References

https://developer.mozilla.org/en-US/docs/Archive/Web/Scope_Cheatsheet

https://developer.mozilla.org/en-US/docs/Glossary/Hoisting

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/let

https://www.infragistics.com/community/blogs/infragistics/archive/2017/07/18/Easy-JavaScript-Part-1_3A00_-Learn-the-_1C20_let_1D20_-Statement.aspx

Functional Programming - Find hidden string in the input

This is continuation of my functional programming experiments in JavaScript. This problem is from HackerRank.

The problem

Input is a string and we have to find out whether the letters of word 'hackerrank' are present inside it in the right order. When we say order, there might be some other letters in between the letters of 'hackerrank' the the order should be maintained. More details can be found in the actual problem link. 

Traditional solution

If the guy is not from the computer science background, there is 99% chance that the solution will be based on loops where a lot of states might have stored temporarily. But it looks like a familiar situation of email validation where we are expecting chaaracters in specific order.

The RegExp way

Regular expressions are great. There is no doubt on that. Here goes the regular expression way.

---

function checkForhackerrank(input) {
  if(testForhackerrank(input)) {
    return 'YES';
  }
  else return 'NO';
}
function testForhackerrank(input){
  return /h(.)*a(.)*c(.)*k(.)*e(.)*r(.)*r(.)*a(.)*n(.)*k/g.test(input)
}

---

Works great for the word hackerrank. Lets generalize

Generalized version

Below goes the generalized version which can work for any pattern other than 'hackerrank'.

---

function testForhackerrank(input){
  return getFindPattern('hackerrank').test(input);
}
function getFindPattern(stringToSearch) {
  return getGlobalRegExp(stringToSearch.split('').map(a=> `(.)*${a}`).join(''));
}
function getGlobalRegExp(pattern){
  return new RegExp(pattern,'g');
}

---

This doesn't need anything special regarding functional paradigm as the magic is done by regular expression. Plnkr is here.

Enjoy coding.

Functional Programming - Finding unique characters in string

These are small code snippets created as part of learning functional JavaScript. As we know when we say functional programming, we have to create pure functions, avoiding assignment etc...It really require practice to write the functional way especially, if we started in imperative languages. This is first in the series where some algorithms are written using functional style.

1. Functional JavaScript - Finding unique characters in string.

The problem

The problem is simple. Have to find out unique chars in string. eg: Hello has to return Helo. 

Functional ways

The easy way is to loop through the characters and push items into another array, if it is not already present. Finally create string by joining array.

Below is slightly different version where the characters are filtered, if the index found in original string is different than current index.

function uniq(str) {
    return str.split('').filter(function(item, index, self) {
      return self.indexOf(item) == index;
    }).join('');
}

Runnable plnkr below. 

JavaScript Access inner objects with string expression

JavaScript is becoming more and more importance for developers. Even it can be used as DSL with the help of embedded V8 runtime engine. Below is one scenario which may occur when using JavaScript as DSL

The DSL is another program fragment, which our program has to execute.
Suppose there is an JavaScript object structure as follows

---

var complexObject = {
 prop0:"prop0 value",
  prop1: {
    prop1prop1: {
     prop1prop2prop3: "prop1prop2prop3 value"
    },
    prop1prop2:"prop1prop2 Value"
  },
  prop2:[
   {prop2prop1:"prop2[prop2prop1] value"},
   {prop2prop2:"prop2[prop2prop2] value"}
  ]
};

---

Lets see how this can be accessed.

Normal JavaScript property access

Now lets see how can we access the values using normal property dot(.) notation

---

var testInnerObjectProperty = function() {
  alert(complexObject.prop0);
  alert(complexObject.prop1.prop1prop2);
  alert(complexObject.prop2[0].prop2prop1);
};

This works perfect.

Access JavaScript property using string property path

But in DSL the path to the value will be available as string. So how to access the value using string property path?

---

var testInnerObjectPropertyWithIndexer  = function(){
  alert(complexObject["prop0"]);
  alert(complexObject["prop1.prop1prop2"]);
  alert(complexObject["prop2[0].prop2prop1"]);
}

We can see the first alert returns propert value as its direct string path access. But the next 2 will not work as JavaScript runtime is not able to find a property names "prop1.prop1prop2" on complexObject. In other words it doesn't have the intelligence to understand the hierarchy. May be its by design.
Lets see another way to access using string string.

Access property using string path via Evil Eval()

This is the easiest method pops up into a medium JavaScript programmers mind though he know its not the good way.

---

var testInnerObjectPropertyWithIndexerEval = function() {
 alert(eval('complexObject.'+'prop0'));
  alert(eval('complexObject.'+'prop1.prop1prop2'));
  alert(eval('complexObject.'+'prop2[0].prop2prop1'));
}

---

Since it works but not the good way what is next.

Access property using string path via parsing

This is the hard way. We must test is property for all combinations. Lets see one implementation.

---

var testInnerObjectPropertyByParsingPath = function() {
 alert('Going to parse prop path');
 alert(Object.getValue(complexObject,'prop0'));
  debugger;
  alert(Object.getValue(complexObject,'prop1.prop1prop2'));
  alert(Object.getValue(complexObject,'prop2[0].prop2prop1'));
}

---

Above is the API interface expected. Below goes the implementation of getValue()

---

Object.getValue = function(obj, path) {
    path = path.replace(/\[(\w+)\]/g, '.$1'); // convert [] access to property access with number
    path = path.replace(/^\./, '');           // trim leading dot
    var props = path.split('.');
    
    for (var propIndex = 0, noOfProps = props.length; propIndex < noOfProps; ++propIndex) {
        var propName = props[propIndex];
        if (propName in obj) {
            obj = obj[propName];
        } else {
            return;
        }
    }
    return obj;
}

---

This can be written using recursion too. It replace the array access [] with property access by replacing corresponding number there. Then loops into the object hierarchy till it finds required object.

Please note this is not the battle tested code. This is rather to give the idea.
https://jsfiddle.net/joygeorgek/s5egwa5o/8/

My JavaScript Module experiments - CSS

This post is continuation of my JavaScript module experiments series with Webpack. Earlier posts below.

1 My JavaScript Module experiments
2 My JavaScript Module experiments - CI & CD
3 My JavaScript Module experiments - CSS

This post assumes the basic knowledge about NodeJS, NPM, Webpack such as its config file and require mechanism.

Should we bundle css?

It really depends. 

Traditional CSS development

If we take the scenario of simple web site development, we may not need bundling provided entire CSS is in single file. But that is not at all the scenario right now. We need the sites responsive and for that we may have to add dedicated styles for different screen resolutions. One way is to write everything in the same css or separated files loaded on demand. But keeping in different files and downloading all to client and deciding which to use will slow down the performance. Because it needs more http file requests. Keeping in separate files is easy in terms of maintaining. Normally the decision to what style needs to be applied is taken at client side based on the screen size. It will be overhead to request the css for different sizes when the user resize the browser window. So it is better to bundle the css for different sizes at server and download as one file. 

Modern CSS

Modern CSS means writing CSS with the help higher level languages such as LESS or SASS. They help us to write less reusable code and get things done. Since the poor browser cannot understand this LESS or SASS languages, we have to convert to CSS. That process is called transpilation. Here there are chances that some styles will be in pure CSS, some in LESS and some in SASS. As we have to convert to CSS and refer into the application, it is better to bundle the outputs to one css file and refer same.

HTML component

Another scenario is development of web components or on-demand view development. In those scenarios we don't have to keep entire CSS in single place and refer from the beginning. We have to bundle the css along with the JavaScript component which only loaded when needed.

Webpack and CSS

Now lets see how the Webpack and CSS works together. 3 different scenarios are listed below to understand at the simplest level. To keep it simple no other things like ECMA 6 or TypeScript is used.

Simple scenario - webpack and css

Here we are simply going to require .css files from JavaScript to get it available in the html. Below are the steps

• Import the npm packages named css-loader & style-loader. These are the Webpack specific NPM packages to process .css . The first one is to recognize the .css file and other is to load the .css as <style> tag.
• Modify the webpack.config.js file to have loader. For syntax refer to the sample given below
• Add require statements in to the JavaScript file
• require('./stylesheet.css');

Below is the link to sample.

https://github.com/joymon/webpack-starters/tree/master/ECMA5-CSS

To run the sample. 

• Make sure there is NodeJS and NPM
• Clone the repo to the local machine
• Open NodeJS command prompt and navigate to the \ECMA-CSS folder
• run 'npm install' command to download dependencies.
• run 'webpack' command.
• If the webpack is not available at system level, '.\node_modules\.bin\webpack' is required
• This will produce bundle.js file
• Just run the index.html file from file system. Make sure there are permissions to run JavaScript from local file

When running we can see that there are <style> tags added to the <head> which are not present in index.html file. Those are added by the style loader at runtime.

Combined css

Injecting <style> at runtime is not a good solution. Better is to have a final combined css file. The below sample shows that mechanism using one more npm package called 'extract-text-webpack-plugin'. Since the sample uses webpack 1, the version of 'extract-text-webpack-plugin' should be 1.0.1.

https://github.com/joymon/webpack-starters/tree/master/ECMA5-CSS-Combined

Here the index.html is referring to a style file called bundle.css which is not there in repository. When this sample is run using the same steps mentioned for above previous scenario, the bundle.css will be created by Webpack. Once that file is present, we can run the index.html as usual.

The bundling of multiple css files to one is done by using ExtractTextPlugin configured in webpack.config.js file and here we don't need the style-loader.

CSS & LESS together

This is the next level where we mix plain .css with advanced .less file and finally produce combined .css file. Sample is available in the below location.

https://github.com/joymon/webpack-starters/tree/master/ECMA5-CSS-LESS

In this sample, instead of earlier childstylesheet.css, childstylesheet.LESS is used. The npm packages used are less, and less-loader for compiling .less files and to load the same respectively. The webpack.config.js also modified to tell how to handle the .less files and to combine using ExtractTextPlugin.

During the compilation the .LESS file will be compiled to .css and then combined with other .css file to produce bundle.css file.

Further reading

https://webpack.github.io/docs/stylesheets.html
http://jamesknelson.com/writing-happy-stylesheets-with-webpack/
https://css-tricks.com/css-modules-part-2-getting-started/
http://stackoverflow.com/questions/39015521/import-css-and-js-files-using-webpack
https://github.com/webpack/file-loader/issues/32

My JavaScript Module experiments - CI & CD

This is second in the series "My JavaScript Module experiments". This post talks about setting up a simple CI & CD pipeline in modularied JavaScript environment using AppVeyor as CI & CD runner.

Other posts in this series below.

1 My JavaScript Module experiments
2 My JavaScript Module experiments - CI & CD

Introduction

This post directly jumps to setting up CI & CD for modularized JavaScript. Here also Webpack is used as module loader as well as to do other operations such as TypeScript transpilation, minification etc...

AppVeyor & GitHub Pages

AppVeyor & GitHub pages are 2 external services used here. AppVeyor provides free CI & CD service for open source projects and GitHub Pages provides free hosting from Github repositories. Basic knowledge about these services are considered as prerequisites.

AppVeyor is mainly targeted to Microsoft .Net stack and it automatically takes care, if there is a Visual Studio solution file. But we can make it work for Node as well by disabling the default build process. That can be done by simple changes inside appveyor.yml file. AppVeyor supports configuring CI & CD activities either in a appveyor.yml file found in the GitHub repo or through their UI. Here we are using the appveyor.yml  file as its easy to understand. Though is a file publicly accessible file in repo, we can include sensitive information inside it with encryption.

Separation of Concerns - What does what

When we look at the technologies used, we can see those can so same tasks. AppVeyor can run normal commands as well as PowerShell and even start NodeJS which in turn starts Webpack. Using normal commands, PowerShell & Node we can do file copy operations. Webpack which can be started as command can also execute code which does file copy. What is the problem if all the technologies can do same task? The problem is lack of clarity on which component does what.

For small projects, it is fine. We can have AppVeyor scripts and Webpack do file copy. But when we think of larger projects we need to have clear distinction on that should do what. Else it would be difficult to maintain in future.

The simple rule can be to assign the dev related tasks such as compilation and bundling  to Webpack and deployment tasks to the AppVeyor scripts such as pushing to GitHub. Ultimately AppVeyor is ruling the environment. So there should be one AppVeyor command which starts Webpack. Summarized duties as follows.

• AppVeyor
• Downloads the source code
• Install proper Node version and setup environment by installing packages.
• This include the Webpack NPM package too.
• Starts Webpack
• Run tests.
• Collect the artifact and deploy  
• Webpack
• Compilation
• Do minification / Uglify
• Create bundle
• Emit the output to \dist folder. (\dist is just a convention. It can be any folder)

The CI & CD workflow

The CI & CD workflow can be simple. After the transpilation &  bundling the output HTML app which includes html, js, images etc...needs to be pushed to the \docs folder of repository. The GitHub repository can be configured to serve web pages from \docs folder.

Sample

It is easy to explain the but there are high possibility for missing steps. So to make it more understandable, please refer sample repo located at below location

https://github.com/joymon/webpack-starters

• It does CI & CD on 'TypeScript-Angular1-Adv folder'.Other folders are showing different Webpack samples.
• Push the output to the \docs folder in same repo. \Docs folder is configured to GitHub pages to serve from.
• It is little advanced Angular 1 sample done using TypeScript.
• Node & NPM used as dev technology.
• AppVeyor.yml has all the CI & CD steps. The GitHub token is encrypted and kept inside it.
• Since the aim of that folder is to minimally show the webpack features, at this point there are no tests included.

My JavaScript Module experiments

Background

After I wrote my post about the selecting Angular 1 for my personal site when Angular 2 is out, I seriously started thinking about myself. Am I too joining the group who hate changes? As mentioned in that post, the main bottleneck is not the concept or syntax of ng2 nor it's performance and stability. The problem was around tools. What are those tools doing? They help us to write modularized JavaScript. They do transpilation, bundling, minification etc...

If we download standard ng2 project starter template, we will get pretty much a good running starter. The tools will be already configured. From there we can simply add our own functionalities. Unless we have crazy requirements, that configuration will work. We just need to add features. Then what is the trouble? The trouble, at least I had was that I am not getting how those tools are doing the magic. I checked with some other people who has problems with ng2. I could see they also have trouble understanding the tool chain. Its very difficult to move ahead in that magic setup especially, if we are coming from little old school who want to know exactly how the software works. For most of the newbies, it doesn't matter. As far as the setup/tool chain is working they can code new features.

Is there a way to survive in web development without knowing bundling? The answer is 'no' as that is the way forward to build huge apps.

So what is the solution. Learn the JavaScript module system from basics with simple samples than  starters which are configured to use features such as transpilation, source map, minification, unit testing, dev & prod  modes etc...

This post explains how I learned module system in my way, though there are tons of tutorials explaining the JavaScript module system. Most of the steps will be pointing to external sites to avoid duplicate content and of course they explain better than me.

What is JavaScript module system

If we have background in Java, it is the equivalent of Java package system. For .Net people, it can be compared to assemblies. Simple as that. But since the JavaScript engines don't have native support for modules, we have to rely on different tools which does the module implementation. To me its like workaround till the browsers or the JS engines know what is module. May be at least in the case of module loading. I am not expecting any JS execution environment will ever help us to create a bundle.

What is the relation to Angular 2 and these module system? Angular 2 uses these module concept and it is good to follow the same for our app as well. So if we download any Angular 2 sample or starter, we can see it uses SystemJS or Webpack. As mentioned earlier, either we can just download a starter and extend it without knowing how the magic works or learn what is the module system from the scratch.

Selecting WebPack

If we google for JavaScript module tools we will see many. Browserify, RequireJS, SystemJS, webpack to name few. We can debate which one is better for days and that itself require one separate post. So for the time being selecting webpack as it is selected by Angular-cli and at least it is not in beta.

NodeJS n NPM are everywhere

It is very difficult now a days to do web development without knowing node and its package management tool NPM. Though Node & NPM doesn't have any role inside browser they are used in tooling side. Webpack also comes as NPM package. So this post assumes reader has knowledge about NodeJS & NPM 

Step 1 Understand via simple ECMA 5

The first thing as JS developer unless we are born to ES6 is to learn how we can modularize ECMA5 or ES5 or simply the traditional JavaScript code. There is no need to write one more tutorial to understand how ES5 can leverage webpack. Just go to 'getting started' tutorial of webpack and try it.

Please try till Section "THE FIRST  LOADER" in the below 'getting started' link.

http://webpack.github.io/docs/tutorials/getting-started/

Before using loader, it is good to know something else. Obviously we should come back to loaders. Because loaders does magic. 

After trying the above, remove the .js extension from require and run. Sample below.

var content = require("./content.js");

to 

var content = require("./content");

We can see still it is working. The file extension is something optional.

Step 2 - webpack.config.js

If we go through any webpack tutorial, we can understand that most of the issues using webpack is because of wrong webpack.config.js. So it is important.

Minimum config

Lets see what is the minimum config required for this webpack.config.js file. Have a look at the below link starting from "Defining a config file" section where it is explained clearly. Read till the "Webpack loaders and preloaders".

https://medium.com/@dabit3/beginner-s-guide-to-webpack-b1f1a3638460

Watch mode

This is a useful mode in development. This is explained in the above tutorial.

Dev Server & hot reload

These are again development time features where we can start a light web server which monitors the development changes and reloads for us. This save good amount of time in switching to command window and typing commands. Yet to see what will happen on performance, if there are 100s of files.

The feature is explained in the above tutorial.

Step 3 - Enter TypeScript

A big no to ES6 at this point only because it lacks compile time type system. This is again debatable. If we are dealing with small applications, it is better to be dynamic typed ie pure JavaScript. But for bigger apps, type checking is must. Otherwise it will take a good amount of time  ie time needed for finding out what are the members of objects and what a function accepts and returns.

In this section we are going to see how TypeScript and webpack works together.

The below article below explains TypeScript and how to use TypeScrpt with Webpack. Read from beginning till "Watch Support"
http://www.jbrantly.com/typescript-and-webpack/

Why till "Watch Support". Because it then includes jQuery. Our focus is to understand how webpack can be used to process TypeScript not with jQuery.

Now its time to clone my webpack repo given below. This will be used going forward in this post.
https://github.com/joymon/webpack-starters

It has a folder named ECMA5-TypeScript. Point command line to that folder and run webpack command. We could see it transpiles the logger.ts & tscomponent.ts and output bundle.js along with other ES5 modules. Running the index.html from disk will show some log statements.

"TypeScript Worked !!! - logger module loaded" - Means a ES5 module was able to load TS module.
"TS component which has dependency on JS component worked" - Hope it is self explanatory

Lets see how it worked

Loaders in webpack - ts-loader

Loaders help to process the file, if those are in different formats. For example, today browsers don't know how to read and execute the TypeScript. So the TypeScript needs to be transpiled to JavaScript. That can be done by using the webpack loader mechanism. In this "webpack-starters" repo, ts-loader is used for that purpose. The configuration is present in the webpack.config.js.

After loader does its job, the final bundle.js is created as usual. index.html only knows bundle.js.

TypeScript loading ES5 modules

If we need ES5 code to load a TS module it is easy. Since ES5 don't care about types, we can write any valid js and at runtime only it checks the objects. But if we want to use ES5 module in TS, the first thing is to tell, what is require() function. TypeScript as language don't know what is require(). So to tell TypeScript about require() we have to use nodejs typing. Node typing has the definition for require(). 

To add Node typing the sample is using a package called  @types/node. Refer package.json. 

Please note since this post is not aiming at teaching TypeScript language and what are typing files,  more details on @types/node are omitted.

Step 4 - TypeScript with AngularJS 1.x

Here we are going to next step of using AngularJS 1.x via webpack module mechanism. The sample is present in the TypeScript-Angular1 folder of the "webpack-starters" repo mentioned above. Sample just shows a simple Angular 1 application with one component.

Setting up ng1+TS+webpack from scratch involves multiple steps which are given below

1. Initial setup of TypeScript+webpack
2. Include the NPM package for Angular
3. Include the @types/angular.
4. A module where angular.module() is called to setup angular. In the sample refer to ngModule.ts
5. An entry point(bootstrap.ts) module where we can call the above module and add other Angular constructs such as components, services, filters etc...Big sorry for directives for easy conversion to ng2.
6. Index.html to use required Angular directives.Minimum ng-app.
7. While adding components export them

Now have a look at the TypeScript-Angular1 folder of the "webpack-starters" if not done already. Below are some questions which might have popped up. The answers are also given along with questions,

• Why ngmodule.ts? 
• This is just a class for wrapping the original angular object. This gives us a common place for registering the angular object such as registering components, filters, services etc...This is optional. Just make sure the code which is executing angular.module and other registration functions are getting called.
• Why bootstrap.ts
• This acts as entry point for the application. It imports all the other modules which contains the app is ngmodule and all the angular constructs. For simplicity the sample only includes a simple component.
• How did the require("angular") worked? Who exported the angular library for webpack?
• This is done by the node_modules\angular\index.js .It exports as angular. If we change this we can see the module is not found.
• Why webpack-dev-server is needed. Can't we double click and run from the disk?
• For just ng-app to work, there is no need of web server. But for angular to locate and load the html template, it requires app to be served via url not just from disk. We can even use IIS to serve the folder. Only requirement is to navigate via http protocol.

Any new questions can be asked in the comments which will be included into the post later with answers?

More

This post shows the tip of iceberg. There are many more things which are required to setup a professional application for development and production. With this basic understanding of module system, learning those will be easy.

Some links are given below which are related to JavaScript modules and bundling

https://medium.freecodecamp.com/javascript-modules-part-2-module-bundling-5020383cf306#.nz4zy82x9

TypeScript : Writing first typedefinition file for our JavaScript library

TypeScript doesn't have its own runtime. But it makes the JavaScript coding much easier and maintainable. TypeScript and JavaScript goes hand in hand as every valid JavaScript is valid TypeScript. But what if we already have a JavaScript library and we don't have budget to convert that to TypeScript. Can TypeScript call JavaScript functions inside the library?

A JavaScript Library

Wiki says "A JavaScript library is a library of pre-written JavaScript which allows for easier development of JavaScript-based applications..". Mainly JavaScript libraries expose their functionalities via objects or functions. Below is a JavaScript library to do some math functions.

//Code math opeations

var MathLib = {
    Add: function (n1, n2) { return n1 + n2; },
    Subtract: function (n1, n2) { return n1 - n2; }
}

This can be easily invoked from anywhere in JavaScript as its a global variable.

function btnclick() {
    alert(MathLib.Add(2,3));
}

Entering the TypeScript world

Now lets see how the above library can be used from TypeScript. If we simply copy paste the JavaScript function to call the library, it won't compile. That is because TypeScript don't know what is the type of MathLib object and don't know whether there is a function called Add() is there in that object.

So what to do? If we want to call any JavaScript library we need to make sure its available in TypeScript. That can be done by writing type definitions for JavaScript library.

declare var MathLib: IMathLib;
interface IMathLib{
    Add(n1: number, n2: number);
    Subtract(n1: number, n2: number);
}

Include the above declarations into TypeScript file then we can see it is compiling and producing same JavaScript code.

What magic happened here? We told TypeScript compiler that there is an variable called MathLib and its type is IMathLib. That type has 2 methods and the signature of those methods. We can tell that the arguments are 2 numbers. Now TypeScript passes the compilation assuming that during runtime there would be an valid object in the variable MathLib with 2 functions with proper names.

The interface name can be anything but the function names should be exactly same. the parameter type can be any valid type in TypeScript including the 'any' type.

Virtual TypeSystem of TypeScript

The type system in TypeScript exists only during compilation. During runtime nobody is enforcing types. This is because the runtime is JavaScript and that is not a hard typed system.

What is <filename>.d.ts

d.ts extension is a convention in TypeScript used to group all the TypeDefinitions. So that our application code will never be confused with TypeDefinition files.

Just move the declare var... line to a file named defs.d.ts and include to the project. It will compile. If there are any compilation error make sure your .ts file which consumes the library has a referene to the type definition. Sample code below

/// <reference path="defs.d.ts" />
function btnclick() {
    alert(MathLib.Add(2, 3));
}

Writing TypeDefinitions for third party JavaScript library

The above clearly shows in the TypeScript world the consumer of the library can simply create the type definitions for any JavaScript library. When we want to write type definitions for large libraries better make sure we are creating proper modules so that there is a clear separation. Also use exports properly.  

The type definitions for all most all famous libraries are already available in internet. One of the repository for type definitions is DefinitelyTyped

References

http://peter.grman.at/how-to-write-typescript-definition-files/
http://www.codeproject.com/Articles/528295/ModularizationplusinplusTypeScript

Why can't we use click() as button's event handler function name in JavaScript?

As JavaScript developers, many of us might have encountered situation where we defined a function for handling events such as Button's onclick and the function never gets executed. After long time of googling, when we change the name of function, it just starts working and we move to next step without digging why it didn't work for that function name. This post is a small journey towards the answer of why we can't have click() to handle button.onclick.

Lets start with code to see what is this issue. Below code is a simple HTML file which has 2 buttons with ids btn1 and btn2 respectively. Both the buttons have onclick event handled.

---

<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
    <meta charset="utf-8" />
    <script type="text/javascript">
        function click() {
            alert("click");
        }

        function click_handler() {
            alert(this);
            alert("click_handler");
        }
    </script>
</head>
<body>
    <button id="btn1" onclick="click()">btn1 uses click()</button>
    <button id="btn2" onclick="click_handler()">btn2 uses click_handler()</button>
</body>
</html>

---

For btn1 click function is used as onclick hander and for btn2 the handler function is click_handler. If we save these contents as html file and run, we could see that the btn2 click is handled properly by calling the click_handler(). But btn1's click() is never invoked. What is happening here?

Quick google may tell us that there is already a click() method defined globally or at document level to call the event handlers and that is the reason why our click() is not working.

But if we use F12 developer tools and check click function on window and document we could see its undefined.

Note:We have to check for system defined click function by removing our click function. Else it will see our click function.

Before going to next steps lets see what will happen if we introduce below line to the above HTML page just above closing of body tag ie above </body>

---

    <button id="btn3" onclick="window.click()">btn3 uses click()</button>

---

We can see the btn3 click is invoking the event handler function click(). So there is nothing special about the function name click(). Hence we can conclude that the issue of conflicting names is somewhere else.

Is there any click function on the button element? 

Now lets check if there is any click function defined already in the button element? For that put a break point anywhere in the click_handler() and run with F12 developer tools open.

Click on the btn2 and type the below line in the F12 console window.

---

   document.getElementById('btn1').click 

---

We can see the below output which confirms there is a system defined click function on the button object.

---

function click() { [native code] }

---

This confirms the reason for name conflict. Below code lists all the functions defined on button element.

for(var p in document.getElementById('btn1'))
    if(typeof document.getElementById('btn1')[p] === "function") 
      console.log(p);

Happy Scripting...

Caution JavaScript Ahead - Difference between >> & >>>

Introduction

As we know inside the computer everything is kept as binary 1 or 0. We can shift the bits to left or right using our program for accomplishing various tasks. In JavaScript there is one more feature for right shifting called unsigned shifting that is what we are going to discuss in this post. Below are some prerequisites before we discuss the same.

Bitwise operations happen on 32 bit numbers only

The first thing we need to know about JavaScript bit manipulation is that it works for 32 bit numbers.

Negative numbers are represented as 2's complement

When the negative numbers are stored, they use 2's complement form.

Signed right shifting >>

Here the sign of the number will not be changed due to shifting. The sign bit will make copies and propagate to the right side when the bits are shifted. An example below. 

---

var binary= (7>>1).toString(); 
console.log(binary);
binary= (-7>>1).toString(); 
console.log(binary);

---

Output - 

3

-4

How we got 3 is clear as its simple right shift. The -4 came instead of -3 because it uses 2's complement. That details are there in one of my previous post.

Unsigned right shifting >>>

Lets see what will happen in unsigned shifting. The main difference is that the sign bit just won't propagate to the right side without making copy. Hence the sign will change. Since the bits added to the left are 0s the sign of -ve number will change to +ve. 

---

var binary= (7>>>1).toString(); 
console.log(binary);
binary= (-7>>>1).toString(2); 
console.log(binary);

---

Output - 

3

2147483644

The negative number became positive. Lets see how that happened.

11111111 11111111 11111111 11111001 - 2's complement of -7

01111111 11111111 11111111 11111100 - After shifting bits one time to right

Look at the left most sign bit it changed to 0 which means its a +ve number now. What is that number? Its 2147483644.

Left shifting <<

The point to remember here is that the unsigned shifting is not there for left shifting operation. The sign bit can be overwritten by the bit right to that and that becomes the new sign of the number.

---

var binary= (1<<31).toString

console.log(binary);

---

Output - 

-2147483648

How this -ve number came?

00000000 00000000 00000000 00000001 - 1 in 32 bit rep

10000000 00000000 00000000 00000000 - shifted left 31 times 

The sign bit is negative hence its in 2's complement. Get the number from this 2's complement format. Its -2147483648

References

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Bitwise_Operators
http://stackoverflow.com/questions/14061999/why-does-0x80000000-1-in-javascript-produce-a-negative-value
http://blog.revathskumar.com/2014/03/javascript-shift-operators.html
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

What is side effect of a function in programming?

Introduction

This articles explains what is side effect of function and advantages of avoiding side effects

Background

We all work with object oriented programming for years and our normal thinking goes to desinging application as similar to real world as much as possible. We use object oriented design principles to achieve the same. Shoud we really make everything object with its own class level properties? We are trying to investigate is there any better way to write code which is maintainable

What is side effect of a function

When we write a function / method in the first place, what is our intention? It does a job. It may or may not accept parameters. Sometimes it returns a value. That is the effect of the function invocation.

But there might be situations, that a function may start doing things which are not controlled by that  function. Look at the below code. 

---

//
// ES6 code which has a function with side effect.
//
class Calculator {
  constructor() {
    this.op = "add";
  }
  calculate(n1, n2) {
    if (this.op === "add") {
      this.op = "sub";
      return n1 + n2;
    } else {
      this.op = "add";
      return n1 - n2;
    }
  }
}

---

Here what is the effect? As consumers we call the calculate() the effect we are expecting is the calculated value. But what is happening here?

We get the calculated value. But in the mean time its doing something else on class level variable which we as consumers, don't expect.

It makes our next call gives different output for the same input values. Those are considered as side effect of the function. Side effect will be seen, if we look from the consumer's view.

Below is the code to invoke the function 3 times with same input.

---

var calc = new Calculator();
alert(calc.calculate(20, 12));
alert(calc.calculate(20, 12));
alert(calc.calculate(20, 12));

---

Drawbacks on functions with side effect

The functions with side effects make it difficult to reason about code. What is reasoning about code? If we simulate the working of code, it is little difficult as we need to remember the state too along with the steps. This makes it difficult to achieve development speed and often cause bugs as it is difficult to think about all the possibilities.

This again brings constraint on calling order especially when there are multiple threads using same object to call the methods. When those kind of scenarios occur, it would be difficult to reproduce the bug at the developer end. Normally when developers encounter these types of thread issues, they simply put a lock around critical section. That simply brings us the effect of old single core processors. Our program will not be able to take advantage of multi cores if we just put locks in our code as it make the threads wait for each other. This may lead to deadlocks as well.

What else brings side effect to a function

Not only changing the global state is considered side effect. Whatever we are doing to the state of outside world is considered side effects. Consider the modified code.

---

// ES6 code which has a function with side effect. It doen't alter class level variable.
class MyCalculator {
  calculate(n1, n2, op) {
    var result;
    if (op === "add") {
      result = n1 + n2;
    } else {
      result = n1 - n2;
    }
    console.log("success");
    return result;
  }
}

---

This seems like side effect free. But if we look at the logging code, we can see its changing the outer world. So has side effect. If somebody has run the below code before invoking above method, only alternate calls will succeed.

---

// Redirect the console.log
var shouldSuccess = true;
console.realLog=console.log;
console.log = function(msg) {
  if (shouldSuccess) {
    shouldSuccess = !shouldSuccess;
    console.realLog(msg);
  } else {
    shouldSuccess = !shouldSuccess;
    throw "I am tired!!"
  }
};

---

In the real scenarios, there might be situations similar to this and that will cause the function fail inconsistently.

Can we write everything without side effects

The world is stateful. Its not possible to write every method in production code to be side effect free. But we should try to minimize the side effect as much as possible. Especially when we are writing sequential algorithms, transformations etc...Also we can we immutable objects to avoid risks.

Points of Interest

There are concepts like pure functions, idem potent functions which are related to side effects. The function programming concept too trying to solve the side effects to an extent.

Communication between WPF Windows App & HTML Page hosted in WebBrowser control

Context

This can be considered as a continuation of one old post in this same blog. In that post we can see the technique to establish communication between HTML Page inside WebBrowser controls and and .Net application. In this post we are going to see the approach in detail. Code is available for download and is self explanatory.

Technical details

• For WPF to JS communication
• WebBrowser control has a method called InvokeScript.
• This can be used to invoke any JavaScript function present in the rendered web page and pass parameters.
• For Web page/JS to WPF .Net communication
• WebBrowser has a property called ObjectForScripting which will be available in JS as window.external
• If we set an object to ObjectForScripting property, that .Net object will be available in JS and from JS we can call any method on that object which will in turn call the .Net code.

What’s not done

This just proves the technique. Performance testing is not part of this.

Source code

• The HTML page is inside the WPF.
• WebBrowser loads the HTML page by using pack: uri syntax.
• WebBrowser's ObjectForScripting Object is set when the window loads
• A class which is comvisible is used for the above property
• From JavaScript the SetAValue() is called to pass data from JS to WPF
• For WPF to JS myWebOC.InvokeScript() is invoked
• In JS the data is received in the setSomeText() function.

Download from the below url.

https://www.dropbox.com/s/gcuzbgphk5a2nn9/WPF_HTML_Communication.zip?dl=0

Converted HTML5+JavaScript app to TypeScript

Below are me experiences in converting one of my small personal project from HTML,JavaScript to TypeScript.

Converting Karel to TypeScript

The application is simple one screen. Its used for teaching programming basics.

The conversion has mainly 2 steps.

Get the .js to .ts

1. Create new TypeScript project in Github. Checkout the same in local machine.
2. Create a new TypeScript project in Visual Studio. Better have Web Essentials installed.
3. Copy paste the existing JavaScript code.
4. Rename the .js files to .ts
5. There may be validation errors on compilation. But it will generate .js files.
6. Add jQuery map file via nuget 
1. install-package jquery.TypeScript.DefinitelyTyped -version 2.0.0
7. Better use Local IIS server setup from Visual Studio
8. Run the app and make sure it works as usual.
9. Commit the project.
10. Host to staging. No need to deploy any .ts files to server.

Some steps are solely related to Github source control repository not at all related to TS. The project is live at https://github.com/joymon/karel-typescript

Use the .ts features

Now leverage TypeScript features one by one. Below are some examples.

world.ts - added class support

https://github.com/joymon/Karel-TypeScript/commit/9a4d0bd9a1d8eb279ac584bc35d61dfc2c972f2f?diff=split

https://github.com/joymon/Karel-TypeScript/commit/bc80e03ad8c112df82a791a77c2b43568a5e26dc?diff=split

https://github.com/joymon/Karel-TypeScript/commit/c4667d3f765e183db9d85938488a760cd526485d?diff=split

TypeScript clearly shows the dead code. Remove everything. Refactor with confidence as it will show errors early.

How to convert from JavaScript to TypeScript is now a solved question. This approach can be used on even large apps. There are chances that somethings may show compilation errors due to lack of type definitions. eg: FileReaderSync class. Ignore those as any valid .js is valid .ts and .ts compiler will produce the code for everything.

Happy conversion.