Tuesday, May 3, 2016

TypeScripting AngularJS 1.x - Define module - improvements for big enterprise apps

Improvements for enterprise apps

We have see that there is a typed way to define modules and setup the Angular constructs such as services, controllers etc...This works great in small apps. But there is a problem, if we are developing an big enterprise app with many developers adding new controllers, directives frequently. In that situation, everybody will be competing for the one app / module file.

Below can be our aim when designing TypeScript Angular module class.
  • No need to repeat the ng-app name in every file.
  • There should be one central place for knowing what are the components registered.
  • Every angular component registered in the system should be testable using dependency injection.

How to fix it?

What if we expose a method in the Module to registerController, Directive etc...Those methods can be called by each file which contains different Angular components to register themselves? The object of Module can be exposed via singleton. Lets see the code below for the Angular module wrapper class.

CountryModule.ts

module CountryApplication {
    "use strict";
    export class CountryModule {
        //Do not expose the property as it is not expected to change from outside.
        private static currentCountryModule: CountryModule;
        public static getInstance(): CountryModule {
            return CountryModule.currentCountryModule;
        }
        private app: ng.IModule;
 
        constructor() {
            this.app = angular.module("CountryApp", []);
            CountryModule.currentCountryModule = this;
        }
        /** Acts as central place for registration. But somebody adds new controller they don't need to edit this file.
        */
        registerService(name: string, serviceConstructor: Function): void {
            this.app.service(name, serviceConstructor);
        }
        registerController(name: string, controllerConstructor: Function): void {
            this.app.controller(name, controllerConstructor);
        }
    }
    var hrApp: CountryModule = new CountryModule();
}

The code is self explanatory. We are just wrapping the actual Angular application inside CoutryModule class. Lets see how services can register to the ng application.

CountryMetadataService.ts

module CountryApplication {
    export class CountryMetaDataService {
        employees: Array<Country> = new Array<Country>(
            { id: 1, DisplayName: "India", Currency: "Indian Rupee", CurrencySymbol: "R" },
            { id: 2, DisplayName: "United States", Currency: "Dollar", CurrencySymbol: "$" }
        );
        GetAll(): Array<Country> {
            return this.employees;
        }
    }
    // Registering with angular app name in every file is difficult to maintain.
    // angular.module('CountryApp').service("CountryMetaDataService", CountryMetaDataService);
    CountryModule.getInstance().registerService("CountryMetaDataService", CountryMetaDataService);
}

An end to end template can be found in Github.com. This will get more scenarios on how to use TypeScript in AngularJS applications in TypeScript way.

Welcome to enterprise web app development.

Tuesday, April 26, 2016

Web App or Web Site ie Generating HTML at server v/s client

Most of the tech people know what is what and when to use what. But there are many developers out there in delivery companies who don't know what is the difference and they all call same thing web site sometimes and web app otherwise. This is just to consolidate my understanding about the difference between these 2

Web Site

This is collection of HTML web pages linked together. The pages may be residing as .html page in the web server or web server generates the pages based on the request. For generation of html pages dynamically server side technologies uses html templates which contains tokens. The tokens will get replaced based on the incoming request. Sometimes it reads database for the token value or calls another service for the same. The driving factor is the incoming request.

There were islands of more interactive components using embedding technologies such as Flash, Silverlight or native ActiveX components. JavaScript was used mainly to do simple nice to have things such as validation. The aim was even if the JavaScript is disabled in the browser the app should function properly.

These web sites were catering application needs as well. Eg: we can develop a Employee payroll application using this model.

Technologies
  • PHP
  • ASP.Net

Pros

  1. The site will function even if the JavaScript is disabled in browser
  2. Easy to develop with little js as js was behaving differently in different browsers

Cons

  1. Web Servers have to fetch data from database and mix it with html templates to produce final html which is directly viewable in browser. This overloads the web servers.

Web App

Here mode of accessing the application doesn't change from the end user side. They have to navigate to a provided URL. The content seen in browser is still html. But the difference is, the html template and data are downloaded separately from the server and the mixing happens in the browser. Sometimes if only needs data and instead of html templates it can use JavaScript programs to generate the viewing html.

Here JavaScript plays the key role. If the browser don't have JavaScript support, the application will not function.

Gmail Web interface is a classic example of a Web App.

Technologies

The same server technologies can be used. Just tell them we need only data. Preferably in JSON format. Don't mix with html template as we know how to display your data in browser.

There are technologies such as ASP.Net Web API which are more tailored to web apps.

At client side
  • JavaScript - This is essential
  • Client side frameworks such as AngularJS, knockout js, Bootstrap etc...

Pros

  • Web Servers act as just data provider. Their throughput increases. ie with same no of servers we will be able to serve more users
  • Easy to handle the browser specific formatting such as different view for mobile, tablet etc...Other words responsive UI.
  • Since the server only gives data, it can be consumed by desktop and native mobile apps.

Cons

  • Development team needs to have strong understanding of JavaScript. JavaScript is the tricky language as it contains unwanted things from the beginning. Also this is the language which has a book explaining good parts
    • There is a new trend of transpilation to get rid of this language problem. We can write in different language and then converted to JavaScript for execution.
  • Client side technologies are not stabilized yet. Everyday there are new frameworks getting introduced.

How to select?

We should always choose right technology to right solution. Most of today's apps demand Web App methodology. But there are scenarios where we can use WebSites. For example, simple sites created for personal / business profiles can follow web site methodology. This is because there are only 5-10 pages expected and those can be generated by web servers quickly.

Conclusion

WebSites and Web Apps differ mainly on where the html is generated. There is no change happened to browser during this shift except they increased the speed of JavaScript engine. HTML is HTML always. Most of today's client side machines are capable of generating html in their browsers without any performance lag. Choose the approach appropriately especially when choosing client side JavaScript framework.

References


Tuesday, April 19, 2016

Deserialize JSON string to C# dynamic type

Introduction

This is the age of APIs in computer science. To be more specific web APIs. Don't get confused with the specific technology of .Net Web API. Any methods or resources which are exposed via URL in a structure is an API. From our apps we can call them to do achieve the functionality.

Resources are now a days exposed via ReST protocol and the format is JSON. It works well with JavaScript. But when coming to .Net world, its little difficult to keep the JSON as is in our app. Often we need to convert those resources into entity classes. Creating entity class for each API response is a tricky tasks. It puts more time to complete the API integration tasks. Often after using one API for months we may need to switch to another API providing same service. At that time we need to rewrite it. Creating classes to get exact replica of response is really difficult to manage.

But its always suggested to have entity classes which are for our application regardless of the API we are using. As soon as we got the response from the web API, we should convert the response to our model and that should go on from that point on wards.

Dynamic object to hold JSON

In the beginning people were programming without type safety. Then the strong typed languages were introduced and everybody went behind that. Now its the time everybody again going to dynamic typing. May be its because of JavaScript's success.

VB always gave us the option to have dynamic typing. But in C# its introduced with .Net 4.0. The dynamic keyword is introduced to denote the variables which are not type safe. In other words a type which we don't know at compile time, what it is going to contain. Kind of key value store / dictionary.

Enough theory. Below is the code to convert a GitHub API response. The API url is 

private Project GetProjectFromJSON(string json)
{
    JavaScriptSerializer ser = new JavaScriptSerializer();
    dynamic projectNode = ser.Deserialize<dynamic>(json);
    return new Project()
    {
        Name = projectNode["name"],
        Description = projectNode["description"],
        SourceUrl = projectNode["html_url"],
        Url = projectNode["homepage"]
    };
}

JavaScriptSerializer is a .Net framework class. We don't need to add any dependency to third parties. There are other .Net libraries which are helping to convert JSON to dynamic object but those will add unwanted dependencies. Hope this is self explanatory.

Important thing to note here is the usage of dynamic. Do not ever return dynamic to outer world. This method should hide the complexities of getting the project from GitHub. The consumers of this method then don't need to change when the logic of GitHub API interaction changes.

More details about usage in real application can be seen in this file.

Tuesday, April 12, 2016

Caution JavaScript Ahead - atob() and btoa() to deal with base64 encoding

Setting the ground

JavaScript is always tricky. Because it has many garbage with it, its dynamic etc...This post aims towards how to work with base64 encoding in JavaScript.

History

In JavaScript if we want to work with base64 the standard methods are 
  • atob() - Used to decdode
  • btoa() - Used to encode to base64
It seems there are not much about this naming. Some people says it stands for Binary To ASCII when encoding and vice versa. Another thought is it was following C function naming. We can also think that when this was developed the aim was to write small JavaScript programs. If the programs are big it will take more time to transmit from the server to client. Language designers were concerned about that.

Usage

Simple

It's just a function which can be called normally.

var output=btoa("Joy");

Output
Sm95

var output=btoa("Sm95");

Output
Joy

Unicode

The above works well if the input is not unicode. If the input is unicode there will be an exception and the message goes as follows
In Chrome

InvalidCharacterError: Failed to execute 'btoa' on 'Window': The string to be encoded contains characters outside of the Latin1 range.

In IE (11)
InvalidCharacterError

Base64 with unicode

The fix to this problem goes as follows for encoding


var encoded = btoa(unescape(encodeURIComponent("ജോയ് മോൻ")));


Output

4LSc4LWL4LSv4LWNIOC0ruC1i+C1uw==


To decode use the below


decoded = decodeURIComponent(escape(atob("4LSc4LWL4LSv4LWNIOC0ruC1i+C1uw==")));

Output
ജോയ് മോൻ

Problem

The problem with this is same. It works differently in different browsers. If the encoding and decoding happens in same place, it works. But suppose if the server encodes the data and JavaScript gets it via Ajax,what will happen?

Ideally atob() should decode it and it does in Chrome and Firefox. But its not working in IE. Even in IE 11.

To be more precise the scenario is to decode the content coming from Github API. Below is a typical Github API url and it gets context in base64.

https://api.github.com/repos/joymon/joyful-visualstudio/readme

When we try to decode the base64 using atob() it will work well in Chrome and Firefox. But throw the the same error message in IE 11.

Solution - Use custom base64 library

The solution to this is to use a custom base64 library. Below is one url to such a library.


Refer this library in to the site and use below line to encode to base64

encoded = Base64.encode(inputText);

To decode

encoded = Base64.decode(encodedText);

Refer the source code of library to get more understanding of how its done.

Sample

Sample code is in jsFiddle. Url below
http://jsfiddle.net/joygeorgek/tv3b7feL/10/

Tuesday, April 5, 2016

What is Microservice Architecture to me?

Microservice is a buzzword in current Software engineering world. Everyone wanted to say we are building microservices or eagerly want to adopt it.

Reasons for micro services

Microservice - Bring Agile to Architecture

If we look at definition, we can see its an architectural style, where system is divided to small autonomous processes which are coupled with programming language/platform independent API. But we had similar kind of architecture in SOA (Service Oriented Architecture). They were also communicating kind of stable independent APIs. But what is the main difference or advantage.

Frequent releases

Yes the ability to release  more frequently is the key. We cannot deny other advantages that system is clean with small services which knows how to do its duty well, system can easily be built with different programming languages, scale only required components/services instead of whole system. etc...

If we remember the introduction of agile, there were so many advantages advertised. Working code over documentation, customer collaboration, responding to change were some to mention. But what is the real key? The key was 2-3 week based sprints which delivers something. As of my understanding most of the service companies adopted agile only because there will be a release in 2-3 weeks and developers will work night and weekend every 2nd week. In waterfall, the night and weekend parties were once in 6 months. Most of the places only the concept of sprint was taken but other concepts like maintaining backlogs, avoiding documentation over code were discarded. There would be no release happened in the world without documentation saying that code is working. 

What is there in Microservice? Independent teams who take ownership of small services. The size of the service, kind of relates to the size of the team who delivers it. And deliver daily, twice a day or if possible hourly.

Yes most of the places we can see an emphasis on TDD when adopting Microservice. But that is something might be discarded to rush to release.

What is the bottom line? The architecture will be agile. A Microservice could be introduced at any time in the development phase. Servers are going to be flooded with services. The complexity of the system is going to be in the orchestration of Microservices.

Design Microservice

Whatever we think, the Microservices architecture is a reality and soon most of our apps are going to be Microservice based. As the software engineering is still evolving, only fittest will survive. Currently it seems Microservice is a fittest one. Of course there are some problems to which Microservices approach is not the best solution.

Here lets see what are the characteristics of Microservices? 

Independent release

If a service can be released independently of the other parts of the system, it can be considered as Microservice. Sometimes the new API methods we released in the Microservice might not have any consumer components as the consumers can release only after the service release. 

The concept of independence is really important, if we want to release too often. If it depend on other components we cannot release fast & often.

Independent data store

If we want to have independent release of a service, it's data store should be independent. If there is a database table and a component release upgrade the schema causing other components to fail, we cannot consider that component as a Microservice. In short Microservice should be the only one to own it's data store. If other components in the system want to write something to the store they should call service.

Independent development team

If we want independence, probably it should be done by independent team. This is something not with the technology. Organization structure needs refactoring and is tough. 

The speed of delivery is prone to errors as well and it is advised to have development team supporting the service. If the team is shuffled after each release or members change often, they cannot support service as their own baby and the micro service architecture might feel a failure.

Independent framework/platform/language

This is not a mandatory thing. If there is a pressing need for adopting a particular technology which is different from regular technology stack, Microservice is recommended. But only if the above 3 criteria are met. Even we can have Microservice based systems with one language and technology.

Adopt only if

Do not jump just into Microservice without proper assessment and planning. If everything is independent who coordinates all these to work as a complete system which cater business need? Below are the things to make sure before jumping into Microservice world.

Defined integration guidelines between components. http(s)

If you don't have defined integration strategy of Microservices, it will simply fail. The recommended integration if through http(s) protocol if the interaction is synchronous. If the integration if asynchronous, adopt message queuing, service bus etc... But make sure there are enough design sessions and agreements before adoption.

Upfront design on the API

The real world systems / business problems are always complex. In monolithic system, we were handling the complexity in big components. But in Microservice era, the individual components are simple, but the complexity is shifted towards the interaction of services.

This requires carefully designed APIs for each Microservice. API is always like Mumbai underworld. Once we enter, there is no coming back. Once we publish an API, we don't know who are all calling it. Correcting the API and communicating the same to all consumers to redirect to new API is always costly.

Deployment is easy and automated.

We have seen one of the major advantage of Microservice is the frequent releases. Is the installation going to be done manually every time? If so we will immediately feel that the Microservice is a bad idea.

Deployment should always be automated. A continuous integration and delivery pipeline is always essential.

Strong automation testing. Prefer 100% test coverage

There is no human required for day to day deployment. They just setup the CI & CD in the beggining and system will do deployment automatically. What about testing? Are we expecting a manual tester to test all the changes before we release daily?

That's again should be automated. Developers should write proper test cases for the API. Most of the companies are employing a separate team / person for testing, is on the assumption that a developer doesn't want to break his own code or psychologically they cannot find out errors in the system what they build . When developer test, he tests only positive or usual scenarios because he knows how the system works. But here in Microservice world. we are not entertaining separate manual testing team,So how to tackle it?

Its very easy to tackle. Some other developer needs to write the tests. If the existing testing team knows how to code API tests, integration tests, they can also do. Please note we are against manual click QA. Automation QA is always welcome.

Service monitoring & management

We are talking about explosion of services. There are chances that each class/component in your existing system such as logging, caching will become a service. Some services will be versioned services, some will be cumulative/in place update. We need to add more service instances and load balance when services show load. Some might not be even used. A machine which hosted a service might go down and we need to relocate the service without zero or a little downtime. How do we deal with all these scenarios? 

The answer is a strong service monitoring and management infrastructure. Easy way is to host in cloud, where this is free with cloud provider. We don't need to develop infra management application for increasing number of machine and deploying the service to it. Or to move service from one machine to another machine etc...

Versioning, dependency management, retirement strategy

V2 of Service A may require V4 of Service B to be present in the system where V1 of Service C require V3 of Service B. This means V3 of Service B cannot be retired until Service C is retired.

Why we need to retire a service. Over the time some services will be legacy and there will be nobody calling it. But it just stays in the production environment. It is the decision of the stakeholders to retire or not. If we plan to retire everybody should be notified that, it is retired so that no new service call these retire ones.

Another interesting scenario is whether the a service say Service X should know the location of other service Y? Though the Location has many things to consider, here let's take as the url of service. If we decide to keep the location of Y in X, we cannot move the Service Y to other machine or environment. Hard coded location will cause trouble. Suppose we started with all the services on premises and due to load, if we decide to move Service Y to cloud Service X, will simply fail in calling Y.

This requires a service registry which keeps track of all the services and their dependency. This can be another service but in fixed location of course.

If you have access to production

If the development is for a product your company owns and is server based, Micro services works great. If the product is delivered as an installer to your clients, it would be really difficult to manage, as they may be still installing the morning release when you give evening release.

So basically better choose, if the CI & CD pipeline can reach till production. We can refactor the other systems to Microservice based but will never get the advantage of frequent release.

What to do with UI

Till now, we were talking mainly about services, what about UI? Are we going to deliver UI twice a day? That is something interesting and will discuss in a future post.

Disclaimer - I work in a project which have 100+ services, though we officially not adopted the Microservices architecture. It just came like that and many services are sharing same data store. We are not doing independent release of services due to data coupling. So the above view is not from a real time successful Microservie based project rather from reading and sufferings of service management.

Monday, March 28, 2016

TypeScripting AngularJS 1.x - Define module

As we know developing and maintaining a large application in pure JavaScript is really tough. One of the reason is its dynamic typing or its not TypeSafe. If we want to change a variable or function name that's it. There is no way to identify where all it used until we run. Of course we can do text search. But its difficult from the tooling perspective. So people going to TypeScript to develop JavaScript applications.

AngularJS is a JavaScript framework for developing enormous apps. If we use TypeScript there, we get more advantage. Since any valid JavaScript is valid TypeScript, people tend to write JavaScript inside TypeScript files without leveraging TypeScript features. But it works. It will definitely give difficulties in long term when maintaining the code.This series is aimed at how we can leverage TypeScript to code AngularJS application in the TypeSafe (compile time) way

Defining module 

It is easy.to define an Angular module via JavaScript.
var app = angular.module("<moduleName>", []);

But lets see how can we do it via TypeScript. Please note this require TypeDefinitions for AngularJS. If we are using Visual Studio, this can be installed via 'angularjs.TypeScript.DefinitelyTyped' package
var app: ng.IModule = angular.module("<moduleName>", []);

Then we can register controllers, services and directives to it. by calling app.controller(), service(), directive() respectively. But is that an object oriented way? Absolutely not. We need to wrap it inside a class. Lets see one sample below.
module CountryApplication {
    "use strict";
    export class CountryModule {
        private static _constructor = (() => {
            var countryApp: CountryModule = new CountryModule();
            countryApp.setup();
        })();
        app: ng.IModule;
        constructor() {
            this.app = angular.module("CountryApp", []);
        }
        setup() {
            var app1: ng.IModule = angular.module("<moduleName>", []);
            this.setupServices();
            this.setupControllers();
            this.setupDirectives();
            this.setupRoutes();
        }
        private setupServices(): void {
            this.app.service("CountryMetaDataService", ["$http",'$q',CountryMetaDataService]);
        }
        private setupControllers(): void {
            this.app.controller("CountryController",
                ['$scope', '$http','CountryMetaDataService', CountryController]);
        }
        private setupDirectives(): void {
            this.app.directive
        }
        private setupRoutes(): void {
        }
    }
}
We can see the things we were doing in Angular with JavaScript. But here we are just organizing, as a class. This gives us one view about our services, directives, controllers and how they are related each other.

The downside of this is every time a developer adds a new controller or service they need to modify this file.

Links

Tuesday, March 22, 2016

Caution JavaScript Ahead - toString(2) is not returning right output always

Introduction

There is Number.toString() method in JavaScript. Passing 2 to this function convert number to binary string. But is it returning right value always?

Background

This require electronics or computer science background as its talking more about internal of computing machine such as 2's complement. But people who do programming without that background can also have a try reading this article.

Converting positive numbers toString(2)

Look at the below code which convert number to string by passing 2 as argument to get binary string representation.

var binary= (6).toString(2); 
alert(binary);

Output
110

This works perfect.

Converting negative numbers toString(2)

Lets convert a negative number to binary using the same technique.

var binary= (-6).toString(2); 
alert(binary);

Output
-110

This looks simple as we are getting the binary digits which is equal to absolute value(value without sign) of number. This is easy to understand for a person who don't have computer science of electronics background. But are we getting right binary representation of -6? No. Its wrong.

2's complement representation

In computer, it stores the data including numbers in binary as it knows only 2 states. On or Off or 1 or 0 respectively. We can store signed numbers as well as unsigned numbers. For signed numbers it require one bit to denote the sign. If the number variable has maximum 4 bits to store, it can store up to -8 to +7 in case, it needs to support -ve numbers. If it is unsigned data type, it can store from 0-15 ie 0 to 24 -1.

Another point to note is the storage mechanism of -ve numbers. One bit is not just kept for sign. Instead it stores as 2's complement. It does so to do add operation easily. Lets see how its represented by taking 4 bit number data type for easy understanding

6 - 0110 - a positive number
-6 - 1010 - Representing -6 in 2's complement

But look at the out put which showed -110 earlier for -6. Ideally the output should be the 2's complement (1010) based on the data type length.

Finding 2's complement

The steps are simple. Find the 1's complement by inverting bits. Then add 1 to the result. Lets see how -5 can be written in 2's complement way. Here first write the binary representation of 5 without sign.

0101 - Leading 0 is required as the data type length we took here is 4.
1010 - Just replace 0s with 1s and 1s with 0s.This is 2s complement.
1011 - Added 1 to get 2s compliment.

Hope its clear and we can easily calculate it for any number. There are online tools available to convert. Try those to learn faster.

Trouble of showing (-binaryNumber) in toString(2)

Normally there is no problem when the negative number is converted as shown in the above sample. -110 for -6 instead of 1010. But it gets complicated when we deal with bit-wise operations such as shifting bits in -ve number and convert to string. Look at the below snippet.


var binary= (6>>1).toString(2); 
alert(binary);
binary= (-6>>1).toString(2); 
alert(binary);

Output - 
11
-11

This gives an impression that the number is just converted to binary regardless of sign and shift the bits right once.

00000000 00000000 00000000 00000110 - Binary of 6
00000000 00000000 00000000 00000011 - After shifting bits one time to right

Added zeros as JavaScript does bit wise operations using 32 bit.

Now check the below example

var binary= (7>>1).toString(2); 
console.log(binary);
binary= (-7>>1).toString(2); 
console.log(binary);

Output - 
11
-100

We could see that the +ve number 7 is simply shifted right and -7 is not. This is because we are expecting just -11 which is nothing but adding -ve sign to  11. 

00000000 00000000 00000000 00000111 - Binary of 6
00000000 00000000 00000000 00000011 - After shifting bits one time to right

We may think that -100 is wrong only because we have understanding that the computer is handling -7 as just 7 with a sign in front of it. In reality, it stores 2's complement which is not what we are seeing when we displayed it using toString(2). Lets see how we got -100

11111111 11111111 11111111 11111001 - 2's complement of -7
11111111 11111111 11111111 11111100 - After shifting bits one time to right

Finding the number from 2's complement 

Its the simple 'reverse process in order' what we did for finding 2's complement.
11111111 11111111 11111111 11111011 - Subtracted one
00000000 00000000 00000000 00000100 - Inverted bits (replace 0s with 1s and viceversa)

This is why we are seeing -100 when we converted to binary using toString(2) after shifting. Try how its working for -6 to get same -11 for +3 and -3.

Conclusion

Number.toString(2) is not showing the 2's complement representation of number which is used internally. But the bit wise operations work on 2's complement.

If we don't understand the 2's complement and we do shifting business with -ve numbers thinking they just have additional sign. Sometime we will see its working (see the case of +6 & -6). But it may fail for other numbers.

Points of Interest

If (-7).toString(2) returned 11111111 11111111 11111111 11111001 it would have been easier to understand how the numbers are stored inside computers. But in the other hand this makes it difficult to understand for common people. Now it is not advisable to change the behavior as there are so many apps written on the current working of toString(2). Those may fail because of correction. May be another overload will help.

There are improvement done in JavaScript which does in new ways to fix issues in old implementation. == and === is the famous example.