JavaScript Access inner objects with string expression

JavaScript is becoming more and more importance for developers. Even it can be used as DSL with the help of embedded V8 runtime engine. Below is one scenario which may occur when using JavaScript as DSL

The DSL is another program fragment, which our program has to execute.
Suppose there is an JavaScript object structure as follows

---

var complexObject = {
 prop0:"prop0 value",
  prop1: {
    prop1prop1: {
     prop1prop2prop3: "prop1prop2prop3 value"
    },
    prop1prop2:"prop1prop2 Value"
  },
  prop2:[
   {prop2prop1:"prop2[prop2prop1] value"},
   {prop2prop2:"prop2[prop2prop2] value"}
  ]
};

---

Lets see how this can be accessed.

Normal JavaScript property access

Now lets see how can we access the values using normal property dot(.) notation

---

var testInnerObjectProperty = function() {
  alert(complexObject.prop0);
  alert(complexObject.prop1.prop1prop2);
  alert(complexObject.prop2[0].prop2prop1);
};

This works perfect.

Access JavaScript property using string property path

But in DSL the path to the value will be available as string. So how to access the value using string property path?

---

var testInnerObjectPropertyWithIndexer  = function(){
  alert(complexObject["prop0"]);
  alert(complexObject["prop1.prop1prop2"]);
  alert(complexObject["prop2[0].prop2prop1"]);
}

We can see the first alert returns propert value as its direct string path access. But the next 2 will not work as JavaScript runtime is not able to find a property names "prop1.prop1prop2" on complexObject. In other words it doesn't have the intelligence to understand the hierarchy. May be its by design.
Lets see another way to access using string string.

Access property using string path via Evil Eval()

This is the easiest method pops up into a medium JavaScript programmers mind though he know its not the good way.

---

var testInnerObjectPropertyWithIndexerEval = function() {
 alert(eval('complexObject.'+'prop0'));
  alert(eval('complexObject.'+'prop1.prop1prop2'));
  alert(eval('complexObject.'+'prop2[0].prop2prop1'));
}

---

Since it works but not the good way what is next.

Access property using string path via parsing

This is the hard way. We must test is property for all combinations. Lets see one implementation.

---

var testInnerObjectPropertyByParsingPath = function() {
 alert('Going to parse prop path');
 alert(Object.getValue(complexObject,'prop0'));
  debugger;
  alert(Object.getValue(complexObject,'prop1.prop1prop2'));
  alert(Object.getValue(complexObject,'prop2[0].prop2prop1'));
}

---

Above is the API interface expected. Below goes the implementation of getValue()

---

Object.getValue = function(obj, path) {
    path = path.replace(/\[(\w+)\]/g, '.$1'); // convert [] access to property access with number
    path = path.replace(/^\./, '');           // trim leading dot
    var props = path.split('.');
    
    for (var propIndex = 0, noOfProps = props.length; propIndex < noOfProps; ++propIndex) {
        var propName = props[propIndex];
        if (propName in obj) {
            obj = obj[propName];
        } else {
            return;
        }
    }
    return obj;
}

---

This can be written using recursion too. It replace the array access [] with property access by replacing corresponding number there. Then loops into the object hierarchy till it finds required object.

Please note this is not the battle tested code. This is rather to give the idea.
https://jsfiddle.net/joygeorgek/s5egwa5o/8/

Embed Google V8 JavaScript engine in .Net application

Background

JavaScript is everywhere whether we accept or not. Now a days the trend in new languages is to transpile to JavaScript. That's the way they don't need to worry about runtime. Thanks to fast JavaScript engines such as V8. People are sure once their code is in JavaScript, the engines will run it in the highest speed, even it works in server side with the help if NodeJS technology. For client side desktop development also we can rely on JavaScript using the Electron technology. In all these places JavaScript engines power the execution.

So what about DSLs? Why can't we write our DSL in JavaScript and execute via JavaScript engines? The only constraint here is from our technology / runtime we should be able to send and receive objects to and from JavaScript engine. In simple terms, in our C#.Net application there is a salary calculation which is dynamic and needs to be customizable by user. If we can pass the Employee object to and from JavaScript engine, we can use JavaScript as DSL for our app.

The problem narrows down to

• Starting JavaScript engine from our technology of choice. Send the JavaScript code to the engine.
• Passing objects from our application JS engine to manipulate.

Since JS is dynamic and not strictly typed, we can pass any string as program into the engine. Engine will fail the code only during the execution. In this post, we can see how such an interaction can be done between .Net and Chrome V8 JavaScript engine.

Selecting Chrome V8 Engine

There are many implementations out there for Chrome V8 engine. Most of those are compatible directly with native code. From .Net its little difficult to interface with those.

We are going to use one out there named JavaScriptDotNet.

Setting up the environment

Download the binaries from the below repository. They moved the code to Github. But unfortunately the ready made binaries are not there.

http://javascriptdotnet.codeplex.com/

Create our .Net project and refer corresponding binaries. There are .Net 3.5 & 4.0 folders and each folder has 32 and 64 bit versions. Since Chrome is a native unmanaged library and JavaScriptDotNet only provides a managed wrapper over that, we need to be careful in selecting which version. The assembly we need to refer to our .Net project is Noesis.JavaScript.The other dlls should be present in the \bin or running directly of our ,Net application.

How to run JavaScript

Its simple as creating JavaScriptContext and calling Run().

using (JavascriptContext context = new JavascriptContext())
{
    var result = context.Run("3+2");
    MessageBox.Show(result.ToString());
}

How to pass .Net objects to JavaScript and vice versa

Now lets see how can we pass .Net objects into JavaScript. If we can't pass the objects there is no meaning in embedding JavaScript engine inside .Net

Lets take one simple Employee class

public class Employee : INotifyPropertyChanged
{
    public int Id { get; set; }
    public string Name { get; set; }
    public double Basic { get; set; }
    public double HRA { get; set; }

}

Employee emp = new Employee() {Basic=100, Name = "Joy" };

Object is also created with one property 'Basic' set to 100. Now lets see what the below code will do on this object.

using (JavascriptContext context = new JavascriptContext())
{
    // Setting the externals parameters to the context
    context.SetParameter("emp", emp, SetParameterOptions.RejectUnknownProperties);
    try
    {
        // Running the script
        context.Run("emp.Total = emp.Basic * 2; emp.HRA = emp.Basic / 2;");

        MessageBox.Show(emp.HRA.ToString());
    }
    catch (Exception ex)
    {
        MessageBox.Show("Error");
    }
}

It will show 50 as the emp.HRA is calculated when JavaScript engine run. Yes now we have embedded JavaScript engine in .Net and JavaScript can now be used as DSL in our C# application.

http://stackoverflow.com/questions/172753/embedding-javascript-engine-into-net

Why I recommend Javascript to my wife? And to everybody

Almost everybody who born as human will think about their partners in advance. Only the depth and frequency of thinking might vary. When I graduated and joined first company, I was thinking about a girl who is working in a technology company but not related to software. The reason behind that is, if both of us are in software, we might talk about software development always which will affect some other developments. As I switched companies and when my parents really started searching bride for me, my thought process was different. (Non Indians may not understand what is arranged marriage.Its not at all forced marriage. In our arranged marriage, couples meet twice or thrice alone to talk before they decide to marry. Mostly parents screen the girls first and arrange meetings. More details see wiki).

I was thinking about a girl who knows how software development works or minimum graduated in computer science but not working in software development / coding! Can be computer science teacher in schools or colleges, working in software administration in other industries such as banking to a level of QA in software industry. The reason behind my change was the culture of the company I was working. Most of the days, need to work or I will say need to be in office for more than 12-14  hours. A normal girl who don't know the peculiarities of software development process will have tough time understanding, why I am staying late in the office.

Finally got one who just completed Masters in computer science and offered from Infosys, one of the well known companies for software development. But fortunately or unfortunately after the marriage, the decision has changed to try in teaching field. She got it quickly in a startup engineering college. That college doesn't have computer science dept. But in the first and second years, all the other departments have CS paper. She find it cool teaching C/C++ etc...

The college was far away from where we stayed and it became difficult to travel more than one and half hours when she conceived baby. She had to quit and became house maker. All went well and we now in US obviously she is in H4 jail. Getting into teaching position seemed tough, so she thought of coming again to development. The tough question came. Which language? Which technology & Platform?

When she asked the same to me, I had no confusion on saying 'Learn Javscript'. Not just learn, master it.

Why Javascript?

If we are passionate about software engineering and up to date with the industry this question will be already solved in our minds. For those who are still thinking about learning Javascript, below section s added which explains why to learn Javascript to secure in software development.

Javascript is everywhere

Yes this is the main one reason to learn Javascript. Lets see where all we can run Javascript.

1. Javascript in web development
With the arrival of faster and multi core processors browsers are now able to render html and process Javascript with very less time. This makes us write javascript driven applications which can run with high response. Another reason is the HTML5 which has more features targeting towards web site to web app transformation. Now we can leverage 3D capabilities via HTML APIs and javascript 3D libraries. It is real and there are many sites out there which uses it. It was not the case earlier. Now we can see lot of process heavy apps are getting written in Javascript and rendered in browser.
All the mail apps such as gmail, outlook etc...runs smoothly in browser with heavy Javascript. Even there are Javascript based game available which runs smoothly in any average machine.

2. Javascript in mobile development (cross platform)
Everyone wants to run their application everywhere. But nobody wants to develop more than once. Especially in mobile platform where there are 3 clean leaders whose platforms are not at all compatible except rendering HTML. We can develop apps for mobile in html, but it may not take the capabilities of hardware especially sensors. So selection of mobile platform was very critical in early mobile development. People tried many solutions and one of the best solution is to develop in HTML and Javascript itself and compile into native form. Phonegap is one leading platform in this and there are many technologies in this area.
3. Javascript in server side
With the launch of Node.JS, Javascript gained popularity in server side as well. It works using chrome V8 Javascript engine and 2 simple C++ libraries for accomplishing event model. It has already proven its scaling abilities and many web apps are already running using it.
4. Javascript for desktop application development
This is another platform where software is running today and may run for some more years. Earlier there was no popular way to write desktop applications using Javascript. But as of my understanding Microsoft changed the model by bringing support for writing Windows 8 desktop apps using HTML5 and Javascript. They did it in very simple way. Extracted the browser rendering mechanism and hosted it in windows app container. They established binding of native resources / objects into Javascript so that Javascript which is running inside can manipulate anything in the machine. This is what any desktop application platform does.
But is it not cross platform. We cannot run out Windows 8 application in Linux. It is been addressed by another similar technology named Electron came out from Github. Github used Chromium as their UI container. Chrome already knows how to render HTML and it has good Javascript engine as well. Connected the dots and we got a nice Javascript + HTML based desktop application development platform. Even Microsoft uses this new technology in their cross platform Visual Studio called VS Code.
5. Javascript for Robotics & IoT / Micro controller boards
Another area where software can run is micro controller boards which is expected to be the heart of IoT. Micro controller means a small computer which has input and out ports, memory and processor to  execute instructions read from memory. Details on what is IoT is available in wiki. In next 5-10 years with the help of IPv6 all the things / devices such as microwave oven, house doors, chairs, tables, fridge,car, medical devices will be connected to internet and all needed to be programmed. If those devices require different languages and tools, it would be difficult to survive both for us as well as companies. Companies needs to hire new people with high salary or spend on training. But the smart people are trying to unify the language and that language is going to be Javascript 99% of time. There are enough controller boards available now which out of the box supports Javascript. eg: Espruino. Even the famous boards such as RapberryPi, Arduino can also use Javascript. This pretty much shows that robotics programs can be written using Javascript.
6. Javascript as DSL
Domain Specific Language is everywhere and we use it knowingly or unknowingly. It is not a language by itself. Its the way we model our business domain and expose it to the users. Users can be our fellow team mates or actual users. Once we have our domain objects, the code to manipulate it can be written in any language such as Powershell, Javascript etc...provided there is DSL run-time to execute the DSL scripts from our host application. In fact if we think deep we can realize that in browser we are using Javascript as DSL and the domain objects are the HTML elements and web request. Similarly when it comes to Node.JS same Javascript and engine process different set of domain objects which are nothing but the server side resources such as files, databases etc...

Are there any other areas left where software runs?

Javascript language has less and rest done by frameworks & engines

The core language is always simple with functions as first class citizens. It can simply do manipulations and call methods on the objects which are passed into it during run-time. But its getting all things done and run everywhere with the help of libraries, frameworks and run-time engines.

If it is running in browser the objects passed to it are the DOM, XHR etc...If its running in server side as Node.JS the objects are http request, http response, database driver, file system object etc...If its in mobile the objects available are the sensors, phone calling and messaging system, file system inside mobile, network IO, display provider etc...finally if its running as DSL, the objects passed are the domain objects such as list of customers, employees which are specific to the business.

Why we need to be masters?

Anybody can write code in Javascript and people were surviving because javascript was mainly used to get some interactivity to the web pages. More than that, it was used to implement kind of 'nice to have' features such as client side validation where we already have a server side validation. Even if the Javascript is disabled at client side our web sites should work. In another words, as a second class citizen. The main reason was slow internet connectivity and even if someone tried to develop Javascript driven web site, the chances for a large crowd using it was very low. Because most of the public had no good connectivity. And due to that reason obviously nobody put attention to improve the Javascript language specs as well as the run-time engines. Big issue was, the rendering browsers were not standardized. If we wanted to make Javascript run in all the browsers, we had to work lot and often will miss deliveries.

But now as we seen earlier things are different. Internet connections became faster. People foreseen the change and they put effort in standardizing the browsers, improved the language specs and engine. Finally framework development started which ensured the compatibility among browsers. We got confidence that our code will work in all the browsers, if we use these frameworks such as jQuery . That enabled us to focus on the features instead of spending time on compatibility issues.

Web sites to web applications

Another important change which already seen is the shift from web sites to web apps. Earlier web development means creating bunch of web pages which will be delivered to browser in the form of HTML. Whether we use ASP.Net, PHP or Java all were just mechanism to generate pages dynamically. The business always remained in the server side. One of the belief was Javascript is not secure and we should never keep our business in it.

But now things changed. The server side slowly retired from its web page generation and business logic container duties and the new role is 'web service' or data provider. If the server just keep the data we can develop application in any client device provided there is a constant connectivity to the server and secure communication mechanism. Connectivity now is not at all a problem. Security is enforced via secure communication protocols such as TLS.

Need for writing good Javascript code

If we agree or not Javascript is everywhere. It is now becoming essential part of our application. In such a scenario, if we don't write the application well, it going to fail. As we all know we are paid to develop applications which works and scale. Due to lack of Javascript knowledge, in the earlier days  the failure may happen only to some nice to have client side validations. But in this new era, entire app will not work. If our app is not working, it leads to less revenue to the company and finally it threatens our job.

The natural selection allows only survival of the fittest species. Similarly if we don't know Javascript there are high chances that the software evolution may eliminate us from the industry.

I have seen many people sitting in HR and admin sections who were once programmers and they talk about their programming history with proud. Should they be proud of their programming history which they were not able to survive? Sometimes they might have left the software development due to other reasons. Whatever if we don't want to be in same place after 10 years, it is very much advisable to learn how to write quality Javascript code.

Of course if we are into system side programming using assembly,c or c++ or go languages, it may be slightly different.

Javascript has lot of garbage

Though the language is getting improved, lot of frameworks and standardized browsers, writing good Javascript is still hard. It is because the language contains lot of garbage from the past. It is not going to be removed from the language to maintain backward compatibility. ie There are lot of applications written using the existing / bad language features. They are all working because they are expecting the language to work in that specific way eg == and ===. All of a sudden if the language got breaking changes or remove bad features all those applications will stop working. In Javascript versioning is too not possible. If we had below features we could write better Javascript in new version. But unfortunately that is not possible and doesn't makes sense in Javascript world.

<script type='text/javascript' version='6.0'>
//Source which contains new clean Javascript language(where == acts like === and === is removed)
</script>

Due to backward compatibility, there are lot of ways to achieve same thing in Javascript. Some ways, we can simply avoid and some needs to be careful selected.

References

Books

• Javascript:The Good Parts Paperback – Douglas Crockford

Links

• https://developer.mozilla.org/en-US/docs/Web/JavaScript

---

Disclaimer

This doesn't mean that JS will survive for ever. When a better new language is born or existing language is enhanced, we should focus on that. But be assured that at least for next 3-5 years, Javascript will survive. As everybody knows The only thing which don't change is CHANGE.

XAML as DSL - Load and saving Windows workflow foundation workflows

Context

Windows Workflow Foundation has got better after MSFT introduced the WWF 4.0. I don't think, it was stable earlier. We are now using WWF, in one of our project. The main reasons are as follows

• There is a complex business scenario (similar to recommendation algorithm) which business experts itself, is not clear how to do it. They want to try/test the algorithm in production data using a different tool. During that test they may need to alter the algorithm and change configurations to try various possibilities. Since the production data is spread across different countries and some don't allow the data to be carried out of the country, its not feasible to have a developer going with the business testing team and alter the algorithm.
• During the above testing conducted by business experts, they want to try different algorithms (copy and modify original) on same data and compare the results.

Why XAML as DSL

This clearly points to the need of a Domain Specific Language to code the algorithm. As mentioned in one of my previous posts about DSL, either we can develop a DSL with its own parser and all or leverage existing languages to manipulate the entities. Creating own domain specific language with its own parser is not feasible as it takes more time.We could easily use PowerShell or Javascript as DSL, but the problem is the tooling. So we took XAML as DSL to code the algorithm. Decision factors below.

• The business want to change the algorithm on the fly.
• Business people are not good coders and its difficult for them to understand by looking at program as text. So they need a UI to edit algorithm (Implemented using DSL)
• There is not much time to create tooling for existing languages such as PowerShell or Javascript

Advantages

XAML as DSL is being executed by the WWF engine. If we see the WWF as technology, it has many inbuilt features we can leverage to have it as DSL. Some items listed below.

• It stores the workflow in clear XAML format
• It comes with out of the box workflow designer control based on WPF. This workflow editor control can be easily hosted in our WPF application.

How to load and save WWF workflow

Lets come to the topic of saving and loading windows workflow foundation .xaml workflow files. As we saw earlier, the business want to edit the algorithm on the fly. We are keeping the xaml file in file system itself , even though the application has database.Below are the code snippets

Load the WWF workflow.xaml from file

Mainly we need to create the instance of WorkflowDesigner class and invoke it's Load() method with string argument. We can pass the file name as string.

Important thing here is, we don't need to read the file ourselves and pass the workflow XAML. For doing that there is another method.

---

            _workflowDesigner = new WorkflowDesigner();
            _workflowDesigner.Load(_fileName);
            //Code to Load _workflowDesigner related controls such as _workflowDesigner.View,_workflowDesigner.PropertyInspectorView 
            //into the WPF container

Save modified workflow to file.

Saving the workflow into file too simple. Just invoke the method. Main thing to make sure the permissions on the file. The user who is running the process should have write permission on the file.

---

            _workflowDesigner.Save(_fileName);

---

Load and save the workflow from database or any other source

There is Text property in WorkflowDesigner class. Assign the workflow xaml content in string form to the Text property and call Load() method to load the XAML string directly. Read the XAML string from database using traditional techniques and assign to Text property. We may get the modified XAML via the same property to save back to database.

---

            _workflowDesigner = new WorkflowDesigner();
            _workflowDesigner.Text = GetWorkflowFromDatabaseAsString();
            _workflowDesigner.Load();

---

References

Loading workflow in WorkflowDesigner

http://odetocode.com/blogs/scott/archive/2007/03/19/windows-workflow-foundation-and-domain-specific-languages.aspx

Happy coding.

My presentation about DSL @ K-MUG user meeting

First time in my 8 years software engineering career, I took a session to public audience. The event was the monthly user meeting of Kerala Microsoft User Group K-MUG community. There were 3 another useful  sessions which were about ASP.Net Web API, CDN via Azure and Using Fiddler.

It was a wonderful experience talking to the audience who comes from different backgrounds and different knowledge levels. Even though everyone uses DSL in one or another form, it was little difficult to make them understand that they are using DSL in their day to day life.

I started by comparing the software engineering with civil engineering in terms of requirement changes during project execution and as a solution to that introduced DSL concept. After explaining the concept along with 2 types internal and external DSL, I moved to using DSL concept in .Net applications. Explained what are the criteria we should consider before selecting a DSL and selected PowerShell as DSL script for a sample.

The sample was the same which I explained in one of my previous post. Explained where the DSL comes into picture in that simple salary calculation sample and showed how the reference to the PowerShell runtime is added.

I believe everyone got the idea of DSL but not the exact idea how they can implement in their project. Feel free to contact me on using DSL in your projects.

Slides has been uploaded to slide share.

Using PowerShell as DSL in .Net applications from Joy K

Passing .Net objects as arguements to PowerShell

In the previous posts about PowerShell to .Net integration, the samples were simply invoking PowerShell scripts from .Net code. In the first post, the sample don't have any parameter at all. It just executed a get-alias command from .Net code. The second which I wrote in relation with DSL, passes the arguments as parameters to a dynamic block which uses '&' for script block execution. The below code snippet is just for  a reference regarding dynamic script block execution.

---

         private  static  void  RunPowerShellScriptDynamicallyWithParameter(int  inputNumber)

         {

             PowerShell  ps = PowerShell .Create();

             //The format needed for PS to execute dynamic scipr block is 

             //&{param ($paramsAsCommaSeparated);statements;} inputParameters as string 

             string  script = "&{param ($number);if (($number % 2)-eq 0)'}} " +inputNumber.ToString();

             ps.AddScript(script);

             ps.AddParameter("number" , inputNumber);

             Collection <PSObject > result = ps.Invoke();

             //Process logic for resultant PSObjects. 

         }

---

C#/VB.Net objects to PowerShell

There was a small limitation that we can append only string variables / primary datatypes as it uses string concatenation for passing the arguments.To achieve actual object parameter passing in a simple manner you need to define the parameters at the beginning of the script block you want to execute. Then from the .Net code add those parameters.Here is a small code block which validates an .Net FixedDeposit object in powershell code.

---

         FixedDeposit  fdAccount = new  FixedDeposit () { Customer = new  Customer  { Name = "George" , Age = 70 }, Interest = 9.0 };

         private  void  RunPowerShellScriptWithParameter()

         {

             PowerShell  ps = PowerShell .Create();

             string  script = "param($fd); if ($fd.Customer.Age -ge 60) {$fd.Interest=$fd.Interest+.5}" ;

             ps.AddScript(script);

             ps.AddParameter("fd" , fdAccount);

             Collection <PSObject > result = ps.Invoke();

             //Inspect the fdAccount object for changes 

         }

---

In India all the banks give additional .5 interest to senior citisons (above 60) for their fixed deposits. If this rule comes via a script it can be modified at any time.PowerShell scripts can be also applied to the UI elements ie we can modify / control the .Net UI elements from PowerShell

DSL Demo : First C# application which uses PowerShell as DSL

As I promised in my previous post, this post is dedicated to fully cover a practical application of DSL by combining 2 of my earlier posts.

http://joymonscode.blogspot.in/2013/01/what-is-domain-specific-language-dsl-in.html
http://joymonscode.blogspot.in/2012/12/running-powershell-script-from-c-vbnet.html

The idea was to give a kick start sample for C# developers into the world of DSL. But before going into the details I would like to add couple of theory titles.

How to select the DSL (Domain Specific Language)

This is a big question before you select the implementation method of DSL. According to Martin Fowler there are 2 categories of DSL. External and Internal DSL.

Internal DSL means adjusting our existing programming language practice to look more like business friendly. ie using plain English constructs to express the business cases. Linq is a best example for that. Also people are using fluent api for achieving internal DSLs. But here there is a considerable limitation. It is little difficult to add support for runtime changes. You can create the framework as a DSL and the tracks ie other developers can use that to achieve their tasks.

External DSL employees same programming language or different language altogether along with its own execution mechanism to run DSL program. SQL is a good example for that. We write C# code and for communicating with DB we need to write entirely different program called SQL queries. The execution engine and the model of SQL execution itself different from C#.

According to me if you are planning to select a DSL to conquer the constant requirements change better go for external DSL. This allows the end users, ability to change the program or behavior at run time. There is another psychological factor. If we ask your users write in C# or any other main stream programming language, they will protest first. But if you ask them to write in javascript or PowerShell they will be OK.This is because according to the end users, the responsibility of writing main stream code always lies with programmers. Can't believe? 

This is proved last week in our office.We have an installation system which expects so many parameters to configure the application such as DB name, IIS application pool name etc...Earlier this has to be entered by the installation guy (QA people in case of testing) at the beginning of an .vbs file.That same vb script file contains the processing logic after the initial declaration section. They usually complain about this process of giving values in .vbs file. According to them its complicated. Now the process changed little bit. We leveraged MS Excel to enter parameter values and wrote vbs to read the values from there. They became very happy. The complication is washed out by introduction of excel. But did we do anything? Yes we removed a mental roadblock of modifying a program file by QA and other installation people. Still they need to enter DB name, application pool name etc...But at a different place.Now they are happy with the installation procedure.

Another thing to remember is, never go for creating your own parser or lexers to execute that external DSL. Of course if you have research time and lavish time you can go.But as an Architect / Technologist the best way is to leverage an existing well proven mechanism to achieve execution of DSL.

In the book "Wings of fire", former Indian president Dr. A.P.J Abdul Kalam is clearly defining the 'difference between science and technology'. According to him, if we are doing experiments to create something really new and we are not sure whether we can produce positive result, it is called as science. In the other hand, if we are doing something with limited resource and we must produce the positive output in time, its called technology.A scientist can go up to any limit to experiment what he wants. But technologist mostly works in constraints, so he needs to be aware of all the available proven options and use those wisely to achieve the goal.

Creating a new DSL and writing its parser and related things is a science. But leveraging the power of PowerShell, Javascript or any other language which can be embedded in our program to achieve the DSL execution is technology. Now you might have understood why the heading says PowerShell as DSL.

C# Salary calculation using PowerShell as DSL

In the previous post I had told about a salary calculation scenario where we can apply DSL to solve the requirement change issue. Basically requirement change is the nature of software engineering rather than  issue. I am not good in explaining theories so I managed to create a C# program which uses PowerShell to accomplish the salary calculation based on dynamic logic written in PowerShell.

Why PowerShell

There is no special interest on PowerShell. You can select any scripting language such as Javascript, Lua etc..Only constraint is the scripting language should support embedding into C# application or there should be some methods to execute the script from C# and communicate back and forth.

The reasons for selecting PowerShell are

• Its becoming more and more popular like .bat file
• Easy to write programs using the pipline support.
• Recently I took a PowerShell hands on training session to a selected audience in our company :)

About the sample

Few words about the embedded sample. Its a simple program to demonstrate how to visualize the working of DSL. This is because most of the C# / VB.Net programmers might have heard about DSL but didn't experienced. It uses simple PowerShell invocation method from C#. In the formula textbox you can enter any PowerShell script snippet which returns a double value. The values for Basic, HRA & DA are available to the PowerShell context as variables $basic, $hra & $da respectively. Try changing the PowerShell code to calculate total salary by considering a 5% basic as metro allowance.

Happy coding...

What is Domain Specific Language (DSL) in software development

Wish you a very very happy new year to all.Let god make all your ventures a great success.

To me, its more like a technical new year. Last year the main plan was to get married. This year needs to improve more on application architecture side of software engineering. OK. Leave that with me.Lets get into the topic.

It took roughly 2 years to realize that we are using DSL in our project. I don’t know, why I didn't realize it in the first 2 years of project in which I was fixing DSL issues and adding more features to that. Reasons may be

• In the first 2 years I was / had to concentrating more on the deliverable and the technical (WPF,Silverlight,WCF,Linq etc…) aspects of project rather than its logical architecture design and now I am more focused on Architecture.
• The trainers (we had few in the beginning) didn’t tell us that we are using DSL in the project. Or nobody thought of the word DSL even though it is using in the project. (I had seen people using Singleton pattern without knowing the technical term ‘Singleton’)
• There are 2 types of developers in the project. I would say application creators. One group is hard core .Net developers who work on the framework using VB.Net and other team uses a tool to create business screens ,its validation and the business flow.If I spent some time on thinking what is the difference between these 2 groups, I would have realized that we are using DSL in the project.

Its all fine.Now lets see what is DSL in programming ? According to Martin Fowler “DSL is a language for solving a specific problem rather than general purpose language which can aim at any problem.It’s presence is there from the beginning of computing”.The examples are SQL - which is aimed to manipulate the data storage domain, Excel formulae – Aimed to mathematical domain, CSS etc…More details on wiki

Are we using DSL in our day to day projects?

Yes of course .All of us uses DSL in our daily programming. Just think about the below scenarios and confirm yourself that you are using DSLs.

• The framework you are created to solve the problems in your domain. The track components call the framework to fulfill business need.
• The business layer methods you are writing to meet specific business requirement.
• The configuration sections which are specific for business domain.

All the above scenarios are aiming to a particular domain and it’s execution sequence can be altered or configured. A language doesn’t needs to be with line numbers such as VB or enclosed in curly braces like C family languages. “Language is a medium to communicate”. By changing a configuration entry you are talking to the computer.Isn’t it?

Importance of DSL in Application development

Before we see what is the importance of DSL we should know what is special in software development which is not in other engineering branches.

Software development v/s other Engineering_disciplines

Its simple. As everybody knows the major difference is ‘constant requirement changes’. A ship construction or bridge plan will not change, if its on half way. But the software does. Can we say no to those changes? Never. Because if we say no, the project will go to another vendor and we had to look for another job. Can we collect all the requirements from the users and document the same before we start development? 80% of requirements we can collect.But the chance to collect 80% requirement is only 50%. We cannot blame anybody as the users don’t know what they want from software or they don’t know what software can do to make their life easier because they are not familiar with computerized system of their work.

If we take a civil engineering graduate, he learns how to construct a bridge in the class room and the same can be applied in real time when he works for a company project. I agree that there can be very slight variation. But majority are same. But take a look at software engineering graduate. He learns how to find out factorial of a given number during college days and he is assigned to create a site which needs to serve 1000hits/second. This is the case in almost all Indian institutions. No idea about foreign institutions.

The same applies to the business cases as well. The business of civil engineering will be same always or atleast engineer is familiar with the terms. But in case of software its is different. He needs to solve the issues of accounting, auditing or insurance and make their life easier without any prior knowledge in the business domains. Yes there are business analysts who are supposed to bridge the gap. But I don’t know how far they are good in bridging. May be I have never seen a real BA. All the BAs I have seen in may programming life are just requirement writers who write down requirements from users. They are not requirement creators. I strongly believe that only requirement creating BAs can bridge the gap between original domain expert users and pure programmers.

How to tackle the requirement change scenario in development

Can we solve all the above said issues using a magic word DSL? No.We can solve the first one and partially the last one. The solution to second problem is with institutions changing their course structure and how interested the student in knowing about industrial software development. Ok let’s concentrate on how to solve the first and last problems using DSL.

Solution – Convert your software users to programmers.

This sounds crazy.Are they going to write C# code?
No.They are going to write DSL programs

Is it practical ?
Yes. It is and we can see so many examples around. By introducing formulae and macros Microsoft excel made so many programmers. By writing cell formulae and macros aren’t the accountants and other general people became programmers? Yes they are.

May I go to users and tell them you need to learn programing?
It depends based on the level of your clients. In most of the scenarios you cannot tell them that they need to program. You need to use the words customization, configuration etc…When client reaches a mature state you may tell that there is a DSL which they can use to modify the application.

So what is the magic : Whenever you had to create / design software, model a domain language and program using it. Thanks to my former colleague Praseed Pai for the sentence.

Example - DSL in salary calculation scenario

Lets look at a real time scenario for a normal developer other than taking scenarios like excel macro .

First we need to see, how it is working without DSL. You first met the customer who is located in Kochi .He told the formulae to calculate the salary according to the on going process / circumstances.  And the formulae is

Salary=Basic+HRA+DA (It’s a small company .So no PF and other factors)

You created the software in C#.Net and the technology used is windows forms . All went well and got a good payment.

After some months Kochi became a metro and your client needs to add one more component into the salary called “Metro allowance”. Client again approached you to modify the software to add the new salary component. You modified the software and got payment for that too. Client was able to reduce (I don’t want to use the word fired) work load of 2 people in the administration department after introducing your software so that they were able to concentrate on growth aspects of company and it grown to a bigger size.

The next change came when they grew. In India if the number of employees is greater than 20, (don’t know the exact number) companies needs to give Provident fund (PF) to the employees. They approached again and this time the change is little more complicated. PF needs employer and employee contribution which leads to calculation of gross salary, deduction and net salary.

From here story can be diverted in 2 ways. You charged more by saying that, we introduced a configurable salary calculation technique so that you can enter the formulae in a particular text area and the system will calculate based on that. For example in the formulae you can refer the Basic pay using _BASIC_ . Obviously client became happy and as they are growing, they decided to create a web site and gave the contract to you. Your client is growing and also you. You will be contacted someday by other companies to purchase the same software. There is no best advertisement than your customer’s.

The other kind of the story can be like this. You charged same or little more and delivered the support for PF. Client is satisfied with the change. But they started thinking, how their competitors and other companies calculate salary. One day they will hear from others that “you know one thing we are using a salary calculation software which is configurable .There is no need to call software guys for formulae changes”. Obviously next time client will purchase that software or ask you to convert existing one in a configurable way. If you say no, that is the end. Your client will grow definitely but without you.

How was the configurable salary calculation happened? Its nothing but a DSL. You modeled / created it knowingly or unknowingly. _BASIC_ is part of that language.

Can I get a sample which uses DSL in C#

Its all good. Theories are always good to listen or sleep. How can I use this in real time as a .Net programmer ? What are the technologies I need to learn? In what language I can write DSLs? Any example ? Am I supposed to create a language and its parser, compiler etc…!!! It seems people are not suggesting that as it involves more effort.  Also when I googled people were suggesting to embed Lua, use Antlr to create parsers,define xml schema to write DSL and .net code to evaluate xml etc…How to pass my domain objects to the DSL code? How to execute the DSL?

Hopefully you can expect a post which answer the same next week same day.With the salary calculation sample of course.