Monday, December 31, 2012

What is Domain Specific Language (DSL) in software development

Wish you a very very happy new year to all.Let god make all your ventures a great success.

To me, its more like a technical new year. Last year the main plan was to get married. This year needs to improve more on application architecture side of software engineering. OK. Leave that with me.Lets get into the topic.

It took roughly 2 years to realize that we are using DSL in our project. I don’t know, why I didn't realize it in the first 2 years of project in which I was fixing DSL issues and adding more features to that. Reasons may be
  • In the first 2 years I was / had to concentrating more on the deliverable and the technical (WPF,Silverlight,WCF,Linq etc…) aspects of project rather than its logical architecture design and now I am more focused on Architecture.
  • The trainers (we had few in the beginning) didn’t tell us that we are using DSL in the project. Or nobody thought of the word DSL even though it is using in the project. (I had seen people using Singleton pattern without knowing the technical term ‘Singleton’)
  • There are 2 types of developers in the project. I would say application creators. One group is hard core .Net developers who work on the framework using VB.Net and other team uses a tool to create business screens ,its validation and the business flow.If I spent some time on thinking what is the difference between these 2 groups, I would have realized that we are using DSL in the project.
Its all fine.Now lets see what is DSL in programming ? According to Martin Fowler “DSL is a language for solving a specific problem rather than general purpose language which can aim at any problem.It’s presence is there from the beginning of computing”.The examples are SQL - which is aimed to manipulate the data storage domain, Excel formulae – Aimed to mathematical domain, CSS etc…More details on wiki

Are we using DSL in our day to day projects?

Yes of course .All of us uses DSL in our daily programming. Just think about the below scenarios and confirm yourself that you are using DSLs.
  • The framework you are created to solve the problems in your domain. The track components call the framework to fulfill business need.
  • The business layer methods you are writing to meet specific business requirement.
  • The configuration sections which are specific for business domain.
All the above scenarios are aiming to a particular domain and it’s execution sequence can be altered or configured. A language doesn’t needs to be with line numbers such as VB or enclosed in curly braces like C family languages. “Language is a medium to communicate”. By changing a configuration entry you are talking to the computer.Isn’t it?

Importance of DSL in Application development

Before we see what is the importance of DSL we should know what is special in software development which is not in other engineering branches.

Software development v/s other Engineering_disciplines

Its simple. As everybody knows the major difference is ‘constant requirement changes’. A ship construction or bridge plan will not change, if its on half way. But the software does. Can we say no to those changes? Never. Because if we say no, the project will go to another vendor and we had to look for another job. Can we collect all the requirements from the users and document the same before we start development? 80% of requirements we can collect.But the chance to collect 80% requirement is only 50%. We cannot blame anybody as the users don’t know what they want from software or they don’t know what software can do to make their life easier because they are not familiar with computerized system of their work.

If we take a civil engineering graduate, he learns how to construct a bridge in the class room and the same can be applied in real time when he works for a company project. I agree that there can be very slight variation. But majority are same. But take a look at software engineering graduate. He learns how to find out factorial of a given number during college days and he is assigned to create a site which needs to serve 1000hits/second. This is the case in almost all Indian institutions. No idea about foreign institutions.

The same applies to the business cases as well. The business of civil engineering will be same always or atleast engineer is familiar with the terms. But in case of software its is different. He needs to solve the issues of accounting, auditing or insurance and make their life easier without any prior knowledge in the business domains. Yes there are business analysts who are supposed to bridge the gap. But I don’t know how far they are good in bridging. May be I have never seen a real BA. All the BAs I have seen in may programming life are just requirement writers who write down requirements from users. They are not requirement creators. I strongly believe that only requirement creating BAs can bridge the gap between original domain expert users and pure programmers.

How to tackle the requirement change scenario in development

Can we solve all the above said issues using a magic word DSL? No.We can solve the first one and partially the last one. The solution to second problem is with institutions changing their course structure and how interested the student in knowing about industrial software development. Ok let’s concentrate on how to solve the first and last problems using DSL.

Solution – Convert your software users to programmers.

This sounds crazy.Are they going to write C# code?
No.They are going to write DSL programs

Is it practical ?
Yes. It is and we can see so many examples around. By introducing formulae and macros Microsoft excel made so many programmers. By writing cell formulae and macros aren’t the accountants and other general people became programmers? Yes they are.

May I go to users and tell them you need to learn programing?
It depends based on the level of your clients. In most of the scenarios you cannot tell them that they need to program. You need to use the words customization, configuration etc…When client reaches a mature state you may tell that there is a DSL which they can use to modify the application.

So what is the magic : Whenever you had to create / design software, model a domain language and program using it. Thanks to my former colleague Praseed Pai for the sentence.

Example - DSL in salary calculation scenario

Lets look at a real time scenario for a normal developer other than taking scenarios like excel macro .
First we need to see, how it is working without DSL. You first met the customer who is located in Kochi .He told the formulae to calculate the salary according to the on going process / circumstances.  And the formulae is

Salary=Basic+HRA+DA (It’s a small company .So no PF and other factors)

You created the software in C#.Net and the technology used is windows forms . All went well and got a good payment.

After some months Kochi became a metro and your client needs to add one more component into the salary called “Metro allowance”. Client again approached you to modify the software to add the new salary component. You modified the software and got payment for that too. Client was able to reduce (I don’t want to use the word fired) work load of 2 people in the administration department after introducing your software so that they were able to concentrate on growth aspects of company and it grown to a bigger size.
The next change came when they grew. In India if the number of employees is greater than 20, (don’t know the exact number) companies needs to give Provident fund (PF) to the employees. They approached again and this time the change is little more complicated. PF needs employer and employee contribution which leads to calculation of gross salary, deduction and net salary.

From here story can be diverted in 2 ways. You charged more by saying that, we introduced a configurable salary calculation technique so that you can enter the formulae in a particular text area and the system will calculate based on that. For example in the formulae you can refer the Basic pay using _BASIC_ . Obviously client became happy and as they are growing, they decided to create a web site and gave the contract to you. Your client is growing and also you. You will be contacted someday by other companies to purchase the same software. There is no best advertisement than your customer’s.

The other kind of the story can be like this. You charged same or little more and delivered the support for PF. Client is satisfied with the change. But they started thinking, how their competitors and other companies calculate salary. One day they will hear from others that “you know one thing we are using a salary calculation software which is configurable .There is no need to call software guys for formulae changes”. Obviously next time client will purchase that software or ask you to convert existing one in a configurable way. If you say no, that is the end. Your client will grow definitely but without you.

How was the configurable salary calculation happened? Its nothing but a DSL. You modeled / created it knowingly or unknowingly. _BASIC_ is part of that language.

Can I get a sample which uses DSL in C#

Its all good. Theories are always good to listen or sleep. How can I use this in real time as a .Net programmer ? What are the technologies I need to learn? In what language I can write DSLs? Any example ? Am I supposed to create a language and its parser, compiler etc…!!! It seems people are not suggesting that as it involves more effort.  Also when I googled people were suggesting to embed Lua, use Antlr to create parsers,define xml schema to write DSL and .net code to evaluate xml etc…How to pass my domain objects to the DSL code? How to execute the DSL?

Hopefully you can expect a post which answer the same next week same day.With the salary calculation sample of course.

Monday, December 24, 2012

PowerShell to start Lync group conversation and send message

In previous blog post I had described how I wrote code for myself to achieve a personal task. As people says “Code for yourself before you code for others”, I took that opportunity to improve my life, before I go to solve problems and improve other’s life with my code. But did my problem solved with that small C# code?
I would say 50/50 .Its really difficult to add one more person on the go. I had to open the solution in VSTS then modify the list and run again. I can add users manually with less time than the time taken for Visual Studio to open. I love VS as developer.But as a user I hate as a user. Again keeping an app.config and editing it is equally time consuming like keeping a script which can also be edited on the fly. So what is best.
I would say script. More general PowerShell which has the capability to seamlessly interact with .net.In simple words its very easy to convert a C# / VB.Net code PS. Here goes the PS equivalent of what I posted last week in C#.
$lyncDLLPath = “C:\Program Files\Microsoft Lync\SDK\Assemblies\Desktop\Microsoft.Lync.Model.DLL”
Import-Module $lyncDLLPath
$client = [Microsoft.Lync.Model.LyncClient]::GetClient()

$conv = $client.ConversationManager.AddConversation()

foreach ($contactUri in $contactUris)
$m = $conv.Modalities[[Microsoft.Lync.Model.Conversation.ModalityTypes]::InstantMessage]
$m.BeginSendMessage("Hi guys this is Joy from Powershell,,,Shall we have lunch....", $null, $null);

Monday, December 17, 2012

C# to start Lync group conversation and send message

Lunch with colleagues is really fun. We can get tasty food from different homes and different cultures. Some items we can try preparing in home and also will be sharing tips to increase taste and keep food undamaged etc… 1 PM is the time for lunch and somebody will initiate a group chat requesting the group to assemble in the pantry. Obviously we cannot have lunch with all 300 people so there are some local groups formed. Our group is little big of size around 8 and its little difficult to add people one by one in to group chat.

Around a year back company introduced Lync replacing old Office Communicator. One of the advantage of Lync we found is the client side programmatic support. But the chance to program for Lync, came recently. Installed the prerequisite 2010 Lync SDK and started coding using C# console application. Note that the Silverlight 4.0 sdk referred as prerequisite in some sites, is not needed or at least I didn’t install to program Lync.

There is not much code involved .We can summarize the steps as below.

The code explains everything else.

private static void SendLyncMessage()
//Uri format in my environement works with sip: protocol'
string[] targetContactUris = { "sip:<email1>", "sip:<email2>" };
LyncClient client = LyncClient.GetClient();
Conversation conv = client.ConversationManager.AddConversation();
foreach (string target in targetContactUris)
InstantMessageModality m = conv.Modalities[ModalityTypes.InstantMessage] as InstantMessageModality;
m.BeginSendMessage("Hi this is from C#", null, null);

Monday, December 10, 2012

Running PowerShell script from C# / VB.Net

Recently I had to take series of PowerShell sessions to one of the non-programmers team in my current organization. Actually its the build and installation team who wanted to automate their installation using PowerShell. It was a new experience to me as most of them are non programmers or haven't coded for years. So I started with very basics such as what is programming, conditional branching statements if ... else,switch case ,looping constructs such as while,do ... while, for, for each, functions, passing arguments to functions etc to creating and running dynamic scripts.Actually I would not have been chosen for teaching  basics of programming as I am working as Junior architect. By the by I got promoted as Junior Architect after the last performance review which is a rare designation with very less people around the world. Since the technology is not familiar to most of the programmers I had to be the trainer.

Whether I took session or not has nothing to do with this post. But there is a small relation. When I explained one of my .Net colleague about new PowerShell training to installation team and the power of PowerShell, he told me one statement. "Yes bringing PowerShell into installation is good so that, we can call that PowerShell code from our .Net code". I said "why we need to call PowerShell from .Net because PowerShell is from .Net and it uses .Net code to accomplish the tasks. Why should we go back again to C# .Net to call the script. We can execute the PS scripts directly like an exe". Then he told. "We are .Net devs and PS needs to be started from our code.What is stopping us?"

It was a partial joke .But I started thinking about it seriously .Will there be any benefits, if we call PowerShell from our code? It seems yes. We can leverage scripting to our application. But before that I need to confirm whether it is callable from C# / VB.Net code. Initial google gave me somewhat big code block which is   using RunSpace and all. But it seems we just need to create PowerShell object to execute simple scripts.

Adding reference to PowerShell assembly

You need to add reference to the below assembly to create the object of PowerShell class.

<InstallDrive>:\Program Files\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0\System.Management.Automation.dll

Invoke PowerShell script from C#

It is again a matter of calling the invoke method in PowerShell class. But be sure that you had added the script using the AddScript method available in the same class.

The RunPowerShellScript method

         private  static  IEnumerable  RunPowerShellScript(string  script)
                 PowerShell  ps = PowerShell.Create();
                 Collection <PSObject > result= ps.Invoke();
                 foreach  (PSObject  psObj in  result)
                     yield  return  (psObj);
Simple isn't it.This uses the LocalRunspace to execute the script .ie the local machine where this C# .Net code is running

Calling RunPowerShellScript method

             foreach  (var  v in  RunPowerShellScript("get-alias" ))
                 Console .WriteLine(v);

I am looking for some practical scenarios where we can use this. Hopefully can find one soon.

Monday, December 3, 2012

Regular expression - Replace exact word matched string

Regular expression are really helpful .There is no second thought on it. But as the fundamentals of regex is little difficult to digest for non-computer science graduates and most of the computer programmers are from non computer science background, it is very difficult to see regex in the production code. I don't know about the percentage of non computer graduates in programming in other countries, but in India its high. But there are exceptions as well, some of them spent time for learning these kind of things and applies in their programming life.

Anyway it is not the intention of this post to discuss about non computer science programmers using regex or not. This post will be listing out scenarios where a regular reducing a bulk amount of code

Find out exact word and replace

Lets consider the sentence "During blogging joymon's soul is filled with joy". We need to change the word 'joy' with 'happiness' .If we just use string.replace in .net it willl replace as "During blogging happinessmon's soul is filled with happiness".

         Dim  orgSentence = "During blogging joymon's soul is filled with joy" 
         Dim  replacedSentence = orgSentence.Replace("joy" , "happiness" )

Out put will be "During blogging happinessmon's soul is filled with happiness"

It did the task well but we didn't want 'joymon' to be replaced with "happinessmon".Unfortunately string.replace don't have flexibility to say replace only exact word matches. We can write a big function to achieve the same.But should we really need that as there are regular expression? Certainly not.We can achieve the same easily using regex.The code is as follows.

Dim  orgSentence = "During blogging joymon's soul is filled with joy" 
Dim  pattern As  String  = "\b"  + "joy"  + "\b" 
Dim  replaceWith = "happiness" 
Dim  reg As  Regex  = New  Regex (pattern)
Dim  replacedSentence As  String  = reg.Replace(orgSentence, replaceWith, 1, 0)

'\b' refers to word boundary.For more details on reg expressions visit the below links