Monday, July 30, 2012

Not able to compile due to missing temp files

Have you ever encountered an error like below which makes your beautiful C# or VB.Net code as just txt file. I believe if the code is not getting compiling its just like a txt file. You must have seen if you are a frequent user of windows disk clean up utility or CCleaner.

“Visual Studio 2010 “cannot find” C:\Users\[USERNAME]\AppData\Local\Temp\1\.NETFramework,Version=v4.0.AssemblyAttributes.vb “

Please don’t doubt your code when you see problems like this after cleaning your temp files using any cleaning utilities. Why this is happening is because of Microsoft Visual Studio’s usage of temp space to achieve its functionalities. When VSTS cannot find the temp file which was copied earlier it reports errors.

Do a clean and rebuild on your solution to solve this issue. Never doubt your code or your team members like what I did when I first saw this error Smile. Goto the below link to see what MSFT is saying about the same error.

http://connect.microsoft.com/VisualStudio/feedback/details/515041/unable-to-open-module-file-when-an-existing-solution-is-loaded

Monday, July 16, 2012

Facts about yield keyword.

When I take interview for SE(Software Engg) or above, I normally ask about less frequently used keywords such as yield to asses about candidate’s C# language skills. A normal person who reads theory will start talking like Yield is something we can use with for each statements . But when the next question arrives they will go into mute and the question is “What is the difference in returning a List<T> and using yield return?” .If they had used yield they will say like we can have more than one yield returns but List<T> can be returned once. But is that the only difference?


Some facts about yield
  • Introduced in .Net 2.0 with the intention to make the iteration easier by eliminating use of IEnumerator derived class.
  • Yield can only be used in methods or property get assessors which are returning IEnumerable,IEnumerator or their generic variants.
  • It is not a .Net runtime feature .It is implemented at the language level which will be converted to IEnumerator derived implementation during compilation.ie you can’t see the yield keyword, if you open the assembly in reflector.
  • Usage of each yield keyword adds one compiler generated private class to your assembly.
  • The generated class will have implementation of IEnumerator and most probably you can see a goto statement in its MoveNext method.
  • There is no one to one relation between for…each and yield. ie you can use for…each without yield and yield without for…each
  • Yield depends on IEnumerable but IEnumerable is not depended on yield.
  • There is a usage “yield break” to stop the returns.
  • Yield allows you to write a C# function containing a code path without return statement.
  • VB.Net has got the yield support recently with VB 11 with .Net 4.5
  • Cannot yield return from try {} block if it has a catch block. catch and finally blocks never allows to write yield return.
  • Cannot be used inside anonymous methods or lambda expressions.
I don’t think explanation is needed on each and every item listed above .So let me go over some items.


History of yield keyword
Earlier if we wanted to iterate something we had to implement the IEnumerable interface which will return an IEnumerator for the iteration purpose. IEnumerator interface has some members named Current,MoveNext to achieve the for…each functionality. Look at a sample implementation here .It is little difficult.Isn’t it? This caused the invention of yield keyword which makes the iteration easier.

Dependency
Yield is not a dependency for anything in the C# language. Even if there is no yield keyword we can accomplish all the tasks. But think about IEnumerable. If it is not there for…each and yield cannot exist.


Why goto comes into generated code?
As I told earlier, the yield is not a .net framework /  runtime feature. It’s a C# & VB language feature which will be converted to IEnumerator based equivalent after compilation. This means you cannot expect this keyword in other .Net compatible languages. Sometimes if you look at the generated IEnumerator derived class you can see a goto statement. When it is coming ? Lets take case by case. The below method returns 2 integers using yield.
private IEnumerable<int> GetFirst2Integers()
{
        yield return 0;
        yield return 1;
}



If you open the assembly in reflector you can see a compiler generated private class GetFirst2Integers which implements IEnumerator and the MoveNext method will look as follows.

private bool MoveNext()
{
    switch (this.<>1__state)
    {
        case 0:
            this.<>1__state = -1;
            this.<>2__current = 0;
            this.<>1__state = 1;
            return true;

        case 1:
            this.<>1__state = -1;
            this.<>2__current = 1;
            this.<>1__state = 2;
            return true;

        case 2:
            this.<>1__state = -1;
            break;
    }
    return false;
}
 



No goto statement.Now lets involve a parameter of method into return logic.Say we pass a startFrom parameter.

private IEnumerable<int> Get2Integers(int startFrom)
{
    int i = startFrom;
    while (i < startFrom + 2)
        yield return i++;
}



This produces the MoveNext method with goto.

private bool MoveNext()
{
    switch (this.<>1__state)
    {
        case 0:
            this.<>1__state = -1;
            this.<i>5__a = this.startFrom;
            while (this.<i>5__a < (this.startFrom + 2))
            {
                this.<>2__current = this.<i>5__a++;
                this.<>1__state = 1;
                return true;
            Label_0055:
                this.<>1__state = -1;
            }
            break;

        case 1:
            goto Label_0055;
    }
    return false;
}



I was not able to conclude on what are the exact scenarios which will put goto statement in the generated code. But most of the cases processing the method parameters cause a goto statement.
http://blogs.msdn.com/b/oldnewthing/archive/2008/08/12/8849519.aspx

Returning infinite times
So what are the actual differences .One we saw is the ability for a method to have multiple returns. Another is the ability to return infinite times. For example we can have a method which can yield return the odd numbers infinitely. The process will be stopped only when the for...each iterator stops by break / return keyword.

Delayed return
If the method which returns using yield takes a lot of time to compute the return value ,it is advisable to use IEnumerable with yield as the return value can be processed by the for...each loop before the next value is computed. That doesn't mean we are getting any parallel processing. But a chance to delay the return value generation till the previous value is processed. If the for...each thinks that I am satisfied with the current value ,it can even stop the loop. If we are return as List this is not at all possible. So in this way the yield saves lot of time in the generation method.

Some more links

http://msdn.microsoft.com/en-us/library/9k7k7cf0(v=vs.80).aspx
http://msdn.microsoft.com/en-us/library/hh156729(v=vs.110).aspx

Monday, July 9, 2012

Asynchronous programming model in Node.JS

Sometime back I had attended a session conducted by K-MUG and one of the session was ‘How to integrate Node.JS with Windows Azure’. Shiju who is a MVP took the session really good and as the result I started learning the Node.JS technology.I am not the right person to talk about the technology and compare its merits with other existing technologies. But the idea of asynchronous event driven programming really interested me as that is something new,comparing with the existing programming approach.(I had worked with threads,events,callbacks etc…in .Net.But Node.JS seems fully leveraging the async)
Some points what I understood about Node.JS
  • It helps to write javascript at server side.
  • It has or starts its own webserver like IIS and Apache
  • Even IIS can relay the request to NodeJS (node.exe) via IISNode module.
  • NodeJS can be used to develop web sites as well as web services.
  • How Node will handle different protocols / compete with WCF is still confusing me. May be there will be extensions for handling that. Don’t ask why Node.JS need to support .Net specific remoting via net.tcp because I don’t want to write services twice for my external clients and internal clients.
  • Everything except our code runs in different thread.For example all the IO and DB related code runs in different thread. Our code always runs in single thread.
  • The above can be understood by an example of 2 concurrent requests. If there are no IO or other parallelizable requests Node will process requests one by one.But if there is an IO operation in first request ,Node will put that IO operation into another thread and takes the second from the event loop. This gives us a virtual feeling of requests/our code being executed in parallel. This is achieved by using 2 libraries called libev & libeio
  • Better for simple web site / ReST kind of service applications. ie Only if our code which is executing in the single thread completes as soon as possible. If we can route the long running process to another thread such as how IO, DB,Network related operations are performed, its fine.
  • Its production ready and so many big busiest business sites are using it. Check out NodeJS site for the list of big users.
  • Since it uses same language (javascript) the training cost is less and easy for the new developers or even designers who know js.
Ensuring that our code in Node.JS is not running in parallel
This is just a code snippet to prove that our code is not running in parallel in Node.JS. Main reason is Node.JS user code don’t have capability to create it’s own threads. Below is a code snippet which puts a delay of 5 seconds in a normal request. If you hit the URL (http://localhost:8000) from your browser you will see that the response is coming after 5 seconds.
var http=require('http')
http.createServer(function (req, res) {
    var startTime= new Date();
    console.log("Process started at :"+startTime.toString());
    
    res.writeHead(200, {'Content-Type': 'text/plain'});
    while(new Date().getSeconds() < startTime.getSeconds() + 5) {
        // I should have googled for a sleep method.
    }
    res.end('Start time:'+startTime.toLocaleTimeString()+",EndTime:"+new Date().toLocaleTimeString());
    }).listen(8000, "127.0.0.1");
    console.log("Server started @ 127.0.0.1:8000");

The response in browser will be.

Start time:08:46:21,EndTime:08:46:26

Now Open 2 tabs in your browser and hit the same url simultaneously. The result will be

Start time:08:47:21,EndTime:08:47:26

Start time:08:47:26,EndTime:08:47:31

It clearly says that the Node.JS is single threaded from our application point of view and it needs to wait to complete current user code to take another request.If my delay code was to fetch contents from file it should have processed differently as the I/O code will go into another thread.Think about ASP.Net.If you write the same code in ASP.Net and hit simultaneously from 2 browser tabs, the results would be

Start time:08:47:51,EndTime:08:47:56

Start time:08:47:52,EndTime:08:47:57

Ok. What about seeing the Node.JS code execution in parallel.I am altering the delay code to execute a sql query in SQL server. Query is nothing but a WAIT FOR DELAY statement.

var http=require('http')
var sql=require('node-sqlserver')
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.write("Process started at :"+new Date().toString()+'<br/>');
    var conn_str = "Driver={SQL Server Native Client 10.0};Server=(local);Database=master;Uid=sa;Pwd=Password!"
    sql.open(conn_str, function (err, conn) { 
        if (err) { 
            res.end('Error in DB opening'+err.toString());
            return; 
        }
        conn.queryRaw("WAITFOR DELAY '000:00:10'", function (err, results) { 
            if (err) { 
                    res.end('Error in query execution' + err);
                    return; 
            }
            res.end('DB execution completed @'+new Date().toLocaleTimeString());
        }); 
        res.write('DB Execution (WAIT FOR DELAY "000:00:10") started @ '+new Date().toLocaleTimeString() +'<br/>');        
    }); 
}).listen(8000, "127.0.0.1");
console.log("Server started @ 127.0.0.1:8000");

If you are not familiar with how to setup SQL Server with NodeJS please refer the below link.
http://weblogs.asp.net/chanderdhall/archive/2012/06/19/microsoft-sql-server-driver-for-nodejs.aspx

After this I tried hitting the url from different browser tabs and I got the below output.

Process started at :Fri Jul 06 2012 20:45:51 GMT+0530 (India Standard Time)
DB Execution (WAIT FOR DELAY "000:00:10") started @ 20:45:51
DB execution completed @20:46:01


Process started at :Fri Jul 06 2012 20:45:53 GMT+0530 (India Standard Time)  
DB Execution (WAIT FOR DELAY "000:00:10") started @ 20:45:53
DB execution completed @20:46:03

Hope you understood the output. The second request was able to get into execution only because the first request entered into SQL execution which happens in different thread. For more details refer the below link.

http://www.quora.com/How-does-IO-concurrency-work-in-node-js-despite-the-whole-app-running-in-a-single-thread

The programming model

All the operations which are to be done after async calls like I/O calls needs to be inside the event handler / callbacks. in simple words nested callbacks. So lets see how to read a file after a sql database call where the file read is depend on the first sql execution result. 

var http=require('http')
var sql=require('node-sqlserver')
var fs=require('fs')
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    res.write("Process started at :"+new Date().toString()+'<br/>');
    var conn_str = "Driver={SQL Server Native Client 10.0};Server=(local);Database=master;Uid=sa;Pwd=Password!"
    sql.open(conn_str, function (err, conn) { 
        if (err) throw err;
        conn.queryRaw("WAITFOR DELAY '000:00:10'", function (err, results) { 
            if (err) throw err;
            res.end('DB execution completed @'+new Date().toLocaleTimeString());
            fs.readFile('joy.txt', function (err, data) {
                if (err) throw err;
                console.log(data);
            });
        }); 
        res.write('DB Execution (WAIT FOR DELAY "000:00:10") started @ '+new Date().toLocaleTimeString() +'<br/>');        
    }); 
}).listen(8000, "127.0.0.1");
console.log("Server started @ 127.0.0.1:8000");

Simple isn’t it? Lets consider one more scenario where there is parallelism in SQL and File operations and another operation needs to be performed after these 2 operations.

But this needs an additional check to ensure that both the operations are completed.ie NodeJS don’t have native beautiful way of handling multiple async callbacks and do operations based on that.So inject our own logic.Keep 2 variables to hold the return state and in the post processing function check the variables. 

var http=require('http')
var sql=require('node-sqlserver')
var fs=require('fs')
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    var bSQLCompleted=false;
    var bFileCompleted=false;
    var conn_str = "Driver={SQL Server Native Client 10.0};Server=(local);Database=master;Uid=sa;Pwd=Password!"
    sql.open(conn_str, function (err, conn) { 
        if (err) throw err;
        conn.queryRaw("WAITFOR DELAY '000:00:05'", function (err, results) { 
            if (err) throw err;
            res.end('DB execution completed @'+new Date().toLocaleTimeString());
            bSQLCompleted=true;
            postSQLnFileReadOperation(bSQLCompleted,bFileCompleted);
        }); 
    }); 
    fs.readFile('joy.txt', function (err, data) {
        if (err) throw err;
        bFileCompleted =true;        
        postSQLnFileReadOperation(bSQLCompleted,bFileCompleted);
    });
}).listen(8000, "127.0.0.1");
console.log("Server started @ 127.0.0.1:8000");
//Accept the http variable if you want to do something specific to output
function postSQLnFileReadOperation(bSQL,bFile){
    if(bSQL && bFile) console.log("Operation after SQL & File read");
}


More links below.
http://raynos.github.com/presentation/shower/controlflow.htm
http://stevehanov.ca/blog/index.php?id=127
http://stackoverflow.com/questions/4234619/how-to-avoid-long-nesting-of-asynchronous-functions-in-node-js
http://stackoverflow.com/questions/5172244/idiomatic-way-to-wait-for-multiple-callbacks-in-node-js

Tuesday, July 3, 2012

Net.TCP in Silverlight from IIS+WAS or ConsoleHost

Background
As usual a requirement came from onsite to one of the development team that they need to host WCF service using net.tcp and access the same from Silverlight 5. A developer spent couple of days but no luck. Then it came to me as I am serving as technical lead. First itself I raised the concerns on using net.tcp with Silverlight as Silverlight implementation lacks security features of net.tcp. Only advantage is the reduced payload size and speed. But there were no clear answer. So started debugging session to correct the sample.

There are so many articles and forum posts about the same with step by step explanation. But if we look closely at the comments, it says most of the people were not able to accomplish the task of setting net.tcp in IIS 7 and call the same from Silverlight. Also we were not able to find any good sample which does this. So planned to post the sample we used for net.tcp to Silverlight 5 communication which I think will help a lot of people.

Hosting net.tcp in IIS + WAS Host and accessing from Silverlight
I don't want to duplicate the contents here as you can get detailed steps with screen shots from the links given below. But would like to highlight the points which we noticed while doing the task.
  • Make sure you have enabled non-http activation in Add remove programs->Turn windows features on or off-> Microsoft .Net framework 3.5.1
  • Make sure the net tcp related listener and adapter services are running in Services.msc
  • The clientaccesspolicy.xml file should be present in the root of IIS with default port 80.
    • http://localhost/clientaccesspolicy.xml must be present regardless you are pointing to c:\inetpub\wwwroot\clientaccesspolicy.xml or some other path
    • The contents should be same as described in any of the links below. Main thing is the tcp port range 4502-4534.
  • The tcp port range which Silverlight supports for net.tcp communication is 4502-4534 .So your service need to use port number which comes in this range. Eg net.tcp://localhost:4503/
  • The port which you are using in the above step must be open in the firewall. Better switch off firewall if you are facing any issue.Once you get it running on firewall and add exception.
  • If you are hosting the services in "Default Web Site" of IIS make sure the edit site->bindings are pointed to the port numbers in the range 4502-4534. eg: 4504:*
  • If you are in Visual studio make sure you are using IIS for running WCF services. Create the IIS web application from Project properties->Web->Use  local IIS web server
  • Make sure the "Default Web Site" and web application in IIS has net.tcp in the enabled protocols section.
  • Turn off the security features of net.tcp in the web.config as Silverlight doesn't support the same .By default in .Net the tcp security will be on. Better make all the security related attribute in the netTcpBinding to "None".
  • A <clear/> tag is advised in the <Service> element in web.config before adding the end points.
  • Try to "Add Service Reference" using the net.tcp://site/service url itself. This will make sure that the server side is correct.
  • In ServiceReferences.ClientConfig use the CustomBinding and inside that binaryMessageEncoding & tcpTransport. There is no security related element in Silverlight config.
Hosting net.tcp  in ConsoleHost and accessing from Silverlight
This is another scenario which I tried along with hosting in IIS and though of sharing the same. Below are the points to note.
  • There is no need to run any additional exe as socket policy server which was there till Silverlight 4 beta.
  • While adding service reference you can use the mex url.eg:net.tcp://localhost:4503/mex
Challenges faced
  • We added the service reference with http:// url which worked fine.But when we tried to call using net.tcp url it failed. So always use net.tcp url to add service reference.
  • We tried to add some security related attributes in the ServiceReferences.clientconfig file which caused more errors. Leave it minimal.
  • There were some exceptions which we cannot catch in Visual Studio.Use the diagnostic trace listeners on System.ServiceModel so that debugging will be easier.For example the below one was never caught by VS
  • We were getting an exception at the server side (service svclog) that there is a protocol exception .We resolved it by making all the security related attributes of nettcpbinding to None. The exception was System.ServiceModel.ProtocolException, System.ServiceModel, Version=4.0.0.0."Stream Security is required at http://www.w3.org/2005/08/addressing/anonymous, but no security context was negotiated. This is likely caused by the remote endpoint missing a StreamSecurityBindingElement from its binding."
Running the attached sample

Download the sample and read the below points about the same.
  • WCFWASService project contains the WCF service .Make sure this web project is running using IIS.
  • ConsoleHost project contains a service which is exposed via net.tcp://localhost:4503/GetData .Goto the bin/debug folder of this project and double click on the ConsoleHost.exe to start the service.
  • SilverlightApplication1.Web is the web host for silverlight application.
  • SilverlightApplication1 is the Silverlight 4 app from which we are going to access the IIS hosted and Console hosted net.tcp services. The url to IIS hosted service is in the ServiceReferences.clientconfig file and the other is hard coded in the method Implementation.TestGetData()
  • If you want to add the service reference of ConsoleHost to any other app use the url net.tcp://localhost:4503/mex
Links which explains step by step
http://tomasz.janczuk.org/2009/11/pubsub-sample-with-wcf-nettcp-protocol.html
http://www.ditran.net/self-hosted-nettcp-wcf-silverlight-4
With screen shots - http://www.silverlightshow.net/items/WCF-NET.TCP-Protocol-in-Silverlight-4.aspx