Tuesday, September 27, 2011

How to know when .Net GC is working

One of my previous posts I was explaining how to learn the working of JIT during an application run. That gives us actual feeling of how the concept if working in the real world. If you are interested to know how the other concepts are working you can monitor using the windows tool called perfmon.

Perfmon.exe can be found inside the system32 folder in Windows installation


You can easily open the application by just typing perfmon in the search box of start menu.

Using Perfmon to monitor the .net Activities

Perfmon is a great tool and its not possible to cover all the aspects of it in this post. Just google for its full usage scenarios. I am just concentrating on how to use it to monitor when GC is working for our application. You can use the same method to know when the jitting happens.

Ok. Lets open the perfmon and watch what are the options in it.There are PerformanceMonitor, ReliabilityMonitor, Data collector sets and Reports.We are interested in the performance monitor where we get the status of GC using a counter.What is a counter in perfmon?Counter is a criteria or specification what to monitor which can be added to a graph to see its current value. By default the processor time counter will be available there. Below are the easy steps to add a counter for monitoring GC of an application.

Goto Performance monitor->Click ”+” button at top –>Expand “.Net CLR Memory” in the available counters->Select “# Gen 0 Collections”->Select your application in the “Instances of selected object” section->Click on “Add” –>Click “Ok”

Remove the processor time counter for easy monitoring. Now the graph will reflect the change when GC comes into picture. For learning purpose better put a button to invoke GC using GC.Collect method .So that you can see the counter increments when you click on that button.

When you add the counter you might have noticed so many counters available there .Play with those and conquer the technology with its inner working.

Tuesday, September 20, 2011

Learning Objective-C Setting environment & HelloWorld.m

Couple of weeks back I happened to meet one of my relative in a family function who is in college studying computer science. He is a tech kid who expected so many things from me when I told I am working in computer field for the last 6 years. He is more interested in creating his own computer game rather than playing .When in college people might not have selected a technology path and will be having knowledge in all most all the areas such as Embedded Systems,.Net,Networking,Java etc...He was not an exception and he started shooting his questions when he met a 6 years experienced professional. The conversation (mainly his doubts) touched so many areas from creating small web sites to programming artificial intelligence systems and controlling device through parallel port to creating a robot.

I was able to survive on the programming topics related to assembly language,C,C++,Java & .Net and how the computer functions internally at the register level. But really failed on Obj C used in iPhone,iPad, Symbian in Nokia, embedded systems and robotics. Finally he asked me “What are you doing in your company related to computer science?” .I had to admit that I am developing a highly generic .Net application for executing workflows which is now targeting auditing workflows. Even I don’t know the auditing business because there is another application to author auditing workflow which is carried out by business analysts. One more thing is we are programming one level up from the real machine and OS .ie we are talking /programming to the .Net and .Net will talk to the machine through OS to carry out the tasks.

The conversation stopped there. But when I think about that I feel there are so many things in high level language club which I don’t know. Learning robotics and programming for embedded systems are not at all in the scope at least now. I applied filters again and came to 3 topics VC++,Android & Obj C. Learning VC++ is delayed because I didn’t get a chance to work. It will not be a problem, if I get a chance. Android uses Java which is again .Net like language. So the remaining is Obj-C which I never tried. Yes my next is Objective-C.

Why should I learn Obj C / benefits of Obj-C
My first reason is the popularity of Apple products and this is the way I can make use of their framework.There are so many iPhone clones. But they cannot get into an agreement that which is the real clone. So iPhone is still the king .As everybody knows Cocoa is the framework to program iPad and iPhone and it need Obj-C. When I started ObjC the usage of class got my attention on the first day itself.We need to have an interface and implementation when we write a simple class itself. We need to share that interface only when we distribute that class. There are lot more reasons when we look at the language level.If you are interested please check the below links

Starting Objective-C
As everybody when we start something fresh we look for teachers. Likewise I checked with the iPhone team in my company .Its 2 new joiners fresh out of college with one Mac and one iPad who got trained in WPF and Silverlight by me .According to them XCode is the only way to execute Objective C and now its licensed and it needs a MAC machine. My upgrade request toWin7 and if possible i7 is pending for months.In this situation asking MAC machine from the company to learn Obj C is not at all a question.Again I am alone and only way is google. I could see couple of ways which helps us to learn objective C in windows. Let’s start with  GNUStep which I feel the simple way.

Setting up environment in Windows
You can get detailed instructions and download links from the below sites.

According to my experience you need to install 3 things in the given order

  • GNUstep MSYS System
  • GNUstep Core
  • GNUstep Devel
This will give you the shell which can be opened by StartMenu->Programs->GNUStep->Shell

Compiling & Running first ObjC program
Just write a small C program to output HelloWorld which can be compiled in normal c compilers or through visual studio.Don’t add any ObjC features such as classes.Below is a small sample.

#include <stdio.h>

int main(void)
printf("God is Love\n");
return ;
void Foo()
printf("Inside Foos\ns");

I normally starts with “God id Love” rather than “Hello World”.The extension for objective C programs is dot(.)m.Save the above program in helloworld.m.Now we have the sample. Then compile using gcc  compiler which will give the .exe file.

gcc helloworld.m

If the environment variable is not set properly this may cause some errors.In that case set the environement variables properly. It will be the PATH variable in the environment setup.Successful compilation will give an .exe file. Now execute that exe same as normal exe. Now we are in a position to learn Obj C.Next time other language features such as loops ,select etc…Then go to class and objects.

Wednesday, September 14, 2011

ICorProfilerCallBack and return to unmanaged code

As a .net developer if you just want your code to be run by the .net runtime and get your work done please don’t read this further.This is for the developers who really want to know what is happening inside the computer when they run a .net application. After reading you may think, why we need .net which is managed by somebody else and we need to code within it’s boundaries rather than exploring the system’s capabilities.So be careful.

Practical feeling v/s theories

From my childhood days itself, I have a habit of practically feeling things before confirming the theories.When my parents said there is shock so don’t touch the electric wire ,I didn’t believe.But realized when I touched myself.(Its 250volts in India).I am sure that ,this nature will be there with many people. The same problem appeared when I entered into the computer programming world. I started programming in my school days directly in the BASIC language where it was really hard for me to write program because I don’t know how this T.V is going to add 2 numbers which I am entering through keyboard. The teachers were simply saying computer will identify your command and process accordingly.But again how it knows how to manage carry and all.

All these questions were answered when I learned about the logical gates in digital electronics during the second year of diploma.Then I learned how these gates techniques can be leveraged to 8085 processor instructions which I executed in a small kit which has 7 segment display .There was no software simulators for 8085  available at that time for us.Third year I wrote and executed assembly instructions in an actual computer by compiling that into .exe.That put me in a stage to think about a compiler or interpreter which can understand a high level language and convert to machine instructions.The high level languages which was confortable for me was C,C++& VC++ in which we can easily plan how our code is going to execute in the physical machine. JAVA and .Net I learned as backups.

Things will not be always good for us.When I came out of the college the world needs .Net and Java programmers than traditional unmanaged coders. Since the bread and butter is important than the passion, I started with C# and .Net since it allows me to write C++ type language in my favorite development tool Visual Studio.But the same issue came there. How my code is being executed by the .Net?

As like any other .net fresher in India, my starting days were hard. So many things to do.No time to think whether its logical or not.That compels people to swallow the theories like how they believe in God. I had to believe that there is something called IL code to which we are compiling our source code which is machine independent. This helps us to write code in different languages such as VB.Net and C# in a single solution.It was really interesting for me.

Also theories said there is a garbage collector which will collect all the free memory so no need to worry about the memory management while coding. That’s great,but I got so many memory leak issues which I was not able to understand at that time. As a developer support engineer I was not supposed to fix the issues, only forward to the development team.

Another doubt was when we double click on the exe how the .net runtime takes control of the exe and manage execution of it.In Java when we run we specify Java <AppName>.class which clearly says we are executing Java.exe which is responsible for executing the IL code.

Things became clear when I tackled my developer support job. In free time googles I came to know there is a tool called ILDasm.exe in the .net framework which is present in my machine itself using which I can see the IL code.This made the concept of language interoperability strong in my mind.ie I realized myself that all our code which is written in VB or C# will compile into this same format.Or even we can write code in IL itself and compile using ILAsm. Then I thought about reverse engineering the IL to C# since its in a readable form which lead me to a nice tool called Reflector to reverse engineer the .Net code. This tool also showed me scenarios where the compiler optimize our code because some code which I wrote came in different form on reflection.

Again the season changed to busy. Microsoft introduced new technology called WPF for presentation purpose.As a control vendor my company also jumped into WPF where I was coordinating the development team and the release. As a .net engineer I had to learn one more technology keeping so many queries in mind.WPF introduced more queries into my mind like where is the WndProc function in WPF?,how the DP is working internally,who is invoking the PropertyChanged event in INotifyPropertyChanged,Is WPF really making use of my graphics card etc…But WPF helped me in one way which is nothing but to understand the working of GC. Since WPF comes with memory leaks if we use bindings ,I got opportunities to explore the .net process memory using another nice tool called WinDbg which showed me what are gen0 objects and when they goto  gen1 and gen2 also what is LOH and why its getting fragmented which leads to OutOfMemory exception.

But still so many queries are left.When a type is being initialized? if our assembly contains so many types will it decrease the performance due to loading time? How the .net runtime takes control when we double click on the .net exe? When a function is being JITted? When the GC starts its operation? etc… and most of the queries were answered when I work with Microsoft for my current company.

Working with Microsoft people

As I mentioned in some of the previous posts ,I got an excellent opportunity to work with Microsoft in their Charlotte,NC campus which leveraged my knowledge about the internals of .Net .Especially working with Wade Mascia who codes the DebugDiag was really wonderful.He introduced me one interface called ICorProfilerCallBack which has enough functions which will be called by the CLR when it performs its different operations.

Using ICorProfilerCallback in managed code

But fortunately or unfortunately we cannot implement ICorProfilerCallback in managed code such as C# or VB.net.I am looking for anyways to do so which I know not logical since writing profiler in managed code to profile managed code makes no sense.We are going to see when our method is going to be JITted and if we write the profiler in managed code how CLR can call the profiler callback method which needs JITting itself. That means we are back to unmanaged world.

Implementing ICorProfilerCallback

Its my continuous 6th year, I am writing  managed .net code.That made it difficult for me to write the VC++ code in a day.So as a normal developer, I googled for the code. There was not much luck except one article in CodeProject.It was simple and has basic functionalities of ICorProfileCallback interface like logging method calls.So I took that code and started scratching it to add my requirements.Hope scott_hackett will not mind, I am modifying his codeSmile.

When we implement the ICorProfilerCallback, we need to know about one more interface which is ICorProfilerInfo which is used as a helper when we deal with the profiler such as to get the method name from the ID and all.We will get an object of ICorProfilerInfo as parameter in the initialize method of ICorProfilerCallback interface.The Initialize method is called by CLR.

ICorProfilerCallback2 & ICorProfilerInfo2

These are the types introduced in .Net2.0 which contains some more functionalities. These are inherited interfaces of their counterparts in  old framework versions. So to get the new features cast the argument coming though Initialize method to ICorProfilerInfo2.

Steps to implement ICorProfilerCallback

For detailed description please see the original article.Below is a summary

  • Implement the interface ICorProfilerCallback /ICorProfilerCallback2 in a VC++ Com project as normal.
  • If you want your profiler to be recognized by the CLR and call the profiler methods,you need to set environment variables appropriately.
  • Simple implementation of ICorProfilerCallback will not give us the call back to our methods.For that we need to register which events we need using the SetEventMask method of ICorProfilerInfo object.Do that in the ICorProfilerCallback::Initialize itself.It accepts a DWORD and each bit controls one callback feature.
  • Now our methods in the implementation class will get called based on the mask we set except the MethodEnter and Leave.For that we need to hook separately using the ICorProfilerInfo::SetEnterLeaveFunctionHooks method.I don’t know why method execution callbacks need a separate way.

About the attached sample

The sample which I got from code project article doesn’t filter namespaces based on our needs. What I wanted was to filter how my application is getting JITed and the method calls.So I modified the ProfilerLauncher to accept namespace and passed that namespace to profiler through the environmental variable ‘NS_FILTER’. Environment variables are the only one way we can communicate to the profiler from outside world. Files are there, but I don’t want to use.

Also modified the profiler to log based on the namespace and added logging mask for the JITting events.As I told earlier its my second visit to C++ after 6 years.Things seems changed / I forgot C++ which made me write code like a fresher.So don’t expect a professional style coding in the sample.

Running the sample

Before running my modified sample I would recommend running the sample attached in the code project article which gives you an idea how many methods are being called when we run a simple win forms application. There are 3 files needed to run the sample.

  • DotnetProfiler.dll :This is the unmanaged profiler.
  • ProfilerLauncher.exe : This is the runner wrote in .Net which accepts target .net exe path and the namespace to filter.Then it create a process with required environment variables and starts execution of the target .net application.
  • Helloworld.exe : Just a .net application which we are going to be profiled.Put your application instead of this. Don’t forget to enter the correct namespace in the launcher screen.

This logs the profile information in a text file in the same folder.Download the sample from here.

Tuesday, September 6, 2011

Creating non zero based arrays in .Net

Last week when I was hunting for .Net performance tips and an array related tip got my attention.It says “Non-zero based arrays will reduce performance”.ie the current default lower bound of .net array is 0.If we change this lower bound of array it will cause performance penalty.But luckily there are no places in our project where developers used non-zero based arrays. I should say thanks to .Net architects in Microsoft than our developers, for making the creation of non-zero based arrays bit complicated. According to my current knowledge, using the normal declaration syntax of array, its not possible to create non-zero based arrays. The only way now is to use the Array.CreateInstance method.Look at the below code snippet.

Array arrayObject = Array.CreateInstance(typeof(int), new int[] { 5 }, new int[] { 2 });

This creates single dimension integer array of size 5 with lower bound as 2.ie you need to access the array as arrayObject[2] to [6] not arrayObject[0].

Normal array and Array.CreateInstance

As you know all the arrays are of class type Array which gives them the ability of iteration and all. We can create array using the CreateInstance method, just like how we create normal array with square bracket.If we display the type of Array object we can see its exactly same as the type of normal array object with [].We can cast the output to normal array object and access the elements using [].Look at the below code it just creates an array with zero as lower bound and access the element at position 0.

private void TestBase0Array()
Array arrayObject=Array.CreateInstance(typeof(int), new int[] { 5 }, new int[] { 0 });
int[] intArray = arrayObject as int[];
if (intArray != null)
Console.WriteLine("TestBase0Array(): ArrayCreateInstance() succeded for base 0, Array type :" + intArray.GetType().FullName);
intArray[0] = 10;
Console.WriteLine("TestBase0Array():Write & Read from array [0] - " + intArray[0]);
catch (Exception ex)
Console.WriteLine("TestBase0Array():Not able to access array");
Console.WriteLine("TestBase0Array(): Not able cast to int[]");

Type of non-zero based arrays

When we come to arrays who’s lower bound is not zero, the type will be different. it will contain a ‘*’.If its an integer array the type will be ‘System.Int32[*]’. This means we cannot cast the output of Array.CreateInstance to our normal array object which is declared using [].(But I am not sure whether there are any hidden methods to cast to normal array).So we cannot access the array elements using [].Then the final option is to use the SetValue and GetValue methods of Array class.Look at the below code.

private void TestBase2Array()
Array arrayObject=Array.CreateInstance(typeof(int), new int[] { 5 }, new int[] { 2 });
int[] intArray = arrayObject as int[];
if (intArray != null)
Console.WriteLine("TestBase2Array(): ArrayCreateInstance() succeded for base 0, Array type :" + intArray.GetType().FullName);
intArray[0] = 10;
Console.WriteLine("TestBase2Array():Write & Read from array [0] - " + intArray[0]);
catch (Exception ex)
Console.WriteLine("TestBase2Array():Not able to access array using []");
Console.WriteLine("TestBase2Array(): Not able cast to int[].Array Type :"+arrayObject.ToString());
Console.WriteLine("TestBase2Array(): Lower bound :" + arrayObject.GetLowerBound(0).ToString());
Console.WriteLine("TestBase2Array(): Length :" + arrayObject.Length.ToString());

arrayObject.SetValue(10, 0);
Console.WriteLine ("TestBase2Array(): Element at 0"+ arrayObject.GetValue(10, 0).ToString());
catch (Exception ex)
Console.WriteLine("TestBase2Array(): Not able to access element at 0.Trying 2nd element");
arrayObject.SetValue(10, 2);
Console.WriteLine("TestBase2Array(): Element at "+arrayObject.GetValue(2).ToString());

dynamic keyword and non-zero based arrays

Its really difficult to use 2 methods to access an array for a developer who would like to access the array as it is.So I tried assigning the output of Array.CreateInstance to a dynamic variable and used the square brackets.It never gives a compilation error.But at runtime it fails with an exception “Unable to cast object of type 'System.Int32[*]' to type 'System.Int32[]'”.

private void TestBase2ArrayUsingDynamic()
Array arrayObject = Array.CreateInstance(typeof(int), new int[] { 5 }, new int[] { 2 });
dynamic intArray = arrayObject;
intArray[2] = 10;
Console.WriteLine("TestBase1ArrayUsingDynamic():Write & Read from array [0] using dynamic keyword- " + intArray[2]);
catch (Exception ex)
Console.WriteLine("TestBase1ArrayUsingDynamic():Exception :"+ ex.Message +
Environment.NewLine + "Not able to access non zero based array even using dynamic keyword!!!");

That means dynamic keyword also failed. We need to use the SetValue and GetValue if we want to use non-zero based arrays.

Non-Zero based arrays and VB.Net

But in VB.Net the story is different.If you are working in VB.Net its all easy.You can use non-zero based arrays in the normal array. I don’t know whether it’s the beauty or curse of unsafe programming.

Dim arrayObject = Array.CreateInstance(GetType(Integer), New Integer() {5}, New Integer() {2})
arrayObject(2) = 10

I don’t think we need to use non-zero based arrays in any of the situations of our normal programming life. One more thing is this non-zero based array creation is not supported by Silverlight. That means if you are developing in common code base which is targeted for WPF,Silverlight and WP7 you will not encounter this scenario.