Showing posts with label Linq. Show all posts
Showing posts with label Linq. Show all posts

Tuesday, September 11, 2018

Functional Programming - Randomize IEnumerable

.Net has IEnumerable to represent a sequence. Though it is not advertised as a functional helper, we can use IEnumerable to get really clean functional programming in .Net. It has so many methods to manipulate and select elements but it really lacks a mechanism to take random numbers from the sequence. Below is one which gives us somewhat random elements from the IEnumerable sequence.

public static IEnumerable<TResult> Randomize<TResult>(this IEnumerable<TResult> source)
{
            return source.
                Select((sourceItem, index) => new
                {
                    Item = sourceItem,
                    Id = Guid.NewGuid()
                }).
            OrderBy(t1 => t1.Id).Select(t1 => t1.Item);
}

How to use the above?

IEnumerable<int> input = new List<int>() { 1, 2, 3, 4 };
int randomElement = input.Randomize().FirstOrDefault();

As seen in the source the randomization is depended on the GUID generation. If the GUIDs are generated in increasing order the randomization will not work.

The advantage of this method is to randomize as lazy collection.

Nuget support

The above is available as nuget. Below is the URL.

https://www.nuget.org/packages/DotNet.Helpers


Tuesday, February 10, 2015

WCF DataServices - Date Time comparison in OData query

The concept of WCF data service which uses OData is really great. But I doubt whether that is ready to be used in an enterprise production applications. The main reason is its immaturity and lack of support on some Linq queries and operators. One of the major item is lack of joins. I believe OData specification itself doesn't have join. Below is another scenario which is not supported by WCF Data Services.

Requirement - Compare date in database against current date

Lets take and scenario where a MessageProcessAttempt table is exposed via WCF Data service. There is a field called DequeuedTime which tells when a message is dequeued. We need to list down all the messages which dequeued in last 1 hour.

In normal sense we can obtain the list by subtracting the de-queued time from current time and checking if the difference < 1 hour. The query can be as follows.

List<MessageProcessingAttempt> msgs = (from msg
                           in qProxy.MessageProcessingAttempts
                           where (DateTime.UtcNow- msg.DequeuedTime.Value).TotalHours<1
                           select msg).ToList();

But it will fail with below error message
"No property 'TotalHours' exists in type 'Edm.DateTime' at position 56."
Position will change based on the code.There will be no compile time errors. It just fails at runtime. Tried to find the details about Edm.DateTime class in msdn but able to find out except one EdmDateTimeConstant class.So what can we do

Solution - Calculate target date and compare

Find the target date and do comparison rather than finding the difference in the OData query.


DateTime timeToFilter = DateTime.UtcNow.Subtract(TimeSpan.FromHours(1));
List<MessageProcessingAttempt> msgs = (from msg
                           in qProxy.MessageProcessingAttempts
                           where msg.DequeuedTime.Value >= timeToFilter
                           select msg).ToList();

Here the major difference we can notice that the date subtraction is not happening via operator and its not producing the TimeSpan obejct. We can have DateTime.UtcNow.Subtract(TimeSpan.FromHours(1)) in OData as well instead of variable and it will work. This points to the fact that TotalHours property is not there in a special type called Edm.TotalHours which is produced as part of subtraction operator.

This was found by our development team in India and I didn't agree to them in the beginning. I spend around 2-3 hours and could see WCF Data Service is still not suitable for enterprise apps. Thanks to the team for finding the workaround.

Happy coding...

Monday, March 31, 2014

Getting unique records based on property from list without using Distinct() in Linq

The scenario is very simple. We have a collection of Person objects. We want to know the unique PersonTypes from the collection. Below the Person class definition.

    public class Person
    {
        [Display(Name = "First Name")]
        public string PersonType { get; set; }
        public XElement Properties { get; set; }
    }
    public class PersonType
    {
        public string TypeName { get; set; }
    }

I don't think nobody will write a loop to achieve this. The normal solution we may think of is to get the distinct Person objects using the Distinct function in Linq. But it requires an IEqualityComparer derived object to handle the comparison which is employed to find the duplicates by the Distinct function. I have explained about IEqualityComparer in one of my previous posts related to merging 2 collections without duplicates.

        static internal IEnumerable<PersonType> GetUniquePersonTypes()
        {
            IEnumerable<PersonType> personTypesList = _persons.Distinct(new PersonComparer())
                                                                                                .Select(per=>new PersonType(){TypeName=per.PersonType});
            return personTypesList;
        }
   public  class PersonComparer:IEqualityComparer<Person>
    {
        bool IEqualityComparer<Person>.Equals(Person person1, Person person2)
        {
            return person1.PersonType.Equals(person2.PersonType);
        }
 
        int IEqualityComparer<Person>.GetHashCode(Person person)
        {
            return person.PersonType.GetHashCode();
        }
    }

GroupBy to find distinct objects based on entity property

Should we really need Distinct and another class which implements IEqualityComparer? If we group the list based on the property which is key to decide duplicate and build a collection out of the group keys, won't that be same result of Distinct with comparer? Lets try it.

        static internal IEnumerable<PersonType> GetUniquePersonTypes()
        {
            IEnumerable<PersonType> personTypesList= _persons.GroupBy(x => x.PersonType)
                                                                                            .Select(group => new PersonType() 
                                                                                                                { TypeName = group.Key }
                                                                                            );
            return personTypesList;
        }

Is it working? Happy coding

Monday, December 30, 2013

Merging 2 collections without duplicates in C# .Net using Linq

Before Linq there was only one method to join 2 collections without duplicate elements. Loop through the first list and check whether the element exists in second list,if not add element into second list. But Linq makes it easier to achieve this requirement. Its mainly done by using union method. Lets see how a simple integer list can be merged without duplicates.

            int[] list1 = {2,3,1 };
            int[] list2 = { 2, 3, 10 };
            foreach (int number in list1.Union<int>(list2))
            {
                Console.WriteLine(number);
            }

Output
2
3
1
10

By default Union removes the duplicates. Its easy to remove duplicates in a primitive type collection. What about dealing with a custom type such as Employee class with 2 properties EmpId and Name? How do the .Net linq library function Union knows 2 Employee objects are equal? Basically when we say 2 objects are same, one more more of their properties should be equal.Lets see how an custom type can tell the framework about its equality.

Object.GetHashCode()

This is a method which is used to retrieve the identity of an object as an integer value. When we use the union internally its calling this method in our Employee class and if 2 objects return same value the framework consider the objects as same

Object.Equals()

Whenever GetHashCode returns a value which is same as another object's hash code the framework calls this method to confirm the object equality. Equals method gets 2 objects and we can do comparison here and return a boolean value which tells whether the objects are equal or not. If we return true the framework discards the second object from the union result. So our Employee class will become as follows

    class Employee
    {
        public string Name { get; set; }
        public int EmpId { get; set; }
        public override bool Equals(object obj)
        {
            bool isEqual = false;
            Employee emp = obj as Employee;
            if (emp != null)
            {
                isEqual = emp.EmpId == this.EmpId;
            }
            return isEqual;
        }
        public override int GetHashCode()
        {
            return this.Name.GetHashCode(); 
        }
    }

It first checks for the HashCode. In this case the hash code is computed by considering only the Name property. If the hash codes returned are same, the framework calls the equals method and takes decision based on its return value.

In the first look it may seem that why we need both these methods? But if we omit the Equals and use only GetHashCode which returns different values, the framework will not consider it as equal objects. Lets see how this can be used to combine 2 Employee lists.

            IList<Employee> empList1 = new List<Employee>() 
            { 
                new Employee(){EmpId=1,Name="joy"},
                new Employee(){EmpId=2,Name="george"}
            };
            IList<Employee> empList2 = new List<Employee>() 
            { 
                new Employee(){EmpId=1,Name="joy"},
                new Employee(){EmpId=2,Name="mon"} 
            };
            foreach (Employee emp in empList1.Union<Employee>(empList2))
            {
                Console.WriteLine("{0},{1}",emp.EmpId,emp.Name);
            }

Output
1,joy
2,george
2,mon

This is the simplest method. If you are a strict follower of SOLID principle, you can move the comparison part to a different class.Only thing you need is to implement IEqualityComparer interface in it.

    class Employee
    {
        public string Name { get; set; }
        public int EmpId { get; set; }
    }
    class EmployeeComparer : IEqualityComparer<Employee>
    {
        bool IEqualityComparer<Employee>.Equals(Employee x, Employee y)
        {
            return x.EmpId == y.EmpId;
        }
 
        int IEqualityComparer<Employee>.GetHashCode(Employee obj)
        {
            return obj.Name.GetHashCode();
        }
    }
///////////////////////////////////////////////////////////////////////////////////////////
//Testing
            foreach (Employee emp in empList1.Union<Employee>(empList2,new EmployeeComparer()))
            {
                Console.WriteLine("{0},{1}",emp.EmpId,emp.Name);
            }
        

If you want to alter the comparison method in such a way that it should only consider the EmpId, we can have that logic inside the GetHashCode & Equals methods.I have included only those methods below for reference.This works with the earlier simple sample as well where we haven't used IEqualityComparer.
        

    class EmployeeComparer : IEqualityComparer<Employee>
    {
        bool IEqualityComparer<Employee>.Equals(Employee x, Employee y)
        {
            return x.EmpId == y.EmpId;
        }
 
        int IEqualityComparer<Employee>.GetHashCode(Employee obj)
        {
            return obj.EmpId.GetHashCode();
        }
    }

Output
1,joy
2,george        

Happy coding.

Sunday, February 27, 2011

What is Dynamic Linq Expression and creating DynamicMethods

Before going to details of dynamic concepts lets make our understanding about static more clear.Below is a normal method we all learned when we started programming.

public int Add(int a, int b)
{
return a + b + 10;
}
private void Window_Loaded(object sender, RoutedEventArgs e)
{
MessageBox.Show(Add(1, 2).ToString());
}


This is nothing but 2 methods.One is  Add which adds 10 to the sum of the passed arguments.Another is to call the Add method by passing parameters 1,2.The result will be 13.No doubt in that.Lets look at the below code which I will say equivalent of the same but less lines.

private void Window_Loaded(object sender, RoutedEventArgs e)
{
MessageBox.Show(
new Func<int, int, int>((a, b) => a + b + 10)(1, 2)
.ToString());
}


Where is the Add method? You can see it as a lambda expression.The lambda function is executed by passing the arguments 1,2 and displays 13 as result.Don’t confuse on the Func.Its a delegate type not any special keyword.

These are static  : You can see that the execution logic is hard coded at the compile time itself.Or in other words by looking at the code, we can tell that how this is going to execute.Hope the word static is clear.Lets go to the dynamic world.

Dynamic Method

Here the method body is constructed at the runtime.The compiler is not compiling this methos and making any byte code for storing in assembly file. This is implemented through Expression classes available in .Net.

Concepts of Expression

If you had studied the compiler subject in your college days properly, you know that our code is constructed of expressions.If we write a=b in C# ,its an assignment expression.for loop is another type of expression.Try block is again another expression type.In short all the statements which we writing in our code is seen as an expression by the compiler.

That’s fine.Have you ever thought of a scenario where you need to create the code dynamically and compile as a method which should execute on the fly? If yes the System.Linq.Expressions namespace is for you.

Expression allows us to create code blocks dynamically as a expression tree using different types of sub expressions which can be compiled as a delegate object for execution.

We cannot say we are creating C# code ,but the equivalent object structure using classes which are derived from the base class Expression.ie we are creating objects which represent our code statement and relating them as how we write the code.The addition statement (a+b) is represented using an object of BinaryExpression. This has 2 properties Left and Right of type Expression again which are nothing but the parameters(a,b) to the + operator. While evaluating it behaves same as how our normal code runs.

using System.Linq.Expressions

This namespace contains a bunch of classes which helps us to create language expressions through code.The expression types include ParameterExpression, ConstantExpression, TryExpression and even the LambdaExpression. While creating your code expression tree use the appropriate classes and connect them properly as how you write code and finally create the LambdaExpression.LambdaExpression has the Compile method to get the delegate out of it. Execute the delegate to execute the created expression code.

Below is the Expression equivalent of the above static code.



private Delegate GetExpression()
{
ParameterExpression p1=ParameterExpression.Parameter(typeof(int),"a");
ParameterExpression p2=ParameterExpression.Parameter(typeof(int),"b");
ConstantExpression c=ConstantExpression.Constant(10);
BinaryExpression be1=BinaryExpression.Add(p1,p2);
BinaryExpression be2 = BinaryExpression.Add(be1, c);
LambdaExpression lambda = LambdaExpression.Lambda(be2, p1, p2);

return lambda.Compile();
}
private void Window_Loaded(object sender, RoutedEventArgs e)
{
MessageBox.Show(GetExpression().DynamicInvoke(1, 2).ToString());
}


I don’t think there will be confusion in creating the Expressions and linking them.There are no constructors.You need to use the factory methods.Once the LambdaExpression is created you can Compile it to get the delegate.Invoke that delegate to execute your dynamic method code.

Expression Tree

By closely observing the object creation we could see the expression objects are linked as a tree structure .The root is the LambdaExpression.This is why it is called as expression tree.

Expression Tree is nothing but an object oriented representation of code.

I hope nobody have doubt on dynamism.By looking at this code we cannot say how the generated method will work.

Applications

There are lot of applications such as creating expression evaluators, even a new programming language of your own!!! According to my understanding the DLR is using these expressions.

If you had started in .Net from its 4.0 version you will say wow! what a feature.But if you are a developer who had worked in earlier versions one question might have popped up in your mind.Do we really need Linq Expression to generate code on the fly where we have CodeDom? Whether you have or not I have this question.Hope I can find the answer soon and share here.

Wednesday, December 15, 2010

Using XML literals in VB.Net with Linq

The XML literal support is the second thing I have seen special in Visual Basic which is really amazing.The first feature was the support for multiple indexer properties .XML literals helps us to create XElement by just writing plain XML in the VB.net code.This simple definition will not help anybody to learn what is XMl literals and their importance in dealing with xml.So let’s consider an example of XML serialization.
I have a class say Employee with 2 properties Name as string and ID as integer.The requirement is to get a XML representation of the employee collection.

Public Class Employee
Public Property Name As String
Public Property Id As Integer
End Class


If I pass a list of employee class I need out put like.


<Employees>
<Employee Name="Joy" Id="Joy" />
<Employee Name="George" Id="George" />
</Employees>


Here is the traditional code.


Private Function GetXMLTraditional(ByVal employees As List(Of Employee)) As String
Dim xEle As New XElement("Employees")
For Each emp In employees
Dim xEleEmp As New XElement("Employee")
xEleEmp.SetAttributeValue("Name", emp.Name)
xEleEmp.SetAttributeValue("Id", emp.Id)
xEle.Add(xEleEmp)
Next
Return xEle.ToString()
End Function


Look at the code which uses XML literals.


Private Function GetXMLUsingXMLLiterals(ByVal employees As List(Of Employee)) As String
Return <Employees>
<%= From emp In employees _
Select <Employee Name=<%= emp.Name %> Id=<%= emp.Name %>/>
%>
</Employees>.ToString
End Function



I don’t think I need to explain anything further about the advantages.If you are a ASP.Net developer you might already understood the use of “<%= %>”

Thursday, March 4, 2010

Using ‘IN’ clause in Linq

Everybody knows how to use ‘in’ clause in sql.This is a frequently used keyword but unfortunately not available in Linq as it is.But don’t worry we can simulate the same in Linq as follows.

List<Employee> emps=new List<Employee>()
{new Employee{Name="Joy"},new Employee {Name="George"}};
string []filters={"Joy","George"};

IEnumerable<Employee> result = from li
in emps
where filters.Contains(li.Name)
select li;



This is the reverse process of in clause.It is checking whether the filter array contains the given value.