Published: 10 Nov 2007
By: Mehfuz Hossain

Learn how to create custom LINQ providers.

Introduction

LINQ (Language Integrated Query) works as a middle tier between data store and the language environment. From a developer's point of view, it is just a new pattern for querying data from multiple data structures directly in the IDE. Behind the scenes it does a whole lot of tasks like expression processing, validation and calling the right routine to fetch data or build a query to run in SQL Server. In short, LINQ stands as common query gateway between the language and the data store.

Figure 1: LINQ workflow (from language to data store)

fig01.jpg

The purpose of this article is to teach you how to extend this query pattern in order to create your own custom provider. The examples in this article have been taken from my LINQ.Flickr project at CodePlex (the link can be found at the end of the article).

Creating and Executing a Query

LINQ is a new level of language abstraction that supports querying virtually any data source - such as SQL, XML and in-memory data structures like Dictionary, List and more. We can even create our own LINQ providers to give querying capabilities to our API, services and data access layer.

The creation of a LINQ provider starts with a Query class, which will be exposed to the user and will be responsible for building and executing a query expression. This class implements two interfaces defined in the System.Linq namespace, namely IQueryProvider and IQueryable.

Note that a class must implement IQueryable to be queried by a user. Without implementing this interface, the complier will throw an exception and complain with the message Could not find an implementation of the query pattern for source type <SourceObject>.

If we look closely at the IQueryable interface, we can see that it needs a provider of type IQueryProvider, which is called by the system to initialize and execute LINQ expressions.

IQueryProvider returns a reference to IQueryable with the constructed expression-tree passed by the LINQ framework, which is used for further calls. In general terms, each query block is converted to a bunch of method calls. For each method call, there are some expressions involved. While creating our provider - in the method IQueryProvider.CreateQuery - we run through the expressions and fill up a filter object, which is used in the IQueryProvider.Execute method to run a query against the data store.

In addition, if we use some custom method calls along with our query; those are translated to different method calls. For example, the below query is followed by two filters: Take and Skip respectively.

In LINQ, this query generates three method call expressions. For each block, IQueryProvider.CreateQuery will be invoked. Now, consider the following query, taken from the IQueryProvider.CreateQuery method in the FlickrQuery class.

Before I describe the code, it is worth mentioning that in LINQ keywords like where or select are transformed to MethodCallExpression as well, along with explicit calls like take or skip. Now, if we look at the code we can see that when the Take method is called, the value is extracted from argument[1] and stored in a local variable for later use.

In the query we are using FlickrContext.Photos, which is the instance of the FlickrQuery class (which implements IQueryable). Every part of the query block in LINQ - which is turned into MethodCallExpression - has two arguments. One holds a reference to the query class itself (in this case FlickrQuery), which is returned back at the end of CreateQuery with the following statement.

This IQueryable reference is either passed into CreateQuery (again) for successive method calls, or is passed to IQueryProvider.Execute during query iteration or single call.

Note that the final output of FlickrQuery will have proper data in itemsToTake, itemsToSkip and dummyObject (whose value will be queried on Flickr). This happens after the LINQ runtime makes all the calls to CreateQuery for the whole expression.

Similarly, the second argument (argument[1]) of the MethodCallExpression holds the query expression. In the case of Take and Skip from the above query, the argument is a ConstantExpression for which the value has been extracted with the following statement:

Therefore, any MethodcallExpression looks like so:

Figure 2: Expression tree (MethodCallExpression)

fig02.jpg

So far, we have built data from an expression tree. Now, the query can be executed in two ways, either by implementing the GetEnumerator method (defined in the IEnumerable interface) in the Query class, (which inherits from IQueryable); or it can be executed by the LINQ runtime directly.

In either case, the IQueryProvider.Execute method is called. The GetEnumerator method calls it when the query is used inside an iterator (e.g. using foreach). In this case, IQueryProvider.Execute has to be called from within GetEnumerator manually.

Alternatively, when query.Single<anyObject> or query.First<anyobject> is invoked, IQueryProvider.Execute is called from the LINQ runtime, with a specific object type. Since IQueryable does not have such method defined in it, we need to create an interface and delegate the call to our version of the method.

In the LINQ.Flickr project I have created an interface called IPhotoList, which defines a Single method that returns a single photo object from the collection. For example, we may need to query a single photo based on a particular photo ID. The corresponding query will look like the following:

The Single method lets the runtime call the Execute method directly with a Photo type. We must implement a mechanism in IQueryProvider.Execute that delegates that to our version of the Single method. Therefore, we need the following code in IQueryProvider.Execute:

Basically, it compares the called method with our implemented method and invokes it to get the result.

Expression processing

As we have seen, the starting point of Expression is MethodCallExpression, which is branched into a ConstantExpression that points to the Query class itself; and the expression for the query.

Each query is then divided into expressions. To make it clearer, let's consider the following code:

Figure 3: Expression tree for the previous query

fig03.jpg

Let's examine the expression at different steps for the above query:

Note that when Binary expression type is Equal, the tree can further branch into unary and then constant expressions. This happens if enum comparison and variable cast are used in the query. Also, a property/local variable can be used in an expression for comparison. In that case, we have to check if the expression is a member access or not. If so, then we have to further process the expression unless a constant expression is found, in order to get the value out of it.

The following snippet shows how to find a ConstantExpression from a MemberAccessExpression:

Finally, the filter object should be reflected to find the property of the filtered object for which the comparison is made. If there is a valid match, then the value is set to the filter object's property, which will be queried back to data store. A similar code snippet for that will be:

The following snippet shows how to get the name of the property for which the comparison is made in a BinaryExpression:

In the above code, expression is the instance of the BinaryExpression class.

Summary

This article has shown the basics of creating a LINQ provider. It briefly showed how the creation and execution of an expression is carried out and how expressions can be parsed while building a custom provider. As said earlier, the examples are taken from my LINQ.Flickr project at CodePlex. Feel free to download it from here: www.codeplex.com/linqflickr and give it a try!

<<  Previous Article Continue reading and see our next or previous articles Next Article >>

About Mehfuz Hossain

Passionate about cutting edge technologies and a .net enthusiast. He played role in variety of products starting from University automation to web 2.0 start-page (www.pageflakes.com). Currently, working at Telerik Inc (www.telerik.com), the premium rad control provider for asp.net and winforms. He i...

This author has published 2 articles on DotNetSlackers. View other articles or the complete profile here.

Other articles in this category


C# 4.0 Reflection Programming - Part 2
As introduced in the first article, the most typically-used tools associated with .NET reflection ar...
C# 4.0 Reflection Programming - Part 3
In the previous article, we used the reflection to obtain the information of an assembly, module, ty...
C# 4.0 Reflection Programming - Part 4
In this last article of this series, we will learn what to do with reflection. But before making the...
Understanding and Using Extension Methods
Extension methods were new to C# 3.0. They allow you to add a method to an existing type without hav...
Introduction to C# 3.0 features
C# 3.0 introduced some of very useful features built on top of 2.0. This article explains the usage,...

You might also be interested in the following related blog posts


New versions of dotConnect ADO.NET Providers and LinqConnect 4.2 with LINQPad Integration and Entity Developer 5.5 read more
Writing LINQ providers easily and elegantly (LinqExtender 3.0) read more
Connecting to SQL Azure with Telerik OpenAccess read more
Lightweight Linq Parser read more
Index for Business Apps Example for Silverlight 3 RTM and .NET RIA Services July Update read more
Generating Dynamic Methods with Expression Trees in Visual Studio 2010 read more
BLinq - Linq to Bing Search APIs read more
OAuth in action Linq2Twitter read more
Important Entity Framework Query Improvements for .NET 4.0 read more
Business Apps Example for Silverlight 3 RTM and .NET RIA Services July Update: Part 18: Custom Linq Provider read more
Top
 
 
 

Discussion


Subject Author Date
placeholder Great Article!!! Kazi Manzur Rashid 11/10/2007 4:53 PM
RE: Great Article!!! Mehfuz Hossain 11/10/2007 10:15 PM
placeholder RE: RE: Great Article!!! Sonu Kapoor 11/11/2007 11:27 PM
RE: RE: RE: Great Article!!! Sonu Kapoor 11/11/2007 11:28 PM
placeholder RE: RE: RE: RE: Great Article!!! Mehfuz Hossain 11/12/2007 12:33 AM

Please login to rate or to leave a comment.