LINQ (Language Integrated Query) works as a middle tier between data store and the language environment. From a
developer's point of view, it is just a new pattern for querying data from multiple data structures directly in the IDE.
Behind the scenes it does a whole lot of tasks like expression processing, validation and calling the right routine to fetch
data or build a query to run in SQL Server. In short, LINQ stands as common query gateway between the language and the data
Figure 1: LINQ workflow (from language to data store)
The purpose of this
article is to teach you how to extend this query pattern in order to create your own custom provider. The examples in this
article have been taken from my LINQ.Flickr project at CodePlex (the link can be found at the end of the article).
Creating and Executing a Query
LINQ is a new level of language abstraction that supports querying virtually any data source - such as SQL, XML and
in-memory data structures like Dictionary, List and more. We can even create our own LINQ providers to give querying
capabilities to our API, services and data access layer.
The creation of a LINQ provider starts with a Query class, which will be exposed to the user and will be responsible for
building and executing a query expression. This class implements two interfaces defined in the System.Linq namespace, namely
IQueryProvider and IQueryable.
Note that a class must implement IQueryable to be queried by a user. Without implementing this interface, the complier
will throw an exception and complain with the message
If we look closely at the IQueryable interface, we can see that it needs a provider of type IQueryProvider, which is
called by the system to initialize and execute LINQ expressions.
IQueryProvider returns a reference to IQueryable with the constructed expression-tree passed by the LINQ
framework, which is used for further calls. In general terms, each query block is converted to a bunch of method calls. For
each method call, there are some expressions involved. While creating our provider - in the method IQueryProvider.CreateQuery
- we run through the expressions and fill up a filter object, which is used in the IQueryProvider.Execute method to run a
query against the data store.
In addition, if we use some custom method calls along with our query; those are translated to different method calls. For
example, the below query is followed by two filters:
In LINQ, this query generates three method call expressions. For each block, IQueryProvider.CreateQuery will be
invoked. Now, consider the following query, taken from the IQueryProvider.CreateQuery method in the FlickrQuery class.
Before I describe the code, it is worth mentioning that in LINQ keywords like
select are transformed to MethodCallExpression as well, along with explicit calls like
skip. Now, if we look at the code we can see that when the
Take method is called, the value is
argument and stored in a local variable for later use.
In the query we are using
FlickrContext.Photos, which is the instance of the FlickrQuery class (which
implements IQueryable). Every part of the query block in LINQ - which is turned into MethodCallExpression - has two
arguments. One holds a reference to the query class itself (in this case FlickrQuery), which is returned back at the end of
CreateQuery with the following statement.
This IQueryable reference is either passed into CreateQuery (again) for successive method calls, or is passed
to IQueryProvider.Execute during query iteration or single call.
Note that the final output of FlickrQuery will have proper data in
dummyObject (whose value will be queried on Flickr). This happens after the LINQ runtime makes all the calls to
CreateQuery for the whole expression.
Similarly, the second argument (
argument) of the MethodCallExpression holds the query expression. In the
Skip from the above query, the argument is a ConstantExpression for which the
value has been extracted with the following statement:
Therefore, any MethodcallExpression looks like so:
Figure 2: Expression tree (MethodCallExpression)
So far, we have built
data from an expression tree. Now, the query can be executed in two ways, either by implementing the GetEnumerator method
(defined in the IEnumerable interface) in the Query class, (which inherits from IQueryable); or it can be executed by the
LINQ runtime directly.
In either case, the IQueryProvider.Execute method is called. The GetEnumerator method calls it when the query is used
inside an iterator (e.g. using foreach). In this case, IQueryProvider.Execute has to be called from within GetEnumerator
invoked, IQueryProvider.Execute is called from the LINQ runtime, with a specific object type. Since IQueryable does not have
such method defined in it, we need to create an interface and delegate the call to our version of the method.
In the LINQ.Flickr project I have created an interface called IPhotoList, which defines a
Single method that
returns a single photo object from the collection. For example, we may need to query a single photo based on a particular
photo ID. The corresponding query will look like the following:
The Single method lets the runtime call the Execute method directly with a Photo type. We must implement a
mechanism in IQueryProvider.Execute that delegates that to our version of the
Single method. Therefore, we need
the following code in IQueryProvider.Execute:
Basically, it compares the called method with our implemented method and invokes it to get the result.
As we have seen, the starting point of Expression is MethodCallExpression, which is branched into a ConstantExpression
that points to the Query class itself; and the expression for the query.
Each query is then divided into expressions. To make it clearer, let's consider the following code:
Figure 3: Expression tree for the previous query
Let's examine the
expression at different steps for the above query:
Note that when Binary expression type is
Equal, the tree can further branch into unary and then
constant expressions. This happens if enum comparison and variable cast are used in the query. Also, a property/local
variable can be used in an expression for comparison. In that case, we have to check if the expression is a member access or
not. If so, then we have to further process the expression unless a constant expression is found, in order to get the value
out of it.
The following snippet shows how to find a ConstantExpression from a MemberAccessExpression:
Finally, the filter object should be reflected to find the property of the filtered object for which the
comparison is made. If there is a valid match, then the value is set to the filter object's property, which will be queried
back to data store. A similar code snippet for that will be:
The following snippet shows how to get the name of the property for which the comparison is made in a
In the above code,
expression is the instance of the BinaryExpression class.
This article has shown the basics of creating a LINQ provider. It briefly showed how the creation and execution of an
expression is carried out and how expressions can be parsed while building a custom provider. As said earlier, the examples
are taken from my LINQ.Flickr project at CodePlex. Feel free to download it from here: www.codeplex.com/linqflickr and give it a try!
Passionate about cutting edge technologies and a .net enthusiast. He played role in variety of products starting from University automation to web 2.0 start-page (www.pageflakes.com). Currently, working at Telerik Inc (www.telerik.com), the premium rad control provider for asp.net and winforms. He i...
This author has published 2 articles on DotNetSlackers. View other articles or the complete profile here.
You might also be interested in the following related blog posts
New versions of dotConnect ADO.NET Providers and LinqConnect 4.2 with LINQPad Integration and Entity Developer 5.5
Writing LINQ providers easily and elegantly (LinqExtender 3.0)
Connecting to SQL Azure with Telerik OpenAccess
Lightweight Linq Parser
Index for Business Apps Example for Silverlight 3 RTM and .NET RIA Services July Update
Generating Dynamic Methods with Expression Trees in Visual Studio 2010
BLinq - Linq to Bing Search APIs
OAuth in action Linq2Twitter
Important Entity Framework Query Improvements for .NET 4.0
Business Apps Example for Silverlight 3 RTM and .NET RIA Services July Update: Part 18: Custom Linq Provider
Please login to rate or to leave a comment.