Searching source code on Koders.com

Lately Phil Haack has blogged about his new job at Koders.com. As far as I can see they provide a search engine for open source code and let people submit their repositories so that their code can be indexed.

I've developed some interest in code searching since I'm going to implement a code search engine for my Information Retrieval course here in Italy, and hope Phil will give me some ideas in his future blog posts about Koders. In fact, I'm going to use Lucene.NET for indexing the source files - which I think they are using, too - and maybe I can learn some good practices and get some tips from them. 

Though it doesn't have a slick interface as Krugle, Koders provides a service for querying its engine from the web, returning the search results as RSS, so I thought I could write a wrapper on top of it. I've come up with a fluent interface query class based on strongly typed classes generated with the ASP.NET RSS Toolkit.

Koders service allows refining search results by programming language, license, logical placement of the code in the source file (class name, method name, interface name) and by file name.

In the wrapper I created the querying is a two-step process. First you need to create a KodersQuery class supplying the parameters of the search, then you use one of the strongly typed classes generated by the ASP.NET RSS Toolkit to load and parse the results:

KodersQuery q = new KodersQuery()
.ProgrammingLanguage(ProgrammingLanguage.Csharp)
.SearchTerms("Subtext");

KodersRss rss = KodersRss.Load(q);

The KodersRss class wraps the xml returned by the query and provides the items found via the KodersRss.Channel.Items property.

Given the profileration of LINQ to X implementations it would be cool to create such an API for search source code, either on Google Code Search or Koders, but I'm still waiting for the final bits of LINQ before to attempt doing it. In fact I've seen the source code of some LINQ to X libraries and it looks like they won't compile with Orcas if they were created with the May CTP.

At the moment the source code is on my svn repository.

kick it on DotNetKicks.com

Published 22 May 2007 02:49 AM by simoneb
Filed under:

Comments

# DotNetKicks.com said on 21 May, 2007 06:39 PM

You've been kicked (a good thing) - Trackback from DotNetKicks.com

# Phil Haack said on 21 May, 2007 08:07 PM
That is really cool! I'm hoping to give it a try when I get a moment's rest here. As for using Lucene.NET, you are correct. We are using Lucene.NET. I wrote about that in a recent post here: http://tinyurl.com/2juc5u I hate to say it, but I agree with you. In some ways the Krugle interface is much slicker. However, I think our project browser has an edge. Especially when you look at our syntax highlighting of an actual code page. I think we win there. :) I just discovered that Krugle uses Lucene, the original Java version that Lucene.NET is based on. Lucene is a powerhouse in open source search!
# Phil Haack said on 21 May, 2007 08:15 PM
Hey Simone, what's the license for the source code?
# simoneb said on 21 May, 2007 08:32 PM

Yes you're right, the project browser is cool, and I think that having you as a developer (well, I don't know what is your exact role there) is a big advantage for Koders ;-) Krugle has been integrated with Sourceforge lately, that's a big advantage for them on the other hand...

The code is public domain, there's nothing special in it and some weird plumbing on resource files where I stored the list of programming languages.

As I said, it would be cool to create a Linq to Koders dialect but I have no time for it right now.

Ok, you pass your secrets about code indexing and parsing to me and I write a Linq to Koders for you :D

JK

Keep up the good work there

# Abi said on 21 June, 2007 12:34 PM
see google's programmers search engine http://devshots.com Devshots returns Google search results, but emphasize on programming related sites and resources. So the search result will be smart and accurate.
# Marc Brooks said on 21 June, 2007 08:26 PM
Cool application! We've just released version 2.0 of the Toolkit on CodePlex... any chance you would like your library added as an example for the next release?
# simoneb said on 21 June, 2007 08:54 PM

Of course Mark, feel free to add it as a sample in next release. That's funny, I was reading your post about the UriTemplate library right now ;) I could have used it to query the Koders service.

# Ultracet. said on 04 August, 2008 06:00 PM

Ultracet addiction. Ultracet.

This site

Search

Go

This Blog

News

Syndication

Sponsors

  • MaximumASP