linqtolucene Wiki & Documentation Rss Feed

Updated Wiki: Documentation

stemarie — Mon, 02 Apr 2012 16:43:13 GMT

Queries
Queries

Where clause
Where clause

Operators
Operators

Updated Wiki: Documentation

stemarie — Mon, 02 Apr 2012 16:43:02 GMT

*Queries
Queries

*Where clause
Where clause

*Operators
Operators

Updated Wiki: Operators

stemarie — Mon, 02 Apr 2012 16:42:33 GMT

Support has been added for multiple operators. It supports querying both directly upon the Query<T> type and from within a Where() clause. Currently, the API looks within the Query<T> member's FieldAttributes for any fields marked IsDefault=true to create the default field list. Some of the query operators listed below support specifying an explicit field for the queryString by using the Where() clause. Other operators will soon work within Where(), all will add support for specific fields when querying directly against Query<T>.

The operators that have been developed include the following: with various overloads for each function (not listed here)

public static Query<T> Search<T>(string queryString)	Applies native Lucene syntax
public static Query<T> Boost<T>(float factor)	Boosts a search phrase
public static Query<T> Like<T>(string phrase, float fuzziness)	Applies Fuzzy or Proximity to a search
public static Query<T> Match<T>(string phrase)	Applies wildcard matches to search
public static Query<T> StartsWith<T>(string phrase)	Finds matches to the prefix of a phrase
public static Query<T> Range<T>(string startRange, string endRange)	Finds matches within the a range, inclusive by default
public static Query<T> Between<T>(string from, string to)	Finds matches within a range, exclusive
public static Query<T> StartGroup<T>()	Begins a query Grouping
public static Query<T> Group<T>(Func<Query<T>, Query<T>> function)	Performs a search function within a group
public static Query<T> EndGroup<T>()	Ends a query Grouping
public static Query<T> Or<T>(Func<Query<T>, Query<T>> function)	Applies OR logic to a set of conditions
public static Query<T> And<T>(Func<Query<T>, Query<T>> function)	Applies AND logic to a set of conditions

Code Samples:

NorthwindIndexContext index = new NorthwindIndexContext();

var query = from c in index.Customers.Search("!myField:Jason test~);  // Output: !myField:Jason test~

var query = from c in index.Customers.StartsWith("mar")               // Output: mar*^2.3 OR asdf* (mo?se)
                                     .Boost(2.3)                                                       
                                     .Or(x => x.StartsWith("asdf")
                                     .Group(x => x.Match("mo?se"));

var query = from c in index.Customers                                 // Output: ContactName:"maria"~0.7
            where c.ContactName.Like("maria", 0.7f)
            select new { Name = c.ContactName, Id = c.CustomerID };

NOTE:
This is in no way a 'stable' release but is a chance for developers to see some of the basics about how the API will work. There are certainly scenarios and types of queries that won't work or will produce unexpected results. Also, Xml commenting is still not anywhere near as thorough as it needs to be, nor is the exception handling and printing. These types of defects will be corrected in future builds.

Also, a lot of the operators currently only support using the default-fields of the type. Applying specific fields to an operation will be supported soon.

Also note that projection and object creation currently only supports translation to string types.

Updated Wiki: Documentation

stemarie — Mon, 02 Apr 2012 16:41:56 GMT

Queries
Where clause

Updated Wiki: Queries

stemarie — Mon, 02 Apr 2012 16:41:36 GMT

In general, the API has changed a lot on the querying part, such that the operator support has been altered to be more integrated with the expression tree.

Some sample queries that you can now do are:

var query = from c in index.Customers
            where c.ContactName.Like("maria", 0.7) && !c.CompanyName == "moose box";

OUTPUT:
((ContactName:maria~0.7 AND -(CompanyName:"moose box")))

var query = from c in index.Customers
            where c.Like("amber") || c.CompanyName.Between("a", "d")

OUTPUT:
((amber) (CompanyName:{a TO d}))

Notice how in the second query, we can query against either a member of the type or the type itself. When querying against the type itself, it creates the query syntax without the field prefix...
c.Like("amber") --> (amber~)
c.CompanyName.Like("amber") --> (CompanyName:amber~)

The operator-methods are:
Like (for fuzzy, proximity)
Match (for wildcard searching)
Between (for exclusive range)
Include (for inclusive range)

You can use where multiple times to create groupings:

var query = from c in index.Customers
            where c.Like("amber") || c.CompanyName.Between("a", "d")
            where !c.CustomerId == "Jason"

OUTPUT:
((amber) (CompanyName:{a TO d})) (CustomerId:Jason)

Note that at the moment, while the AND operator is supported inside of groups, it is not supported between groups yet, making the default between groups OR.

Updated Wiki: Documentation

stemarie — Mon, 02 Apr 2012 16:41:17 GMT

Where clause

Updated Wiki: Where clause

stemarie — Mon, 02 Apr 2012 16:40:48 GMT

The latest release now supports a simple Where() clause in the statement.

What it does

Single perdicate in the Where() statement, which means multiple field comparisons are not yet support with AND|OR clauses.
The only operator supported is the Equals() or == operator
Object Creation or Projection using Type conversion. Currently only supports String types.

Code Sample

NorthwindIndexContext nw = new NorthwindIndexContext();
string name = "Maria";

var query = from c in context.Customers
            where (c.ContactName == name)
            select new { Name = c.CompanyName, Id = c.CustomerID };

ObjectDumper.Write(query);

Support for Additional operators and predicates will be made available in upcoming releases.

Acknowldegments
The majority of the effort requires acknowledgement from the {url:Wayward Weblog|http://blogs.msdn.com/mattwar/archive/2007/07/30/linq-building-an-iqueryable-provider-part-i.aspx] by Matt Warren of Microsoft. Much of the Expression evaluation has used this as a starting point and been converted to code that supports the Lucene API.

As part of the porting process, this particular release is not fully unit-tested and may be a bit buggy along with the fact that many facets are still not yet supported. As a result the source code version is not yet available as additional pieces get stripped from or added to the code-base. Xml Commenting is also required. This release is made available to get any interested developers a small sense of how it will be like when working with the LINQ to Lucene API.

Updated Wiki: Documentation

stemarie — Mon, 02 Apr 2012 16:39:15 GMT

var query = from c in index.Customers
            where c.ContactName.Like("maria", 0.7) && !c.CompanyName == "moose box";

OUTPUT:
((ContactName:maria~0.7 AND -(CompanyName:"moose box")))

var query = from c in index.Customers
            where c.Like("amber") || c.CompanyName.Between("a", "d")

OUTPUT:
((amber) (CompanyName:{a TO d}))

var query = from c in index.Customers
            where c.Like("amber") || c.CompanyName.Between("a", "d")
            where !c.CustomerId == "Jason"

Updated Wiki: Home

stemarie — Mon, 02 Apr 2012 13:50:50 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

Feedback
Your feedback and thoughts on LINQ to Lucene are vital to guiding the direction of this project. Please submit your feedback, feature suggestions and bugs into the Issue Tracker or Discussion Forum

Blog Posts
Here are some useful links that demonstrate the use of LINQ to Lucene:

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Current Release
LinqToLucene v0.61
Monday, April 2, 2012
Azure Blob Support for both Lucene indexes and entities in binary files
Azure Tables Support for both Lucene indexes and entities in binary files

Older Release
LinqToLucene v0.6
Monday, March 26 2012
Much refactoring and... Azure Blob Support for both Lucene indexes and entities in binary files

Older Release
LinqToLucene v0.5
Sunday, March 24 2012

Completed
- Upgrade to .NET 4
- Update to Lucene.Net 2.9.4
Refactoring
- Core Code
- Unit Tests
Optimizations
A lot of code cleanup
Ported unit tests to MS Test
Removed dependency on MbUnit

Older Release
The Refactored Release
Tuesday, 22nd July 2008
A large set of library changes including an API analogous to LINQ to SQL, more supported LINQ methods, powerful LINQ to SQL Data Context support and more unit tests.

Older Release
The Re-Release
As promised, this release merges the features of previous releases into the one bundle.

Older Release
The Type-Conversion Release
Monday, 19 November 2007
TYPES BEYOND STRINGS NOW SUPPORTED
As promised (albeit out of order), support has been added for type-conversion. If the type of the indexed-property uses a type-converter that supports conversion from a string, the type will automatically convert to its appropriate type. This automatically inlcudes string, bool, int, datetime and a number of other .net types that already implement their own type-converters. Custom types can implement their own type-converters. For more information on type-converters please review either of the following article: MSDN: How to Implement a Type-Converter

Older Release
The Query Release
Tuesday, 13 November 2007
This release provides a real focus on the querying abilities of the LINQ to Lucene project and is the first real 'working release', converting LINQ statements to Lucene queries with deferred query execution and object creation or projection. It culminates the majority of the required querying features for LINQ that Lucene provides natively.

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

First Releases
A first release for generating a simple index out of a database for mapping types to Lucene using LINQ is now available.
This is the first small piece of the puzzle.

The Index Release

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

stemarie — Mon, 02 Apr 2012 13:50:33 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Current Release
release85352:
Monday, April 2, 2012
Azure Blob Support for both Lucene indexes and entities in binary files
Azure Tables Support for both Lucene indexes and entities in binary files

Older Release
LinqToLucene v0.6
Monday, March 26 2012
Much refactoring and... Azure Blob Support for both Lucene indexes and entities in binary files

Older Release
LinqToLucene v0.5
Sunday, March 24 2012

Completed
- Upgrade to .NET 4
- Update to Lucene.Net 2.9.4
Refactoring
- Core Code
- Unit Tests
Optimizations
A lot of code cleanup
Ported unit tests to MS Test
Removed dependency on MbUnit

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

stemarie — Mon, 26 Mar 2012 21:12:12 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Completed
- Upgrade to .NET 4
- Update to Lucene.Net 2.9.4
Refactoring
- Core Code
- Unit Tests
Optimizations
A lot of code cleanup
Ported unit tests to MS Test
Removed dependency on MbUnit

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

stemarie — Mon, 26 Mar 2012 21:10:01 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Completed
- Upgrade to .NET 4
- Update to Lucene.Net 2.9.4
Refactoring
- Core Code
- Unit Tests
Optimizations
A lot of code cleanup
Ported unit tests to MS Test
Removed dependency on MbUnit

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

stemarie — Mon, 26 Mar 2012 21:02:59 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Current Release
LinqToLucene v0.6
Monday, March 26 2012
Much refactoring and... Azure Blob Support for both Lucene indexes and entities in binary files

Older Release
The Refactored Release
Tuesday, 22nd July 2008
A large set of library changes including an API analogous to LINQ to SQL, more supported LINQ methods, powerful LINQ to SQL Data Context support and more unit tests.

Older Release
The Re-Release
As promised, this release merges the features of previous releases into the one bundle.

Older Release
The Type-Conversion Release
Monday, 19 November 2007
TYPES BEYOND STRINGS NOW SUPPORTED
As promised (albeit out of order), support has been added for type-conversion. If the type of the indexed-property uses a type-converter that supports conversion from a string, the type will automatically convert to its appropriate type. This automatically inlcudes string, bool, int, datetime and a number of other .net types that already implement their own type-converters. Custom types can implement their own type-converters. For more information on type-converters please review either of the following article: MSDN: How to Implement a Type-Converter

Older Release
The Query Release
Tuesday, 13 November 2007
This release provides a real focus on the querying abilities of the LINQ to Lucene project and is the first real 'working release', converting LINQ statements to Lucene queries with deferred query execution and object creation or projection. It culminates the majority of the required querying features for LINQ that Lucene provides natively.

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Project Roadmap

stemarie — Mon, 26 Mar 2012 16:37:05 GMT

Project Roadmap

NOTE: we're currently naming our release versions using positive integers. We're now designing a roadmap that uses decimal version names

If there's anything we're missing please let us know in the issue tracker or in the discussion forum.

Version 1.0

OrderBy,OrderByDescending - single and multiple order bys
Support IQueryable Methods
- All
- Any
- Contains
- LastOrDefault
- SingleOrDefault
- Cast? - does this even need to be supported?
- Distinct
- GroupBy

Explicitly throw exceptions on these IQueryable methods
- Join
- GroupByJoin
- Intersect
- Reverse? -- Maybe this is supportable
- Sum
- Average
- Concat
- Min,Max
- TakeWhile,SkipWhile
- Everything else not convertable
More unit tests for IQueryable methods
Add implicit Paging support - see Built-in support for PagedList<T>
- But, what if devs use have their own type of paging?
SortFieldType support - Auto, Number, Text, etc
Field Formatters - support System.ComponentModel.TypeConverter stuff for string target conversion and formatting

*

Version 1.1

ObjectQueryTranslator
Batch and merge indexing (Indexers, RamDirectories and Merge) - maybe provide several indexing strategies
Mapping unit tests

*

Version 1.3

Perf tests and comparisons against LINQ to XML, maybe i4o
Compiled Queries - see Compiled Expression/Queries (Performance improvement)

Version 1.5

Object Tracking against DataContext for synchronizing the DataContext changes w/ index
Filters

Updated Wiki: Project Description

stemarie — Mon, 26 Mar 2012 16:34:39 GMT

Lucene is an Information Retrieval System used for full-text searching and scoring results commonly referred to as a search engine (similar to google-type-searching). LINQ is Microsoft's extensible Language Integrated Query platform that provides querying directly in a managed CLR. LINQ comes out of the box with LINQ to Objects, LINQ to DataSets, LINQ to XML, LINQ to SQL and LINQ to Entities. The goal of LINQ to Lucene is to provide developers with the ability to enjoy full-text searching using a fast-proven search-engine within the .Net managed CLR. The plan is to get the LINQ to Lucene code-base pushed up to CodePlex for additional developers to generate feed-back and improve and enhance.

Since the format of Lucene, which stores its data in an Index as Documents with fields, the project will be capable of mapping custom business object types, a kind of Object-Index-Mapper (OIM) similar to the typical Object-Relational-Mapper (ORM) that LINQ to SQL provides. LINQ to Lucene uses attributes and reflection on classes to be indexed and their members, similar to the Table and Column decorators for SQL mapped objects. Types can easily support LINQ to SQL decorations for a larger number of members while co-supporting LINQ to Lucene decorations for the few members that support it. Classes stored in the lucene index are decorated with the [Document] attribute and its properties are decorated with the [Field] attribute.

Similar to the DataContext object, there is an IndexContext class that manages querying, retrieval and mapping of objects to the appropriate index as well as writing the Index. Early releases of this will treat the index as though it is a read-only item, but the IndexContext object will eventually provide a facility for tracking changes and persisting those to the index with a familiar SubmitChanges() method. The implementation is being designed to support the generation of an Expression Tree that can be analyzed and converted into a Lucene query and ultimately executed against the index on first request following the same pattern that LINQ to SQL uses allowing for rich query generation.

The project is built using Test Driven Development with the Visual Studio 2010 Test Projects.
The source code is written in C# 3.0 using the Lucene.Net port version 2.9.4.

Updated Wiki: Home

CVertex — Sat, 24 Mar 2012 13:50:53 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (cached post as author is still finding a new home)

Current Release
The Refactored Release
Tuesday, 22nd July 2008
A large set of library changes including an API analogous to LINQ to SQL, more supported LINQ methods, powerful LINQ to SQL Data Context support and more unit tests.

Older Release
The Re-Release
As promised, this release merges the features of previous releases into the one bundle.

Older Release
The Type-Conversion Release
Monday, 19 November 2007
TYPES BEYOND STRINGS NOW SUPPORTED
As promised (albeit out of order), support has been added for type-conversion. If the type of the indexed-property uses a type-converter that supports conversion from a string, the type will automatically convert to its appropriate type. This automatically inlcudes string, bool, int, datetime and a number of other .net types that already implement their own type-converters. Custom types can implement their own type-converters. For more information on type-converters please review either of the following article: MSDN: How to Implement a Type-Converter

Older Release
The Query Release
Tuesday, 13 November 2007
This release provides a real focus on the querying abilities of the LINQ to Lucene project and is the first real 'working release', converting LINQ statements to Lucene queries with deferred query execution and object creation or projection. It culminates the majority of the required querying features for LINQ that Lucene provides natively.

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

CVertex — Sat, 24 Mar 2012 13:49:15 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (author is still finding a new home for the blog)

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

CVertex — Sat, 24 Mar 2012 13:47:36 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Home

CVertex — Sat, 24 Mar 2012 13:44:10 GMT

What is LINQ to Lucene?

Providing a custom LINQ solution for the Lucene Information Retrieval System, commonly referred to as a search-engine.

LINQ & Lambda, Part 4: Lucene.Net (dead link) see http://web.archive.org/web/20090218220824/http://vijay.screamingpens.com/archive/2008/07/21/linq-amp-lambda-part-4-lucene.net.aspx for snapshot of the post. Author will resurrect this post at some point

	Lucene Syntax	LINQ to Lucene
Terms & Phrases	"test" or "hello dolly"	c.Match("test") or c.Match("hello dolly")
Fields	title:"The Right way" and text:go	c.Title == "The Right way" or c.Text == "go"
WildCard	amb?r	c.ContactName.Match("amb?r")
Prefix	amber*	c.ContactName.StartsWith("amber")
Fuzzy	roam~ or roam~0.8	c.ContactName.Like("roam") or c.ContactName.Like("roam", 0.8)
Proximity	"jakarta apache"~10	c.ContactName.Like("jakarta apache", 10)
Inclusive Range	mod_date:[20020101 TO 20030101]	c.ModifiedDate.Includes("20020101", "20030101")
Exclusive Range	title:{Aida TO Carmen}	c.Title.Between("Aida", "Carmen")
Boosting	jakarta^4 apache	c.Title.Match("jakarta".Boost(4), apache)
Boolean Or	"jakarta apache" OR jakarta	where c.Match("jakarta apache") \|\| c.Match("jakarta")
Boolean And	"jakarta apache" AND "Apache Lucene"	where c.Match("jakarta apache") && c.Match("Apache Lucene")
Boolean Not	"jakarta apache" NOT "Apache Lucene"	where c.Match("jakarta apache") && !c.Match("Apache Lucene")
Required	+jakarta lucene	c.Title.Match("jakarta".Require(), "lucene")
Grouping	(jakarta OR apache) AND website	where (c.Title == "jakarta" \|\| c.Title == "apache") && (c.Title == "website")
Native Syntax	ie. title:{+return +"pink panther")	c.Search("title:(return +\"pink panther\"")

[Document]
public class Customer 
{
  [Field(Store=FieldStore.Yes, Index=FieldIndex.No)] public string CustomerID { get; set; }
  [Field(Store=FieldStore.Yes, Index=FieldIndex.Tokenized)] public string CustomerName { get; set; }
}

IndexContext index = new IndexContext;
index.WriteIndex<Customer>();

Updated Wiki: Project Roadmap

CVertex — Sat, 25 Apr 2009 13:14:12 GMT

Project Roadmap

OrderBy,OrderByDescending - single and multiple order bys
Support IQueryable Methods
- All
- Any
- Contains
- LastOrDefault
- SingleOrDefault
- Cast? - does this even need to be supported?
- Distinct
- GroupBy

Explicitly throw exceptions on these IQueryable methods
- Join
- GroupByJoin
- Intersect
- Reverse? -- Maybe this is supportable
- Sum
- Average
- Concat
- Min,Max
- TakeWhile,SkipWhile
- Everything else not convertable
More unit tests for IQueryable methods
Add implicit Paging support - see Built-in support for PagedList<T>
- But, what if devs use have their own type of paging?
SortFieldType support - Auto, Number, Text, etc
Field Formatters - support System.ComponentModel.TypeConverter stuff for string target conversion and formatting

Version 0.2

ObjectQueryTranslator
Batch and merge indexing (Indexers, RamDirectories and Merge) - maybe provide several indexing strategies
Mapping unit tests

Version 0.3

Build process using MSBuild
- Zipped assembly release (Maybe with ILMerge w/ Lucene.Net.dll)
- Zipped source release

Version 0.4

Perf tests and comparisons against LINQ to XML, maybe i4o
Compiled Queries - see Compiled Expression/Queries (Performance improvement)

Version 0.5

Object Tracking against DataContext for synchronizing the DataContext changes w/ index
Filters
Aggregation <-- what does this mean?