Mongodb Index usage slower with regex anchor - regex

Question

I've got a query that's using a regex anchor and it seems to be slower when running an index scan rather than a collection scan.
A bit of background to the question:
I have a MSSQL database that has approximately 2.8 million rows in a table. We were running the following query against the table to return approximately 2.6 million results in 23 seconds:
select * from table where 'column' like "IL%"
So out of curiosity I decided to see if mongodb could perform this any faster than my MSSQL database and on a new test server I created a mongodb database which I filled 1 collection (test1) with just under 3 million objects. Here's the basic structure of a document in a collection:
> db.test1.findOne()
{
"_id" : 2,
"Other_REV" : "NULL",
"Holidex_Code" : "W8BP0",
"Segment_Name" : "NULL",
"Source" : "Forecast",
"Date_" : ISODate("2009-11-12T11:14:00Z"),
"Rooms_Sold" : 3,
"FB_REV" : "NULL",
"Rate_Code" : "ILM87",
"Export_Date" : ISODate("2010-12-12T11:14:00Z"),
"Rooms_Rev" : 51
}
All of my records have Rate_Code prefixed with IL and I ran the following query against the database which took just over 3 seconds:
> db.test1.find({'Rate_Code':{$regex: /^IL/}}).explain()
{
"cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 2999999,
"nscannedObjects" : 2999999,
"nscanned" : 2999999,
"nscannedObjectsAllPlans" : 2999999,
"nscannedAllPlans" : 2999999,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 4,
"nChunkSkips" : 0,
"millis" : 3398,
"indexBounds" : {
},
"server" : "MONGODB:27017"
}
Out of curiosity I created an index to see if I could speed up the retrieval at all:
> db.test1.ensureIndex({'Rate_Code':1})
However this appears to actually slow down the resolution of the query to approximately 6 seconds on average:
> db.test1.find({'Rate_Code':{$regex: /^IL/}}).explain()
{
"cursor" : "BtreeCursor Rate_Code_1",
"isMultiKey" : false,
"n" : 2999999,
"nscannedObjects" : 2999999,
"nscanned" : 2999999,
"nscannedObjectsAllPlans" : 2999999,
"nscannedAllPlans" : 2999999,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 4,
"nChunkSkips" : 0,
"millis" : 5895,
"indexBounds" : {
"Rate_Code" : [
[
"IL",
"IM"
]
]
},
"server" : "MONGODB:27017"
}
The OS has 2GB of memory and appears to be holding both indexes quite comfortably in memory with no disk usage being recorded when the query is ran:
> db.test1.stats()
{
"ns" : "purify.test1",
"count" : 2999999,
"size" : 623999808,
"avgObjSize" : 208.0000053333351,
"storageSize" : 790593536,
"numExtents" : 18,
"nindexes" : 2,
"lastExtentSize" : 207732736,
"paddingFactor" : 1,
"systemFlags" : 0,
"userFlags" : 0,
"totalIndexSize" : 153218240,
"indexSizes" : {
"_id_" : 83722240,
"Rate_Code_1" : 69496000
},
"ok" : 1
}
I'm thinking the slow down is due to mongodb performing a full scan of the index followed by a full collection scan as it can't be sure that all my matches are in the index but I'm not entirely sure if this is the case. Is there any way that this could be improved upon for better performance?
Thanks for any help.