Efficient way to grab all items form ContentSearchManager SearchIndex in Sitecore? - sitecore

Question

using (var context = ContentSearchManager.GetIndex(index).CreateSearchContext())
{
var items = context.GetQueryable<SearchResultItem>()
.Where(item => item.TemplateId == _templateId)
.ToList();
var resultList = new List<MediaItem>();
I then go through and use conditionals to filter the results more and add them to resultList.
I add them to the list with (MediaItem)x.GetItem();
I have terrible performance, which I am sure is caused by GetItem querying the database 10,000s of times.
I need to have the results as a MediaItem due to needing to edit the fields in these items.
How can I avoid querying Sitecore for each item with GetItem() ?
Thanks,
Sitecore Noob
I tried using .GetItem() with poor performance. I also tried researching how to get back Items instead of SearchResultItem

Answer 1

Yes, this would be a very slow and ineffective way of loading the needed items, since you're basically loading references to all media items of the given type into memory from Solr (GetQueryable<T>) and then load everything again from the database through .GetItem(). But the pattern you're trying isn't too far off actually.
Using the SearchContext, you should do the filtering in where you get all items references you need, letting Solr do the computation. I.e. not the 10,000s of items, just the ones you're editing. Once you get the filtered result back from Solr, you perform the .GetItem() on the items you want to work with. Always check that the item (and version) actually exists after a .GetItem() since the Solr index can sometimes be out of sync with the database, so a Solr response can in rare cases reference an item that doesn't exist.
How you should write the Solr query is a bit hard to describe without knowing more details about what filters you need. In general, I'd recommend using the SolrNet API directly instead of going through the ContentSearchManager wrapper. It's sometimes hard to understand what the search manager does under the hood and you basically need to monitor the Search.log in order to verify that your code does what it's supposed to do. A small mistake in the .Where() could cause the entire media library to be loaded into memory and you'll experience the same performance issue again as you have right now. With the query you have today, it's worth noting the difference between the underlying _template and _templates index fields, as the first one (that's used in your sample code) will return items of exactly that given template while the later one will also include all items inheriting from that template.