Sitecore: Extract Indexed Content of Media Files using MediaItemContentExtractor

Here is something in addition to my previous post regarding indexing associated content:

Here is a common scenario:
Your custom index configuration is set up to crawl all the content for your website which is then used by your site search (keywords search) to fetch search results. In addition to you content item crawlers, you add a crawler for Media Library items as well and Sitecore does a great job of indexing PDF, DOCX, DOC, etc. files automatically, provided your have a valid IFilter installed, and now you have search extended to show file items as search results.

Now consider the following scenario:
One of the lookup fields on your page points to a file in the media library and the new requirement is to show the page item in the search result when the search phrase matches the content in the associated file.

Solution (Lucene & Solr): Create a computed field called "related_content" that stored the crawled content of the associate file and extend the query to now search both "_content" (in Solr and I think it's simply "content" in lucene) and "related_content" fields.

Here is the code:

        
using System;
using System.Text;
using System.Xml;
using OConnell.Domain.Models.OConnell.Intranet.Components;
using OConnell.SC.Extensions;
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.ComputedFields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;

namespace OConnell.SC.Search.ComputedFields
{
    public class RelatedContent : IComputedIndexField
    {
        public string FieldName { get; set; }
        public string ReturnType { get; set; }

        public object ComputeFieldValue(IIndexable indexable)
        {
            // PDF,File,Docx,Document,Doc template IDs (Unversioned)
            private readonly string _mediaItemTemplates =
            "{0603F166-35B8-469F-8123-E8D87BEDC171}|{962B53C4-F93B-4DF9-9821-415C867B8903}|{7BB0411F-50CD-4C21-AD8F-1FCDE7C3AFFE}|{777F0C76-D712-46EA-9F40-371ACDA18A1C}|{16692733-9A61-45E6-B0D4-4C0C06F8DD3C}";
           Item item = indexable as SitecoreIndexableItem;
            if (item == null)
            {
                Log.Debug(string.Concat("Rejected item at path: ", indexable.AbsolutePath));
                return null;
            }
            Log.Debug(string.Concat("Getting related content for item at path: ", indexable.AbsolutePath));
            var sb = new StringBuilder();

            try
            {
                //change the field name to your lookup field
                var mediaItem = item.Database.GetItem(item.Fields["Related File"].Value);
                if (mediaItem != null && _mediaItemTemplates.Contains(mediaItem.TemplateID.ToString()))
                {
                    var indexedContent = GetFileContent(mediaItem);
                    sb.Append(indexedContent.ToString() ?? string.Empty);
                }
            }
            catch (Exception ex)
            {
                Log.Error(ex.Message,item);
            }
           
            return string.IsNullOrEmpty(sb.ToString()) ? null : sb.ToString();
        }

        private string GetFileContent(SitecoreIndexableItem indexableMediaItem)
        {
            XmlNode configurationNode =
                Sitecore.Configuration.Factory.GetConfigNode(
                    "contentSearch/indexConfigurations/defaultSolrIndexConfiguration/mediaIndexing");

            //MediaItemContentExtractor expects the full xml 

            //including the "mediaIndexing" node and GetConfigNode seems to 

            //be ommitting the parent node and hence loading it as XML before passing 

            //passing to MediaItemContentExtractor

            var xmlDocument = new XmlDocument();
            xmlDocument.LoadXml(configurationNode.OuterXml);
            var extractor = new MediaItemContentExtractor(xmlDocument);
            var indexedContent = extractor.ComputeFieldValue(indexableMediaItem);
            return indexedContent == null ? string.Empty : indexedContent.ToString();
        }
    }
}

Don't forget to add the configuration element for your computed field:

 
<fields hint="raw:AddComputedIndexField">
    <field fieldName="related_content">Sitecore.SharedSource.ComputedFields.RelatedContent, Sitecore.SharedSource
    </field>
</fields>
    

Sitecore: Indexing Associated Content

Setting up indexes and indexing content for site search in Sitecore is a pretty straightforward task and there is an extensive knowledge base put together by the community with various examples. One useful construct we often find ourselves implementing for keyword search is setting up a computed field. I typically use this construct to index any additional content referenced by a page, typically (content blocks, promos, callouts) added to the page via presentation details

The Need: Index externally reference content by a page item

Solve: Create a computed field to index TextField and HtmlText type fields of referenced items

This implementation fetches all the renderings for the current item's presentation for the default device and checks their datasource item for index-able content.
As a suggestion, check if the current item inherits from certain page templates else skip the execution.

Step1: Create a class and implemented the IComputedIndexField interface as shown below:

    public class RelatedContent : IComputedIndexField
    {
        public const string DefaultDeviceId = "{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}";
        public string FieldName { get; set; }
        public string ReturnType { get; set; }

        public object ComputeFieldValue(IIndexable indexable)
        {
            Item item = indexable as SitecoreIndexableItem;
            //add condition to skip if the current item does not belong to a page template
            var sb = new StringBuilder();
            var masterDb = Factory.GetDatabase("master");
            DeviceItem defaultDevice = masterDb.GetItem(DefaultDeviceId);
            RenderingReference[] renderings = item.Visualization.GetRenderings(defaultDevice, true);
            foreach (RenderingReference rendering in renderings)
            {
                if (string.IsNullOrEmpty(rendering.Settings.DataSource))
                    continue;

                Item datasourceItem = item.Database.GetItem(rendering.Settings.DataSource);
                if (datasourceItem == null) continue;

                //add an if condition to get indexable content for certain template types
                sb = GetIndexableContent(sb, datasourceItem);
            }
            return string.IsNullOrEmpty(sb.ToString())?null:sb.ToString();
        }

        private StringBuilder GetIndexableContent(StringBuilder sb, Item item,bool indexAllTextFields = true, string fieldName="")
        {
            if(sb==null) sb = new StringBuilder();

            if (!string.IsNullOrEmpty(fieldName))
            {
                if (item.Fields[fieldName] != null
                && !string.IsNullOrEmpty(item.Fields[fieldName].Value))
                    sb.Append(item.Fields[fieldName] + " ");
            }

            if (indexAllTextFields)
            {
                //skip standard fields by checking for "_" in name
                foreach (Field field in item.Fields.Where(x=>!x.Name.StartsWith("_")))
                {
                    var customField = FieldTypeManager.GetField(field);

                    if (!string.IsNullOrEmpty(customField.Value))
                    {
                        if (customField is TextField)
                            sb.Append(customField.Value.Trim() + " ");
                        else if(customField is HtmlField)
                            sb.Append(Sitecore.StringUtil.RemoveTags(customField.Value.Trim()) + " ");
                    }
                }
            }
            return sb;
        }
    }

STEP 2: Add the following to your custom index configuration

For Sitecore Lucene:

<configuration ref="contentSearch/indexConfigurations/defaultLuceneIndexConfiguration" >
    <fields hint="raw:AddComputedIndexField">
        <field fieldName="related_content"></classname/>, </assemblyname/></field>
    </fields>
</configuration>

For Solr:

<configuration ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration" >
    <fields hint="raw:AddComputedIndexField">
        <field fieldName="related_content"></classname/>, </assemblyname/></field>
    </fields>
</configuration>

That's it! oh, and don't forget to rebuild your index.

If you like this approach or would like to suggest a better approach, please leave a comment.