Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Bo Damgaard Mortensen 719 posts 1207 karma points
    Dec 04, 2013 @ 16:06
    Bo Damgaard Mortensen
    0

    Indexing numeric datatypes with Examine/Lucene

    Hi all,

    I'm running my head against the wall here trying to make Umbraco index numeric datatypes to .. well.. numbers in Lucene :-)

    I have the following IndexSet:

    <IndexSet SetName="MyIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/MyIndexSet">
        <IndexUserFields>
          <add Name="prisIAlt" Type="Number"/>     
        </IndexUserFields>
        <IncludeNodeTypes>
          <add Name="MyDocType" />
        </IncludeNodeTypes>
      </IndexSet>    

    The 'prisIAlt' (translated: Total price) field should be used for a range search, something like this: prisIAlt:[0 TO 100]

    When I have a look at the data for this field in Luke, it shows the following:

    and my Lucene query doesn't work (obviously)

    Does anyone know why it doesn't index the 'prisIAlt' field as a number? :-) 

    Any help on this is greatly appreciated!

    Thanks in advance.

    - Bo

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Dec 04, 2013 @ 16:15
    Ismail Mayat
    0

    Bo,

    In lucene its all text. To do range searches and make piece of text numeric you need to inject it into a new field or update existing one into numeric format see http://our.umbraco.org/forum/developers/extending-umbraco/11819-Examine-range-query-numeric I had the issue and sorted myself lol

    I am not sure about what you currently have in there in prisalt field. Is the correct field going in? Is field text field in umbraco and have users put in non numeric content?

    Regards

    Ismail

  • Bo Damgaard Mortensen 719 posts 1207 karma points
    Dec 04, 2013 @ 16:19
    Bo Damgaard Mortensen
    0

    Hi Ismail,

    Thanks a lot for your time and response :-)

    To answer your questions: the 'prisIAlt' field in Umbraco is of type Numeric and I've entered 500 in one field and 1200 in another to check if the ranged search worked. I'll take a look at your post then, thanks!

    What's a bit confusing, though, is that you can actually specify a type on the custom fields and it seems like it's indexed like that. In my case, that would be the Numeric type.

  • Bo Damgaard Mortensen 719 posts 1207 karma points
    Dec 04, 2013 @ 16:24
    Bo Damgaard Mortensen
    0

    Have read your post now, Ismail :-) What exactly was the turning point for you to get it to work? Hook into the ExamineEvents_GatheringNodeData event and index the field using a .ToString("D6") and then use 

    IBooleanOperation Range(string fieldName, string start, string end);

    for the query?

    Thanks again!

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Dec 04, 2013 @ 16:51
    Ismail Mayat
    0

    Bo,

    Yup that sorted it. I am not sure what setting the property to numeric does? I think it was added in later examine versions so at the time i was trying to get this to work padding out the numeric field did the trick and I have been using it ever since.

    Regards

    Ismail

  • Conor Breen 11 posts 100 karma points
    Sep 11, 2017 @ 23:15
    Conor Breen
    0

    This is obvs years too late to be of any use to you guys, but just posting this for the benefit of others...

    I went through this same procedure to try and get int fields in my document type searchable as strongly typed ints so I could use the Range search criteria effectively. So, I created a custom Index set in ExamineIndex.config, like:

      <IndexSet SetName="CarIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/Car/">
        <IndexAttributeFields>
          <!-- Leave empty to include all Internal content fields -->
        </IndexAttributeFields>
        <IndexUserFields>
          <!-- Car Page -->
          <add Name="vehicleType" />
          <add Name="make" />
          <add Name="model" />
          <add Name="year" Type="NUMBER" />
          <add Name="price" Type="NUMBER" />
          <add Name="mileage" />
          <add Name="fuelType" />
          <add Name="transmission" />
          <add Name="engineSize" />
          <add Name="bodyStyle" />
          <add Name="description" />
          <add Name="photos" />
          <add Name="dateSold" Type="DATETIME" />
          <add Name="accessories" />
        </IndexUserFields>
        <IncludeNodeTypes>
          <add Name="car"/>
        </IncludeNodeTypes>
        <ExcludeNodeTypes>
        </ExcludeNodeTypes>
      </IndexSet>
    

    I created a custom indexer in ExamineSettings.config, like:

    <add name="CarIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
           supportUnpublished="false"
           supportProtected="false"
           analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"/>
    

    and I created a search query that used the indexer (or so I thought):

    var criteria = ExamineManager.Instance.SearchProviderCollection["**CarSearcher**"].CreateSearchCriteria(UmbracoExamine.IndexTypes.Content);
     var filter = criteria.NodeTypeAlias("car").And();
    
    if (advancedSearch.SelectedVehicleTypeID > 0)
    {
          filter = filter.Field("vehicleType", advancedSearch.SelectedVehicleTypeID.ToString()).And();
    }
    
    if (advancedSearch.SelectedMakeID > 0)
    {
          filter = filter.Field("make", advancedSearch.SelectedMakeID.ToString()).And();
    }
    
    if (advancedSearch.SelectedModelID > 0)
    {
           filter = filter.Field("model", advancedSearch.SelectedModelID.ToString()).And();
    }
    
    if (advancedSearch.MinYear.HasValue && advancedSearch.MaxYear.HasValue)
    {
           filter = filter.Range("year", advancedSearch.MinYear.Value, advancedSearch.MaxYear.Value, true, true).And();
     }
    
    if (advancedSearch.MinPrice.HasValue && advancedSearch.MaxPrice.HasValue)
     {
            filter = filter.Range("price", advancedSearch.MinPrice.Value, advancedSearch.MaxPrice.Value).And();
    }
    
    ...
    filter = filter.OrderByDescending(new[] { "_umb_updatedate" }).Compile();
    
    var umbracoHelper = new UmbracoHelper(UmbracoContext.Current);
    
    var results = umbracoHelper.TypedSearch(criteria);
    var count = results.Count();
    
    var page = results.Skip(Math.Max(advancedSearch.PageNumber-1, 0) * pageSize).Take(pageSize);
    

    But still wasn't getting any results. I could see from Examine Management tab on Developer section that my index was indexing the pages, and all the fields were there. If I put in a basic search just on node alias type, its matched to all my docs. But anytime I included the range search using the overloads that take INT parameters, I got no results, even if I put in a breakpoint in the code and copied the compiled Lucene query and ran it in Examine management section...

    So I did the above code, using .ToString("D8") to store additional properties in custom fields. Rebuilt index, could see the new fields were there and if I run the same lucene query as before in Umbraco (only with the different names - _year and _price instead of year and price) I got the expected results. However, skip to the page calling my query through the Examine API, and no results... what gives?

    Then it hit me... I hovered over Umbraco.TypedSearch and noticed there was another overload that accepts a SearchIndexer as a parameter. Did some Googling, and yep, unless you pass through your search indexer to the typed search, it will use the default indexer, which of course doesn't contain your custom fields (or indeed the fields where you have added data types in the config, e.g. ).

    Changed this line of code:

    var results = umbracoHelper.TypedSearch(criteria);
    

    To:

    var results = umbracoHelper.TypedSearch(criteria, ExamineManager.Instance.SearchProviderCollection["CarSearcher"]);
    

    And boom, now getting results on the front end. So now I wondered, given the conversation above about what the point of these attributes are and if they do anything, if maybe this is what happened you Bo? So I tried changing my query to use the original fields again ("price" and "year" instead of the padded versions "price" and "year") - and yes, it worked perfectly first time.

    So long story short, the data type attributes do seem to work as expected, but you must pass the custom indexer through to your Umbraco.TypedSearch call, not just when compiling your search criteria!!

  • Ismail Mayat 4511 posts 10090 karma points MVP 2x admin c-trib
    Sep 12, 2017 @ 07:57
    Ismail Mayat
    0

    Conor,

    Wow I did not know you could do that. I thought with TypedSearch you were stuck with whichever index is set to default. Well you live and learn!

    Regards

    Ismail

Please Sign in or register to post replies

Write your reply to:

Draft