Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Patrick 10 posts 33 karma points
    Jun 07, 2011 @ 04:49
    Patrick
    0

    Disable Lucene Examine

    Hi,

    Does anybody know how to turn off the Lucene Examine indexing job? We don't need to use the search function in the backend. Thanks.

  • Jesper Hauge 298 posts 485 karma points c-trib
    Jun 07, 2011 @ 08:36
    Jesper Hauge
    0

    I would think removing (or better: set comment brackets around, in case it crashes the backend and you need to restore) the contents of /config/ExamineIndex.config and /config/ExamineSettings.config should prevent Examine from indexing any content. 

    Whether this will speed up you site in any way I'm not sure, I would think the indexing events still occur, but that nothing wil be indexed, as soon as Examine discovers there's no indexes set up.

    Regards
    Jesper Hauge

  • Ricardo 24 posts 56 karma points
    Jul 04, 2012 @ 12:08
    Ricardo
    0

    @Jesper: Unfortunately this does not completely works. I tried several options to "remove" VirginGames.Web\config\ExamineIndex.config and VirginGames.Web\config\ExamineSettings.config but there are exceptions logged in umbracoLog table.

     

  • Jeroen Breuer 4810 posts 11885 karma points admin mvp c-trib
    Feb 14, 2013 @ 11:39
    Jeroen Breuer
    0

    Did you ever find a way to disable examine? We also would like to disable it because we don't need it and rebuilding the indexes also costs performance.

    Jeroen

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Feb 14, 2013 @ 16:08
    Shannon Deminick
    1

    You set the enableDefaultEventHandler = false for your index. This will essentially just leave Examine doing nothing as it won't listen to Umbraco events and thus no indexing.

    http://examine.codeplex.com/wikipage?title=UmbracoExamine&referringTitle=Documentation

    http://examine.codeplex.com/wikipage?title=Full%20Configuration%20Markup%20%26%20Options&referringTitle=UmbracoExamine

    Can I ask why you want to disable it ?

     

  • Herman van der Blom 10 posts 81 karma points
    Apr 13, 2017 @ 09:25
    Herman van der Blom
    1

    Let me answer: "I want to disable it because its unreliable and it crashed IIS Express"

  • Herman van der Blom 10 posts 81 karma points
    5 days ago
    Herman van der Blom
    0

    In my case I found a solution to make it more stable. When you have a lot updates on your pages Lucene indexes can be corrupted. I saw this happening with bulk loads. Now before I start the bulk loads I reindex first so you will have a clean Lucene index starting point. This works out fine for me.

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Feb 14, 2013 @ 16:09
    Shannon Deminick
    0

    @Jeroen, it only rebuilds indexes once when they are not there. Otherwise will just incrementally add to the index when saving/publishing. The performance should be negligeable. 

  • Jeroen Breuer 4810 posts 11885 karma points admin mvp c-trib
    Feb 14, 2013 @ 16:20
    Jeroen Breuer
    0

    Hello,

    The problem is that we've got a registration form which creates nodes after registration. When a lot of users do this the CPU goes to a 100%. I already started another topic about this (http://our.umbraco.org/forum/developers/api-questions/37022-Publishing-with-API-is-slow), but I recently discoverd that the problem is probably examine which is indexing like crazy. Disabling it probably solves this problem.

    Jeroen

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Feb 14, 2013 @ 16:49
    Shannon Deminick
    0

    Hey Jeroen, that might help the problem with CPU overhead but I suspect the problem is mostly due to the data APIs and their usage. If it's one node created the Examine will only index the one node at a time, not everytime a property is set or anything like that. Let me know how much it does change the performance though as I'd be interested. Examine has also changed the way it works between different versions.

  • Jeroen Breuer 4810 posts 11885 karma points admin mvp c-trib
    Feb 18, 2013 @ 17:59
    Jeroen Breuer
    0

    I've set enableDefaultEventHandler = false and removed all the files from \App_Data\TEMP\ExamineIndexes, but the indexes are still being rebuilded. Does it somehow try to rebuild them when it sees there are no files? I've looked through my code and the indexes aren't being rebuilded anywhere. Currently running on Umbraco 4.9. Do you know which version of Examine is with 4.9?

    Jeroen

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Feb 18, 2013 @ 18:09
    Shannon Deminick
    2

    Examine will rebuild the indexes on startup if they don't exist. If you've disabled event handlers they will not be updated with publishing, etc... so they'll just sit there. The rebuilding is a one time thing only since they don't exist during startup. That said, you can disable that too by adding this attribute to your config:

    <Examine RebuildOnAppStart="false">

  • Stuart Neal 9 posts 29 karma points
    Jan 23, 2015 @ 18:03
    Stuart Neal
    0

    Modify your ExamineIndex.config and ExamineSettings.config file as follows, to disable Lucene indexing entirely. I've done this on Umbraco 6.2.4 If you delete App_Data, the "ExamineIndexes" folder never gets recreated.

    config/ExamineIndex.config

    <?xml version="1.0"?>
    <ExamineLuceneIndexSets>
        <IndexSet SetName="ExternalIndexSet" IndexPath="~/App_Data/TEMP/ExamineIndexes/External/"/>
    </ExamineLuceneIndexSets>
    

    config/ExamineSettings.config

    <?xml version="1.0"?>
    <Examine RebuildOnAppStart="false">
        <ExamineIndexProviders>
            <providers>
            </providers>
        </ExamineIndexProviders>
        <ExamineSearchProviders defaultProvider="ExternalSearcher">
            <providers>
                <add name="ExternalSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine" enableDefaultEventHandler="false"/>
            </providers>
        </ExamineSearchProviders>
    </Examine>
    
  • Shannon Deminick 1387 posts 4598 karma points core hq
    Jan 27, 2015 @ 02:33
    Shannon Deminick
    1

    I'll re-iterate again... there shouldn't be any reason to disable Examine and you shouldn't since that is what is used as the media cache. If you don't have up to date indexes your media access will be slow and consume a lot of memory if you have a lot of media in your media tree. Most CPU problems that people blame on Examine/Lucene are actually not Examine's fault.

  • Herman van der Blom 10 posts 81 karma points
    Apr 13, 2017 @ 09:30
    Herman van der Blom
    0

    I have noticed more of these Messages of Umbraco HQ. That goes by the assumption that everything in Umbraco is "stable". And thats surely is not the case.

    What I miss in Umbraco is "defensive programming". Its means if something can and will crash or is unstable like the indexing of Lucene there must be an easy solution to disable it by design and not by what seams like hacking around.

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Apr 21, 2017 @ 03:10
    Shannon Deminick
    0

    No unfortunately there is not an 'easy' way to do that, if there were I'd definitely let you know. As above, media is retrieved from Lucene, withtout the indexes there you will have a very slow site if you are using media - which most people are.

    There are ways you can 'disable' Examine' but it comes with a cost and some things will simply no longer work.

    I don't know what version of Umbraco or Examine you are using but both of these things become more and more stable when you use the most recent versions.

    Decoupling Lucene from the core is something we want to do but it's not that easy because there are many things that rely on being able to search the indexes and return the correct data. It something we want and will work towards but it's not something that can be done quickly/easily but we'll get there eventually.

    In the meantime, if you are having issues and you can replicate then we will fix them if you let us know what they are.

  • Herman van der Blom 10 posts 81 karma points
    Apr 22, 2017 @ 20:41
    Herman van der Blom
    0

    The funny thing is the same Lucene crashes happens with the expensive SiteCore CMS. Anyway one way to solve it is to minimize the Lucene indexing. In my case I minimized on only one index Field. I had to load 800Mb of XML Legislation files in the Database. FYI, I use 7.5. of Umbraco.

    Problem you will get then is that with searches, you want to display information from those searches. I got that solved by getting the full Record by Id and reading/displaying the Fields I want.

  • Herman van der Blom 10 posts 81 karma points
    5 days ago
    Herman van der Blom
    0

    When there is a heavy update load on pages Lucene does not seems to catch up with indexing or better said keep those in sync.

    Those situations happens to me if I am bulk-loading. I load legislation pages and even try to load multiple pages at ones.

    I developed this so this process can be repeated multiple times. If there was a load before and I deleted this load first (=delete all those pages) then Lucene has problems keeping indexes 100% in-sync.

    A Lucene reindex is stable and syncs it again. So before I start loading I reindex first and then no Lucene errors are showing up.

  • Ian Grainger 60 posts 83 karma points
    May 13, 2015 @ 10:21
    Ian Grainger
    0

    We have a site that doesn't use Umbraco at all except to log in. I've had to make it an Umbraco site to get that working (though not sure why, I asked questions about that but couldn't get anywhere :( ) - so I don't want the indexer to do anything, ever.

    There's simply no point for any overhead at all.

  • Shannon Deminick 1387 posts 4598 karma points core hq
    May 13, 2015 @ 10:24
    Shannon Deminick
    0

    Sounds like a pretty strange case. If Umbraco isn't doing anything (i.e. not saving content or media), then Examine/Lucene won't do anything either, there is no overhead.

    I already mentioned mentioned on the previous page:

    You set the enableDefaultEventHandler = false for your index. This will essentially just leave Examine doing nothing as it won't listen to Umbraco events and thus no indexing.

  • Ian Grainger 60 posts 83 karma points
    May 13, 2015 @ 10:38
    Ian Grainger
    0

    It *is* a strange case! That's why it's frustrating that I had to install the whole of Umbraco to get it working...

    I've now suddenly started seeing a problem with "OwinStartUpAttribute not specified" even though I've set owin:AutomaticAppStartup to false, though. So I guess I actually *still* don't have a solution to the problem of using just the Umbraco membership provider and none of the rest of Umbraco:(

  • Ryan Rahlf 3 posts 74 karma points
    Aug 11, 2015 @ 15:46
    Ryan Rahlf
    1

    I'm currently seeing a "device is not ready" error in a shared hosted environment with multiple Umbraco installs. This is apparently related to the Lucene Examine, which is what brought me to this thread.

    If my choices are "slow media loading" and "yellow error screen" for my client's websites, I guess I'll have to take the slow media. I'd prefer not to disable Lucene since I'm not sure what else I'll be losing, but my Umbraco sites absolutely must stay up.

    Is anyone in this thread familiar with this error? Is there's another way to address this exception without disabling Lucene?

    Related: https://our.umbraco.org/forum/umbraco-7/using-umbraco-7/70038-getting-systemio-error-from-lucene-index

    https://our.umbraco.org/forum/developers/api-questions/43516-umbraco-on-azure-The-device-is-not-ready-bug

    http://issues.umbraco.org/issue/U4-3222

  • Jules 2 posts 73 karma points
    Aug 28, 2015 @ 16:10
    Jules
    0

    I have started to wonder if this issue on shared hosting is actually to do with some process that runs in the background and touches lucene during some operation potentially corrupting the index. Outside of that I have no idea why I keep getting the "Device Not Ready" error messages at seemingly random intervals

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Aug 31, 2015 @ 13:03
    Shannon Deminick
    0

    If your host's server setup has your IIS website running off of a network share, then you might have issues with Lucene and IO latency.

    In this case you have a couple of options:

    In 7.2.8, you can configure your indexers and searchers with the attribute: useTempStorage="LocalOnly"

    This needs to be configured for both your indexers and searchers:

    For example:

      <add name="InternalIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
           supportUnpublished="true"
           supportProtected="true" 
           analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net"
           useTempStorage="LocalOnly"/>
    
      <add name="InternalSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
           analyzer="Lucene.Net.Analysis.WhitespaceAnalyzer, Lucene.Net" 
           useTempStorage="LocalOnly"/>
    

    What this will do is store your indexes on the local hard disk (not the network share) in the ASP.Net temp files location. This also means that anytime you deploy new DLLs or bump your global.asax, the ASP.Net temp files will be removed so when your site restarts it will rebuild the indexes.

    If you are on an older version of Umbraco, you can use this project: https://github.com/Shazwazza/UmbracoExamine.TempStorage

    which does a very similar thing but requires you update some different configuration options (docs are on that page).

    This should resolve IO latency issues for when your hosting provider is hosting your website from a file share (this is true for Azure websites!)

  • Laurence Gillian 572 posts 1082 karma points
    Sep 18, 2015 @ 11:01
    Laurence Gillian
    0

    I'd just like to confirm, I have been seeing this error on Azure since May, with a website that has a lot of media in the backoffice, as well as a lot of media on the frontend.

    Other symptoms we have seen are that the website freezes (waiting for available socket) after a publish event for up to 60 seconds.

    I had originally suspected to be caused by the content editors uploading very large GIF's which ImageResizer was struggle to resize, but disabling cropping on GIF's did not alter performance.

    I have updated our Examine configuration to use the recommendations above, we've got monitoring setup, so should be able to see if this resolves the issue, as it's currently failing over quite frequently!

    Wishing all a good weekend, Laurence

  • Niels Hartvig 1889 posts 2223 karma points admin core hq c-trib
    Sep 20, 2015 @ 19:51
    Niels Hartvig
    0

    @laurence Did the updated settings help?

  • Laurence Gillian 572 posts 1082 karma points
    Sep 21, 2015 @ 08:34
    Laurence Gillian
    0

    Hey @niels,

    1. The site hasn't crashed, (no further IO errors) since
    2. Publish event still slows down (blocks) requests the website, though it's not as bad as before (e.g. 20 seconds, rather than ~5 minutes)

    I'm not sure the following logs are that helpful, however just incase!

    • 2015-09-21 08:15:12,712 [26] INFO Umbraco.Core.Publishing.PublishingStrategy - [T42/D4] Content 'Contact' with Id '4728' has been published.

    • 2015-09-21 08:15:19,683 [26] INFO
      Umbraco.Core.PluginManager - [T42/D4] Starting resolution types of umbraco.interfaces.ICacheRefresher

    • 2015-09-21 08:15:20,093 [26] INFO
      Umbraco.Core.PluginManager - [T42/D4] Completed resolution of types of umbraco.interfaces.ICacheRefresher, found 20 (took 406ms)

    • 2015-09-21 08:15:24,712 [26] INFO
      umbraco.content - [T5/D4 ] Save Xml to file...

    • 2015-09-21 08:15:24,758 [26] INFO umbraco.content - [T34/D4] Saved Xml to file.

    12 seconds between Publish and Save XML to file event.

    Basic text page document type, minimal properties, just strings and a rte.

    Cheers, Laurence

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Sep 21, 2015 @ 08:44
    Shannon Deminick
    0

    @Laurence, you haven't provided the actual problem that you are having. What is your Umbraco version? You've written:

    I'd just like to confirm, I have been seeing this error on Azure since May

    But what is "this error" (there are numerous different errors referenced on this post)

  • Laurence Gillian 572 posts 1082 karma points
    Sep 21, 2015 @ 09:07
    Laurence Gillian
    0

    Ah yes! I was referring to your comment, regarding IO Latency Problems (and Azure)

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Sep 21, 2015 @ 09:12
    Shannon Deminick
    0

    and your umbraco version is ... ? ;)

    Please remember, the more information you can provide about your issues the better everyone can help and the faster you can get an answer.

  • Laurence Gillian 572 posts 1082 karma points
    Sep 21, 2015 @ 09:26
    Laurence Gillian
    0

    Sleepy! ;-) Umbraco version 7.2.6 assembly: 1.0.5632.22681

  • Shannon Deminick 1387 posts 4598 karma points core hq
    Sep 21, 2015 @ 09:48
    Shannon Deminick
    0

    The publish event should not block at all and in fact it far less blocking than it was in 7.2.4.

    It is very strongly advised to upgrade to 7.2.8 asap since 7.2.6 and 7.2.7 have known issues with the xml cache file locking.

    You could also try to change your logging level to Debug instead of Info to see if there's other things happening during this publishing time.

  • denisedelbando 133 posts 330 karma points
    27 days ago
    denisedelbando
    0

    For the device not ready issue for media, what i did was rebuild my index. Click on one of the image then just click save.

    Not the best advise but right now i am still developing locally so this is fine.

    and yes i got this issue on 7.6.4. and to think this is a new installation.

  • claudio morandi 9 posts 76 karma points
    1 week ago
    claudio morandi
    0

    sorry for late reply to this topic. I have a multiple domain instance of umbraco and when restarting the ISS (or the webserver) I notice some errors connected with Examine Lucene Index.

    On some websites (not all of them but just a couple for example), I have an error 500 on the home page and the error is the following: The process cannot access the file 'xxxxxxxxx\App_Data\TEMP\ExamineIndexes\InternalMember\Index\write.lock' because it is being used by another process.

    Do you have any suggestion? Which could be the problem?

  • Herman van der Blom 10 posts 81 karma points
    5 days ago
    Herman van der Blom
    0

    I had lots of these errors to. In mine case reindexing solves the problems.

  • claudio morandi 9 posts 76 karma points
    5 days ago
    claudio morandi
    0

    reindexing on every build of project?

Please Sign in or register to post replies

Write your reply to:

Draft