Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Justin McKinley 12 posts 33 karma points
    Apr 19, 2010 @ 16:35
    Justin McKinley
    1

    Severe Issue with macro caching under heavy load

    Greetings!

    Please let me know if this should go into a different forum, or if this needs to be posted to the issues list in codeplex. Thank you in advance for your time in reading and helping me with this. This is a major issue that we are having, affecting the performance of our production web site (we've had to turn Umbraco caching completely off, and can sometimes get thousands of requests per hour).

    I'm hoping I'm an idiot and there's one setting I'm forgetting to set that will fix all of this!

    We are using Umbraco for a fairly busy website, and we are encountering problems when caching is enabled on our macros - specifically on macros that are .NET user controls. The short version of the problem is: When the website is under moderately heavy load (20+ users at the same time reloading the same page), the content that is cached gets messed up. Sometimes it doesn't load at all, so all of areas in the page where the macros are supposed to be are blank; sometimes the macro content seems to overwrite itself - we'll see data repeated in the same control.

    We are using a fairly-heavily customized installation of Umbraco 4.0.3, using ASP.NET 3.5. We have about 50 user controls (almost all of them pull data from a DB) that we use at many places within the site. I was concerned that maybe we broke this when customizing it, or maybe our user controls were doing something out of the ordinary, but I was able to recreate the problem using a stock installation of Umbraco (installed via Web Installer), and a project with a single user control that just queries the Northwind database (Orders table) and writes the data to the page. Of course, the macro that is this user control has caching enabled. Changing any of the cache settings (time, cache by page/personalized) does not seem to change the outcome.

    To simulate the load, I'm using a load test within Visual Studio Team Edition. If I set the number of concurrent users at 20 or more hitting a page that only contains a single macro/user control (the one that queries the Northwind database) with caching enabled, I see this issue. I don't see the issue if I just try to hit the page as hard as I can (by hitting refresh in the browser repeatedly, or by opening a bunch of browser windows and refreshing them). 

    Our best guess is that the problem is concurrency issues storing content in the cache - normal multi-threaded types of issues. For example:

    1. While under heavy load, request #1 is being processed, it checks the cache to see if the content for macro A is stored in it; it does not find macro A, so it starts rendering macro A.
    2. Before step #1 is complete, request #2 is being processed. It also checks the cache to see if the content for macro A is stored; it doesn't find macro A (because step #1 hasn't completed yet), so it starts rendering macro A.
    3. Request #1 completes rendering of macro A and writes it to the cache.
    4. Before step #3 is complete, request #2 completes rendering of macro A and also writes it to the cache, resulting in the content in the cache being some overlapped combination of the two renderings.
    Note that that's just our educated guess - we could be way off on what is actually happening.

    Some more details:

    • The "Maximum Worker Processes" setting in IIS 7 is set to 1 (that's the first thing I checked)
    • Our production web site is using LINQ-to-SQL to access the data for the user controls/macros, but for simplicity, my test project is just using a SqlDataReader. 
    • I've tested having a user control that is just a GridView hooked to a SqlDataSource that just dumps the Orders table from the database. When I test using this, the problem I most often have is that the macro fails to load occasionally. I then created a new user control that uses the same query, but uses a SqlDataReader (in the code) and manually outputs the data to a PlaceHolder in the user control. This user control more accurately reflects a lot of what we're using in the production web site. The problem I most often have with this method is that the data often overlaps or repeats itself. For example, the user control is supposed to write the first 40 rows from the DB table to the page, but when the problem occurs, I get the first 40 rows correctly, followed by 10-15 of the same rows, with the last row being cut off mid-row. (These same rows are from the beginning of the page, as if the user control got rendered again and started over from the top.)
    • When the problem of overlapping occurs, the overlapped content stays in the cache until the next time the cache expires. (I usually set the cache very low, like 5 or 10 seconds, for testing, but this problem still occurs when the cache is set higher (120 is our preferred timeout).)
    • Compiling/changing configuration to be in debug or release mode does not seem to affect the problem.
    • Changing the Xml Cache settings (enabling or disabling) does not seem to affect the problem.
    • I've been able to recreate this on many different machines, including my development server (a fairly well-powered machine with Windows 7), a desktop machine running Windows 2008 server, and on our production server, a decent machine also running Windows 2008 server. The biggest thing in common that all of the machines I've recreated this on have is that they're all running IIS7. I have not tried recreating this in IIS 6.
    I will try to upload or add our sample code to this thread so you may be able to recreate the problem on your end.
    Thank you again for your time! I really appreciate any help that you can give on this issue.
    Justin

  • Justin McKinley 12 posts 33 karma points
    Apr 19, 2010 @ 16:37
    Justin McKinley
    0

    Here is the code for the simple data user control that we are using to recreate the problem:

     

     

    TestDataUserControl2

    <%@ Control Language="C#" AutoEventWireup="true" CodeBehind="TestDataUserControl2.ascx.cs" Inherits="TestUmbracoUserControlsCache.TestDataUserControl2" %>
    <asp:PlaceHolder ID="phContent" runat="server"></asp:PlaceHolder>

     

    TestDataUserControl2.ascx.cs

    using System;
    using System.Data.SqlClient;
    using System.Collections;
    using System.Configuration;
    using System.Data;
    using System.Linq;
    using System.Web;
    using System.Web.Security;
    using System.Web.UI;
    using System.Web.UI.HtmlControls;
    using System.Web.UI.WebControls;
    using System.Web.UI.WebControls.WebParts;
    using System.Xml.Linq;
    
    namespace TestUmbracoUserControlsCache
    {
        public partial class TestDataUserControl2 : System.Web.UI.UserControl
        {
            protected void Page_Load(object sender, EventArgs e)
            {
                RenderControls();
            }
    
            private void RenderControls()
            {
                try
                {
                    SqlConnection conn = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["NorthwindConnectionString"].ConnectionString);
                    SqlCommand cmd = new SqlCommand("SELECT top 120 * FROM [Orders] ORDER BY [OrderDate]", conn);
                    //SqlCommand cmd = new SqlCommand("SELECT top 5 * FROM [Orders] ORDER BY [OrderDate]", conn);
                    conn.Open();
                    SqlDataReader reader = cmd.ExecuteReader();
                    while (reader.Read())
                    {
                        for (int i = 0; i < reader.FieldCount; i++)
                        {
                            Literal lit = new Literal();
                            lit.Text = reader[i].ToString();
                            phContent.Controls.Add(lit);
                        }
                        phContent.Controls.Add(new Literal { Text = "\n<br />" });
                    }
                    reader.Close();
                    conn.Close();
                }
                catch (Exception ex)
                {
                    Response.Write("<br />XXX Exception: " + ex.ToString() + "<br />");
                    //throw;
                }
    
            }
        }
    }

     

    x

     

  • Justin McKinley 12 posts 33 karma points
    Apr 19, 2010 @ 16:57
    Justin McKinley
    0

    I forgot to add one possibly important point:

    I've tried running the Application Pool in Classic and Integrated mode, and it doesn't seem to change the outcome (i.e., we still have the problem). And I'm using the IIS7/ASP.NET 3.5 -specific web.config file that was provided.

     

  • Tommy Messbauer 16 posts 36 karma points
    May 03, 2010 @ 20:06
    Tommy Messbauer
    0

    I am seeing similar issues except that I have pages that return 500s sometimes.  Also, there are performance spikes where a page will have a 50ms response time, then every few seconds a batch of request will have 200ms-300ms response times.  I have not gotten into turning off macro caching.  I looked at the source code and there are no locks around the cache sections like you say.  This can cause all types of issues.  I am concerned as well.  Please let me know if you would like to talk direct.

  • Tommy Messbauer 16 posts 36 karma points
    May 05, 2010 @ 16:20
    Tommy Messbauer
    0

    I opened up the Umbraco core and fixed this cache race condition.  Source code is checked in here.

    http://umbraco.codeplex.com/SourceControl/changeset/changes/65950

  • SO 17 posts 41 karma points
    Jul 14, 2010 @ 23:15
    SO
    0

    I have the same problem. Cached macros randomly doesn't show under heavy load. It's easily replicable by running apache ab tool and refreshing browser - some cached macros are not rendered. AB tool reports failed requests too. I've tried Tommy's solution, but it doesn't seem to work for me . Maybe there are some configuration issues because not many people reports this problem.

  • Aaron Powell 1708 posts 3046 karma points c-trib
    Jul 15, 2010 @ 06:58
    Aaron Powell
    0

    You should be closing your reader and connection to the database in a finally statement, otherwise they wont be closed when an exception is thrown.

  • SO 17 posts 41 karma points
    Jul 15, 2010 @ 16:36
    SO
    0

    I don't think that reader is the problem in this particular situation. I've just recreated this behaviour in V4.5  (latest post relates to  V4.0.4). After clean installation with runway module I cached top navigation, ran ab tool and refreshed browser. In about 5-10% of requests navigation doesn't appear and ab shows errors. No cache no problems .

  • Jose Reyes 3 posts 23 karma points
    Jul 27, 2010 @ 18:10
    Jose Reyes
    0

    Hello guys, does anyone has find the solution for this issue ?, currenty I'm running a website but it's extremadly slow but when I activate the cache this increase response time up to 50%, but I'm still experiencing this bugs.

  • Eric S 10 posts 30 karma points
    Sep 15, 2010 @ 16:58
    Eric S
    0

    Having a similar issue, where cached macros do not always appear. The load is not especially heavy, but it's on IIS7. Our local dev IIS6 site doesn't have the issues. Maybe it's an IIS7 thing?

  • Eric S 10 posts 30 karma points
    Dec 16, 2010 @ 17:53
    Eric S
    0

    Please upvote the issue on Codeplex if you are experiencing this too.

    http://umbraco.codeplex.com/workitem/27610?ProjectName=umbraco ;

  • Justin McKinley 12 posts 33 karma points
    Mar 29, 2011 @ 18:57
    Justin McKinley
    0

    Tommy,

        The link you posted to code you updated (11 months ago) doesn't work any more. Could you please let us know where that is now? I never got a chance to look at it. 

  • Henri Toivonen 77 posts 111 karma points
    Oct 06, 2011 @ 16:45
    Henri Toivonen
    0

    Any news about this issue? I'm having some problems also with cached macros not rendering anything sometimes under load.

  • Jonathan Cohen 40 posts 115 karma points
    Oct 25, 2011 @ 19:53
    Jonathan Cohen
    0

    Any new info, input from the umbraco core team or any other gurus.

    This is a very importat issuse that have been a problem from 4 - 4.7.1 - anyone that can point in the right direction in the source were the probelm lies (Tommy do you have your fix to share?)  This is a show stopper in high load sites, and there you really want to use cache as mush as possible - but we can't now.

    /Jonathan

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Oct 25, 2011 @ 21:15
    Niels Hartvig
    0

    Any additional info would be greatly appreciated as there's a few reports of issues under heavy load, but much more on the opposite and no body have been able to provide reproduction steps so far. If you could help that would be brilliant - we'd love to fix this!

  • Jonathan Cohen 40 posts 115 karma points
    Oct 25, 2011 @ 21:42
    Jonathan Cohen
    0

    Hi Niels,

    We have a complete site, where the problem exists, but I shall try to "minimize" the projekt and make a zip of web + db for testing that i could provide you for futher investigation. But I get the exact same problem as the initial post in this thread, and as a bit futher down, someone with a clean install and then runaway installation get the sam problem.

    We have a Win 2008 server r2, IIS7.5, SQL2008, .net4.0 and umbraco 4.7.1 enviroment.

    No difference if it is cache by page or for site, when we enter anything else than "0" in duration it randomly brakes when someone browses the page - more often when someone also is editing content is my "feel".

    "Breaks" usually means one of two thing, eihter the macro is multplied or it is blank - most often blank. the UmbracoLog do not show anything when it happens.

    /Jonathan

     

     

     

     

     



  • Henri Toivonen 77 posts 111 karma points
    Oct 25, 2011 @ 21:43
    Henri Toivonen
    0

    This is a tricky one for sure. I have not been able to really reproduce this, other than the fact that I see it happen live on a production server now and again during peak hours.

    I've been thinking recently about making somekind of experiment, make a bunch of macros that do something and display something, put them all on a page, start up a load tester and poke around.

    Niels, lets say I manage to get it to happen more often, what would the next step be? Check the debugtrace? Logs? Is there a debugsetting somewhere that would be helpful?

    I appreciate the fact that bugs you can't see at home are really hard to fix, so I'll do my best to help out instead of just being grumpy. :)

     

    /Henri (eatfrog on twitter)

  • Justin McKinley 12 posts 33 karma points
    Oct 25, 2011 @ 22:02
    Justin McKinley
    0

    Niels,

       My initial 2 posts on this thread attempt to describe how to recreate the problem in a very simple site. I'm able to recreate it using the code and steps I gave there. Let me know if that helps or not, and I can try to provide more details, or I can send you the test site.

       We determined the issue to be in macro.cs, I believe (I don't have the code handy). I believe our fix, which is really a hack for now (it has its own issues, but caching now works), was to do the caching at a different level. Instead of caching the macro how it's done in the original implementation, we change it (in \umbraco\templateControls\Macro.cs and in macro.cs) to cache the actual (rendered) HTML content of the macro. This, along with some locks and a singleton, seems to fix the issue I described.

        I can send my code changes if desired, but like I said, it's not perfect.

  • Jonathan Cohen 40 posts 115 karma points
    Oct 28, 2011 @ 21:14
    Jonathan Cohen
    0

    Reproduce this:
    1. Download 4.7.1 from codeplex
    2. Unzip and set permissions
    3. Create IIS pointing to the folder
    4. Create an empty DB
    5. Install normal way, choose starter kit: Simple and skin: friendly ghost
    6. Create a Razorscript/Macro in back end based on "Navigation" template, no extra code
    7. Set cache to 3600 sec
    8. Choose macro on the master template anywhere between body...
    9. Run LoadUI (http://www.loadui.org/) or any other load tool - with 500 users - browse your site and the meny disapears every now and then 20% of the page loads.

    My environment:
    Windows 7 sp1
    IIS 7.5.7600.16385
    .NET4.0, Integrated, Network service
    SQL2008

    But in all my Umraco installations (from 4 to 4.7.1 on .net 2 tp 4) on different live servers and development I have the problem, it seems only happening to ascx and razor, not xslt macros. And I really only noticed it on our latest release that has a lot of visitors.

    /Jonathan

  • Jonathan Cohen 40 posts 115 karma points
    Oct 30, 2011 @ 19:43
    Jonathan Cohen
    0

    Justin, I would be very happy if you shared your code - what umbrao version did you do your fix for?

     

    /Jonathan

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 03, 2011 @ 14:10
    Dave Woestenborghs
    0

    I had this error too under heavy load. Under heavy load the macros' weren't rendered 50-70% of the time. I wasn't able to find the cause but I found a solution.

    After adding renderEvent="Render" to my macro tags in the rendertemplate I had only 2% failing under heavy load.

    <umbraco:Macro renderEvent="Render" ... />

  • julius 107 posts 289 karma points
    Nov 03, 2011 @ 23:41
    julius
    0

    This is a very interesting read. I have been battling this problem for two days now and I need a fix urgently. Our client's site doesn't perform very well and we promised to improve performance and to provide proof in the form of a load test. We are using Umbraco 4.5.2 and load testing the site causes macro's to disappear and I've seen duplications too.

    The weirdest thing we've seen so far is that one of our macro's is throwing a nullreference exception which we've never seen before and which is actually impossible in normal circumstances. In the codebehind of our Navigation.ascx control we have a list of NavigationPanels which is filled in a loop like this.

    List<NavigationPanel> panels = new List<NavigationPanel>();
    foreach(Section section in sections){
    NavigationPanel panel = someClass.GetPanelForSection(section);
    if(panel != null){
    panels.Add(panel);
    }
    }

    Even though there is a null test we are getting a nullreference exception on one of the panels. This only happens after the combination of Umbraco caching and load testing. It seems as if the ascx is run whil Umbraco has forgotten about the backing code and data? I really have no clue why this happens. Any advice is most welcome. We are currently building our own caching mechanism as an (hopefully) intermediate solution.

  • Jonathan Cohen 40 posts 115 karma points
    Nov 04, 2011 @ 17:48
    Jonathan Cohen
    0

    Hi

    The error is happeningn between CreateChildControls and Render in Macro.cs. Somewhere between there this.Controls.Count() suddenly is "0".

    We are giving up on this and use .Net OutputCaching on pagelevel and this code to kick cache when publish happens:

     Response.AddFileDependency(Server.MapPath(umbraco.GlobalSettings.ContentXML)); 

    Another problem we faced when trying OutputCaching in ascx was that this is not handled in the umbraco core, the following error happens:

    "Unable to cast object of type ‘System.Web.UI.PartialCachingControl’ to type ‘MyUserControl’."

    Read more here http://msdn.microsoft.com/en-us/magazine/cc163577.aspx, eventually this could be a way for the core to handle cache for ascx controls. It is also intressting to see what will happen in version 5 and caching were things like this must be working.

    /J

     

     

     

     

  • Tim 1193 posts 2675 karma points MVP 3x c-trib
    Nov 07, 2011 @ 11:31
    Tim
    0

    I've had this error too on 4.5 and 4.7 sites. When caching is disabled, everything works fine, with caching enabled, macros occasionally vanish. Turn off caching, and everything is fine.

    I think that there is a race condition in the code that loads the Macros (Tommy Messbauer's message early mentions this as well). If we can find his checkin, that should give an idea of where the error lies hopefully! It looks like what's hapening is that when the Macros are loaded, that the macro load is being called twice at the same time, causing a race condition, and the blank/malformed macro content.

    @niels, do you guys have a better search on Codeplex than the default one? I've spent several hours trying to find the checkin that Tommy mentions, do you have any ways of finding it that us general users don't?

    :)

    This issue appears to be related: http://umbraco.codeplex.com/workitem/29068 do we know if the fix mentioned was rolled into the core?

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 07, 2011 @ 12:09
    Niels Hartvig
    0

    I think I've found the bug (line 398 of macro.cs) where the check to see if there's content in the cache is made but not stored in a local variable which means that the cache could be emptied right after the check. I've just committed a fix and hopefully this will fix the issues! 

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 07, 2011 @ 12:11
    Dave Woestenborghs
    0

    I found Tommy's changeset : http://umbraco.codeplex.com/SourceControl/changeset/changes/ec5b367404b5

    Maybe this can help

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 07, 2011 @ 12:13
    Niels Hartvig
    0

    @Jonathan: Thanks for the pointer to the article on PartialCachingControl. Ironically, I couldn't find anything on this when I created Umbraco which was what initially caused my to build the caching engine in Umbraco (would be needed for XSLT, etc anyway).

    I'll see if we can add support for CacheDirectives in 4.7.2 thanks to this. For now it's all about getting 4.7.1.1 out!

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 07, 2011 @ 12:25
    Niels Hartvig
    0

    The fix is in the latest nightly - any takers?
    http://nightly.umbraco.org/umbraco%204.7.1/4.7.1/4.7.1.460.zip

  • Justin McKinley 12 posts 33 karma points
    Nov 07, 2011 @ 15:52
    Justin McKinley
    0

    I believe we will be testing the fix today or tomorrow. 

    Thank you!

  • Jonathan Cohen 40 posts 115 karma points
    Nov 07, 2011 @ 15:53
    Jonathan Cohen
    0

    Hi Niels,

    I did a quick test on one of our "problematic" projekts, and no difference (both ascx and razor) :-(

    I copied the the items in bin folder, rebuild - that should have made it? I will try a bit more on the source later...

    Any one else had eny luck?

    /Jonathan

  • Andreas Gehrke 20 posts 44 karma points
    Nov 07, 2011 @ 17:31
    Andreas Gehrke
    0

    @Niels, how come the macroHtml variable is assigned twice. Once on line 388 and again on line 392. No need for fetching the HTML two times?

  • Justin McKinley 12 posts 33 karma points
    Nov 07, 2011 @ 21:25
    Justin McKinley
    0

    We just tested with the updated macro.cs code (we yanked all of our bugfix code out, of course), and it did not fix the problem for us. We experience the same exact symptoms as before (same as everyone else here is experiencing). 

    Neils, if you read through the theories here (including mine), it seems to be a race condition, with (under load) multiple requests rendering the macro and putting it into the same place in the cache at nearly the same time, causing overlapping (that used to happen in v4.5 for us, doesn't seem to happen in v4.7) or blanks. This theorized cause might have changed between v4.5 and 4.7 with the reworking of the macro.cs code, but since it's behaving nearly the same way, I would assume it's a similar cause, at least.

     

  • Justin McKinley 12 posts 33 karma points
    Nov 07, 2011 @ 21:28
    Justin McKinley
    0

    Here is the code that we updated and added to attempt to fix this macro caching bug. As I've said before, it's not perfect, but it does mostly work for us. It updates the macro.cs and umbraco/templateControls/Macro.cs files, and adds a new file, MacroCacheSingleton.cs. 

    http://www.mediafire.com/?83qma5s7h7tspej

    It basically changes the caching to cache the fully-rendered content of a user control (i.e., the HTML) instead of the user control object, and thus moves most of the work to the umbraco/templateControls/Macro.cs class. I'm not sure if this is the best way to do this, but like I said, it works for us. I'd be interested to hear everyone's thoughts on it.

     

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 07, 2011 @ 21:45
    Niels Hartvig
    0

    Sorry, guys - I was sure I had it nailed. I'll look into any missing locks tomorrow!

    @Justin: doesn't storing the full html of the user control the state to get lost? Ie. if you have buttons or anything else depending on post backs?

  • Justin McKinley 12 posts 33 karma points
    Nov 08, 2011 @ 14:23
    Justin McKinley
    0

    Niels, 

       Yes, the state would get lost. I thought that the cache didn't work with postbacks anyway (it's been a while since I've used it, so I may be wrong). But in any case, it doesn't matter for us, we just use it to cache non-interactive content. I'd love it to work better than that, but this was a quick (hack) solution to get something working for us.

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 09, 2011 @ 10:04
    Dave Woestenborghs
    0

    @Niels,

    Did you have a look at Tommy's changeset ? Maybe this can be a solution:

     umbraco.codeplex.com/.../ec5b367404b5

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 09, 2011 @ 10:11
    Niels Hartvig
    0

    @dawoe: It's already a part of the core and has been since 4.0

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 09, 2011 @ 10:19
    Niels Hartvig
    0

    @Andreas Gehrke: A bug (missed by refactor). I've corrected it - thanks!

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 09, 2011 @ 10:23
    Niels Hartvig
    0

    @dawoe: Just doubled check and in fact it's not. I'll find out what happened!

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 11, 2011 @ 17:42
    Dave Woestenborghs
    0

    @Niels : I just got the latest source code from codeplex and did some testing. 

    In marco.cs constructor this if statement is never called when I do some testing :

     

     if (macroObjectCache.ContainsKey(macroCacheIdentifier + id))
               {
                    var tempMacro = macroObjectCache[macroCacheIdentifier + id];
                    Name = tempMacro.Name;
            Alias = tempMacro.Alias;
                    ScriptType = tempMacro.ScriptType;
                    ScriptAssembly = tempMacro.ScriptAssembly;
                   XsltFile = tempMacro.XsltFile;
                    scriptFile = tempMacro.ScriptFile;
                    Properties = tempMacro.Properties;               
    propertyDefinitions = tempMacro.propertyDefinitions;
                    RefreshRate = tempMacro.RefreshRate;
                    CacheByPage = tempMacro.CacheByPage;
                    CacheByPersonalization = tempMacro.CacheByPersonalization;
                    DontRenderInEditor = tempMacro.DontRenderInEditor;

                    HttpContext.Current.Trace.Write("umbracoMacro",  string.Format("Macro loaded from cache des (ID: {0}, {1})", id, Name));
                }

     

    So all the sql statements in the else branch are executed each time. Maybe this is the bottle neck

     

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 17, 2011 @ 13:21
    Dave Woestenborghs
    0

    @Niels 

    Any update on this issue ?

  • Christophe 9 posts 29 karma points
    Nov 26, 2011 @ 13:13
    Christophe
    0

    I also did some test with this and the information in the macroObjectCache dictionary is always lost when exiting. Its state is not persisted. Due to that, every macro call result in a DB access to retrieve its details and we get exceptions that there are no more connections available in the connection pool.

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 26, 2011 @ 14:23
    Niels Hartvig
    0

    @Christophe: I committed a complete refactor of the macro.cs class yesterday that changes how data is fetched and cached as well as solving a memory leak around xslt extensions. Hopefully this will end all issues:

    http://umbraco.codeplex.com/SourceControl/changeset/changes/20be2d518820

    I'll release a test build on Monday when I've confirmed that all basic functionality works as expected but until then feel free to try the nightly (just *not* in production!):
    http://nightly.umbraco.org/umbraco%204.7.1/4.7.1/4.7.1.466.zip ;

  • Jonathan Cohen 40 posts 115 karma points
    Nov 26, 2011 @ 19:33
    Jonathan Cohen
    0

    Hi Niels,

    I downloaded the latest source from codeplex (.466) and test run underload and the problem still persist with the disapearing  ascx macros.

    I will try some more tomorrow and step/debug the code.

    /Jonathan

     

     

     

  • Christophe 9 posts 29 karma points
    Nov 27, 2011 @ 01:31
    Christophe
    0

    @Niels We very recently identified that problem and we have modified the Macro.cs file to declare the "macroObjectCache" as static + lock around the "Add" method. This solved the performance issues on our test server and we plan to put it in production soon.

    I'll make some test with the .466 change to see the impact on performance.

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 28, 2011 @ 08:11
    Niels Hartvig
    0

    .466 won't affect macro result caching only the loading and caching of macro meta data. Sorry for the confusion (this thread is getting long and with multiple issues :-)). I'm on the macro.cs class today to fix the macro result caching issues!

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 28, 2011 @ 13:09
    Dave Woestenborghs
    0

    Hi Niels,

    I just got the latest source from codeplex. Marco's that are marked to cache by page don't display at all the moment.

    Dave

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 28, 2011 @ 13:10
    Niels Hartvig
    0

    @dawoe: Nope, it's fixed and I'm testing. New commit coming later today.

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 28, 2011 @ 13:42
    Dave Woestenborghs
    0

    Okay,

    I will wait for the new commit then

    Dave

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 12:10
    Niels Hartvig
    0

    Wow, I've finally found the cause of this bug (where cached controls randomly fails under load). Turns out that the control state is shared across threads when it's cached the way we're caching it which makes it fails if multiple threads are manipulating it at the same time. If I clone the cached control everything works fine (not saying that will be the *right* solution to the issue - just that this no longer causes problems under load).

    Getting much, much closer. Boy, this been frustrating :-)

  • Henri Toivonen 77 posts 111 karma points
    Nov 29, 2011 @ 12:12
    Henri Toivonen
    0

    Excellent news Niels! :)

    ETA on 4.7.1.1? Really need to roll out that fix on a few sites asap.. ;)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 12:14
    Niels Hartvig
    0

    4.7.1.1 will come out this week no matter if we nail this issue or not. Fingers crossed we do by the end of the day, then 4.7.1.1 out for testing tomorrow. Friday the latest, at least.

  • sun 403 posts 395 karma points
    Nov 29, 2011 @ 12:43
    sun
    0

    by the way, Please update tinymce and jquery,jqueryUI. It can fix some bugs

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 12:46
    Niels Hartvig
    0

    @sun It can also introduce a lot of new bugs, so we won't do that. We did it with 4.7.1 and it introduced quite a lot of issues in 3rd party packages so for now we'll leave it.

  • sun 403 posts 395 karma points
    Nov 29, 2011 @ 12:51
    sun
    0

    thanks for your reply. can you add Chinese language for backend office?

    if you can, I will give you the newest Chinese language file, it translated by me somedays ago and it refered to some other files.

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 12:57
    Niels Hartvig
    0

    @sun: Yeah, that would be awesome. If you want to (and know how) you can make a pull request so you get creditted in the commit history. Else just email them to me ([email protected]).

  • sun 403 posts 395 karma points
    Nov 29, 2011 @ 13:09
    sun
    0

    The file is send to [email protected], please check your mail.

    Thanks for your team to make this great cms. it's flexible and brief, I love it.

    I'm pleasure to do something for umbraco.

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 16:16
    Niels Hartvig
    3

    An update of good and bad news. The good news is that there's no longer issues with Razor macros as they can safely be rendered as pure text. The bad news is that with general .NET controls it's a very hard nut to crack.

    I can't figure out how to cache a control so that it can be used over multiple simultaneous threads. I’ve also tried with the race condition locks implementation by Tommy Messbauer but that didn’t work as the main issue occurs when the .NET control is added or rendered (inside either the CreateChildControls() or Render() methods in the Macro.cs custom control). I get errors that doesn’t make sense – but seems to relate to threading issues - such as control collections where the exception doesn’t match information in the debugger or where the .NET parser complains about the lack of a server side form despite that in the parent collection in debugger it’s clearly there.

    It’s clear that the general way Umbraco handles .NET controls needs to be modified but there’s *no* way we could do this in a patch release (it would have a big impact on backwards compatibility). So for now – coming 4.7.1.1 – you shouldn’t wrap .NET controls that needs to be cached in Umbraco Macros (you can still add them directly into the mastertemplates and use output directives (ie. pure .NET style).

    If anyone have ideas to this, I’m all ears. Obviously these issues won't be in v5 as it's not using Controls and have a completely different architecture.

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Nov 29, 2011 @ 16:52
    Dave Woestenborghs
    0

    Hi Niels,

    Maybe you can render the control to a string and cache the output

    private string GetControlOutPut(Control control)
            {

                string body;

                Page.Form.Controls.Add(control);

                 TextWriter myTextWriter = new StringWriter();

                HtmlTextWriter myWriter = new HtmlTextWriter(myTextWriter);

                control.RenderControl(myWriter);

    body = myTextWriter.ToString();

    Page.Form.Controls.Remove(control);

     

                     return body;

            }

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 16:54
    Niels Hartvig
    0

    Ye, issue with doing that is that you loose any postback control support which can be a quite big issue?

  • Henri Toivonen 77 posts 111 karma points
    Nov 29, 2011 @ 16:59
    Henri Toivonen
    0

    I am happy with this as an interim solution. For me, personally, as long as the razor macros work im happy.

    I cant't really explain why, but the thought of wrapping controls in a macro makes me feel uneasy. Should you really be doing this at all? I sure wouldn't, even if it did work as intended.

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 21:13
    Niels Hartvig
    0

    @Henri: One is legacy (in (very) old versions you couldn't reference .NET controls directly without macros), but the main would be if you wanted to let the editors insert the .net control into the WYSIWYG editor. In that case you'd need to be able to wrap them.

  • Justin McKinley 12 posts 33 karma points
    Nov 29, 2011 @ 22:14
    Justin McKinley
    0

    @Niels - exactly right (of course) on the inserting into the editor, that's what we do all over the place, along with other ways of inserting user controls/macros into pages, to allow our editors to swap pieces in and out at will. That's why we're so concerned about this caching issue. 

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Nov 29, 2011 @ 22:42
    Niels Hartvig
    0

    @Justin: Does those controls have any postback functionality? One interim option would be to have a setting that'll cause .NET control macros to be cached as text. That way performance would be stellar and no issues would be present at all (and at the same time we'd have backwards compat for sites where this isn't an issue).

    For 4.8 it would be a goal to find a better way to deal with this obviously!

  • Chriztian Steinmeier 2798 posts 8787 karma points MVP 7x admin c-trib
    Nov 30, 2011 @ 01:00
    Chriztian Steinmeier
    0

    @Henri: One very good feature of the macro wrapper is the ability to swap implementation from any one of the supported engines to another, without having to go through all of your templates.

    /Chriztian

  • Justin McKinley 12 posts 33 karma points
    Nov 30, 2011 @ 02:23
    Justin McKinley
    0

    @Niels:

         We only attempt to cache our user controls that do not have postback functionality (there are plenty that do have postback, but we don't cache those), knowing the limitation of the macro caching issues (especiallly our hacked fix of the macro caching).

         The interim option you mention could be a nice option to allow at least some user controls to work. Note that that option is similar to what we implemented (which isn't perfect), as I mentioned much earlier in the thread. The problem with our fix is that if any there is any cached user control/macro on the page, and there is another user control (wrapped in a macro, not cached) on the same page that has a postback, that the postback may break. I'm sure that problem is fixable, we just haven't had the time to dive into it. 

     

  • pvassalo 21 posts 28 karma points
    Dec 06, 2011 @ 15:01
    pvassalo
    0

    Did the 4.7.1.1 came out already?  I can't find it on codeplex to download

    thanks!

  • Henri Toivonen 77 posts 111 karma points
    Dec 06, 2011 @ 15:08
    Henri Toivonen
    0

    Nope, not yet. Closest thing is the latest nightly build, but i wouldn't put that on a production server:

    http://nightly.umbraco.org/umbraco%204.7.1/4.7.1/4.7.1.467.zip

  • Christophe 9 posts 29 karma points
    Dec 13, 2011 @ 21:35
    Christophe
    0

    Hi,

    I tested the latest Nightly build (467) on a test server and I don't have any problem anymore with the "macroObjectCache".

    The SQL Profiler confirmed me that Niels's refactoring works fine.

    Chris.

  • Nikolas van Etten 202 posts 162 karma points
    Dec 22, 2011 @ 02:10
    Nikolas van Etten
    0

    Any news regarding the release of 4.7.1.1?

  • Tom Fulton 2030 posts 4998 karma points c-trib
    Dec 22, 2011 @ 14:14
    Tom Fulton
    0

    I heard a little bird say it'd be released Friday ;)

  • Christophe 9 posts 29 karma points
    Dec 22, 2011 @ 20:17
    Christophe
    0

    If that little bird is not wrong then it will be a nice Christmas present ;-)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Dec 22, 2011 @ 20:18
    Niels Hartvig
    0

    The bird is correct.

  • Nikolas van Etten 202 posts 162 karma points
    Dec 26, 2011 @ 20:40
    Nikolas van Etten
    0

    What happened to the bird?!

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Dec 27, 2011 @ 00:58
  • Henri Toivonen 77 posts 111 karma points
    Dec 27, 2011 @ 14:17
    Henri Toivonen
    0

    Macro parameters are not working for me with that beta release. Or was behaviour changed?

    I am also since before not able to use cookies nor querystrings as parameters to macros. (Trying to work around caching issues).

    edit:

    After further investigation i found that the macro parameters actually do work as long as you create parameters for the macros in the macro settings. I have almost never done this before, since it was not needed unless you wanted to insert a macro with the richtext editor.

    If it's intended behaviour, sure, I'll buy that. But in that case I think some kind of warning would be in order, not just simply dropping the parameter.

  • Dave Woestenborghs 3504 posts 12133 karma points MVP 8x admin c-trib
    Jan 16, 2012 @ 10:12
    Dave Woestenborghs
    0

    I have just tested the 4.1.1.1 release under heavy load and still have the problem of razor macro's that are not rendering. Anyone else having this problem ?

     

  • Christophe 9 posts 29 karma points
    Jan 26, 2012 @ 09:48
    Christophe
    0

    Hello Umbracians,

    We finally upgraded our productions servers to Umbraco 4.7.1.1 and it runs really well. The site is a lot more stable than previously.

    Thanks for the good job.

    Chris

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Jan 26, 2012 @ 09:50
    Niels Hartvig
    0

    @Christophe: Phew.

    @dawoe: Could you provide more info (and is anyone experiencing the same?). We can't reproduce the bug anymore in our tests :-/

  • Justin McKinley 12 posts 33 karma points
    Jan 26, 2012 @ 18:13
    Justin McKinley
    0

    @Niels,

       You mentioned this previously in this thread, and also in the issue tracker item for this on CodePlex, but I want to confirm that this latest version does not fix the issue for user controls (ascx).  In CodePlex, you said that it wouldn't be possible to fix this for user controls without breaking backwards compatibility. How possible would it be to issue a fix that is not part of the main release - i.e., a patch for users that want it (like me)? If it's not something you want to work on, do you have enough knowledge of the cause that you could give me some notes so that I (or someone else here) could attempt to fix it?

        Thanks!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jan 28, 2012 @ 14:06
    Floris Robbemont
    0

    I have the same issues with release 4.7.1.1 and razor macros. When viewing the site with more then 2 people the macros sometimes don't output their HTML (Only when caching is enabled)

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jan 28, 2012 @ 15:48
    Floris Robbemont
    0

    I think i may have found a solution for this problem. I'm doing some additional performance tests right now and checking if my razor code still works as expected.

    I'll post an update here when i know more.

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jan 29, 2012 @ 00:05
    Floris Robbemont
    0

    I might have found the solution and the cause of this problem. My problem was:

    - Under heavy load I got random Razor errors
    - Under heavy load cached macro's (Razor) disapeared at random (no HTML output).
    - To top this: Under heavy load I sometimes got the wrong HTML for a cached macro. For instance: My main navigation showed a different selected page then the actual page (wrong cache loaded, the cache key was correct).

    I went to take a look into macro.cs in Umbraco.Presentation and changed the following things:

    - Line 494 -> Included Script MacroType in order to cache Razor output as String instead of MacroCacheContent.
    - Line 292 -> Changed the check if a macro item was already cached (String output only).
    - Line 440 & 461 -> Changed the Cache priority to NotRemovable. Macro caching should not be removed before it's time runs out (Macro caching is an important part of the performance). This issue occured when the web application itself uses a lot of cache and Umbraco Macro cache objects were puched out of the cache because of their low priority.

    The problem was that this didn't change the odd behaviour of missing HTML and wrong cache. I then changed the two static methods GetMacro(int & string) to get the Macro instances. I removed the cache code from those methods and everything worked fine (lines 153 & 171). My guess is that the RenderMacro method is not thread safen (or any submethods). When storing the instance in the cache the RenderMacro method sometimes cached or outputted the wrong content. I don't have the time at the moment to investigate this issue further (would like to though :)), but the removal of the Caching from these static methods seems to do the trick right now.

    I uploaded a debug DLL (not for production use) and my modified macro.cs file to my home server:
    http://thuis.florisrobbemont.nl/Umbraco.zip

    @Niels & other umbraco devs:

    Could you check if my modifications in any way conlict with the current codebase. I don't have an extensive knowledge of the codebase (yet) so I don't know if my code will break compatibility.

    If my modifications (all marked with the comment: '// FlorisRobbemont: ') are fine could  somebody commit them into source control?? I don't have an HG client setup right now :)

     

    Cheers,

    Floris Robbemont

  • Henri Toivonen 77 posts 111 karma points
    Feb 01, 2012 @ 17:18
    Henri Toivonen
    0

    Yeah.. now i've seen this too again.  I think it's more rare now though than it was before? Diffilcult to say for certain. But something is still funky unfortunately. :/

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 01, 2012 @ 17:36
    Floris Robbemont
    0

    @Henri Toivonen: I could reproduce this problem under heavy load and simple razor macros. The code I uploaded fixes the problem, but it's more a workaround then a real solutions. Over the next few days I have some time to fix this hopefully :)

  • fabrice 104 posts 227 karma points
    Feb 02, 2012 @ 14:23
    fabrice
    0

    Hello,

    I have the exact same problem :"Under heavy load I sometimes got the wrong HTML for a cached macro."

    I had to disable all macro caching for now !!

    I'll try Floris code. Thanks Floris by the way

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 02, 2012 @ 14:27
    Floris Robbemont
    0

    I'm currently workig on a fork to really fix this issue. It'll probably be next week until i have some time to fix this.

     

    http://umbraco.codeplex.com/SourceControl/network/Forks/florisrobbemont/MacroCachingHeavyLoad

  • fabrice 104 posts 227 karma points
    Feb 02, 2012 @ 14:35
    fabrice
    0

    super thank you very much !!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 02, 2012 @ 16:12
    Floris Robbemont
    0

    Pull request created: http://umbraco.codeplex.com/SourceControl/network/Forks/florisrobbemont/MacroCachingHeavyLoad/contribution/1830

    macro.cs in umbraco.presentation is not designed to be cached. the issues were only surfacing when multiple users are browsers through the site. Removing the caching from the static GetMacro methods fixed the issues. I also included some performance updates to the renderMacro method (String caching)

    I can supply a Release Build assembly if needed. My code change has been tested in production now.

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 08:22
    fabrice
    0

    Many thanks, i'll try the new macro.cs today :)

    PS : Personally I don't need any Release Build assembly, I'll recompile umbraco myself. But thanks

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 09:26
    fabrice
    0

    Hello Floris,

    Could you please tell me what version of Umbraco your fork is based on ?

    From the 4.7.1.1 source, in umbraco/presentation/macro.cs , the class macro inherit from the class "page". But not in your fork.

    Thank you

    UPDATE : hmm... I think I found out what the problem is : it seems that the source code you can download in "http://umbraco.codeplex.com/releases/view/73692" is not the one of Umbraco 4.7.1.1 but the one of 4.7.1

     

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 09:31
    fabrice
    0

    hmm... I think I found out what the problem is :

    it seems that the source code you can download in "http://umbraco.codeplex.com/releases/view/73692" is not the one of Umbraco 4.7.1.1

     

     

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 09:32
    fabrice
    0

    hmm... I think I found out what the problem is :

    it seems that the source code you can download in "http://umbraco.codeplex.com/releases/view/73692" is not the one of Umbraco 4.7.1.1

     

     

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Feb 03, 2012 @ 09:34
    Niels Hartvig
    0

    @fabrice: Since 4.7.1.1, macro.cs does *not* inheirt from System.Web.UI.Page

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Feb 03, 2012 @ 09:36
    Niels Hartvig
    0

    @fabrice: Auch, that's a miss. I'll fix once we've merged this pull req!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 03, 2012 @ 09:36
    Floris Robbemont
    0

    @fabrice: True. You need to go to the source code tab in Codeplex, Select the 4.7.1 branch and click download.

    I created a fork based on the latest version of the 4.7.1 branch.

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 09:39
    fabrice
    0

    ok thanks ! the server is busy at the moment but I manage to get the clone url of your fork and then get the source code :)

    PS : sorry for the 2 messages similar before, it was a mistake

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 03, 2012 @ 10:00
    Floris Robbemont
    0

    @Niels Hartvig:

    I was looking arround the macro.cs class again. It's possible that for cached XSLT macro's my code-change (removing the cache) has a negative impact. (saw a lot of XSLT building code that's not cached aymore)

    How much of a performance hit do you think there'll be now the Ctor of the macro.cs class is being called on every macro render?

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Feb 03, 2012 @ 10:04
    Niels Hartvig
    0

    @floris: I'm not done reviewing the pull req, but I'll let you know during the day! Awesome work so far! #h5yr

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 03, 2012 @ 10:06
    Floris Robbemont
    0

    @Niels Hartvig:

    In reply to my previous post:

    I don't think that the current code is compatible for caching. It would take a rewrite of the macro.cs class in order to make it thread safe (the Model property is dangerous at the moment). I have some spare time next week, if you want I can take a swing at it :)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Feb 03, 2012 @ 13:43
    Niels Hartvig
    0

    @floris: Nice updates!

    Ideally the macro class should be splitted in two. One that contained the basic macro meta data and one that handled an instance of a macro rendering (ie. combine the MacroModel (the data of the instance that's being rendered) and Macro (that contains heavily cached meta data about how the macro is supposed to work (xslt, usercontrol, etc). That way the MacroModel won't be cached (which it shouldn't). Now that you found (the obvious!) cause, it should be pretty trivial to do (and make it much easier to get an overview of what macro.cs does!). I'll have a little look.

    #h5yr!

  • fabrice 104 posts 227 karma points
    Feb 03, 2012 @ 13:46
    fabrice
    0

    Hello,

    I have done some stress tests with LoadUI and IIS SEO Toolkit and I could reproduce easily the problem with the 4.7.1.1 dll.

    Then I tried again with the Floris-4.7.1.1 and everything went fine !! No loose of HTML and no HTML mixup.

    Thanks Floris !

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Feb 03, 2012 @ 13:48
    Niels Hartvig
    0

    w00t - floris should get an official #h5yr hat. Gosh, I love being a part of a community of smart people :)

  • Floris Robbemont 57 posts 89 karma points c-trib
    Feb 03, 2012 @ 13:53
    Floris Robbemont
    0

    @Niels:

    I agree with the split-up. One class with heavily cached Macro (static) information. And one class which handles the render process and has an instance for each MacroModel. That would be the ideal situation.

  • Jeroen Breuer 4908 posts 12265 karma points MVP 4x admin c-trib
    Feb 14, 2012 @ 13:47
    Jeroen Breuer
    0

    Is this fix already part of the latest v4 Change Set or do I still need to use the MacroCachingHeavyLoad Fork?

    Jeroen

  • Henri Toivonen 77 posts 111 karma points
    Mar 08, 2012 @ 12:45
    Henri Toivonen
    0

    Shameless bump.

    What is the status on this, is the floris fix merged? Any eta on when this will be out officially?

  • fabrice 104 posts 227 karma points
    Mar 21, 2012 @ 09:44
    fabrice
    0

    Hello,

    For info, I'm using Floris update but i'm experiencing randomly that I need to publish twice sometimes before that the cache is refreshed...

    It's maybe because I need to render a macro inside the RTE from Razor using :

    @Html.Raw(umbraco.library.RenderMacroContent(masterPost.BlogContent.ToString(), Model.Id))

    Can it be that ?

     

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Mar 21, 2012 @ 10:21
    Floris Robbemont
    0

    @fabrice

    I don't think this is due to my change because i only removed caching. It could be that Umbraco has a different caching dictionary for the RenderMacroContent method.

    You could try to subscribe to the Document_Published event and clear some cache yourself. If the problem persists you could enter it as an issue on CodePlex. Maybe some core devs might know why your cache isn't cleared properly.

    Cheers,

    Floris

     

  • fabrice 104 posts 227 karma points
    Mar 21, 2012 @ 10:28
    fabrice
    0

    Super thank you very much for your fast reply.

  • mscommunities 40 posts 95 karma points
    Apr 05, 2012 @ 05:00
    mscommunities
    0

    Any news if this will be part of another minor release, maybe 4.7.1.2 or 4.7.2? We upgraded to 4.7.1.1 on Monday and are seeing a lot of mixed up html as a result of macro caching. We are now forced to rollback to 4.7.1 since we didn't seem to have issues with that build. Any update would be great.

    PS - I hear great things about Floris' fix but would much rather wait for an official release.

  • sun 403 posts 395 karma points
    Apr 05, 2012 @ 05:39
    sun
    0

    Umbraco 4.7.2 is scheduled for March 2012. Where to find it? Now it is April.

  • John 88 posts 112 karma points
    Apr 17, 2012 @ 09:07
    John
    0

    Any answer on how we can get this current update?

  • Floris Robbemont 57 posts 89 karma points c-trib
    Apr 17, 2012 @ 09:15
    Floris Robbemont
    0

    @John

    Do you know how to build Umbraco from a Codeplex fork?

    - Clone from: https://hg.codeplex.com/forks/florisrobbemont/macrocachingheavyload

    - Make sure you update to branch 4.7.1 and have the latest revision.

    - Build on release.

    - Copy all DLL's that have changed in retrospect to you umbracoinstallation/bin folder

    - Check if MacroEnginges.Juno/bin/Release has changed, if so,..copy those as well.

    If you have difficulties with the above, I can supply you with a custom build (zip file with all de DLL's).
    But it might be a good idea to try this first :)

  • John 88 posts 112 karma points
    Apr 18, 2012 @ 02:18
    John
    0

    I wasn't sure about the cloning from that version but I manually made the changes to the latest version of 4.7.1 in macro.cs

     

    Thanks Floris

  • Ben McKean 272 posts 549 karma points
    Apr 19, 2012 @ 16:13
    Ben McKean
    0

    Hi Floris

    Would it be possible for you to send me the updated dlls please?

    [email protected]

    One of our very high profile sites is having this issue and we need to apply the fix as soon as possible.

    Thanks

    Ben

  • Nikolas van Etten 202 posts 162 karma points
    Apr 19, 2012 @ 16:37
    Nikolas van Etten
    0

    I would also be very much interested in the dll.

    [email protected]

    I'm quite surprised that there is still no update from the HQ... Nice with v5, but v4 is what most of us are stuck with for now!

     

    Thanks,

    Nik

  • Floris Robbemont 57 posts 89 karma points c-trib
    Apr 19, 2012 @ 17:55
    Floris Robbemont
    0

    Hi All,

    The fix DLL's can be downloaded from: http://www.lucrasoft.nl/Umbraco4.7.1MacroCachingFix.zip

    Please note that this is not an official build. It has been stable on our production sites though :)

  • Henri Toivonen 77 posts 111 karma points
    Apr 19, 2012 @ 18:02
    Henri Toivonen
    0

    I do agree that the maintenance of 4.7 should not be forgotten. Although 5.x is looking great we still have all of our sites on 4.7 and are still making our new sites with 4.7.

    It is way too early to abandon the older stuff. This patch should be merged, it has been working fine on the site that has the most problems for a couple of months now. :/

  • fabrice 104 posts 227 karma points
    Apr 23, 2012 @ 10:19
    fabrice
    0

    I confirm that it works great in production (www.kilroy.dk)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Apr 23, 2012 @ 13:36
    Niels Hartvig
    2

    I'm sorry with the lack of response from the HQ and I agree that Florris' great patch should have been merged long time ago. Unfortunately, v5 took all resources and in retrospec I should had prioritized differently, obviously. My fault!

    We're releasing 4.7.2 this week (ready for testing Tuesday, so I'll post a link in this thread) and it'll include Florris' fix. Further down the line, I'm happy to say that we're not leaving v4 behind and that there'll be a v4.8 in May which will include upgraded dependencies and the best of uComponents thanks to great support from Hendy Racher and Lee Kelleher.

    v4 isn't dead!

    Cheers,
    Niels... 

  • Floris Robbemont 57 posts 89 karma points c-trib
    Apr 23, 2012 @ 14:54
    Floris Robbemont
    0

    @Niels:

    Could you send me an e-mail to floris[at]lucrasoft.nl?

    I have found another performance issue in the member class. I think emailing it to you is faster then forking.

    Thnx :)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Apr 24, 2012 @ 11:43
    Niels Hartvig
    0

    @floris: Awesome! I've sent you an e-mail yesterday. If you could get back to me quickly (as in today) then we can add it to 4.7.2, else it'll be for 4.8 :)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Apr 24, 2012 @ 14:24
    Niels Hartvig
    1

    4.7.2 should be good to go - all kudos for fixing the load issue should go to Floris - h5yr!

    Anyone care to help testing the nightly and report positive/negatives:
    http://nightly.umbraco.org/umbraco%204.7.2/4.7.2/4.7.2.495.zip

    Changelog:
    http://umbraco.codeplex.com/workitem/list/advanced?keyword=&status=All&type=All&priority=All&release=Umbraco%204.7.2&assignedTo=All&component=All&sortField=Votes&sortDirection=Descending&page=0

    Thanks!
    Niels... 

  • sun 403 posts 395 karma points
    Apr 24, 2012 @ 14:30
    sun
    0

    I hope when it final released, it wan't pop up a box that show upgrade available. 4.7.1 is released.......

    thanks.

  • Carsten Johannesen 35 posts 55 karma points
    Jun 13, 2012 @ 12:45
    Carsten Johannesen
    0

    @Niels

    I am sorry to spoil the fun :-(  but...

    I have just installed 4.7.2 on a testserver, and done some testing of this under load, and the problem has not been solved!

    Our user controls still render double and halfway when the server is under load - but fine with no load.

    The caching facilities will still be unausable, when a server is under load.

    However it seems that the rendering problems are not as severe as I remember them when testing last year on 4.7.0 under load, so Floris's work has not been completely in vain, however the problems persist.

    Carsten

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 13, 2012 @ 13:50
    Floris Robbemont
    0

    @Carsten:

    Are your controls .NET user controls (ASCX or DLL), or are they Razor/XSLT scripts?

    My fix included the following:

    • Cache Razor scripts in the same manner as XSLT script
    • Remove double caching layer to remove caching threading bug
    • Difference between Empty string and Null reference (null reference output from macro == error, Empty string == just no output)
    I can remember that the system didn't work with UserControls as they require things like ViewState and ControlState (ASP.NET WebForms and caching is pretty difficult).
    In any case I would be glad to help you with this. I would need your code (or DLL Builds with PDB) and database to test it with my local build of Umbraco to try and find the cause of this.
    Let me know :)
  • Carsten Johannesen 35 posts 55 karma points
    Jun 13, 2012 @ 13:58
    Carsten Johannesen
    0

    @Floris

    My controls are ascx + dll (from visual Studio vb.net).

    I could probably supply dlls, pdb or code, however the site is pretty complex so as not to "drown you" in data we should probably discuss more precisely how to proceed. Also the macros rely on internal data from our SQLServer databases as well as Umbraco.

    you can see the production site here www.tv2bornholm.dk without the caching :-) to get a feel of the site.

    One of the first macros to break under stress is our user macro which renders the orange top menu navigator for the site. The buttons gets duplicated and/or removed.

    Carsten

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 13, 2012 @ 14:20
    Floris Robbemont
    0

    @Carsten:

    I'll proceed as you want. The main problem is the Macro rendering, so I shouldn't have to focus on any other data. All I need is a running site.

    SQL Databases can be restored on my own dev machine. But without a running site there's no way for me to see how Umbraco is handling the caching of your macros. The best way for me is to get all your code and db's (and run it locally) (I've done this before for one of Jeroen Breuers websites). 

    If you want, I'll sign a non-disclosure agreement before receiving any code :)

     

  • Carsten Johannesen 35 posts 55 karma points
    Jun 13, 2012 @ 14:29
    Carsten Johannesen
    0

    @Floris

    Non-disclosure? :-) I don't think that will be necessary but I'll check with my bosses.

    So should I mail the stuff  to you (and at what address)

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 13, 2012 @ 14:46
    Floris Robbemont
    0

    @Carsten:

    The best way is to RAR it (including all necessary components and databases in BAK format) and send it using dropbox or yousendit. It'll be too  big for emails.

    My e-mail address is: [email protected]

    I probably won't be able to work on it today (or tonight because we have to win our next soccer battle since we lost to some Danish men ;))

    Tomorrow night I have some time though...I'll look into it then :)

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 23, 2012 @ 17:33
    Floris Robbemont
    0

    @Carsten:

    The problem is with the fact that UserControls are instances. To make a long story short: The fact that your control sometimes dissappear or appear double is because something is going wrong with how ASP.NET is handling those controls (there should be more then 1 instance of them, instead...there is not because the Umbraco core caches the instance). 

    I think I've found a way to bridge the caching system of the XSLT and Razor scripts (which is far more superiour due to it's string caching instead of control caching) and the system the Usercontrols use.

    It will take some rewriting on your part, but you won't have to change all your usercontrols to Razor files :)

    I'll keep you posted.

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 23, 2012 @ 18:31
    Floris Robbemont
    0

    @Carsten:

    I'll be sending you instructions and some new DLL's (umbraco core DLL's) today.

    My tests here were not done on your codebase (it took too much to get it working, too much dependencies).
    I copied your TopMenu.ascx code and tested it on one of my sandbox Umbraco sites.

    After implementing the new UserControl and Custom Control caching I got from 41 request/sec to 339 request/sec.
    That's 727% faster :)

    Please note: this only works on UserControls and Custom Controls with static output and no events, like a menu.
    This won't work on anything that requires a postback or other 'outside' variables. You should handle these controls like MacroScripts.
    No state, no postbacks, only rendered HTML output.  

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 23, 2012 @ 19:26
    Floris Robbemont
    1

    Pull request created: http://umbraco.codeplex.com/SourceControl/network/forks/florisrobbemont/MacroCachingHeavyLoadPart2/contribution/3018

    If these changes are approved, it should be in the core when 4.8 is released.

  • Tony Bolton 83 posts 109 karma points
    Jun 26, 2012 @ 14:05
    Tony Bolton
    0

    Great work everyone esp. @Floris - just had the same problem with a cached Razor macro pulling back zero content - looks like this will do the trick!

  • Carsten Johannesen 35 posts 55 karma points
    Jun 28, 2012 @ 10:02
    Carsten Johannesen
    0

     @Floris

    It sounds good that you have located problems. Great work :-)

    @Umbraco/Niels:

    However will the fix specified be the permanent / official one? This sets up some kind of coding requirement for "third party" User Controls if they are to work within Umbraco.

    Umbraco should make it very clear what demands are for user controls within the CMS for macro caching to actually work. (Otherwise you get posts with lots of exclamation marks from people like me who gets a nasty surprise when doing load testing ;-)
    My understanding was (from the official videos for instance) that it should work without any special coding in user controls.

    And I'm not very happy about the "please note"  from Floris above either - I am pretty sure that we have user controls of this kind as well.

    So should my conclusion not be that userControl macro caching is still flawed… unless you have pretty simple User Controls (which output only static content)?

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 28, 2012 @ 10:43
    Floris Robbemont
    0

    @Carsten:

    The problem lies with the fact that UserControls weren't made for caching (not if you want advanced caching like Umbraco does).

    ASP.NET's controls, viewstate, control state, validation, etc all make it pretty much impossible to do caching at all. They need instances of controls.

    The code I added gives developers an option to use some of the Razor goodness (plain HTML output, which is cacheable)  in their usercontrol.
    The only downside to this is that you lose all ASP.NET's attemps at making a more statefull framework. 

    Is the UserControl macro caching flawed? Yes.

    But there's not much we can do about it (apart from static HTML caching)
    It's simply not possible to provide HTML output caching based on instances of controls (outside of razor).

    The exceptions like KeyAlreadyAddedToDictionary and ControlAlreadyInCollection are due to the fact that the same control (instance) is being added to the output on multiple different threads.

    The only choices at the moment are:
    - Use the ICacheableControl interface to output static content. Rewrite some code to support this interface. Maybe split up controls to allow both caching and postback functionality.
    - Rewrite your code to Razor (using normal HTTP posts as postbacks -> which do not require viestate, control state, or any other attempt at state)

    I would think that the first option is the sensible in your situation.

  • Carsten Johannesen 35 posts 55 karma points
    Jun 28, 2012 @ 10:58
    Carsten Johannesen
    0

    @Floris

    Excellent explanation.

    This is important information for developers chosing Umbraco as their platform.

    Would it be possible for Umbraco to let us use the standard ASP.NET caching facilities (the ones you setup using caching directives in the *.ASCX files).

    This functionality is currently not possible to use when running the User controls "thru" Umbraco, but works in "clean" (non-umbraco) ASP.NET sites. (You get an Umbraco error when trying - believe me I have tried when I first discovered the problems :-)

    I am no Umbraco source code expert, but that solution might be worth exploring by the Umbraco team?

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Jun 28, 2012 @ 11:34
    Floris Robbemont
    0

    @Carsten,

    True :)

    But you'll have to through great lengths to get all the Umbraco caching possibilities into that system. I've found the built-in caching capabilities of ASP.NET somewhat basic. ASP.NET 4.0 has some newer caching stuff but it's still based on query string. To get it to work with macro parameters for example would be a real PITA.

    And with Razor scripts being the future ^^

    I am curious about the exception you get when enabling caching on UserControls. I'll check that one out when I find the time :)

  • Niels Hartvig 1951 posts 2391 karma points c-trib
    Jun 28, 2012 @ 11:43
    Niels Hartvig
    0

    I believe it would be possible to support default .NET caching in User Controls. The reason it YSODs is because when you add a cache directive to a user control, its type is changed (thus no longer being a user control but something like ICachedControl (can't remember)). I believe someone in the community posted a possible solution to this (poitning to an MSDN article). I'll see if I can find it a bit later today :)

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 10:28
    Dan Diplo
    0

    Just to report that a client has had the macro-caching bug return in a 4.72 installation. It used to be something that happened fairly regularly in 4.7 and 4.7.1 but when we upgraded them to 4.72 we thought it was fixed. But yesterday it reared it's head again in their 4.72 site.

  • Carsten Johannesen 35 posts 55 karma points
    Sep 06, 2012 @ 10:43
    Carsten Johannesen
    0

    @Dan

    My understanding (from the correspondance just above your report) and from what I talked with Floris about those 2 months ago, is that the problem is unfixable (for usercontrols) but of course only appears under intense load.

    Please see Floris's explanation and hints for avoiding the issue.

    We are now in the process of implementing a thirdparty caching solution (Varnish) for our site - because we have given up on a fix in Umbraco.

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 10:46
    Dan Diplo
    0

    Thanks, Carsten, but the problem is occuring in Razor (CSHTML) macros and not in user-control macros. As you say, it only appears to be under load and so far has only happened once since upgrading to 4.72. However, the client are a large/important client so any problems, however rare, are a problem for them.

  • Carsten Johannesen 35 posts 55 karma points
    Sep 06, 2012 @ 10:49
    Carsten Johannesen
    0

    @Dan

    Then I cannot help you :-( as we don't use Razor macros.

    Hopefully Floris (or Niels) or someone else from the community will assist you

    sorry

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Sep 06, 2012 @ 10:50
    Floris Robbemont
    0

    @Dan

    As far a I know, my proposed fix never made it into the 4.72 release. I'm checking for you if it's present in the 4.8 release.

  • Floris Robbemont 57 posts 89 karma points c-trib
    Sep 06, 2012 @ 11:02
    Floris Robbemont
    0

    There are some changes in de 4.8 release to improve overall stability. Maybe you could check if the bug still exists in that codebase.

    As of 4.9, the pipeline will be completely refactored. Therefor I cannot say if the bug still exists in 4.9

    We haven't had any issies since 4.7 (custom build with my original fix). What symptons are you experiencing?

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 11:16
    Sebastiaan Janssen
    0

    FYI, Floris' changes did make it into 4.7.2, changeset 00c1641350be. And the pipeline will change for 4.10.0, not 4.9.0.

    @Dan Without knowing what the exact symptoms are or what kind of code your running or how the macro is configured there's not much anybody can say about your specific problem, so please elaborate.

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 11:18
    Sebastiaan Janssen
    0

    Oops, scratch that, they were applied to 4.7.2 after it was released, so the fix rolled into 4.8.0 which is probably why you're still seeing these problems.

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 11:33
    Dan Diplo
    0

    OK - thanks, Seb. I was under the impression it had been fixed in 4.72 since the release notes stated, "Umbraco 4.7.2 is a patch release fixing several bugs including major issues with Razor caching as well as minor security fixes."

    Will have to make a decision whether to upgrade the client to 4.8...

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 11:38
    Sebastiaan Janssen
    0

    Hmm, good one, looking at it again (sorry!) I now see it in the 4.7.2 branch and since the release version doesn't seem to be properly tagged I was confused. Looks like it's in 4.7.2 after all. In which case, please provide some more info about your actual problems. :-)

    If you want to make sure the dlls you're using have the fix, you can open them in ILSpy or DotnetPeek, should be umbraco.dll, the macro class, the lines are marked: 

    // FlorisRobbemont: issue #27610 -> Presentation macro not supposed to be cached.
  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 12:22
    Dan Diplo
    0

    Hi Seb. I checked the umbraco.dll and it is version 1.0.4393.24044. I opened it in DoteNetPeek but couldn't see any comments in there at all? All I can see is the at the top of the macro class it has:

    // Type: umbraco.macro

    // Assembly: umbraco, Version=1.0.4393.24044, Culture=neutral, PublicKeyToken=null

    // Assembly location: X:\mypath\httpdocs\bin\umbraco.dll

    I don't have an Umbraco.pdb, though....

  • Tony Bolton 83 posts 109 karma points
    Sep 06, 2012 @ 12:27
    Tony Bolton
    0

    I didn't think ILSpy etc. rendered out the comments as they're not included in the assemblies?

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 12:40
    Sebastiaan Janssen
    0

    Ah yes, comments are optimized out of course. Check for this:

             public static macro GetMacro(string alias)
             {
               return new macro(alias);

    It should NOT say this: 

    return cms.businesslogic.cache.Cache.GetCacheItem(GetCacheKey(alias), macroRuntimeCacheSyncLock,

     

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 14:12
    Dan Diplo
    0

    I get this, so presume this is up-to-date?

    return umbraco.cms.businesslogic.cache.Cache.GetCacheItem<macro>(macro.GetCacheKey(alias), macro.macroRuntimeCacheSyncLock, TimeSpan.FromMinutes(60.0), (umbraco.cms.businesslogic.cache.Cache.GetCacheItemDelegate<macro>) (() =>

     

     

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 14:33
    Sebastiaan Janssen
    0

    Aah, there seems to be your problem, it should say

    return new macro(alias)

    So now I'm wondering how come your version says 4.7.2, I just downloaded and checked 4.7.2 to decompile it and I definitely get the correct code. Did you misread and are you still on 4.7.1 maybe?

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 14:35
    Sebastiaan Janssen
    0

    umbraco.dll file version should be 1.0.4500.21031 by the way! The version you mentioned earlier is 4.7.1.1 actually.

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 15:16
    Dan Diplo
    0

    Good spot! But the web.config has 4.7.2 as the version, hence my assumption it was 4.7.2! 

  • Sebastiaan Janssen 5044 posts 15475 karma points MVP admin hq
    Sep 06, 2012 @ 16:06
    Sebastiaan Janssen
    0

    Really odd though as if the version is wrong you should be getting the installer, might be that some dlls were updated but not all then.

  • Dan Diplo 1554 posts 6205 karma points MVP 5x c-trib
    Sep 06, 2012 @ 21:13
    Dan Diplo
    0

    Yeah, bit odd. But thanks for all your help - I can manually updated the DLL and hope that fixes. Cheers :D

  • Sébastien Richer 194 posts 430 karma points
    Oct 02, 2012 @ 17:43
    Sébastien Richer
    0

    Hello Dan and Seb (most recent peeps in convo.),

    I have this site running on fairly heavy traffic, nothing too intense, but still. I have cachin enabled (900 seconds) on most of my .cshtml razor script macros. Occasionnaly, I get error messages, saying that my macros could not load:

    "Error loading MacroEngine script (file: mycoolmacro.cshtml)"

    This occurs on most of my macros, throughout the site. Reloading the webpage usually fixes the issue. I've confirmed that I'm using the 4.7.2 DLL as mentionned by Seb.

    Using jetbrains dot peek, I could not find the "return new macro(alias)" line Seb mentionned. How should I tackle this issue, I feel uncomfortable as this site is live right now.

    Thanks!

  • Sébastien Richer 194 posts 430 karma points
    Oct 02, 2012 @ 17:55
    Sébastien Richer
    0

    My cms.dll is at version = 1.0.4498.19837 and that's where I can find umbraco.cms.businesslogic.cache.Cache.GetCacheItem[...]. Am I looking at the right place?

    Thanks!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 02, 2012 @ 20:04
    Floris Robbemont
    0

    You can find the correct method in the umbraco dll. Using JetPeek navigate directly in the umbraco name to the macro class.

    There should be 2 methods there called public static macro GetMacro(...). These methods should both contain: return new macro(...);

    If that's incorrect you have the wrong version. If not, there's another issue with the macro caching.

  • Sébastien Richer 194 posts 430 karma points
    Oct 02, 2012 @ 20:11
    Sébastien Richer
    0

    Thanks Floris, yup found them, so I have to correct version then. It's the "occasionnal" part of my problem that haunts me... could it be related to my Examine search engine indexing stuff instead of caching?

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 02, 2012 @ 20:26
    Floris Robbemont
    1

    I might be able to take a look at it tomorrow. Could you send me the full stacktrace and exception message?

  • Sébastien Richer 194 posts 430 karma points
    Oct 02, 2012 @ 21:01
    Sébastien Richer
    0

    Hi Floris,

    I would but the problem is when these macros don't render, all I see is "Error loading MacroEngine script (file: mycoolmacro.cshtml)". The other problem I have is that this only occurs on my live site, so I have less debugging flexibility. Got any tips to get any form of logging or exception message?

    Thanks!

     

    EDIT: I'm trying to catch it with /?umbdebugshowtrace=true, not easy, but we'll see.

  • Sébastien Richer 194 posts 430 karma points
    Oct 02, 2012 @ 21:32
    Sébastien Richer
    0

    Good news, here is the stack trace (kind of not as helpfull as I expected though):

    RazorDynamicNode got datatype ec15c1e5-9d90-422a-aa52-4f7622c63bea for titre on CalendrierEvenementChampionnat 1,0197227   0,000035
    Checking for a RazorDataTypeModel for data type guid ec15c1e5-9d90-422a-aa52-4f7622c63bea...    1,0197399   0,000017
    Checking the RazorDataTypeModelTypes static mappings to see if there is a static mapping... 1,0197519   0,000012
    There isn't a staticMapping defined so checking the RazorDataTypeModelTypes cache...    1,0197682   0,000016
    GUID ec15c1e5-9d90-422a-aa52-4f7622c63bea does not have a DataTypeModel, falling back to ConvertPropertyValueByDataType 1,0197871   0,000019
    umbracoMacro    Loading IMacroEngine script [done]  1,0198538   0,000067
    umbracoMacro Error loading MacroEngine script (file: CalendrierEvenement.cshtml, Type: '' Thread was being aborted. at umbraco.macro.renderMacro(Hashtable pageElements, Int32 pageId) 1,0199489 0,000095
    renderMacro    Rendering started (macro: Calendrier Evenement List, type: 6, cacheRate: 900)   1,0215545   0,001606
    renderMacro Macro Content loaded from cache 'CalendrierEvenementList-3034-'.    1,0215841   0,000030
    renderMacro Rendering started (macro: Sponsor Box, type: 6, cacheRate: 0)   1,0216115   0,000027
    umbracoMacro    MacroEngine script added (SponsorBox.cshtml)    1,0216291   0,000018
    umbracoMacro    Loading IMacroEngine script 1,0216442   0,000015
    RazorDynamicNode got datatype ec15c1e5-9d90-422a-aa52-4f7622c63bea for titre on PartenairesCorporatifs  1,0266815   0,005037

    I put in bold the part that concerns my macro, I could paste the whole thing if someone want's to see it.

    Thanks!

     EDIT: I've found this related issue http://issues.umbraco.org/issue/U4-537

     

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 03, 2012 @ 01:22
    Floris Robbemont
    1

    Hi Sebastien,

    Would it be possible to e-mail me the source for 'CalendrierEvenement.cshtml' ? I think the error is coming from a combination of the macro code and the razor code (not sure though). Could you send it to: floris[at]lucrasoft.nl ?

    If that doesn't bring us a solution another way to go would be to send me a complete copy of the site and database (from the live site). I can run a stress test on that codebase and set some breakpoints to see if and when it goes wrong (using the Umbraco source). I can sign a non-disclosure agreement to protect your source and your customer's content (my company can supply the paperwork for that). I've done this for some other umbraco projects in the past and this seems to work the best.

    Cheers,

    Floris

  • Sébastien Richer 194 posts 430 karma points
    Oct 03, 2012 @ 15:41
    Sébastien Richer
    0

    Hi Floris,

    I'll install the source on my dev environnement and try to reproduce it, but this is not exclusive to this specific cshtml, I get the issue with any / all macros that display data. I'll try a couple of things today and post back here.

    Thanks!

  • Sébastien Richer 194 posts 430 karma points
    Oct 03, 2012 @ 23:50
    Sébastien Richer
    0

    I think I found it, I'll look into this more tomorow, http://umbraco.codeplex.com/SourceControl/changeset/c7190b610ecc, this query sent me in that direction : SELECT TOP (10000) id, userId, NodeId, Datestamp, logHeader, logComment FROM umbracoLog WHERE (logComment LIKE N'%Abort%').

    More on this tomorow! :D

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 04, 2012 @ 14:45
    Floris Robbemont
    1

    Did you use the feed on that page? The ThreadAbort exception is usually the result of Response.Redirect (without the optional boolean telling the server to not end the current request). If you have any Respone.Redirect in your code, check if you have used it like this:

    Response.Redirect("url", false);

    That way, the thread-abort exception won't fire.

  • Sébastien Richer 194 posts 430 karma points
    Oct 04, 2012 @ 14:56
    Sébastien Richer
    0

    I don't use the feed, so it's in fact a bit weird, (This forum post is what sent me this way http://our.umbraco.org/forum/ourumb-dev-forum/bugs/31809-Possible-bug-in-472-upgrades, but come to think of it, is that a forum "about" the forum?). I do use Response.Redirect without those parameters (I'll research that today), but pages that cause the problem I'm having do not redirect, they are regular data displaying pages.

    EDIT: Not sure exactly how this works, could "some other" thread, not directly involved in rendering my macro and getting terminated by .end() or such, cause my problem? Hmmm...

    More news soon!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 04, 2012 @ 15:40
    Floris Robbemont
    1

    I have created a trace listener to track macro errors on production sites (using very smelly code).

    You can implement this easily by putting the a .cs file in your App_Code directory, and registering it in the web.config.
    Maybe it'll help you get to the source of this exception. 

    UmbracoTraceListener.cs

     

    using System.Web;
    using umbraco.BusinessLogic;

    namespace Lucrasoft.Presentation.Modules {
       public class UmbracoTraceListener : WebPageTraceListener
       {
           public override void WriteLine(string message, string category)
           {
                base.WriteLine(message, category);
                if (category.ToLower() == "umbracomacro" && message.ToLower().StartsWith("error loading razor script"))
                {
                   umbraco.BusinessLogic.Log.Add(LogTypes.Error, 0, message);
               }
           }
        }
    }

    And in the web.config:

    <system.diagnostics>
        <trace>
          <listeners>
            <add name="WebPageTraceListener" type="Lucrasoft.Presentation.Modules.UmbracoTraceListener, Lucrasoft.Presentation" />
          </listeners>
        </trace>
      </system.diagnostics>
  • Sébastien Richer 194 posts 430 karma points
    Oct 04, 2012 @ 16:53
    Sébastien Richer
    0

    Looks good I'll give it a try! I got the error this morning with another macro, here is the content of that cshtml file, tell me if you see anything risky (beware, sloppyish/old code, also some extension methods in there, but otherwise very very straightforward):

    @using XYZ.CMS.Common.ExtensionMethods
    @using umbraco.MacroEngines
    @inherits umbraco.MacroEngines.DynamicNodeContext
    
    @{
        var nodeAccueilId = Convert.ToInt32(System.Configuration.ConfigurationManager.AppSettings["NodeAccueil"]);
        var typeList = new List<string> { "TypeA", "TypeB", "TypeC", "TypeD", "TypeE" };
    } <ul class="breadcrumbs"> @foreach (dynamic node in CurrentModel.AncestorsOrSelf()) { var hideThisNode = !node.IsNullOrFalse("umbracoNaviHide"); if (node.Id == nodeAccueilId) { // Home node <li class="home"><a href="/">Accueil</a></li> } else if (Model.Id == node.Id || (hideThisNode && !typeList.Contains(node.NodeTypeAlias))) { // Current node <li>@node.nomMenu</li> } else if (node.nomMenu.ToString() != "") { if (node.NodeTypeAlias == "TypeF" || node.NodeTypeAlias == "TypeG" || node.NodeTypeAlias == "TypeH") { // Ancestor node without anchor <li>@node.nomMenu</li> } else { // Ancestor node <li><a href="@node.NiceUrl">@node.nomMenu</a></li> } } } </ul>

    And here is the extension method (again... old code haha):

    public static bool IsNullOrFalse(this DynamicNode node, string propertyName)
    {
        return (!node.HasProperty(propertyName) || node.GetPropertyValue(propertyName) == "0");
    }
  • Sébastien Richer 194 posts 430 karma points
    Oct 04, 2012 @ 18:04
    Sébastien Richer
    0

    Ok I changed my response.redirect codes to :

    Response.Redirect(sr.url, false);
    Context.ApplicationInstance.CompleteRequest();

    I'll upload this to my live site and see if I still get reports on the problem.

    I managed to reproduce 100% the effect locally, I opened 5 tabs on page /xyz/, then opened 5 other tabs on /abc/ that only has this "Response.Redirect" and not that false parameter.

    But I'm wondering, hitting the redirect (without the bool), could that impact other requests being executed or just those implicated by requests made by me... I'm wondering if one of these calls can affect requests from other users... I guess this is more at the IIS level.

    Thanks for the help Floris!

  • Floris Robbemont 57 posts 89 karma points c-trib
    Oct 17, 2012 @ 13:49
    Floris Robbemont
    1

    HI Sebastien,

    Did this fix the error you were having on your production site?

    To anwser your question:

    This does not effect other requests. Basicly what the boolean does is tell the Reponse.Redirect() method NOT to execute Response.End().
    That methods is responsible for for the thread abort exception. What it will do is send a redirect header to the browser telling it to redirect to a different page.

    I'm not familiar with Context.ApplicationInstance.CompleteRequest(); though. What does it do?

  • Sébastien Richer 194 posts 430 karma points
    Oct 17, 2012 @ 15:05
    Sébastien Richer
    0

    Hi Floris,

    Well I have'nt seen the problem occur on my production site anymore and the client has not yet reported occurances. This looks like it solved the issue for the moment! CompleteRequest will ask .NET to "end" the response process.

    So some (most) code that is after that call will not occur. This is just to save some processing, most of the time, when we redirect like this, very little code is executed after that. Although I've seen people write code they "relied" on mecanics like redirect instead of things like "return" or "continue" which is at best, very bad code / practice haha :)

    I do kind of suspect that this issue / bug was actually only accessible to very busy users, using multiple tabs and hitting redirects while rendering other macros in other windows. Because I don't think that the exception could span multiple users in it's impact.

    I'll ring back here if the problem persists, right now I'm adjusting my various projects to make their redirects "safe".

    Thanks again for the help Floris!! Much appreciated!

  • Jeroen Breuer 4908 posts 12265 karma points MVP 4x admin c-trib
    Nov 25, 2013 @ 10:01
    Jeroen Breuer
    0

    We're currently having the same cache issues on a 4.7.2 website. Should upgrading to 4.8 solve the cache problems? We could also upgrade directly to 4.11.10 if that is just as easy.

    Jeroen

  • Daryl Teo 6 posts 28 karma points
    Oct 20, 2014 @ 07:04
    Daryl Teo
    0

    To future google adventurers:

    also facing this issue on a legacy install, with no upgrade path.

    In my situation, turning OFF cache by page / cache personalization seems to have worked around the issue. Macros are rendering without disappearing. That being said, I'm not sure if I've simply reduced the chances of the problem occurring.

    My settings:

    • period 6000
    • cache by page OFF
    • cache personalization OFF

    Regards, Daryl

Please Sign in or register to post replies

Write your reply to:

Draft