Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • James Strugnell 84 posts 192 karma points
    Dec 21, 2015 @ 15:26
    James Strugnell
    0

    Azure Load Balancing Best Practice with Umbraco Forms

    Hi,

    We're currently running a fairly large site on Umbraco 7.3.1 on an Azure App Service (website). We regularly have to scale this site out to use multiple machines when the client has expected high traffic. Until now we've ignored the guidance to keep the back-office on a separate single server (i.e. no scaling). We had not had any problems and were riding our luck I guess. This last weekend, however, the umbraco.config file locked up, seemingly due to multiple instances accessing it at the same time, and caused the site to crash at a very unideal time.

    So we are now looking at creating a separate instance, that doesn't scale, for the back-office area. In doing this we've realised that we are now going to have problems with the Umbraco Forms package. I believe the schema files that describe each Form are stored as JSON structures on the local file system. Given that we will be using a different file system for the back-office, compared to the front-end how are we supposed to keep the 2 in sync?

    I've seen a previous post here (https://our.umbraco.org/forum/umbraco-pro/contour/69489-best-practice-usage-usage-of-umbraco-forms-on-load-balanced-environment) that suggests replicating a specific folder in the Plugins directory, but I'm not sure this is possible in Azure?

    Has anyone solved this? To be honest I'm still not sure why the File System is used for a centralised repository? Surely this should be in the database?

    I'm predicting other issues like this with other packages that store configuration in files and these then not being picked up by the front-end servers.

    If anyone can help with this it would be most appreciated?

    Thanks, James.

  • Shannon Deminick 1524 posts 5269 karma points MVP 2x
    Dec 21, 2015 @ 18:20
    Shannon Deminick
    0

    Hi,

    I see your concern about Forms storing data in files as opposed to the DB for this particular circumstance.

    This same file issue will be apparent for a few things however and I'm sure there are a few different ways to solve it. I can't tell you 'best practices' for this because this is all rather new.

    • Media -> this will be a problem too, but you can use the Azure Blob Media storage provider which should solve the problem: https://our.umbraco.org/projects/collaboration/umbracofilesystemprovidersazure
    • Deployments -> You'll have to deploy changes to two environments. I'm sure there's a few ways to achieve this with CI build systems.
    • If you are letting your devs/editors edit any css/js directly in the back office (not source controlled), you'll have the same issue you are facing with the Forms files

    There's probably some utilities out there that perform file syncing between Azure webapp instances, or maybe ones that file sync based on Blob storage.

    I'd like to give you a clear/direct answer here but I unfortunately can only provide some suggestions at this time.

    If you are doing Traditional LB, then these issues aren't really a problem because you're files will be either replicated or shared, so we need to figure out a reasonable solution for Flexible LB.

  • James Strugnell 84 posts 192 karma points
    Dec 22, 2015 @ 12:31
    James Strugnell
    0

    Hi Shannon, thanks for your input. We are already using the Media solution you've suggested. For deployments, currently we would just need to deploy twice, not ideal but do-able, and even better if automated. We don't do any code/template editing through the UI, so no problem there.

    My ideal solution would be a means to keep everything on the /umbraco path tied to a single azure instance, even if the rest of the site is scaled. i.e. any traffic that comes in through /umbraco stays on the one server designated as the Master.

    There is a feature in Azure called Traffic Manager that I had hoped might provide this functionality (keeping certain paths on one instance) but sadly it does not. Perhaps it will though. Or perhaps Azure Web Apps will enable this in another way.

    Do you see this as something that could be implemented within Umbraco, or as a package? It seems like it should be possible to obtain the current Master instance ID and then enforce the ARRAfinity cookie to keep admins on that one server. Could that work?

    I've watched your CodeGarden load balancing video and you seem to forcibly connect to the 3 different Azure instances through 3 different browsers. Are you doing some cookie magic there to connect to each individual instance? Or were you just clearing your cookies and retrying until you got the right instance?

    Also, to prevent the umbraco.config file locking is it advisable (with no side-effects) to use umbracoContentXMLUseLocalTemp="true" setting? Presumably the Flexible LB "Cache instructions" will update this file on each server when content gets updated? Are there downsides to that setting (as it seems like an optimal approach)?

    Really appreciate your help with this.

    James.

  • Shannon Deminick 1524 posts 5269 karma points MVP 2x
    Dec 22, 2015 @ 13:31
    Shannon Deminick
    0

    Hi,

    you won't be able to simply redirect any paths that start with /umbraco. This is because SurfaceControllers and UmbracoApiControllers, along with a few other web services are routing under this path. You could however redirect the exact paths: "/umbraco" and "/umbraco/".

    My advise would be to setup your management Azure web apps instance to only listen on a custom DNS address like: admin.mysite.com, then have your front-end Azure web apps instance listen on mysite.com

    Yup, you should set umbracoContentXMLUseLocalTemp="true" on your front-end/scaling instance since the normal path will be shared by all of your websites. You shouldn't need to set this for your management instance since you shouldn't scale that. I'l update the docs ASAP.

    The magic queries that I was executing during my CG demo had quite a bit of magic involved actually. You have to use the Azure APIs to determine the website instances in your web app. This involves installing/configuring certificates (I had help with all that and unfortunately don't remember that), then using the API to query instances, then you can use that instance Id to replace the current ARRAffinity cookie.

    I've found that (horrible code) code again... whether or not it helps you out is a different story. This is the code to display that orange overlay box in my CG demo and the code to handle when you click on an instance to change the ARRAffinity cookie. Again, i can't remember how to do all the certificate stuff unfortunately.

    //NOTE: Can't remember if all of thse packages were required or not
    // I feel like the AzureCLI is not necessary
    @using AzureCLI;
    @using System.Security.Cryptography.X509Certificates;
    @using Microsoft.WindowsAzure;
    @using Microsoft.WindowsAzure.Management.WebSites;
    @using Microsoft.WindowsAzure.Management.WebSites.Models;
    
    @{
        var roots = Umbraco.ContentQuery.TypedContentAtRoot();
        var luceneQuery = "+__IndexType:content +__Path:(" + "-1".Replace("-", "\\-") + "*)";
        var crit = ExamineManager.Instance.SearchProviderCollection["InternalSearcher"].CreateSearchCriteria().RawQuery(luceneQuery);       
        var count = ExamineManager.Instance.SearchProviderCollection["InternalSearcher"].Search(crit).TotalItemCount;       
    }   
    <div style="height: 450px; color: orange;  position: fixed;  top: 0;  left: 0;  z-index: 9999;  border: 2px solid orange;  padding: 10px;  background-color: rgba(0, 0, 0, 0.8);  margin: 5px;">    
        <form action="/" method="post">
        <input type="submit" style="position:absolute;top:80px;right:50px;" />
        <input type="hidden" name="publish" value="true" />
        </form>    
        <ul>
            <li style="color:orange;font-size:2em;">Server: @Environment.MachineName</li>
            <li style="color:orange;font-size:2em;">Content count: @ApplicationContext.Services.ContentService.Count()</li>             
            <li style="color:orange;font-size:2em;">Publish count: @(roots.DescendantsOrSelf<IPublishedContent>().Count())</li>     
            <li style="color:orange;font-size:2em;">Index count: @count</li>        
        </ul>               
        <p style="padding:10px;color:orange;">
            <div style="display:block;font-size:.7em;">
            <strong>Current server affinity:</strong><br /> 
            <span id="CurrentARRAffinity"></span>
            <br />
            <br/>
            </div>
            <strong>All server ids</strong><br />                
            @{              
                var ids = (string[])ApplicationContext.ApplicationCache.RuntimeCache.GetCacheItem("AzureIds", () => {            
                    string certPath = Server.MapPath("~/App_Data/YOURCERTIFICATE.pfx");        
                    var x509Cert = new X509Certificate2(certPath, "YOURPASSWORD", X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.Exportable);            
                    var subscriptionId = "YOUR-SUBSCRIPTION-GUID";            
                    var creds = new CertificateCloudCredentials(subscriptionId, x509Cert);            
                    var client = new WebSiteManagementClient(creds);       
                    //For example, mine was: "shazumblbtest1-AustraliaEastwebspace" and "shazumblbtest1"     
                    return client.WebSites.GetInstanceIds("YOUR-REGION-NAME", "YOURWEBAPPNAME").ToArray();
                }, TimeSpan.FromSeconds(3));
            }
            @foreach(var id in ids) {
                <a style="color:orange;text-decoration:underline;" href="#" class="SetARRAffinity" data-affinity="@id">@id</a><br />
            }
        </p>        
    </div>
    
    <script type="text/javascript">
    $(document).ready(function() {
        $("#CurrentARRAffinity").html($.cookie("ARRAffinity"));
        $("#ClearARRAffinity").click(function () {
            $.cookie("ARRAffinity", "");
        });
        $(".SetARRAffinity").click(function () {
            $.cookie("ARRAffinity", $(this).attr("data-affinity"),  { domain: '.shazumblbtest1.azurewebsites.net' });
            window.document.location.reload();
        });
    });
    </script>
    
  • James Strugnell 84 posts 192 karma points
    Dec 22, 2015 @ 16:07
    James Strugnell
    0

    Thanks Shannon. We've updated to use the Local Temp Cache location so hopefully that will be an improvement and prevent any further file locking issues.

    I might register a feature request with Azure to be able to route specific URLs to specific instances, and prevent scaling. I can imagine this might be a useful feature beyond just Umbraco.

    Scanning that code suggests it might be possible to implement something within Umbraco to keep users on one server, though I'll save that for another (very quiet) day.

    Cheers.

  • James Strugnell 84 posts 192 karma points
    Dec 22, 2015 @ 16:11
    James Strugnell
    1

    I've not marked your answers as "the answer" Shannon, as it seems the current answer (to the original question) is that Umbraco Forms does not play nicely with the recommended approach for Azure Flexible Load Balancing. I think the Forms team should seriously look at switching to, or providing the option of, persisting form schemas in the database.

  • Shannon 24 posts 45 karma points
    Feb 01, 2016 @ 07:21
    Shannon
    0

    I am also having this issue

  • Robert Stigsson 47 posts 158 karma points
    Mar 30, 2016 @ 09:10
    Robert Stigsson
    0

    We're thinking of moving to Umbraco Forms from Contour, but this is the main reason why we keep the Contour-solution for now (and might develop our own solution for it in the future).

    The other problem is that we are not "updating" the server, but we compile everything and then overwrite everything on the server, which would overwrite the Forms-folder. Not ideal from our point of view to lose all forms that aren't versioned in GIT..

    If Forms "just" would support some kind of Load Balanced solution, that would be great. If they solve the "store forms locally"-problem, even better.

  • James Strugnell 84 posts 192 karma points
    Mar 30, 2016 @ 09:21
    James Strugnell
    0

    There was an Umbraco package released recently that claims to enable the file-based Form bits to be kept in sync across multiple nodes -

    https://our.umbraco.org/projects/website-utilities/umbraco-forms-cache-refreshers-for-flexible-load-balancing/

    I'm aware this doesn't solve the source code deployment (overwrite) issue but it does help for when you have a master/slave set-up and for when you have multiple front-end instances.

    Just thought I'd mention it here. Note that I've not actually tried this package yet.

  • Robert Stigsson 47 posts 158 karma points
    Mar 30, 2016 @ 12:05
    Robert Stigsson
    0

    Thank you very much for your answer!

    That was truly something I was looking for! Although, when reading through the code I get the feeling that this is not very secure (might be me reading the code wrong, but got the feeling that anyone from anywhere could update the forms). Do you know if there are any package-thread here at our.umbraco.org, or anywhere else? :)

  • James Strugnell 84 posts 192 karma points
    Mar 30, 2016 @ 12:13
    James Strugnell
    0

    No problem. What do you mean by "Package-thread"? If you mean a Package "Feed" there is an RSS feed for the Umbraco package repository here: https://our.umbraco.org/rss/projects

  • Robert Stigsson 47 posts 158 karma points
    Mar 30, 2016 @ 12:39
    Robert Stigsson
    0

    Oh, sorry for being unclear. I meant a post where he maybe presented his Package and where people could comment or just get in touch with him..

  • James Strugnell 84 posts 192 karma points
    Mar 30, 2016 @ 12:47
    James Strugnell
    0

    There is a github project and Issues list here: https://github.com/gfyans/CacheRefreshersForForms/issues

    If you look at one of those issues you'll see he mentions that Umbraco are working on a core solution for this that will be available soon. Hopefully they are also working on enabling the database as a persistence layer.

    There is an Umbraco Forms issues list here: http://issues.umbraco.org/issues/CON

Please Sign in or register to post replies

Write your reply to:

Draft