Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    Jan 28, 2017 @ 07:36
    Darren Ferguson
    0

    Questions!

    Hi Jeavon,

    Thanks for piecing this all altogether.

    A quick question though? Why do you go to the trouble of having the CDN source type as Blob storage?

    When we've done this setup - we've just pointed the CDN at the Umbraco site, where it requests the image/asset on the original URL the first time and subsequently caches it.

    In this approach, you don't have to upload your CSS, Javascript etc to CDN as the CDN fetches them from Umbraco.

    With images the only modification you have to make is to prefix the original /media URL with the CDN domain and the image processor stuff still works.

    I'm (probably) missing something!

  • Simon Dingley 1171 posts 2634 karma points
    May 11, 2017 @ 09:06
    Simon Dingley
    0

    Hi Darren,

    I'm new to this and I am in the process of trying to implement an Azure based CDN at the moment and your approach seems far easier so wondered if you can point towards more information on that setup? Is this something that can be done for an Umbraco instance not hosted on the Azure platform?

    I wish I'd seen your post before I just transferred 5GB of media to my Azure CDN test account :(

    Cheers, Simon

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 11, 2017 @ 12:24
    Darren Ferguson
    1

    I've been meaning to blog about it!

    You use: https://our.umbraco.org/projects/collaboration/umbracofilesystemprovidersazure/

    To use the backoffice for media in blob storage.

    You set up a CDN endpoint pointing at your umbraco site (Custom endpoint).

    In your templates you put the CDN URL on the front of the URL:

    e.g. http://mycdn.azureedge.net/media/32983/what.jpg

    You need to setup that CDN endpoint to cache every unique request.

    The CDN caches hard, so you may want to put the update date of the media item into the URL e.g.

    http://mycdn.azureedge.net/media/32983/what.jpg?ud=201708091032

    So that you get the new version each time the media item is saved.

  • Simon Dingley 1171 posts 2634 karma points
    May 18, 2017 @ 09:27
    Simon Dingley
    0

    Hi Darren,

    This was super simple to setup with HTTP - so thanks! Currently I can't get it to work with HTTPS due to a gateway error which I am yet to resolve. Some blog posts I have been reading seem to indicate it might be related to the fact we have SNI enabled on the server but that is a discussion for a separate thread.

    You use: https://our.umbraco.org/projects/collaboration/umbracofilesystemprovidersazure/

    To use the backoffice for media in blob storage.

    Is this optional? It seems to still work without this step?

    Initially I thought that this approach was going to copy the assets into my blob storage container but now it seems the storage container is not even required - am I correct? If so, even more reason for this approach to be favoured! It's such a light and easy integration.

    It would be great if Jeavon could contribute to this thread with reasons to favour his approach over yours.

    Thanks, Simon

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 11:19
    Darren Ferguson
    0

    Hi Simon,

    We've not had issues with https....

    I guess you could use the approach without blob and Azure filesystem providers - but Pete makes a good point, now I've read it properly.

    Let us know about the SNI issue, either here or on another thread.

  • Simon Dingley 1171 posts 2634 karma points
    May 11, 2017 @ 13:38
    Simon Dingley
    0

    Thanks Darren, I'll be sure to try this as it seems so much more straightforward with less config to manage in Umbraco.

  • Peter Duncanson 395 posts 1254 karma points c-trib
    May 18, 2017 @ 09:55
    Peter Duncanson
    1

    If you upload to blob you get some advantages if you have multiple developers/editors/installed instances I believe.

    The assets are stored in one place so if you pull the site down to develop on it you don't need to get all the media too. This works with Darren's setup too of course but you end up with one Umbraco site that has the data and all the others that don't so you now have a tricky situation if anyone deletes the wrong site, by having the assets stored shared but separate from Umbraco you keep a nice separation.

    If you then work locally and add some assets for a new feature they get stored locally on your local version not the live site that the CDN will ask for the assets from. If you then forget to copy them over you end up breaking things. Its an easy mistake to make and can be a tricky one to remember and fix come deployment time.

    Both setups work, Darrens is easier (if you only have one developer/site on the whole) but Jeavons is more belt and braces with the complete separation of the assets from Umbraco.

  • Simon Dingley 1171 posts 2634 karma points
    May 18, 2017 @ 11:31
    Simon Dingley
    0

    Centralised media is a bonus, especially as this site, in particular, has over 5Gb of media currently. Say all of your environments are using the shared blob storage, how do you manage the possibility that media will be uploaded from a test/staging environment that is not intended to be used in production? Or, is this where the CDN toolkit comes in handy?

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 11:38
    Darren Ferguson
    1

    You need a blob storage container per umbraco environment/database e.g. dev/stage/prod.

    You can configure the container that is used in the Azure filesystem provider config.

    You probably also want a CDN endpoint per environment.

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 10:55
    Darren Ferguson
    0

    Hi Pete - my solution uses blob storage too??

  • Peter Duncanson 395 posts 1254 karma points c-trib
    May 18, 2017 @ 10:57
    Peter Duncanson
    0

    Oh, from Simons description of it not copying the assets to blob (is it maybe storing resized image in the blob storage?) it sounded like it wasn't.

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 11:17
    Darren Ferguson
    1

    sorry i'll read it properly this time and reply to simon :)

  • Simon Dingley 1171 posts 2634 karma points
    May 18, 2017 @ 11:20
    Simon Dingley
    0

    To remove any confusion, right now I have just setup the origin to point to the staging site and can get media returned via http://.azureedge.net/ with no further configuration. What I am saying is that at this point no media is copied into the blob storage.

    I understand that once I add UmbracoFileSystemProviders.Azure into the mix that any media uploaded will then go to the blob storage - correct? But, without this step I'm curious where any media is being drawn from? Is it simply cached across Azure CDN Servers with no visible storage after the first request?

    Sorry for the noob questions but this is my first time so please be gentle :)

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 11:24
    Darren Ferguson
    0

    Azure CDN will store copy of the media on it's own servers.

    Upon first request - if it doens't hold a copy of the item it will get it from Umbraco.

    If you don't set up Azure filesystem providers - it will get the copy to cache from the local file system where umbraco lives, otherwise it would read it from Blob storage.

  • Simon Dingley 1171 posts 2634 karma points
    May 18, 2017 @ 11:42
    Simon Dingley
    0

    Thanks for the clarification. So in the case of an existing site, I need to still push all existing media up to the blob storage container if I am to enable the Azure filesystem provider?

  • Darren Ferguson 1003 posts 3138 karma points mvp c-trib
    May 18, 2017 @ 11:46
    Darren Ferguson
    0

    Yup - there is a tool called Azure storage explorer.

    We also wrote a console app that does it with multiple threads - and byte size comparison which is quite fast for large amounts of media:

    https://github.com/darrenferguson/Moriyama.BlobCopy

  • Simon Dingley 1171 posts 2634 karma points
    May 18, 2017 @ 12:12
    Simon Dingley
    0

    Azure storage explorer

    Oh I've already been through the pain of that piece of software - it should be READ-ONLY! It could not handle upload of the existing media, looked like it was creating multiple background processes for each item and just hanging! What I used in the end was AZCopy and the following command and it was super quick in comparison:

    AzCopy.exe /Source:<path_to_umbraco>\media /Dest:https://<endpoint>.blob.core.windows.net/media /DestKey:<key> /S
    

    Since I've got to do it again now I will try out your app - thanks!

  • Simon Dingley 1171 posts 2634 karma points
    May 22, 2017 @ 11:17
    Simon Dingley
    0

    Thanks for this Darren, uploading nice and quickly now. I made a small pull request to set the MIME type for files before uploading and it seems to be working well for me so far.

  • Jeavon Leopold 2906 posts 12398 karma points admin mvp c-trib
    May 19, 2017 @ 12:31
    Jeavon Leopold
    0

    First things first, Darren's approach is called origin pull, my approach (or the approach of the CDN toolkit) is called origin push.

    We originally planned to use origin pull as it's very simple however we hit some security issues during testing.

    Therefore the reason we created the CDN Toolkit an was to enable a CDN push approach which would allow us to secure what is placed in the blob storage and therefore onto the CDN.

  • Simon Dingley 1171 posts 2634 karma points
    May 19, 2017 @ 12:47
    Simon Dingley
    0

    Thanks Jeavon, are you able to share any details of the security issues you encountered so that we can make an assessment of whether we are likely to be impacted by it or not?

    Thanks, Simon

  • Jeavon Leopold 2906 posts 12398 karma points admin mvp c-trib
    May 19, 2017 @ 12:48
    Jeavon Leopold
    0

    Can you Slack me on the community channel?

  • Simon Dingley 1171 posts 2634 karma points
    May 19, 2017 @ 12:57
    Simon Dingley
    0

    I will try my best to do that as soon as I can - thanks

  • Jeavon Leopold 2906 posts 12398 karma points admin mvp c-trib
    May 19, 2017 @ 12:58
    Jeavon Leopold
    0

    I have sent you some info on there already :)

Please Sign in or register to post replies

Write your reply to:

Draft