Copied to clipboard

Flag this post as spam?

This post will be reported to the moderators as potential spam to be looked at


  • Murray Roke 502 posts 965 karma points c-trib
    Sep 09, 2014 @ 00:39
    Murray Roke
    0

    Fluent Migrator style Migrations for Umbraco

    Hi everyone.
    I'm undertaking a bit of a proof of concept project to create a migrations framework/api for umbraco. Taking inspiration from Fluent Migrator.

    I'm wondering if anyone has tried this before or if there's something else that could do what I want.

    We've used uSiteBuilder in the past quite heavily, and we're looking for something else... simpler? faster? more control?

    I've not use uSync, it seems simpler and faster, but not more control (this is probably what we'll use if my migrations project fails, or is just too much work.)

    The idea is we have some code which runs for each version (checking a version sequence number in the DB) the code could be arbitrary but usually will contain simple API calls to manipulate the doctype/datatype info in the database.

    This code can be pretty verbose, so I'd like to wrap it in a bit of a fluent API.

    [UmbracoMigration(1)]
    public class ModifyLandingPage : UmbracoMigration
    {
        public override void Up()
        {
            migrator.ContentType("Landing Page")
                .DeleteProperty("Bolox")
                .DeleteProperty("oldProperty");
    
            migrator.ContentType("Landing Page")
                .Property("Alpha")
                    .Alias("beta")
                    .Name("Beta")
                    .Description("Mandatory")
                    .Mandatory();
            migrator.Save();
        }
    }
    

    The advantage here, is we can run any code we need for the migration, we can delete properties, we can re-name properties.

    I am thinking on start any un-run migrations will run. also each start the latest migration will ALWAYS run (you could configure that off if you like eg, for production) the API would encourage idempotent code. Idem-potency makes developing your migrations easier.

    I probably won't implement the 'down' feature that FluentMigrator has simply because we probably wouldn't use that.

    Any thoughts? :-)
    Cheers.
    Murray.

  • James Jackson-South 489 posts 1747 karma points c-trib
    Sep 09, 2014 @ 01:21
    James Jackson-South
    0

    Count me in. I'm definitely intrigued.

  • Jamie Howarth 306 posts 773 karma points c-trib
    Sep 09, 2014 @ 11:25
    Jamie Howarth
    0

    Follow @MagDevsUK and @Type_TR on Twitter, as they're hoping to OSS their efforts that they've been working on over the last few months.

  • Tom 24 posts 69 karma points
    Sep 09, 2014 @ 11:49
    Tom
    2

    Morning guys, as Benjamin said we have a EF code first migration inspired nuget package for umbraco we are using for our current build. Its solid and in use in our live environment, in need of a little love for it to be feature complete and it lacks proper documentation but its not far off. As always before we release this we need to get Legal to do their stuff, but the proposal has been submitted. Fingers crossed it not too far away.

    Quick example.

    public override void Migrate()
        {
            var basePageDt = DocumentType.Get("BasePage");
    
            var zoneHomepageTemplate = Template.Create("ZoneHomepage", "Zone Homepage");
    
            var zoneHomepageDt = DocumentType.Create("ZoneHomepage", "Zone Homepage")
                                             .SetParent(basePageDt)
                                             .AddAllowedTemplate(zoneHomepageTemplate)
                                             .SetDefaultTemplate(zoneHomepageTemplate)
                                             .Save();
    
            zoneHomepageDt
                .CreateTab("Sidekick")
                .AddProperty("SidekickImage", "Sidekick Image", DataTypes.MediaPicker, "Sidekick", false, "Hero Image")
                .AddProperty("SidekickHeader", "Sidekick Header", DataTypes.TextString, "Sidekick", false, "Intro text Header")
                .AddProperty("SidekickRichtext", "Sidekick  Wysiwyg Content", DataTypes.RichtextEditor, "Sidekick", false, "WYSIWYG content")
                .AddProperty("SidekickCallToAction", "Sidekick Call To Action", CustomDataTypeConsts.UrlPicker, "Sidekick", false)
                .AddProperty("SidekickWidgets", "Sidekick Widgets", CustomDataTypeConsts.WidgetPicker, "Sidekick", false, "Widgets to display below sidekick text area")
                .Save();
         }
    

    Keep an eye on @MagDevsUk as we'll no doubt announce it through that.

  • James Jackson-South 489 posts 1747 karma points c-trib
    Sep 09, 2014 @ 21:56
    James Jackson-South
    0

    Does legal getting involved mean that it wouldn't be open source? As in properly feature complete and open source?

  • Murray Roke 502 posts 965 karma points c-trib
    Sep 10, 2014 @ 00:23
    Murray Roke
    0

    @Tom looks interesting, how does it support versioning & running? I'm not familiar with EF's migrations.
    What operations does the fluent API support? rename, edit, delete?
    I guess that those operations it doesn't support we can just fall back to the raw API :-)

  • Tom 24 posts 69 karma points
    Sep 10, 2014 @ 09:30
    Tom
    1

    @James - I don't see why not, its just a hoop we need to jump through, I imagin it will just go under one of the usual open source licences.

    @Murray - The main reason we moved away from uSync/Courier (we tried both) was so we could refactor and CI our application, so we have made sure it allows us to modify everything we have had a need to so far (which is most things). It runs on app start and checks a migration history table to see what migrations it has already ran and executes any it hasn't.

  • James Jackson-South 489 posts 1747 karma points c-trib
    Sep 10, 2014 @ 12:32
    James Jackson-South
    0

    Awesome! I'm really looking forward to seeing the fruits of your labour.

  • Murray Roke 502 posts 965 karma points c-trib
    Sep 10, 2014 @ 23:00
    Murray Roke
    0

    Yep awesome, are your fluent operations idempotent? (can be run twice with the same outcome as running once)?
    I think this is important while in development to re-run the last migration over and over so we don't end up with hundreds if micro migrations.

  • Tom 24 posts 69 karma points
    Sep 11, 2014 @ 11:15
    Tom
    0

    The most important thing to us is having 100% confidence that when migrations execute there is no way anything unexpected can happen. To ensure we are happy to let this run wild and free in our production environments it will immediately bomb out at the first sign of something unexpected and roll back any instructions in the migration that have already been executed. Once a migration has ran, that's it, it won't run again against a db that has it already executed.

    This is not ideal for dev, but early on we opted for robustness over ease of development, that being said other than a few steps we take to ensure our migration is sound there is nothing overly tedious about it. I build up my migration step by step verifying each one has done what I expected. I write some instructions, run the migration, check results, comment out previous instruction then add more. Pretty much rinse and repeat that until all the changes for the piece of work I'm doing is done.

    Our current application has 91 migrations and is probably going to have about 150 before the end of the project.

  • David Brendel 792 posts 2970 karma points MVP 3x c-trib
    Sep 11, 2014 @ 13:07
    David Brendel
    0

    Hi guys,

    don't know if it's important for you but Umbraco ships with its own migration system where you can do a lot of stuff.

    Haven't tryied to change DocumentType or something like this with it but think that could be possible.

    Just search for MigrationRunner or MigrationBase in the Umbraco Source Repository on GitHub.

    Maybe you can find some usefull stuff there.

    Besides of that I'm interested in the outcome of this project.

    David

  • Jamie Howarth 306 posts 773 karma points c-trib
    Sep 11, 2014 @ 13:49
    Jamie Howarth
    0

    MigrationRunner and MigrationBase are for database schema changes and the metadata they use implies upgrading or downgrading between differing Umbraco versions.

    You would have to write your own versioning utilities to override Umbraco's versioning to handle using these classes - which would be less than ideal (i.e. I'm using Umbraco 7.1.4 on a project, I don't like the idea of having to bump version to 7.1.5, 7.1.6 etc. just to get my migrations to run, cause then I'm hacking a version number inconsistent with my runtimes).

  • Tom 24 posts 69 karma points
    Sep 11, 2014 @ 14:34
    Tom
    0

    Yeee we did look at that but it was no go, it wasn't quite what we where looking for. We looked into all the well tried and tested tools people currently use for this kind of thing but they where all dead ends. When we decided to write our own we looking into Umbracos source for any existing functionality we could leverage. We decided in the end to keep it simple and drive the public Umbraco services.

    EDIT - ye, what Benjamin said :P

  • Chris Roberts 74 posts 106 karma points c-trib
    Sep 15, 2014 @ 20:22
    Chris Roberts
    0

    Just followed @MagDevsUK and @Type_TR on Twitter - can't wait to hear some news on this!

    Something like this has been on our, "If only we had time to build..." list for ages now.

  • David Brendel 792 posts 2970 karma points MVP 3x c-trib
    Sep 16, 2014 @ 12:25
    David Brendel
    0

    The MigrationRunner can be used to Run Migrations outsite the Umbraco versions.

    I'm currently using it to make updates to custom database tables for a package I develope.

    But you're right it's more for changing database scheme and not for changing other stuff.

    Just wanted to state that there is something in Umbraco.

  • Murray Roke 502 posts 965 karma points c-trib
    Oct 06, 2014 @ 00:47
    Murray Roke
    0

    @Tom, any news on the possibility of releasing what you guys have built, how are the lawyers ;-)

  • Tom 24 posts 69 karma points
    Oct 09, 2014 @ 11:53
    Tom
    0

    Not date yet unfortunately, we've started some basic docs and all the paperwork is submitted. Hopefully we'll be attending the upcoming Manchester Umbraco meetup, which from its title 'Why code first is bad', sounds like it could be interesting.

    I have created a github repo with a ubr short README with code example https://github.com/dev-mag/uMigrate. Once released it will be available from there.

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 09, 2014 @ 13:28
    Chris Roberts
    0

    Thanks for the update! The 'readme' in the repository looks promising...

    I'm assuming the answer to the "Why code first is bad" topic is, "Because Umbraco sell Courier"?!

    My personal experience of Courier has ranged from disappointing to disasterous - other people's mileage may vary, of course - but I'm looking forward to having a 'proper', robust way to deploy changes out to Umbraco sites in different environments!

    Thanks,
    - Chris

     

  • James Jackson-South 489 posts 1747 karma points c-trib
    Oct 09, 2014 @ 13:33
    James Jackson-South
    0

    Thanks for the update. Really looking forward to seeing some code. :)

    I think "Why Code First is Bad" is a misnomer. It really should be "Why are Current Code First Implementations Bad".

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 09, 2014 @ 13:35
    Chris Roberts
    0

    Ahh - yeah - that'd be fair enough.

    Always fun to try and stir up a bit of trouble, though! ;o)

    As a matter of interest - what do you currently use for your deployments?

  • James Jackson-South 489 posts 1747 karma points c-trib
    Oct 09, 2014 @ 13:49
    James Jackson-South
    0

    Honestly... Our deployment system right now is shambolic.

    We've been using uSiteBuilder but it's dog slow and fraught with concurrency errors that I don't have time to fix. I hate that we have to manually edit the web.config file also.

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 09, 2014 @ 15:02
    Chris Roberts
    0

    Our process isn't too much better.

    We have a system to automatically deploy templates, css, images and config files direct from version control using Jenkins - but we don't have any automation for deployment of changes to document types, etc.

    Feels like the fluent API mentioned in this topic could be the answer!

  • Tom 24 posts 69 karma points
    Oct 09, 2014 @ 15:13
    Tom
    1

    We have ours completely automated from TeamCity to AWS test/live environments. A package is created for each push/checkin and manually selected for deployment to a given environment, if the package includes any number of new migrations they get executed and jobs a gooden. Working well for us so far, Our current live site has executed 127 migrations without any faults (touch wood).

    The key for us has been being able to test the migrations locally, repeatably, so we are sure there will be no issues against any other environment.

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 09, 2014 @ 15:33
    Chris Roberts
    0

    Sounds a lot less painful than what we're doing!

  • Tom 24 posts 69 karma points
    Oct 09, 2014 @ 15:40
    Tom
    0

    Yeeeee, we went through the pain of having to write pages and pages of release instructions for every deployment, and then replay them for every environment. Not fun at all, especially if a typo sneaks in!

  • Tom 24 posts 69 karma points
    Oct 17, 2014 @ 11:05
    Tom
    0

    Looks like somebody may have stolen the name of our framework :(. Although uMigrate would be the obvious name for such a framework it seems a little too much of a coincidence that it appeared shortly after I mentioned it here. I find it hard to believe that after years without a code first migration style deployment framework for Umbraco 2 with the same name with very similar api can come about within weeks of one another.

    Legal have just given us the go ahead to release it too, typical.

  • Murray Roke 502 posts 965 karma points c-trib
    Oct 17, 2014 @ 11:23
    Murray Roke
    0

    So this is someone else's eh. http://our.umbraco.org/projects/collaboration/umigrate

    uMigrage is the first name I thought when I was thinking of making such a thing. I guess it's an overdue idea.
    Now you will be able to come up with a less boring name.:-)

    Now I guess there's going to be 2 to evaluate :-]

    I really appreciate all the work you've put in, can't wait to try it out.

    Cheers.
    Murray.

  • James Jackson-South 489 posts 1747 karma points c-trib
    Oct 17, 2014 @ 15:38
    James Jackson-South
    0

    If yours works in v6 also you can have my first born.

  • Tom 24 posts 69 karma points
    Oct 17, 2014 @ 15:43
    Tom
    1

    haha :), our sites are built in 6 as when we started the project 7 had only just been released. Umbracos testing strategy of letting the users test it in production doesn't cut it for us so we stuck with tried and tested. We will upgrade though in future!

    TL;DR.... it works in 6.

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 17, 2014 @ 15:45
    Chris Roberts
    0

    Well - it's a shame about the naming conflict, but if we've gone from having no solution, to having two potential solutions - that's a win as far as I'm concerned!

    We haven't started using v7 yet for the same reason. Looking forward to trying uMigrate / uGetYourStuffLiveWithoutWantingToKillYourself (or whatever it ends up being called) soon!

    Thanks,
    - Chris

  • James Jackson-South 489 posts 1747 karma points c-trib
    Oct 17, 2014 @ 15:47
    James Jackson-South
    0

    enter image description here

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 17, 2014 @ 15:48
    Chris Roberts
    0

    uRollout

    uDeploy

    uNoGUI

  • Chris Roberts 74 posts 106 karma points c-trib
    Oct 17, 2014 @ 15:52
    Chris Roberts
    0

    So, Tom,

    When do you think we'll be able to see your v6 friendly version?

    Thanks,
    - Chris

  • Tom 24 posts 69 karma points
    Oct 17, 2014 @ 15:53
    Tom
    2

    we have a name, we considered uDeploy haha. Just doing to do some refactoring over the weekend the namespaces and such match. Keeping it a not so well guarded secret until then though in case somebody else uses it for 'inspiration'.

  • Murray Roke 502 posts 965 karma points c-trib
    Oct 20, 2014 @ 05:28
    Murray Roke
    2

    goose. (it migrates)

  • Murray Roke 502 posts 965 karma points c-trib
    Nov 13, 2014 @ 04:24
    Murray Roke
    0

    @Tom, any news to share? :-)

  • Tom 24 posts 69 karma points
    Nov 13, 2014 @ 09:21
    Tom
    1

    Slow but sure, we're just splitting some stuff out and other re factorings. Testing the changes with our application over the next week or so then we'll get it public :).

  • Chris Roberts 74 posts 106 karma points c-trib
    Nov 13, 2014 @ 18:51
    Chris Roberts
    0

    Thanks Tom,

    Keep up the good work - let me know if you need a beta tester! (not that I'm keen, or anything...)

    - Chris

  • Tom 24 posts 69 karma points
    Nov 20, 2014 @ 12:18
    Tom
    0

    We have finally got round to refactoring it ready for release, we are updating our application to use the public version to ensure everything is still working. Once we're using the public one in production i'll let you guys know. Keep in mind its been made for our internal use so the Api isn't 'feature complete', but we've been using it for months so cant be that lacking lol.

  • Chris Roberts 74 posts 106 karma points c-trib
    Nov 20, 2014 @ 12:21
    Chris Roberts
    0

    Awesome - looking forward to seeing it!

    Will you be accepting community contributions? If so then I guess anything it doesn't do we can add!

    Thanks,
    - Chris

  • Tom 24 posts 69 karma points
    Nov 20, 2014 @ 13:19
    Tom
    0

    Its open source on GitHub so I don't see why not :)

  • Tom 24 posts 69 karma points
    Nov 21, 2014 @ 10:30
    Tom
    0

    Morning guys,

    I'll start a new thread in a week or 2 but for now I'll bury these links here to limit visibility for now.

    Source : https://github.com/dev-mag/uFluent

    Api : https://www.nuget.org/packages/uFluent/ Runner : https://www.nuget.org/packages/uFluent.Migrate/

    Any and all feedback is welcomed!

    Documentation is lacking atm, that's on the todoasap list!

    2 important details for uFluent.Migrate are...

    1. Migration classes need to implement IUmbracoMigration
    2. ApplicationStarted will need to run uFluent.Migrate.uFluentMigrate.Run();

    Docs are being worked on, this is very much an early glance for you guys.

  • Chris Roberts 74 posts 106 karma points c-trib
    Nov 27, 2014 @ 10:26
    Chris Roberts
    0

    Hi Tom,

    Thanks for posting this - sorry I didn't jump on it sooner but I must've missed the e-mail alert from this site!

    We'll take a look at this and let you know how we get on!

    Thanks,
    - Chris

  • Tom 24 posts 69 karma points
    Nov 27, 2014 @ 10:50
    Tom
    0

    No problem, hope it is of some use in this early form, we've done 3 deployments on this new package so its usable. Apologies for gross lack of documentation, it is on the board, just prioritized beneath project guff.

  • Chris Roberts 74 posts 106 karma points c-trib
    Nov 27, 2014 @ 10:51
    Chris Roberts
    0

    No worries - you've done loads already by the look of it!

    If I get anywhere with it, I'll try and put together some blog posts to help people get started!

  • Tom 24 posts 69 karma points
    Dec 02, 2014 @ 00:16
    Tom
    0

    Evening guys, there's some basic documentation on github if you want to take a look. https://github.com/dev-mag/uFluent

    Feedback and/or questions welcome.

  • Murray Roke 502 posts 965 karma points c-trib
    Dec 02, 2014 @ 01:26
    Murray Roke
    0

    Awesome, a section for 'Install' which includes the nuget links would be handy. :-)

    I also note that nuget says:
    Dependencies
    UmbracoCms.Core (≥ 6.1.6 && < 7.0.0)

    What's the story with v7 support?

    Cheers.
    Murray.

  • Tom 24 posts 69 karma points
    Dec 02, 2014 @ 09:15
    Tom
    0

    We started the project that ended up spawning uFluent before 7 was out of beta, there's nothing more to it than that really. We are going to be upgrading to 7 soonish and that will include full testing and fixing anything that's broken against 7. But for now unfortunately we have no idea what it would do running on top of 7, it might even work fine, but until we have thoroughly tested it and have it working on our production environment officially it only supports 6.x

  • Tom 24 posts 69 karma points
    Dec 16, 2014 @ 14:51
    Tom
    0

    Oh and added Nuget Links to readme

Please Sign in or register to post replies

Write your reply to:

Draft