Setting up load balanced environments generally takes a lot of planning, design and testing. This document should assist in setting up Umbraco in load balanced environments. This document assumes that you have a fair amount of knowledge about:
* IIS
* Windows Server (2003/2008)
* .Net Framework
* .Net SDK
Design
For the sake if this document, the following assumptions will be made:
* All servers are part of the same domain
* All servers are on the same network/subnet
* You have administration access to all servers
There are 2 design alternatives you can use to effectively load balance servers:
1. The load balanced website files are located on a centralized file share (SAN/NAS/Custered File Server/Network Share)
2. Each server hosts copies of the load balanced website files and a file replication service is running to ensure that all files on all servers are up to date
And, you'll need a load balancer to do your load balancing obviously!
DNS
Each server in your cluster will require a unique DNS name assigned to the host header for the IIS install. This is so Umbraco knows which nodes (server nodes, not content nodes ;) to replicate it's cached content with. Obviously each server has it's own unique domain DNS name but i highly recommend against adding this name to the host headers for IIS.
An example of how to setup DNS and host headers between 3 load balanced servers:
Server 1
* domain DNS name: server1.mydomain.local
* internal website DNS name: server1.mywebsite.com (use for IIS host header)
* IP Address: 192.168.1.10
Server 2
* domain DNS name: server2.mydomain.local
* internal website DNS name: server2.mywebsite.com (use for IIS host header)
* IP Address: 192.168.1.11
Server 3
* domain DNS name: server3.mydomain.local
* internal website DNS name: server3.mywebsite.com (use for IIS host header)
* IP Address: 192.168.1.12
Of course you'll have your public website's DNS/address which you'll also need to add to the host header for each of your IIS server's websites. For instance, if the public website address is: http://www.mywebsite.com then you'll need to add www.mywebsite.com as a host header to IIS website on each server.
Load Balancer
A load balancer is what is going to balance the traffic between your servers. There's a ton of load balancers out there and hardware ones are generally the most effective way to go about balancing traffic. If you don't have a hardware load balancer, don't worry as you can use software. Windows server comes with NLB (Network load balancing). It's relatively easy to setup and it's free. Windows Server 2008 has changed some things with NLB's architecture so if you're using 2008, make sure you read up on it.
Some important notes on NLB:
* Load balancing with VMWare & NLB
* Ensure that the internal IP Addresses for NLB have DNS registration disabled, are not configured to a a client for Microsoft Networks and have Netbios over TCPIP disabled
* Windows Server 2008 changed the way that TCP-IP works and have disabled forwarding. In order for NLB to work with 2 network cards (the recommended way), you have to enable forwarding for the private NIC:
** Article 1
** Article 2
File Storage on SAN/NAS/Clustered File Server/Network Share
This is the recommended approach, however it is probably more difficult to setup.
Pros
* No duplication of files
* Easier to add more servers
* No issues with file replication failures
* Easier to deploy files
Cons
* More difficult to setup
* More hardware required
A note when using this method to store your files centrally, you MUST make sure that your file storage system is HA (Highly Available) which means that there's not single point of failure. If you're hosting your files on a File Server share, you need to make the file share clustered (using MSCS or similar). WIndows server 2008 supports connecting directly to a SAN via iSCSI if your SAN supports it (there are also many other ways to connect to a SAN to share folders), otherwise you should be able to connect to a NAS via a UNC path.
There's a lot of work required to get this working, but once it's done it's fairly easy to maintain. We've this same setup working for many websites so hopefully these notes help you get started:
Windows Setup
* Create domain user account that will run your IIS websites. Example: MyDomain\WebsiteUser
* Grant this domain user FULL access to your file share
* On each web server, add this user to the IIS Security group account. Server 2003: IIS_WPG, Server 2008: IIS_IUSRS
* The .Net Code Access Policy must be updated on each server to run with Full Trust for the UsterNC share:
** EXAMPLE: %windir%\Microsoft.NET\Framework64\v2.0.50727\caspol -m -ag 1. -url "file://\\fileserver.mydomain.local\Inetpub\MySite\*" FullTrust -name "WebsiteUser"
* The IIS user above needs to be granted the appropriate IIS permissions:
** EXAMPLE: %windir%\Microsoft.NET\Framework64\v2.0.50727\Aspnet_regiis.exe -ga ActiveDirectoryDomain\ProcessIdentity
* Restart the server
Much of the above is covered in this Microsoft doc: ASP.Net 3.5 Hosting
IIS Setup
Since the files for the website will be hosted centrally, each IIS website on your servers will need to point to the same UNC share for the files. For example: \\fileserver.mydomain.local\Inetpub\MySite
* point to the shared file server: \\fileserver.mydomain.local\Inetpub\MySite
* "Connect To" this share with the user account created above
* have their application pools run as the user above
* Have the IIS anonymous user account set to the application pool account (IIS 7)
File Storage with File Replication
I've implemented a few clusters with file replication but I find it's harder to maintain and room for more error. Though, it is generally easier to setup.
Pros
* Easier to setup
* No need for additional hardware
Cons
* Harder to maintain
* Harder to add servers
There's
a few file replication services out there, some free, some not so free.
Some people have been known to write their own with command lines like
xcopy or even synctoy. Conveniently, Windows Server comes with a fairly
good solution called DFS which supports file replication over the domain and it works pretty well with Umbraco: Implementing DFS Replication
When using DFS make sure to exclude the ExamineIndexes folder under /app_data/temp from the replication, as explained in this Examine Codeplex thread: https://examine.codeplex.com/discussions/270941.
IIS Setup
IIS configuration is pretty straightforward with file replication because IIS is just reading files from it's own file system just like a non-clustered website.
Umbraco Configuration
Configuring Umbraco to support load balanced clusters is probalby the easiest part. In the /config/umbracoSettings.config file you need to updated the distributed call section to the following (as an example)
<distributedCall enable="true">
<user>0</user>
<servers>
<server>server1.mywebsite.com</server>
<server>server2.mywebsite.com</server>
<server>server3.mywebsite.com</server>
</servers>
</distributedCall>
As you can see in the above XML the distributed server names are the custom DNS names created for each IIS host name for each server. Don't forget to enable the distributedCall.
In addition to enabling the distributed calls above, there are some additional items that should be modified as well. Each server will need a unique copy of the /App_Data/TEMP directory for Examine and Client Dependency Cache to work correctly. Further, the umbraco.config xml cache file should also be unique to limit any issues with file locks.
Configuring Temp files with a virtual directory
- Edit /web.config and change the umbracoContentXML to use ~/App_Data/TEMP/umbraco.config.
- Copy the /App_Data/TEMP directory to each server, outside of any replication areas or to a unique folder for each server.
- Create a virtual directory (not a virtual application) in the /App_Data folder, and name it TEMP. Point the virtual directory to the folder you created in step 2.
- You may delete the /App_Data/TEMP folder from the file system (not IIS as this may delete the virtual directory) if you wish.
Umbraco Administration
It is highly recommended to designate ONE of the load balanced servers as your administration server. The reason for this is a known issue with Umbraco in load balanced environments when writing to the umbraco.config cache file. If you are administrating Umbraco via the public website address (which means it's load balancing requests), then a file locking issue will occur because multiple servers are attempting to write to the same file simultaneously.
To get around this issue, editors and administrators should administrate umbraco from one of the servers. This would mean that your firewall will need to allow public traffic to one of the internal DNS address such as http://server1.mywebsite.com/umbraco/umbraco.aspx
Testing
You'll need to test this solution A LOT before going to production. You need to ensure there are not windows security issues, etc... The best way to determine issues is have a lot of people testing this setup and ensuring all errors and warnings in your application/system logs in Windows are fixed.
To test Umbraco distributed calls, just create and publish some content on one server (i.e. http://server1.mywebsite.com/umbraco/umbraco.aspx), then browse to the front end content on another server (i.e. http://server2.mywebsite.com/public/page1.aspx if page1 was the newly published content). If the page shows up on the 2nd server, though it was published from the 1st server, then distributed calls are working! You'll need to thoroughly test this though.
Conclusion
Though this is somewhat details, this is still a basic overview. Setting these environments up for production is not an easy task and requires a lot of testing. Hopefully this guide will point you in the right direction!