Amazon AWS Elastic Load Balancers and MSBuild

Our jobs take us to some interesting places sometimes - me, well, recently I've spent a fair amount of time stuck in the land of Amazon's cloud offering AWS.

Right now I'm working on a release process based around MSBuild that can deploy to a farm of web servers at AWS. As with any large online offering ours makes use of load balancing to provide a reliable and responsive service across a set of web servers.

Anyone who has managed deployment of a solution in this scenario is most likely familiar with this approach:

Remove target web host from load balancing pool.
Update content on the web host and test.
Return web host to load balancing pool.
Repeat for all web hosts.
(Profit! No?!)

Fantastic Elastic and the SDK

In AWS-land load balancing is provided by the Elastic Load Balancing (ELB) service which, like many of the components that make up AWS, provides a nice programmatic API in a range of languages.

Being focused primarily on .Net we are we are happy to see good support for it in the form of the AWS SDK for .NET. The AWS SDK provides a series of client proxies and strongly-typed objects that can be used to programmatically interface with pretty much anything your AWS environment is running.

Rather than dive into the detail on the SDK I'd recommend downloading a copy and taking a look through the samples they have - note that you will need an AWS environment in order to actually test out code but this doesn't stop you from reviewing the code samples.

Build and Depoy

As mentioned above we are looking to do minimal manual intervention deployments and are leveraging MSBuild to build, package and deploy our solution. One item that is missing in this process is a way to take a target machine out of the load balancer pool so we can deploy to it.

I spent some time reviewing existing custom MSBuild task libraries that provide AWS support but it looks like many of them are out-of-date and haven't been touched since early 2011. AWS is constantly changing so being able to keep up with all it has to offer would probably require some effort!

The result is that I decided to create a few custom tasks so that I could use for registration / deregistration of EC2 Instances from one or more ELBs.

I've included a sample of a basic RegisterInstances custom task below to show you how you go about utilising the AWS SDK to register an Instance with an ELB. Note that the code below works but that it's not overly robust.

The things you need to know for this to work are:

Your AWS Security Credentials (Access and Secret Keys).
The names of the ELBs you want to register / deregister instances with.
The names of the EC2 Instances to register / deregister.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Build.Utilities;
using Microsoft.Build.Framework;
using Amazon.ElasticLoadBalancing;
using Amazon.ElasticLoadBalancing.Model;

namespace TheFarm.MsBuild.CustomTasks.Aws.Elb
{
   /// <summary>
   /// Register one or more EC2 Instance with one or more AWS Elastic Load Balancer.
   /// </summary>
   /// <remarks>Requires the AWS .Net SDK.</remarks>

   public class RegisterInstances : Task {
    /// <summary>
    /// Gets or sets the load balancer names.
    /// </summary>
    /// <value>
    /// The load balancer names.
    /// </value>
    /// <remarks>The account associated with the AWS Access Key must have created the load balancer(s).</remarks>
    [Required]
    public ITaskItem[] LoadBalancerNames { get; set; }

    /// <summary>
    /// Gets or sets the instance identifiers.
    /// </summary>
    /// <value>
    /// The instance identifiers.
    /// </value>
    [Required]
    public ITaskItem[] InstanceIdentifiers { get; set; }

    /// <summary>
    /// Gets or sets the AWS access key.
    /// </summary>
    /// <value>
    /// The aws access key.
    /// </value>
    [Required]
    public ITaskItem AwsAccessKey { get; set; }

    /// <summary>
    /// Gets or sets the AWS secret key.
    /// </summary>
    /// <value>
    /// The aws secret key.
    /// </value>
    [Required] public ITaskItem AwsSecretKey { get; set; }

    /// <summary>
    /// Gets or sets the Elastic Load Balancing service URL.
    /// </summary>
    /// <value>
    /// The ELB service URL.
    /// </value>
    /// <remarks>Will typically take the form: https://elasticloadbalancing.region.amazonaws.com</remarks>
    [Required]
    public ITaskItem ElbServiceUrl { get; set; }

    /// <summary>
    /// When overridden in a derived class, executes the task.
    /// </summary>
    /// <returns>
    /// true if the task successfully executed; otherwise, false.
    /// </returns>
    public override bool Execute()
    {
      try {
         // throw away - to test for valid URI. new Uri(ElbServiceUrl.ItemSpec);
        var config = new AmazonElasticLoadBalancingConfig { ServiceURL = ElbServiceUrl.ItemSpec };

        using (var elbClient = new AmazonElasticLoadBalancingClient(AwsAccessKey.ItemSpec, AwsSecretKey.ItemSpec, config))
        {
          foreach (var loadBalancer in LoadBalancerNames)
          {
            Log.LogMessage(MessageImportance.Normal, "Preparing to add Instances to Load Balancer with name '{0}'.", loadBalancer.ItemSpec);
            var initialInstanceCount = DetermineInstanceCount(elbClient, loadBalancer);
            var instances = PrepareInstances();
            var registerResponse = RegisterInstancesWithLoadBalancer(elbClient, loadBalancer, instances);
            ValidateInstanceRegistration(initialInstanceCount, instances, registerResponse);
            DetermineInstanceCount(elbClient, loadBalancer);
          }
        }
      }
      catch (InvalidInstanceException iie)
      {
        Log.LogError("One or more supplied instances was invalid.", iie);
      }
      catch (LoadBalancerNotFoundException lbe)
      {
        Log.LogError("The supplied Load Balancer could not be found.", lbe);
      }
      catch (UriFormatException)
      {
        Log.LogError("The supplied ELB service URL is not a valid URI. Please confirm that it is in the format 'scheme://aws.host.name'");
      }

      return !Log.HasLoggedErrors;
    }

    /// <summary>
    /// Prepares the instances.
    /// </summary>
    /// <returns>List of Instance objects.</returns>
    private List<Instance> PrepareInstances()
    {
      var instances = new List<Instance>();

      foreach (var instance in InstanceIdentifiers)
      {
        Log.LogMessage(MessageImportance.Normal, "Adding Instance '{0}' to list.", instance.ItemSpec);
        instances.Add(new Instance { InstanceId = instance.ItemSpec });
      }
      return instances;
    }

    /// <summary>
    /// Registers the instances with load balancer.
    /// </summary>
    /// <param name="elbClient">The elb client.</param>
    /// <param name="loadBalancer">The load balancer.</param>
    /// <param name="instances">The instances.</param>
    /// <returns>RegisterInstancesWithLoadBalancerResponse containing response from AWS ELB.</returns>
    private RegisterInstancesWithLoadBalancerResponse RegisterInstancesWithLoadBalancer(AmazonElasticLoadBalancingClient elbClient, ITaskItem loadBalancer, List<Instance> instances)
    {
      var registerRequest = new RegisterInstancesWithLoadBalancerRequest { Instances = instances, LoadBalancerName = loadBalancer.ItemSpec };
      Log.LogMessage(MessageImportance.Normal, "Executing call to add {0} Instances to Load Balancer '{1}'.", instances.Count, loadBalancer.ItemSpec);
      return elbClient.RegisterInstancesWithLoadBalancer(registerRequest);
    }

    /// <summary>
    /// Validates the instance registration.
    /// </summary>
    /// <param name="initialInstanceCount">The initial instance count.</param>
    /// <param name="instances">The instances.</param>
    /// <param name="registerResponse">The register response.</param>
    private void ValidateInstanceRegistration(int initialInstanceCount, List<Instance> instances, RegisterInstancesWithLoadBalancerResponse registerResponse)
    {
      var postInstanceCount = registerResponse.RegisterInstancesWithLoadBalancerResult.Instances.Count();
      if (postInstanceCount != initialInstanceCount + instances.Count)
      {
        Log.LogWarning("At least one Instance failed to register with the Load Balancer.");
      }
    }

    /// <summary>
    /// Determines the instance count.
    /// </summary>
    /// <param name="elbClient">The elb client.</param>
    /// <param name="loadBalancer">The load balancer.</param>
    /// <returns>integer containing the instance count.</returns>
    private int DetermineInstanceCount(AmazonElasticLoadBalancingClient elbClient, ITaskItem loadBalancer)
    {
      var response = elbClient.DescribeLoadBalancers(new DescribeLoadBalancersRequest { LoadBalancerNames = new List<string> { loadBalancer.ItemSpec } });
      var initialInstanceCount = response.DescribeLoadBalancersResult.LoadBalancerDescriptions\[0\].Instances.Count();
      Log.LogMessage(MessageImportance.Normal, "Load Balancer with name '{0}' reports {1} registered Instances.", loadBalancer.ItemSpec, initialInstanceCount);
      return initialInstanceCount;
    }
  }
}

So now we have a task compiled into an assembly we can reference that assembly in our build script and invoke the task using the following syntax:

<RegisterInstances LoadBalancerNames="LoadBalancerName1"
                   InstanceIdentifiers="i-SomeInstance1"
                   AwsAccessKey="YourAccessKey"
                   AwsSecretKey="YourSecretKey"
                   ElbServiceUrl="https://elasticloadbalancing.your-region.amazonaws.com"
/>

That's pretty much it... there are some vagaries to be aware of - the service call for deregistration of an Instance returns prior to the instance being fully de-registered with the load balancer so don't key any deployment action directly off of that return - you should perform other checks first to make sure that the instance *is* no longer registered prior to deploying.

I hope you've found this post useful in showing you what is possible when combining the AWS SDK with the extensibility of MSBuild.