Category Archives: Web Apps

Continuous Deployment for Docker with VSTS and Azure Container Registry

I’ve been watching with interest the growing maturity of Containers, and in particular their increasing penetration as a hosting and deployment artefact in Azure. While I’ve long believed them to be the next logical step for many developers, until recently they have had limited appeal to many every-day developers as the tooling hasn’t been there, particularly in the Microsoft ecosystem.

Starting with Visual Studio 2015, and with the support of Docker for Windows I started to see this stack as viable for many.

In my current engagement we are starting on new features and decided that we’d look to ASP.Net Core 2.0 to deliver our REST services and host them in Docker containers running in Azure’s Web App for Containers offering. We’re heavy uses of Visual Studio Team Services and given Microsoft’s focus on Docker we didn’t see that there would be any blockers.

Our flow at high level is shown below.

Build Pipeline

1. Developer with Visual Studio 2017 and Docker for Windows for local dev/test
2. Checked into VSTS and built using VSTS Build
3. Container Image stored in Azure Container Registry (ACR)
4. Continuously deployed to Web Apps for Containers.

We hit a few sharp edges along the way, so I thought I’d cover off how we worked around them.

Pre-requisites

There are a few things you need to have in place before you can start to use the process covered in this blog. Rather than reproduce them here in detail, go and have a read of the following items and come back when you’re done.

  • Setting up a Service Principal to allow your VSTS environment to have access to your Azure Subscription(s), as documented by Donovan Brown.
  • Create an Azure Container Registry (ACR), from the official Azure Documentation. Hint here: don’t use the “Classic” option as it does not support Webhooks which are required for Continuous Deployment from ACR.

See you back here soon 🙂

Setting up your Visual Studio project

Before I dive into this, one cool item to note, is that you can add Docker support to existing Visual Studio projects, so if you’re interested in trying this out you can take a look at how you can add support to your current solution (note that it doesn’t magically support all project types… so if you’ve got that cool XAML or WinForms project… you’re out of luck for now).

Let’s get started!

In Visual Studio do a File > New > Project. As mentioned above, we’re building an ASP.Net Core REST API, so I went ahead and selected .Net Core and ASP.Net Core Web Application.

New Project - .Net Core

Once you’ve done this you get a selection of templates you can choose from – we selected Web API and ensured that we left Docker support on, and that it was on Linux (just saying that almost makes my head explode with how cool it is 😉 )

Web API with Docker support

At this stage we now have baseline REST API with Docker support already available. You can run and debug locally via IIS Express or via Docker – give it a try :).

If you’ve not used this template before you might notice that there is an additional project in the solution that contains a series of Docker-related YAML files – for our purposes we aren’t going to touch these, but we do need to modify a couple of files included in our ASP.Net Core solution.

If we try to run a Docker build on VSTS using the supplied Dockerfile it will fail with an error similar to:

COPY failed: stat /var/lib/docker/tmp/docker-builder613328056/obj/Docker/publish: no such file or directory
/usr/bin/docker failed with return code: 1

Let’s fix this.

Add a new file to the project and name it “Dockerfile.CI” (or something similar) – it will appear as a sub-item of the existing Dockerfile. In this new file add the following, ensuring you update the ENTRYPOINT to point at your DLL.

This Dockerfile is based on a sample from Docker’s official documentation and uses a Docker Container to run the build, before copying the results to the actual final Docker Image that contains your app code and the .Net Core runtime.

We have one more change to make. If we do just the above, the project will fail to build because the default dockerignore file is stopping the copying of pretty much all files to the Container we are using for build. Let’s fix this one by updating the file to contain the following 🙂

Now we have the necessary bits to get this up and running in VSTS.

VSTS build

This stage is pretty easy to get up and running now we have the updated files in our solution.

In VSTS create a new Build and select the Container template (right now it’s in preview, but works well).

Docker Build 01

On the next screen, select the “Hosted Linux” build agent (also now in preview, but works a treat). You need to select this so that you build a Linux-based Image, otherwise you will get a Windows Container which may limit your deployment options.

build container 02

We then need to update the Build Tasks to have the right details for the target ACR and to build the solution using the “Dockerfile.CI” file we created earlier, rather than the default Dockerfile. I also set a fixed name for the Image Name, primarily because the default selected by VSTS typically tends to be invalid. You could also consider changing the tag from $(Build.BuildId) to be $(Build.BuildNumber) which is much easier to directly track in VSTS.

build container 03

Finally, update the Publish Image Task with the same ACR and Image naming scheme.

Running your build should generate an image that is registered in the target ACR as shown below.

ACR

Deploy to Web Apps for Containers

Once the Container Image is registered in ACR, you can theoretically deploy it to any container host (Azure Container Instances, Web Apps for Containers, Azure Container Services), but for this blog we’ll look at Web Apps for Containers.

When you create your new Web App for Containers instance, ensure you select Azure Container Registry as the source and that you select the correct Repository. If you have added the ‘latest’ tag to your built Images you can select that at setup, and later enable Continuous Deployment.

webappscontainers

The result will be that your custom Image is deployed into your Web Apps for Containers instance and which will be available on ports 80 and 443 for the world to use.

Happy days!

I’ve uploaded the sample project I used for this blog to Github – you can find it at: https://github.com/sjwaight/docker-dotnetcore-vsts-demo

Also, please feel free to leave any comments you have, and I am certainly interested in other ways to achieve this outcome as we considered Docker Compose with the YAML files but ran into issues at build time.

Tagged , , ,

Deploy a PHP site to Azure Web Apps using Dropbox

I’ve been having some good fun getting into the nitty gritty of Azure’s Open Source support and keep coming across some amazing things.

If you want to move away from those legacy hosting businesses and want a simple method to deploy static or dynamic websites, then this is worth a look.

The sample PHP site I used for this demonstration can be cloned on Github here: https://github.com/banago/simple-php-website

The video is without sound, but should be easy enough to follow without.

It’s so simple even your dog could do it.

Dogue

Tagged , , ,

Zero to MySQL in less than 10 minutes with Azure Database for MySQL and Azure Web Apps

I’m a long-time fan of Open Source and have spent chunks of my career knocking out LAMP solutions since before ‘LAMP’ was a thing!

Over the last few years we have seen a revived Microsoft begin to embrace (not ’embrace and extend’) the various Open Source platforms and tools that are out there and to actively contribute and participate with them.

Here’s our challenge today – setup a MySQL environment, including a web-based management UI, with zero local installation on your machine and minimal mouse clicks.

Welcome to Azure Cloud Shell

Our first step is to head on over to the Azure portal at https://portal.azure.com/ and login.

Once you are logged in open up a Cloud Shell instance by clicking on the icon at the top right of the navigation bar.

Cloud Shell

If this is the first time you’ve run it you will be prompted to create a home file share. Go ahead and do that :).

Once completed, run this command and note down the unique ID of the Subscription you’re using (or note the ID of the one you want to use!)

 
az account list

MySQL Magic

Now the fun begins! I bet you’re thinking “lot’s of CLI action”, and you’d be right. With a twist!

I’m going to present things using a series of simple bash scripts – you could easily combine these into one script and improve their argument handling, but I wanted to show the individual steps without writing one uber script that becomes impenetrable to understand!

Here’s our script to setup MySQL in Azure.

Now I could get you to type that out, or upload to your cloud share via the Portal, but that’s no fun!

At your Cloud Shell prompt run the following (update the last command line with your arguments):

 
curl -O -L https://gist.githubusercontent.com/sjwaight/0ba37e3c3522aeaebf6bd20c9f3895b0/raw/a9fccb9126330983b8a84a71f07dea2b1c583c68/create-azure-mysql.sh
chmod 755 create-azure-mysql.sh

# make sure you update the parameters for your environment
# change 'mysqldemo01', 'yawadmin' and '5ecurePass@word!'
./create-azure-mysql.sh 368ff49e-XXXX-XXXX-XXXX-eb42e73e2f25 westus mysqldemorg mysqldemo01 yawadmin 5ecurePass@word!

Note that the server name (mysqldemo01) must be gobally unique – if you get an error try adding some random text to the server name.

The other two items ‘yawadmin’ and ‘5ecurePass@word!’ are Deployment Credentials that must be set in the App Service before you can use them – if you are using a new Subscription you can go ahead and set them. Note!!! if you are already using Deployment Credentials in the Subscription you shouldn’t change them without understanding the impact this will have across other App Service instances as these Credentials are shared.

After a few minutes you will have a MySQL server ready for use. Note that by default you won’t be able to connect to it as the firewall around it is shut by default (which is a good thing). We’ll rectify connectivity later. For now, on to the next piece of the puzzle.

Manage MySQL from a browser

No, not via some super-duper Microsoft MySQL tooling, but via everyone’s old favourite phpMyAdmin.

Surely this will take a lot of work to install I hear you ask? Not at all!

Enter Azure Web App Site Extensions! Specifically the phpMyAdmin extension.

Let’s get started by creating a new App Service Plan and Web App to which we can deploy our management web application.

We’re going to re-use the trick we learned above – pulling a Gist from Github using curl. First have a ready through the script :).

You can download this Gist from within your Cloud Shell and execute it as follows. Make sure to update the command line arguments

 
curl -O -L https://gist.githubusercontent.com/sjwaight/d65a04036cd6ceda63bf90485e22adb4/raw/6802a35d21a9d46e0abb5e3f1c88f32551f325da/create-azure-webapp.sh
chmod 755 create-azure-webapp.sh

# make sure you update the parameters for your environment
./create-azure-webapp.sh 368ff49e-XXXX-XXXX-XXXX-eb42e73e2f25 westus mysqldemorg mysqldemo01 yawadmin 5ecurePass@word! mydemoapplan msqlmgewebapp

In order to deploy the Web App Sit Extension we are going to dig a bit behind the covers of Azure App Services and utilise the REST API provided by the kudu site associated with our Web App (this appears as ‘Advanced tools’ in the Portal). If you want to understand more about its capabilities you can, and specifically about how to work with Site Extensions read their excellent documentation.

Note: if you haven’t previously setup a Git / FTP deployment user you should uncomment the line that does this. Note that this step sets the same credentials for all instances in the Subscription, so if you already have credentials defined think twice before uncommenting this line!

 
curl -O -L https://gist.githubusercontent.com/sjwaight/8a530aff0e5f3991484176825b49a563/raw/be8cfa68f95a29687f2dae6af9ca163e8e7877ac/enable-phpmyadmin.sh
chmod 755 enable-phpmyadmin.sh

# make sure you update the parameters for your environment
./enable-phpmyadmin.sh 368ff49e-XXXX-XXXX-XXXX-eb42e73e2f25 mysqldemorg msqlmgewebapp deployuser d3pl0yP455!

Done!

Browse to your freshly setup phpMyAdmin instance.

Connection Error

Oh noes!

Yes, we forgot to open up the firewall surrounding the Azure Database for MySQL instance. We can do this pretty easily.

Remember those ‘outboundIpAddresses’ values you captured when you created the Web App above? Good, this is where you will need them.

You should find that you have four source IP addresses from which outbound traffic can originate. These won’t change as long as you don’t “stop” or delete the Web App.

Here’s our simple script to enable the Web App to talk to our MySQL instance.

Now the usual drill.

Note: You might have more than four outbound IP addresses to allow – if so simply edit the script to suit.

 
curl -O -L https://gist.githubusercontent.com/sjwaight/bbd71ca3f094abe5cf438c1b826b83fe/raw/4b6aaddee8f6a21b81061ed4c4085258c0271068/update-mysql-firewall.sh
chmod 755 update-mysql-firewall.sh

# make sure you update the parameters for your environment
./update-mysql-firewall.sh 368ff49e-XXXX-XXXX-XXXX-eb42e73e2f25 mysqldemorg mysqldemo01 192.168.1.1 192.168.2.2 192.168.3.3 192.168.4.4

Once the rules are applied, try refreshing the web browser and you will now see the you are viewing the glorious phpMyAdmin.

phpMyAdmin on Azure!

Congratulations, you now have a fully functional MySQL environment, with no installs and minimal configuration required!

Tagged , , , , , , ,

Speaking at Office 365 Saturday

If you’re interested to learn more about Microsoft Graph API and how you can leverage it to build compelling solutions in the form of Bots in Microsoft Teams, I’ll be speaking at Office 365 Saturday in Sydney this week on June 3rd.

Tickets are free, but get in while there are still some left!

O365 Saturday Sydney

Saturday, Jun 3, 2017, 8:45 AM

Clifton’s Sydney
60 Margaret Street Sydney, AU

115 Members Went

Welcome to the 2017 edition of Sydney Office 365 Saturday.Join administrators, end users, architects, developers, and other professionals that work with Microsoft Technologies for a great day of awesome sessions presented by industry experts.Did you attend and want to leave feedback.Leave feedback here.O365 Saturday is a fantastic day to learn…

Check out this Meetup →

Demo

If you are interested in the demonstration I ran during my talk you can download the code from Github.

The way the bot hangs together is shown below.

How is the bot built?

Tagged

Per-environment config value tokenization for Azure Web Apps using VSTS Release Management

For the majority of the last ten years I’ve been working with delivery of solutions where build and deployment comes from some centralised location.

When Microsoft made InRelease part of TFS as Release Management, I couldn’t wait to use it. Unfortunately in its state at that time the learning curve was quite steep and the immediate value was outweighed by the effort to get up and running.

Roll forward to 2016 and we find Release Management as a modern, web-based feature of Visual Studio Team Services (VSTS). The cherry on the cake is that a lot of the learning curve has dropped away as a result.

In this post I’m going to look at how we can deploy a Web Deploy (or MS Deploy) packaged Web Application to an Azure Web Application and define different deployment environments with varying configurations.

Many people would apply configuration transformations at build time, but in my scenario I want to deploy the same compiled package to multiple environments without the need to recompile anything.

My Challenge

The build definition for my Web Application results in a package that allows it to be deployed to an Azure Web App by Web Deploy. The result is the web.config configuration file is in a zip file that is transferred to the server for deployment by Web Deploy.

Clearly at this point I don’t have access to the web.config file in the drop folder so I can’t transform it with Release Management. Or can I?!

Using Web Deploy Parameters

Thankfully the design of Web Deploy provides for the scenario I described above though use of either commandline arguments or a specially formatted input file that I will call the “SetParameters” file.

Given this is a first-class feature in the broader Microsoft developer toolkit, I’d expected that there would be a few Tasks in VSTS that I could use to get all of this up and running… I got close, but couldn’t quite get it functioning as I wanted.

Through the rest of this post I will walk you through the setup to get this going.

Note: I am going to assume you have setup Build and Release Management definitions in VSTS already. Your Build should package to deploy to an Azure Web App and the Release Management definition to deploy it.

VSTS Release Management Setup

The first thing to get all of this up and running is to add the Release Management Utilities extension to your subscription. This extension includes the Tokenizer Task which will be key to getting the configuration per-environment up and running.

You also need to define an “Environment” in Release Management for each deployment target we have, which will also be used as a container for environmental configuration items to replace at deployment time. A sample is shown below with two Environments defined

Environments

We’ll come back to VSTS later, for now, let’s look at the project changes you need to make.

Source Project Changes

For the purpose of this exercise I’m just worrying about web.config changes.

First of all, you need to tokenise the settings you wish to transform. I have provided a sample below that shows how this looks in a web.config. The format of two underscores on either side of your token placeholder is required.

The next item we need to do is to add a new XML file to our Visual Studio project at the root level. This file should be called “Parameters.xml” and I have included a sample below that shows what we need to add to if it we want to ensure we replace the tokens in the above sample web.config.

You’ll notice one additional item in the file below that isn’t related directly to the web.config above – the IIS Website name that will be used when deployed. I found if I didn’t include this the deployment would fail.

When you add this file, make sure to set the properties for it to a Build Action of “None” and Copy to Output Directory of “Do not copy”.

Note: if you haven’t already done so, you should run a Build so that you have Build Artifacts ready to select in a later step.

Add the Tokenizer to your Release Management Definition

We need now to return to VSTS’ web interface and modify our existing Release Management definition (or create a new one) that adds the Tokenizer utility to the process.

You will need to repeat this so all your environments have the same setup. I’ve shown how my Test environment setup looks like below (note that I changed the default description of the Tokenizer Task).

Release Management Definition

Configuration of the Tokenizer is pretty straight forward at this point, especially if we’ve already run a build. Simply select the SetParameters.xml file your build already produced.

Tokenizer setting

Define values to replace Tokens

This is where we define the values that will be used to replace the tokens at deployment time.

Click on the three dots at the top right of the environment definition and from the menu select “Configuration variables…” as shown below.

Variable Definition

A dialog loads that allows us to define the values that will go into our web.config for this environment. The great thing you’ll note is that you can obfuscate sensitive details (in my example, the key to access the Document DB account). This is non-reversible too – you can’t “unhide” the value and see the plain-text version.

Token Values

We’re almost done!

Explicitly select SetParameters file for deployment

I’m using the 3.* (preview) version of the Deploy Azure App Service Release Management Task, which I have configured as shown.

App Service Task

At this point, if you create a new Release and deploy to the configured environment you will find that the deployed web.config contains the values you specified in VSTS and you will no longer need multiple builds to send the same package to multiple environments.

Happy Days! 🙂

Tagged , , , , , ,