Tag Archives: Microsoft Azure

Understanding Azure’s Container PaaS Capabilities

If you’ve been using Azure over the past twelve months, you can’t but have the feeling that it’s become a bit like this…

Containers... Containers Everywhere

.. and you’d be right.

To be fair, though, Containers have been one of the hot topics in computing in general and certainly one that’s been getting the most interest in my recent Azure Open Source Roadshows.

One thing that has struck me though is that people are not clear on the purpose of all the services in Azure that have ‘Containers’ listed as a capability, so in this post I am going to try and review the Azure Platform-as-a-Service offerings that have Container capabilities and cover what the services can be used for.

First, before we begin, let’s quickly get some fundamentals under our belts.

What is a Container?

Containers provide encapsulation and isolation for workloads and remove the need for a complete Operating System image to be deployed in order to manage resource allocation.

They have proven popular because they typically have smaller footprints than Virtual Machines, boot much faster as a result and have a modern build process based on composition that gels well with software development.

A Container still needs to “run” somewhere – this “somewhere” is what I will call a “Container Host” through the rest of this post.

So where does Docker fit into all of this? Docker provides tooling for the creation, running and management of Containers and is by far the best known tech in this space. Microsoft has worked with Docker to ensure the Docker tooling supports Windows and Windows Containers.

Our most basic Container workload setup then would be: one Container Host running one Container.

What is a Container Orchestrator?

A big part of running Containers at scale is their management which is where technologies like Kubernetes (k8s), Docker Swarm and DC/OS come into play. These technologies allow you to manage multiple Containers and their workloads, performing orchestration of deployments and controlling connectivity between Container instances running on Container Hosts.

An Orchestrator typically runs more than one node to ensure availability, but nothing stops us from running a simple single node setup like Minikube to start to learn about them.

Right, now we have some fundamentals in place, let’s take a look at what Azure offers.

Azure’s Container Offerings

Note that we are going to focus on PaaS services – you can of course still run Containers on Virtual Machines, or deploy something like OpenShift in Azure if you wish.

Please note: any service listed as ‘Preview’ should not be used for Production deployments!


Azure Container Registry (ACR)

What is it? ACR is an Azure-hosted Container Registry based on the open-source Docker Registry v2 spec. This is a turnkey part of Azure’s Container story.

Why use it? When you build a Container Image you need a place to store it. Docker Hub is the Registry where you pull all your public Images from and which is run by Docker. ACR provides you with a private Docker-compatible Registry that you can push Images to and use as a deployment source.

Benefits:

  • Private Registry you configure that is not published on a well-known endpoint that is a source for public Images
  • Provides a unique *.azurecr.io Registry endpoint which can be used to store Images that can be deployed *anywhere* (not just Azure)
  • Webhook support that can be used for Continuous Deployment, particularly with Azure’s Web Apps for Containers (see below)
  • Control access to the Registry using Azure Active Directory Credentials
  • ACR provides seamless authentication (i.e. no configuration) with other Azure services like Azure Container Instances, Azure Container Services, App Service and Batch.
  • Geo-replication is the hotness! (requires Premium level) * Preview

Restrictions:

  • Cooler features (like geo-replication) are at higher price point only
  • I’m struggling here for others! 🙂

> ACR Documentation.


Azure Container Instances (ACI) * Preview

What is it? An Individual Container Host that can run one or more Container. No need for you to manage the Host.

Why use it? These are probably the easiest way to get going running Container-based workloads in Azure. If you have a simple workload that needs a public IP and which can talk to various Azure PaaS services then consider ACI over Web Apps for Containers or Azure Container Service.

Benefits:

  • No need for you to manage the Container Host – tell it which Containers to run and that’s it!
  • Pay per-second for use with customisable CPU Core and memory options
  • Supports multiple Containers in single ACI Host
  • “Whole of Azure” scale: deploy ACI workloads in any Azure Region (where ACI is available).

Restrictions:

  • No production Orchestrator support: there is an experimental Kubernetes Connector, but apart from this you cannot bolt an ACI Host into an Orchestrated environment
  • No VNet support: you can’t connect an ACI Host to Azure VNets.

> ACI Documentation.


Web Apps for Containers (App Service on Linux)

What is it? A Container Host that runs on a Linux-based variant of Azure’s App Service that is aimed at web-centric workloads (hence the name). Like ACI, you still don’t need to manage the Host.

Why use it? If you have a website or HTTP API workload you traditionally host on Linux and that you can (or have) containerised, then this is a good spot to start as it limits your workloads exposure to HTTP (80) and HTTPS (443) ports.

Additionally, even if you haven’t containerised your solution, you can still use this service to host it. When you select a framework to use to host your solution (like Java or PHP) the framework is deployed to Web Apps for Containers as a Docker Image!

Benefits:

  • Get access to standard App Service features like Autoscale, Custom Domains, SSL and Continuous Deployment
  • No need for you to manage the Container Host – tell the Web Apps Instance which Container to run and it will do that
  • Deploy existing Docker images from Docker Hub, Azure Container Registry or from Azure’s pre-built framework images
  • Troubleshoot containers using SSH from Kudu.

Restrictions:

  • No Orchestrator support: what you gain through using App Service you give up in not being able to bolt the Container Host into an Orchestrator like Kubernetes
  • Multi-Container deployments are not supported
  • Not all Windows-based App Service features are supported (yet)
  • Not currently supported in App Service Environments.

> Web Apps for Containers Documentation.


Azure Container Services (ACS)

What is it? A service that allows you to run Container Hosts that are managed by an Orchestrator of your choice (Kubernetes, Docker or DC/OS).

Why use it? If you already run Container workloads on VMs (regardless of hosting location) that use an Orchestrator, or you’d like to start using Containers at scale and need an Orchestrator, then this is the service to use.

Benefits:

  • No need for you to manage the underlying Virtual Machine infrastructure
  • Orchestrator and Container Host setup is managed for you by the ACS engine (which is open source)
  • Container Host scalability is supported via use of Virtual Machine Scale Sets (VMSSs)
  • All hosts (Orchestrator and Container) are vanilla instances – this is not a special “Azure release”
  • Orchestrators have Azure extensions allowing them to perform actions such as creating Azure Load Balancers when you specify load balanced workloads in your setup.
  • Integration with Azure Container Registry for Image deployments.

Restrictions:

  • You pay for both the Container Hosts and Orchestrator Nodes (they are just VMs after all)
  • You can’t increase the number of Orchestrator / Cluster Masters after you have initially created an ACS cluster
  • You can’t upgrade the Orchestrator once you have created an ACS cluster – you need to create a new ACS cluster to gain access to a newer release.

> ACS Documentation: Kubernetes | Docker | DC/OS.


Azure Container Services – Managed Kubernetes (AKS) * Preview

What is it? This service is similar to ACS (above), however in this service (which only supports Kubernetes) the Orchestrator Nodes are managed on your behalf by Microsoft.

Why use it? If you are invested in Kubernetes (or intend to use it as your Orchestrator) and would prefer not to have to manage the Orchestator Nodes then you should select this over standard ACS with “unmanaged” Kubernetes. If you are using the ACS Kubernetes offering already then this is a logical place to migrate to once AKS is Generally Available.

Benefits:

  • You don’t pay for the Orchestrator Nodes running Kubernetes
  • Orchestrator Node availability, patching and upgrading is managed by Microsoft
  • No need to create a new ACS cluster to pick up new Kubernetes releases
  • Will support 100% of the standard Kubernetes API
  • All other ACS features remain in place!

Restrictions:

  • Only supports Kubernetes!
  • During preview does not support all Kubernetes features.

> AKS Documentation.

On a side note: AKS + ACI (with its Kubernetes connector) + ACR will be an amazing PaaS Container story once all these components are all Generally Available! 😎


Azure Service Fabric

What is it? Service Fabric is both a cluster Orchestrator and a development framework for delivery of highly available, distributed applications. It pre-dates the current Container hype cycle and is used to deliver services in Azure such as CosmosDB.

Why use it? If you want to leverage the Reliable Actor and Service patterns offered by the Service Fabric development framework. Also worth considering if you haven’t yet started with an Orchestrator like Kubernetes.

Benefits:

  • Mature product that underpins key Microsoft cloud-scale services
  • Runs in Azure or on-premises.

Restrictions:

  • Container workloads can’t benefit from the development framework as they run as ‘guest executables’ on cluster nodes (this will change in future as you will be able to Containerise Reliable Actors and Services)
  • Developers using Windows 10 can’t deploy Container-based solutions to local Service Fabric clusters.

> Service Fabric Container Documentation.


Azure Batch

What is it? The name says it all really – you can use Azure Batch to run compute workloads that can be broken into lots of concurrent processes. Examples include payroll runs, animation rendering or research modelling. Batch sits well within the High Performance Compute (HPC) landscape.

Why use it? If you have a batch-style workload with processors that can be Containerised then this is a service you should seriously be considering. More-so if you wish to consider a hybrid scenario where you run some of your workload in-house and burst to Azure as required. As you have Containerised workload you can ship dependencies in a single bundle.

Benefits:

  • You can use Docker Hub as a source for Images (yes, you could pull tensorflow and run it in Azure 😉 ), in addition to ACR and any other compatible Registry
  • Use Singularity in addition to Docker Containers with Batch
  • Run processes on low-priority VMs to reduce the cost (best for non-time sensitive operations).

Restrictions:

  • RDMA (high performance networking) support only available for Containers running on Linux.

> Azure Batch Container Documentation.


So there we are – hopefully the Azure Container story now makes much more sense and you can pick between the services to understand those that would be most appropriate to your use case.

Happy days! 😎

Tagged , , , , , ,

Get Started with Docker on Azure

The most important part of this whole post is that you need to know that the whale in the Docker logo is officially named “Moby Dock“. Once you know that you can probably bluff your way through at least an introductory session on Docker :).

It’s been hard to miss the increasing presence of Docker, particularly if you work in cloud technology. Each of the major cloud providers has raced to provide container services (Azure, AWS, GCE and IBM) and these platforms see benefits in the higher density hosting they can achieve with minimal changes to existing infrastructure.

In this post I’m going to look at first steps to getting Docker running in Azure. There are other posts about that will cover this but there are a few gotchas along the way that I will cover off here.

First You Need a Beard

Anyone worth their take home pay who works with *nix needs to grow a beard. Not one of those hipstery-type things you see on bare-ankled fixie riders. No – a real beard.

While Microsoft works on adding Docker support in the next Windows Server release you are, for the most part, stuck using a Linux variant to host and manage your Docker containers.

The Azure Cross-Platform Command-Line Interface teases you with the ability to create Docker hosts from a Windows-based computer, but ultimately you’ll have a much easier experience running it all from a Linux environment (even if you do download the xplat-cli there anyway).

If you do try to set things up using a Windows machine you’ll have to do a little dancing to get certificates setup (see my answer on this stackoverflow post). This is shortly followed by the realisation that you then can’t manage the host you just created by getting those nice certificates onto another host – too much work if you ask me :).

While we’re on Docker and Windows let’s talk a little about boot2docker. This is designed to provide an easy way to get started with Docker and while it’s a great idea (especially for Windows users) you will have problems if you are running Hyper-V already due to boot2docker’s use of Virtualbox which won’t run if you already have Hyper-V installed.

So Linux it is then!

Management Machine

Firstly let’s setup a Linux host that will be our Docker management host. For this post we’ll use a CentOS 7 host (I’ve avoided using Ubuntu because there are some challenges installing and using node.js which is required for the Azure xplat CLI).

Once this machine is up and running we can SSH into it and install the required packages. Note that you’ll need to run this script as a root-equivalent user.

Now we have our bits to manage the Docker environment we can now build an image and actual Docker container host.

Docker Container Host

On Azure the easiest way to get going to with Docker is to use the cross platform CLI’s Docker features.

As a non-root user on our management linux box we can run the following commands to get our Docker host up and running. I’m using an Organisational Account here so I don’t need to download any settings files.

# will prompt for username and password
[sw@sw1 ~]$ azure login

# set mode to service management
[sw@sw1 ~]$ azure config mode asm

# get the list of Ubuntu images - select one for the next command
[sw@sw1 ~]$ azure vm image list | grep Ubuntu-14_04

# setup the host - replace placeholders
[sw@sw1 ~]$ azure vm docker create -e 22 -l "West US" {dockerhost} "b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04_1-LTS-amd64-server-20141125-en-us-30GB" {linxuser} {linxpwd}

At this point we now have a new Azure VM up and running that has all the necessary Docker bits installed for us. If we look at the VM’s entry in the Azure Portal we can see that ports 22 and 4243 are already open for us. We can go ahead and test that everything’s good. Don’t forget to substitute your hostname!

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 info

Deploy an Image to a Container

As we have our baseline infrastructure ready to rock so let’s go ahead and deploy an image to it. For the purpose of this post we are going to use the wordpress-nginx image that can be built using the configuration in this Github repository.

On our management host we can run the following commands to build the image from the Dockerfile contained in the Git repository.

[sw@sw1 ~]$ git clone https://github.com/eugeneware/docker-wordpress-nginx.git

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 build -t="docker-wordpress-nginx" docker-wordpress-nginx/

Note: you need to make sure you run this as the user who setup the Docker container host and that you do it in the home directory of the user. This is because the certificates generated by the container host setup are stored in the user’s home folder in a directory called .docker. Also, expect this process to take a reasonable amount of time because it’s having to pull down a lot of data!

Once our image build is finished we can verify that it is on the Docker host by issuing this command:

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 images

Let’s create a new containerised version of the image and map the HTTP port out so we can access it from elsewhere in the world (we’re going to map port 80 to port 80). I’m also going to supply a friendly name for the container so I can easily reference it going forward (if I didn’t do this I’d get a nice long random string I’d need to use each time).

[sw@sw1 ~]$  docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 create -p 80:80 --name="dwn01" docker-wordpress-nginx

Now that we have created this we can start the container and it will happily run until we stop it :).

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 start dwn01

If we return to the VM management section in the Azure Management Portal and add an Endpoint to map to port 80 on our Docker container host we can then open up our WordPress setup page in a web browser and configure up WordPress.

If we simply stop the container we will lose any changes to the running environment. Docker provides us with the ‘commit’ command to rectify this. Let’s go ahead and save our state:

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 commit dwn01 sw/dwn01

and then we can stop the Container.

[sw@sw1 ~]$ docker --tls -H tcp://{dockerhost}.cloudapp.net:4243 stop dwn01

We now have a preserved state container along with the original unchanged one. If we want to move this container to another platform that supports Docker we could also do that, or we could repeat all our changes based on the original unchanged container.

This has been a very brief overview of Docker on Azure – hopefully it will get you started with the basics and comfortable with the mechanics of setting and up and managing Docker.

Tagged , ,

Microsoft Azure: 2014 Year in Review

What a massive year it’s been for Microsoft’s Azure public cloud platform. Running the Azure Sydney User Group this year has been great fun and seeing the growing local interest has been fantastic.

The focus from Microsoft has really changed in this space and has been clearly signalled with the change in name of Azure from Windows Azure to Microsoft Azure during the year and an increasingly broad set of non-Microsoft services offered on it.

2015 promises to be another big year, but let’s look back at what happened during 2014 with Azure.


January

The year got off to a fairly quiet start, but as we’ll see, it soon ramped up.

Preview

Everything this month was under GA only, so see below!

Generally Available

  • Websites:
    • staged publishing support
    • Always On support *
    • more frequent metric updates and monitoring alerts
  • SQL Database: new metrics and alerts
  • Mobile Services: SenchaTouch support
  • Cloud Services: A8 and A9 machine sizes now supported.

* If you’re using New Relic there are some known issues with this feature.

Other News

The Azure platform received PCI-DSS compliance validation and introduced reduced pricing rates for storage and storage transactions.


February and March

The headline item in this period was the launch of the Japan Geography with Japan East (Saitama Prefecture) and West (Osaka Prefecture) providing that market with in-country services. Also during this period we had the following announcements and launches:

Preview

Generally Available

Other News

Local gamers unhappy not to have a local Xbox server platform to run on. Who knew it was such an issue having lag and big ping times 😉

Can we haz l0c4l serverz?


April

The big change this month was the change in name for Azure. Guaranteeing a million-and-one outdated websites, slides and documents in one swoop, the service name was changed from Windows Azure to Microsoft Azure. Just for fun there is no “official” logo, just text-based branding.

This change was a subtle nod to Azure’s ability to run Infrastructure-as-a-Service (IaaS) workloads on platforms other than Windows – something it had been doing for quite some time when this change was made.

Preview

  • Newly designed management portal
  • Mobile services: documented offline support and role-based Azure AD authentication
  • Resource Manager via PowerShell
  • SQL Database: active geo-replication (read replicas); self-service restore; 500GB support; 99.95% SLA
  • Media Services: secure delivery and Office 365 Video Portal.

Generally Available

  • Azure SDK 2.3: increased Visual Studio support – create VMs using Server Explorer
  • Autoscale – Virtual Machines, Cloud Services, Web Sites and Mobile Services
  • Azure AD Premium – Multi-factor Authentication (MFA) and security reporting
  • Websites: SSL bundled; Java support; Web Hosting Plans; Available in SE Asia
  • Web Jobs SDK
  • Media Services: Live Streaming; Partnerships for Content Management and Analytics (Ooyala) and Live Ingest (iStreamPlanet)
  • Basic Tier introduction: lower cost for dev/test scenarios. Applies to VMs and Websites
  • Puppet and Chef support on Azure VMs via VM Agent Extensions
  • Scheduler Service
  • Read Access Geo Redundant Storage (RA-GRS).

May and June

The pace from the first quarter of the year carried over into these two months! The stand out amongst the range of announcements in this period was the launch of the API Management service which was the result of the October 2013 acquisition of Apiphany.

Preview

  • Azure API Management – publish, manage and secure your existing REST APIs
  • Azure File Service (SMB shares) – even use on Linux VMs
  • BizTalk Hybrid Connections – on-prem connects without the secops guys 😉
  • Redis Cache support – now the preferred caching platform in Azure
  • RemoteApp – Lay down common Apps on demand
  • Site Recovery – backup your on-prem VMs to Azure
  • Secure VMs using security extensions from Microsoft, Symantec and McAfee
  • Internal Load Balancing for VMs and Cloud Services
  • HDInsights: Apache HBASE and Hadoop 3.1
  • Azure Machine Learning (or as I like to call it “Skynet”).

Generally Available

  • ExpressRoute – WAN and DC cross-connects
  • Multi-connection Virtual Networks (VNET) and VNET-to-VNET connections
  • Public IP Address Reservation (IPv4 shortage anyone?)
  • Traffic Manager: use Azure and non-Azure (“external”) endpoints
  • A8 and A9 VM support – lots of everything (8 / 16 cores – 7 GB RAM per core)
  • Storage Import/Export service – check region availability!

Other News

MSDN subscribers gained the ability to deploy Windows 7 and 8 images onto Azure VMs for dev/test scenarios and Enterprise Agreement (EA) customers were given the ability to purchase add-ons via the Azure Store which had previously not been possible.

We also learned about availability of IPv4 addresses with some US-based services being issued IPv4 addresses assigned to South America, causing many LOLs for service admins out there who thought their services were in Brazil!


July and August

This period’s summary: Ice Bucket Challenge.

Preview

  • Event Hubs: capture data from all the Internet connected things!
  • Redis cache: in more places and sizes
  • Preview management portal: manage Azure SQL Database
  • DocumentDB
  • Azure Search.

Generally Available


September

No single announcement jumps out so I was going to put a picture of a kitten here but I thought you might want to see this (even if it is from 2012).

Preview

  • Role-based access control (RBAC) for Azure management in preview portal only
  • Resource Tagging support: filter by tag – useful for billing and ops
  • Azure SQL Database – Elastic Scale preview. Replaces Federations model
  • DocumentDB – enhanced management tooling and metrics
  • Azure Automation – AD auth; PowerShell converter; Runbook gallery and scheduling
  • Media Services – Live Streaming and DRM, faster encoding and indexer.

Generally Available

  • ‘D’ Series VMs: 60% faster CPU, more RAM and local SSD disk
  • Redis Cache: recommended cache solution in Azure. 250MB – 53GB! support
  • Site Recovery: on-prem DR with Azure – Win / Linux
  • Notification Hubs: Baidu Push (China)
  • Virtual Machines: instance-level public IPs (no NAT/PAT)
  • Azure SQL Database: three new service tiers and hourly billing
  • API Management: added OAuth support and REST Management API
  • Websites: VNet support, “scalable CMS” with WordPress and backups improvements
  • Management Services Alerts.

October and November

Pretty hard to go by this news it terms of ‘most outstanding announcement’ for these two months, especially for those of us in Australia!

Preview

  • ‘G’ Series VMs – (“Godzilla” VM) more CPU/RAM/SSD than any VM in any cloud *
  • Premium Storage – SSD-based with more than 50k IOPS *
  • Marketplace changes – CoreOS and Cloudera
  • Increased focus on Docker including portal support
  • Cloud Platform System (CPS) from Dell.
  • Batch: parallel task coordination
  • Data Factory: build data processing pipelines
  • Stream Analytics: analyse your Event Hubs data.

* Announced but not yet in public preview.

Generally Available

  • Australia Geography launches!
  • Network Security Groups
  • Multi-NIC Support in VMs (VM size dependent)
  • Forced Tunnelling (route traffic back on-prem)
  • ExpressRoute:
    • Cross-Subscription Sharing
    • Multi-connect to an Azure VNET
  • Bigger Azure Virtual Gateways
  • Ops Logging for Gateways and ExpressRoute
  • More control over Gateway encryption
  • Azure Load Balancer Source IP Affinity (“Sticky Sessions”)
  • Nested Traffic Manager Profiles
  • Preview Portal: Internal Load Balancing and Instance / Reserved IP Management
  • Automation Service: PowerShell Service Orchestration
  • Microsoft Antimalware Extension on VMs and Cloud Services (for free)
  • Many more VM Extensions available (PowerShell DSC / Octopus Deploy Tentacle)
  • Event Hubs: ingest more messages; SLA-backed.

Other News

We always have this vision of large-scale services being relatively immune to wide-ranging outages, yet all the main cloud platforms have regular challenges resulting in service disruptions of some variety.

On November 18 (or 19 depending on your timezone) Azure had one of these events, causing a disruption across many of its Regions affecting Storage and VMs.

The final Root Cause Analysis (RCA) shows the sorts of challenges involved in running platforms of this size.


December

You can almost hear the drawing of the breath before the Azure team starts 2015…

Preview

  • Premium Storage
  • Azure SQL Database: better feature parity with SQL 2014 and better large DB support.
  • Search: management via portal, multi-lingual support.
  • DocumentDB: better management via portal.
  • Azure Data Factory: integration with Machine Learning.

Generally Available

  • RemoteApp: run desktop apps anywhere
  • Azure SQL Database: new auditing features
  • Live Media Streaming: access the same platform as used at the World Cup and Olympics
  • Site Recovery: supported without SCVMM being deployed
  • Active Directory: App Proxy and password write-back enabled
  • Mobile Services: Offline Sync Managed SDK
  • HDInsight: Cluster customisation.

Other News

Another big announcement for the Australian cloud market was the news that from early 2015 Microsoft would be offering Office 365 and CRM Online from within Australia’s borders. What a great time to be working in this market!


There we have it! What a year! I haven’t detailed every single announcement to come out from the Azure team (this post would easily be twice as long), but if you think I’ve missed anything important leave a comment and I’ll update the post.

Simon.

Tagged , , , ,