Tag Archives: Azure

Speaking: Azure Functions at MUG Strasbourg – 28 September

I’m really excited about this opportunity to share the power of Azure with the developer and IT Pro community in France that is soon to gain local Azure Regions in which to build their solutions.

If you live in the surrounding areas I’d love to see you there. More details available via Meetup.

Tagged , ,

Secure your VSTS Release Management Azure VM deployments with NSGs and PowerShell

One of the neat features of VSTS’ Release Management capability is the ability to deploy to Virtual Machine hosted in Azure (amongst other environments) which I previously walked through setting up.

One thing that you need to configure when you use this deployment approach is an open TCP port to the Virtual Machines to allow remote access to PowerShell and WinRM on the target machines from VSTS.

In Azure this means we need to define a Network Security Group (NSG) inbound rule to allow the traffic (sample shown below). As we are unable to limit the source address (i.e. where VSTS Release Management will call from) we are stuck creating a rule with a Source of “Any” which is less than ideal, even with the connection being TLS-secured. This would probably give security teams a few palpitations when they look at it too!

Network Security Group

We might be able to determine a source address based on monitoring traffic, but there is no guarantee that the Release Management host won’t change at some point which would mean our rule blocks that traffic and our deployment breaks.

So how do we fix this in an automated way with VSTS Release Management and provide a secured environment?

Let’s take a look.

The Fix

The fix is actually quite straightforward it turns out.

As the first step you should go to the existing NSG and flip the inbound rule from “Allow” to “Deny”. This will stop the great unwashed masses from being able to hit TCP port 5986 on your Virtual Machines immediately.

As a side note… if you think nobody is looking for your VMs and open ports, try putting a VM up in Azure and leaving RDP (3389) open to “Any” and see how long it takes before you start seeing authentication failures in your Security event log due to account enumeration attempts.

Modify Project Being Deployed

We’re going to leverage an existing Release Management capability to solve this issue, but first we need to provide a custom PowerShell script that we can use to manipulate the NSG that contains the rule we are currently using to block inbound traffic.

This PowerShell script is just a simple wrapper that combines Azure PowerShell Cmdlets to allow us to a) read the NSG b) update the rule we need c) update the NSG, which commits the change back to Azure.

I usually include this script in a Folder called “Deploy” in my project and set the build action to “Copy always”. As a result the file will be copied to the Artefacts folder at build time which means we have access to it in Release Management.

Project Setup

You should run a build with this included file so that it is available in your

Modify Release Management Defintion

Note that in order to complete this step you must have a connection between VSTS and your target Azure Subscription already configured as a Service Endpoint. Typically this needs to be done by a user with sufficient rights in both VSTS and the Azure Subscription.

Now we are going to modify our existing Release Management definition to make use of this new script.

The way we are going to enable this is by using the existing Azure PowerShell Task that we have available in both Build and Release Management environments in VSTS.

I’ve shown a sample where I’ve added this Task to an existing Release Management definition.

Release Management Definition

There is a reason this Task is added twice – once to change the NSG rule to be “Allow” and then once, at the end, to switch it back to “Deny”. Ideally we want to do the “Allow” early in the process flow to allow time for the NSG to be updated prior to our RM deployment attempting to access the machine(s) remotely.

The Open NSG Task is configured as shown.

Allow Script

The Script Arguments should match those given in the sample script above. As sample we might have:

-resourceGroupName MyTestResourceGroup -networkSecurityGroupName vnet01-nsg 
-securityRuleName custom-vsts-deployments -allowOrDeny Allow -priority 3010

The beauty of our script is that the Close NSG Task is effectively the same, but instead of “Allow” we put “Deny” which will switch the rule to blocking traffic!

Make sure you set the “Close” Task to “Always run”. This way if any other component in the Definition fails we will at least close up the NSG again.

Additionally, if you have a Resource Group Lock in place (and you should for all production workloads) this approach will still work because we are only modifying an existing rule, rather than trying to add / remove it each time.

That’s it!

You can now benefit from VSTS remote deployments while at the same time keeping your environment locked down.

Happy days 🙂

Tagged , , , , ,

Azure Security Fundamentals: Azure SQL Database

I’m continuing my Kloud blog series on the security fundamentals for various Microsoft Azure services with my most recent post being for Azure SQL Database.

Please go and have a read!

Tagged , ,

Setting Instance Level Public IPs on Azure VMs

Since October 2014 it has been possible to add a public IP address to a virtual machine in Azure so that it can be directly connected to by clients on the internet. This bypasses the load balancing in Azure and is primarily designed for those scenarios where you need to test a host without the load balancer, or you are deploying a technology that may require a connection type that isn’t suited to Azure’s Load Balancing technology.

This is all great, but the current implementation provides you with dynamic IP addresses only, which is not great unless you can wrap a DNS CNAME over the top of them. Reading the ILPIP documentation suggested that a custom FQDN was generated for an ILPIP, but for the life of me I couldn’t get it to work!

I went around in circles a bit based on the documentation Microsoft supplies as it looked like all I needed to do was to call the Set-AzurePublicIP Cmdlet and the Azure fabric would take care of the rest… but no such luck!

Get-AzureVM -ServiceName svc01 -Name vm01 | `
Set-AzurePublicIP -PublicIPName vm01ip -IdleTimeoutInMinutes 4 | `
Update-AzureVM

When I did a Get-AzureVM after the above I got the following output – note that I did get a public IP, but no hostname to go along with it!

DeploymentName              : svc01
Name                        : vm01
Label                       :
VM                          : Microsoft.WindowsAzure.Commands.ServiceManagement.Model.PersistentVM
InstanceStatus              : ReadyRole
IpAddress                   : 10.0.0.5
InstanceStateDetails        :
PowerState                  : Started
InstanceErrorCode           :
InstanceFaultDomain         : 1
InstanceName                : vm01
InstanceUpgradeDomain       : 1
InstanceSize                : Small
HostName                    : vm01
AvailabilitySetName         : asn01
DNSName                     : http://svc01.cloudapp.net/
Status                      : ReadyRole
GuestAgentStatus            : Microsoft.WindowsAzure.Commands.ServiceManagement.Model.GuestAgentStatus
ResourceExtensionStatusList : {Microsoft.Compute.BGInfo}
PublicIPAddress             : 191.239.XX.XX
PublicIPName                : vm01ip
PublicIPDomainNameLabel     :
PublicIPFqdns               : {}
NetworkInterfaces           : {}
VirtualNetworkName          : Group demo01
ServiceName                 : svc01
OperationDescription        : Get-AzureVM
OperationId                 : 62fdb5b28dccb3xx7ede3yyy18c0454
OperationStatus             : OK

Aaarggh!

The Solution

It turns out, after a little experimentation, that you all you have to do to get this to work is to supply a value to an undocumented parameter DomainNameLabel for the Set-AzurePublicIP Cmdlet.

Note: there is also no way to achieve this at time of writing via the Azure web portals – you have to use PowerShell to get this configured.

Let’s try our call again above with the right arguments this time!

Get-AzureVM -ServiceName svc01 -Name vm01 | `
Set-AzurePublicIP -PublicIPName vm01ip `
   -IdleTimeoutInMinutes 4 -DomainNameLabel vm01ilpip | `
Update-AzureVM

Success!!

DeploymentName              : svc01
Name                        : vm01
Label                       :
VM                          : Microsoft.WindowsAzure.Commands.ServiceManagement.Model.PersistentVM
InstanceStatus              : ReadyRole
IpAddress                   : 10.0.0.5
InstanceStateDetails        :
PowerState                  : Started
InstanceErrorCode           :
InstanceFaultDomain         : 1
InstanceName                : vm01
InstanceUpgradeDomain       : 1
InstanceSize                : Small
HostName                    : vm01
AvailabilitySetName         : asn01
DNSName                     : http://svc01.cloudapp.net/
Status                      : ReadyRole
GuestAgentStatus            : Microsoft.WindowsAzure.Commands.ServiceManagement.Model.GuestAgentStatus
ResourceExtensionStatusList : {Microsoft.Compute.BGInfo}
PublicIPAddress             : 191.239.XX.XX
PublicIPName                : vm01ip
PublicIPDomainNameLabel     : vm01ilpip
PublicIPFqdns               : {vm01ilpip.svc01.cloudapp.net , vm01ilpip.0.svc01.cloudapp.net}
NetworkInterfaces           : {}
VirtualNetworkName          : Group demo01
ServiceName                 : svc01
OperationDescription        : Get-AzureVM
OperationId                 : 62fdb5b28dccb3xx7ede3yyy18c0454
OperationStatus             : OK

Now that I have this information I can setup DNS CNAMEs against the PublicIPFqdns and use DNS to manage the invariable IP address change between instance recycles. Happy days!

Tagged , , , , , ,

Azure’s G Series VMs – Prime Compute Only One Click Away!

Kloud Blog

I’m going to start this blog post by making one thing clear. My intent in writing this post is light-hearted – I had some spare time on my hands over a lunch break and I wondered what I could do with it. The result was this blog post :).

Ever since Microsoft announced their G Series Virtual Machines for Azure I’ve been looking for a good reason to fire one up and kick the tyres. Today while I was skimming through my Twitter feed I came across a tweet showing the time it took to calculate the trillionth prime number on a 16 vCPU Linux instance running on GCP.

As any good propeller head knows, the first rule of having access to massive raw compute is to put it to use solving mathematical challenges. This may take the form of a pure maths challenge like finding the n-th digit of…

View original post 602 more words

Tagged ,

Microsoft Azure: 2014 Year in Review

What a massive year it’s been for Microsoft’s Azure public cloud platform. Running the Azure Sydney User Group this year has been great fun and seeing the growing local interest has been fantastic.

The focus from Microsoft has really changed in this space and has been clearly signalled with the change in name of Azure from Windows Azure to Microsoft Azure during the year and an increasingly broad set of non-Microsoft services offered on it.

2015 promises to be another big year, but let’s look back at what happened during 2014 with Azure.


January

The year got off to a fairly quiet start, but as we’ll see, it soon ramped up.

Preview

Everything this month was under GA only, so see below!

Generally Available

  • Websites:
    • staged publishing support
    • Always On support *
    • more frequent metric updates and monitoring alerts
  • SQL Database: new metrics and alerts
  • Mobile Services: SenchaTouch support
  • Cloud Services: A8 and A9 machine sizes now supported.

* If you’re using New Relic there are some known issues with this feature.

Other News

The Azure platform received PCI-DSS compliance validation and introduced reduced pricing rates for storage and storage transactions.


February and March

The headline item in this period was the launch of the Japan Geography with Japan East (Saitama Prefecture) and West (Osaka Prefecture) providing that market with in-country services. Also during this period we had the following announcements and launches:

Preview

Generally Available

Other News

Local gamers unhappy not to have a local Xbox server platform to run on. Who knew it was such an issue having lag and big ping times 😉

Can we haz l0c4l serverz?


April

The big change this month was the change in name for Azure. Guaranteeing a million-and-one outdated websites, slides and documents in one swoop, the service name was changed from Windows Azure to Microsoft Azure. Just for fun there is no “official” logo, just text-based branding.

This change was a subtle nod to Azure’s ability to run Infrastructure-as-a-Service (IaaS) workloads on platforms other than Windows – something it had been doing for quite some time when this change was made.

Preview

  • Newly designed management portal
  • Mobile services: documented offline support and role-based Azure AD authentication
  • Resource Manager via PowerShell
  • SQL Database: active geo-replication (read replicas); self-service restore; 500GB support; 99.95% SLA
  • Media Services: secure delivery and Office 365 Video Portal.

Generally Available

  • Azure SDK 2.3: increased Visual Studio support – create VMs using Server Explorer
  • Autoscale – Virtual Machines, Cloud Services, Web Sites and Mobile Services
  • Azure AD Premium – Multi-factor Authentication (MFA) and security reporting
  • Websites: SSL bundled; Java support; Web Hosting Plans; Available in SE Asia
  • Web Jobs SDK
  • Media Services: Live Streaming; Partnerships for Content Management and Analytics (Ooyala) and Live Ingest (iStreamPlanet)
  • Basic Tier introduction: lower cost for dev/test scenarios. Applies to VMs and Websites
  • Puppet and Chef support on Azure VMs via VM Agent Extensions
  • Scheduler Service
  • Read Access Geo Redundant Storage (RA-GRS).

May and June

The pace from the first quarter of the year carried over into these two months! The stand out amongst the range of announcements in this period was the launch of the API Management service which was the result of the October 2013 acquisition of Apiphany.

Preview

  • Azure API Management – publish, manage and secure your existing REST APIs
  • Azure File Service (SMB shares) – even use on Linux VMs
  • BizTalk Hybrid Connections – on-prem connects without the secops guys 😉
  • Redis Cache support – now the preferred caching platform in Azure
  • RemoteApp – Lay down common Apps on demand
  • Site Recovery – backup your on-prem VMs to Azure
  • Secure VMs using security extensions from Microsoft, Symantec and McAfee
  • Internal Load Balancing for VMs and Cloud Services
  • HDInsights: Apache HBASE and Hadoop 3.1
  • Azure Machine Learning (or as I like to call it “Skynet”).

Generally Available

  • ExpressRoute – WAN and DC cross-connects
  • Multi-connection Virtual Networks (VNET) and VNET-to-VNET connections
  • Public IP Address Reservation (IPv4 shortage anyone?)
  • Traffic Manager: use Azure and non-Azure (“external”) endpoints
  • A8 and A9 VM support – lots of everything (8 / 16 cores – 7 GB RAM per core)
  • Storage Import/Export service – check region availability!

Other News

MSDN subscribers gained the ability to deploy Windows 7 and 8 images onto Azure VMs for dev/test scenarios and Enterprise Agreement (EA) customers were given the ability to purchase add-ons via the Azure Store which had previously not been possible.

We also learned about availability of IPv4 addresses with some US-based services being issued IPv4 addresses assigned to South America, causing many LOLs for service admins out there who thought their services were in Brazil!


July and August

This period’s summary: Ice Bucket Challenge.

Preview

  • Event Hubs: capture data from all the Internet connected things!
  • Redis cache: in more places and sizes
  • Preview management portal: manage Azure SQL Database
  • DocumentDB
  • Azure Search.

Generally Available


September

No single announcement jumps out so I was going to put a picture of a kitten here but I thought you might want to see this (even if it is from 2012).

Preview

  • Role-based access control (RBAC) for Azure management in preview portal only
  • Resource Tagging support: filter by tag – useful for billing and ops
  • Azure SQL Database – Elastic Scale preview. Replaces Federations model
  • DocumentDB – enhanced management tooling and metrics
  • Azure Automation – AD auth; PowerShell converter; Runbook gallery and scheduling
  • Media Services – Live Streaming and DRM, faster encoding and indexer.

Generally Available

  • ‘D’ Series VMs: 60% faster CPU, more RAM and local SSD disk
  • Redis Cache: recommended cache solution in Azure. 250MB – 53GB! support
  • Site Recovery: on-prem DR with Azure – Win / Linux
  • Notification Hubs: Baidu Push (China)
  • Virtual Machines: instance-level public IPs (no NAT/PAT)
  • Azure SQL Database: three new service tiers and hourly billing
  • API Management: added OAuth support and REST Management API
  • Websites: VNet support, “scalable CMS” with WordPress and backups improvements
  • Management Services Alerts.

October and November

Pretty hard to go by this news it terms of ‘most outstanding announcement’ for these two months, especially for those of us in Australia!

Preview

  • ‘G’ Series VMs – (“Godzilla” VM) more CPU/RAM/SSD than any VM in any cloud *
  • Premium Storage – SSD-based with more than 50k IOPS *
  • Marketplace changes – CoreOS and Cloudera
  • Increased focus on Docker including portal support
  • Cloud Platform System (CPS) from Dell.
  • Batch: parallel task coordination
  • Data Factory: build data processing pipelines
  • Stream Analytics: analyse your Event Hubs data.

* Announced but not yet in public preview.

Generally Available

  • Australia Geography launches!
  • Network Security Groups
  • Multi-NIC Support in VMs (VM size dependent)
  • Forced Tunnelling (route traffic back on-prem)
  • ExpressRoute:
    • Cross-Subscription Sharing
    • Multi-connect to an Azure VNET
  • Bigger Azure Virtual Gateways
  • Ops Logging for Gateways and ExpressRoute
  • More control over Gateway encryption
  • Azure Load Balancer Source IP Affinity (“Sticky Sessions”)
  • Nested Traffic Manager Profiles
  • Preview Portal: Internal Load Balancing and Instance / Reserved IP Management
  • Automation Service: PowerShell Service Orchestration
  • Microsoft Antimalware Extension on VMs and Cloud Services (for free)
  • Many more VM Extensions available (PowerShell DSC / Octopus Deploy Tentacle)
  • Event Hubs: ingest more messages; SLA-backed.

Other News

We always have this vision of large-scale services being relatively immune to wide-ranging outages, yet all the main cloud platforms have regular challenges resulting in service disruptions of some variety.

On November 18 (or 19 depending on your timezone) Azure had one of these events, causing a disruption across many of its Regions affecting Storage and VMs.

The final Root Cause Analysis (RCA) shows the sorts of challenges involved in running platforms of this size.


December

You can almost hear the drawing of the breath before the Azure team starts 2015…

Preview

  • Premium Storage
  • Azure SQL Database: better feature parity with SQL 2014 and better large DB support.
  • Search: management via portal, multi-lingual support.
  • DocumentDB: better management via portal.
  • Azure Data Factory: integration with Machine Learning.

Generally Available

  • RemoteApp: run desktop apps anywhere
  • Azure SQL Database: new auditing features
  • Live Media Streaming: access the same platform as used at the World Cup and Olympics
  • Site Recovery: supported without SCVMM being deployed
  • Active Directory: App Proxy and password write-back enabled
  • Mobile Services: Offline Sync Managed SDK
  • HDInsight: Cluster customisation.

Other News

Another big announcement for the Australian cloud market was the news that from early 2015 Microsoft would be offering Office 365 and CRM Online from within Australia’s borders. What a great time to be working in this market!


There we have it! What a year! I haven’t detailed every single announcement to come out from the Azure team (this post would easily be twice as long), but if you think I’ve missed anything important leave a comment and I’ll update the post.

Simon.

Tagged , , , ,

Use Azure Management API SDK in an Entity Framework custom database initializer

A post over on Stack Overflow got me thinking about how you can override the default behaviour of the Entity Framework code first database initializer so that the tier of the database created is something other than the deprecated ‘Web’ tier. Here’s one way to go about it.

Required bits

There are a few things to get going here – you’ll need to add the the Microsoft Azure SQL Database Management Library nuget package to your solution which will install a bunch of dependencies required to interact with the Azure Management API.

You should also familiarise yourself with how to create and use Management Certificates which will be required for all interactions with the Azure Management API.

Once you’ve looked through that I suggest having a good read of Brady Gaster’s blogs on using the Management API in which he gives some good overviews on working with Azure SQL Database AND on how you can go about uploading your Management Certificate to an Azure Website.

For the purpose of the remainder of this post we’ll be using the sample MVC / EF code first sample application which can be downloaded from MSDN’s code site.

Now you’ve done that, let’s get started…

Create a custom EF initializer

Entity Framework provides a nice extensibility point for managing initialisation of databases amongst other items (primarily to allow you to use the latest hipster database of choice and roll your own supporting code) and we’re going to use a simple sample to show how we could change the behaviour we’re seeing above.

In the below sample we create Standard tier databases – we could just as easily change this to a configuration element and modify which database we wish to create. Note that I load a lot of information from configuration – in the below sample I can deploy those configuration elements at the Cloud Service level and manage via the Azure Management Portal. I could just as easily leave them in the web.config if I wanted to.

A sample of what appears in the configuration (this is from a web.config)

  <appSettings>
    <add key="AzureSqlDatabaseServerName" value="t95xxttjmj"/>
    <add key="AzureSqlDatabaseName" value="SchoolDemo"/>
    <add key="AzureSubscriptionId" value="00000000-0000-0000-0000-000000000000"/>
    <add key="AzureSubscriptionCertThumbprint" value="61b463082dcb0198aab451c14efb7ff4b83a42b4"/>
  </appSettings>

In our global.asax of our web application we then need to include the following code:


Database.SetInitializer(new ContosoCustomDatabaseInitializer());

At this point when EF attempts to fire up a new database instance it will call our custom code and initialise a new database on the specified server using the management libraries.

Hopefully in a future release we’ll see an update to the default database setting to use the new Standard tier instead.

Tagged , , , ,

TechEd 2014: Azure API Management Talk now available.

For those of you who didn’t get a chance to come along to my presentation on Azure API Management at TechEd in Melbourne this year the session is now up on Channel 9. Slides to go along with the session are also available.

Enjoy!

http://channel9.msdn.com/Events/TechEd/Australia/2014/DPP406

Tagged ,

See the other side of my TechEd API Management Competition

Now that TechEd Melbourne is done and dusted I thought I’d publish the results of my competition and share some of the cool configuration and analytics that Azure API Management provides to API Publishers.

First off, congratulations to Kieran and Lachlan for being the two agents of change who were prepared to get in and get dirty!

As you read through the below you can click the images below to see them at a larger resolution.

Who used the API Proxy?

As API Management requires callers to pre-register and identify themselves using keys it is possible for the proxy to identify who is connecting and making the most calls and what they are doing.  I put a limitation on the API calls that can be made (see later in this post) so I can see that the top developer was blocked after he hit the 50 calls per day limit.

Top Developers

What did they call?

We also get some metrics on which API operations are being invoked the most by our trusty band of developers (shown above). The calls to VIEWCATALOG were all down to my demonstration during my session and the two calls to UPDATEPRODUCTDESCRIPTION were Lachlan trying his hand (well done!)

Top Operations

At-a-glance Analytics

The Azure Management Portal gives me some coarsely-grained statistics so I can see what’s going on at a glance as shown below.

Azure Portal Traffic View

If I want more detailed analytics though I need to open the API Management Publisher Portal and show the Analytics for the Proxy (the below graph shows all traffic across all APIs including the API from my session in addition to the competition API).  This is really one of the valuable pieces of API Management and one reason businesses could adopt this so they can start to understand what parts of their existing APIs are seeing the most demand.

Graphical Analytics

Controlling Usage

As this was a demo API I wanted to make sure that I didn’t end up with a hefty bill by someone accidentally pounding away on the API so I put a few things in place to protect the proxy and the service it was proxying.

Caching Responses

Firstly I dropped in a day-long cache of the responses from the backend API.

Cache Setup

Rate Limiting and Quota for Calls

I cloned the existing ‘Starter’ Product that comes as a sample with API Management and put it in place for my API.  The product itself is just a container and I need to then apply a Policy to it which I did using the following setup.

<policies>
	<inbound>
		<base />
        <!-- 10 calls per hour -->
		<rate-limit calls="10" renewal-period="60" />
		<!-- 50 calls per day -->
		<quota calls="50" renewal-period="86400" />
	</inbound>
	<outbound>
		<base />
	</outbound>
</policies>

This was to make the competition a little more interesting (as Lachlan found out :)).

So there we have it, a bit of an insight into what the metrics are that were being generated by the API Management setup that you may have all seen during my demo at TechEd.

If you have any questions please get in touch.

Tagged , ,

I’m speaking about Azure API Management at TechEd Melbourne 2014

Given the changes to TechEd this year and the smaller set of sessions available I’m exceptionally happy to be coming back for another year to talk more about Azure.

I’ll be speaking in Melbourne on 7 October on the topic “Microsoft Azure API Management: Win Friends and Make Money”. You can find out more about my session here: http://techedmelbourne.azurewebsites.net/SessionDetail.aspx?id=19015

Hope to see you there!

I'm speaking at TechEd Australia 2014

Tagged , ,