“The best journeys answer questions that in the beginning you didn’t even think to ask.” – Jeff Johnson – 180° South
I thought with the announcement at Build 2017 of the preview of the Functions tooling for Visual Studio 2017 that I would take a look back on the journey I’ve been on with Functions for the past 12 months. This post is a chance to cover what’s changed and capture some of the lessons I learned along the way, and that hopefully you can take something away from.
In the beginning
I joined the early stages of a project team at a customer and was provided with access to their Microsoft Azure cloud environment as a way to stand up rapid prototypes and deliver ongoing solutions.
I’ve been keen for a while to do away with as much traditional infrastructure as possible in my solutions, primarily as a way to make ongoing management a much easier proposition (plus it aligns with my developer sensibilities). Starting with this environment I was determined to apply this where I could.
In this respect the (at the time) newly announced Function Apps fit the bill!
The First Release
I worked on a couple of early prototype systems for the customer that leveraged Azure Functions in their pre-GA state, edited directly in the Browser. I’m a big C# fan, so all our Functions have been C# flavoured.
After hacking away in the browser I certainly found out how much I missed Visual Studio’s Intellisense, though my C# improved after a few years away from deep use of the language!
We ran some field tests with the system, and out of this took some lessons away.
I blogged about Functions with KeyVault and also on how to deliver emails using Functions with SendGrid.
At this point I also learnt a key lesson: Functions are awesome, but may not be best suited to time-critical operations (where the trigger speed is measured in milliseconds or low seconds). This was particularly the case with uncompiled Function Apps (which at the time was the only option).
I wrote a Function to push a sythentic transaction into our pipeline periodically to offset some of this behaviour. Even after multiple revisions we’re still doing this certain activities such as retrieving and caching Graph API access tokens.
Another pain point for us at this juncture was the lack of continuous deployment options. Our workaround was a basic copy / paste into a VSTS Web project held in Git repository in VSTS.
Around the end of our initial field trials Functions hit GA which was perfect for us!
The Second Release
I now had a production-ready system running Functions at its core. At this stage we had all our Functions in a single App because the load wasn’t sufficient to break them out.
Our service was yet to launch when the Functions team announced a CI / CD experience using the ‘Deployment Options’ setting in the Azure Portal.
We revved to a second release, using the new CI / CD deployment option to enforce deployments to a production App Service Plan. We were still living with on-demand compilation which we found impactful at deployment time as everything had to be restored and recompiled.
The Third Release (funproj all round!)
Visual Studio 2015 tooling was announced!
Yep, we used it – welcome to funproj Revision (3) of the codebase!
About now, one of my earlier blogs on sending email came back to haunt me as I began to (and still do periodically) receive emails from people clearly trying out my sample code!
Ah… the dangers of writing demonstrations that have your real email address in the To: field.
One item we’ve done with each major revision of the Functions code is to publish a new App Service Plan and use this is as our deployment target. We paramaterise as much as we can so we can pre-populate things like Service Bus connection strings. Our newly deployed Functions are disabled and then manually cut over one at a time until we’re satisfied that the new deployment is in a happy state. This n-1 Function App will live for a couple of weeks after initial deployment with its Functions disabled. This was all decided before deployment slot support which may change this design 🙂
We have a Function we can run that checks that all your environment variables are setup in a new App Plan – hopefully in future we’ll get this automated away as well!
Additionally, if we have a major piece of code changing, but not sufficient for a major revision, we’ll do an Azure-level backup of the App before we run the deployment so we have a solid recovery point to return to if need be.
Function runtime versioning
We had one particularly bad day where our Functions started playing up out-of-the-blue. It turned out that an unintended breaking change had been made by the Functions team when they did a new release.
The key learning from this experience for us was that our production Function Apps always run a known-good version of the runtime which is set as a specific version by setting the FUNCTIONS_EXTENSION_VERSION to something like ‘1.0.10576’ rather than ‘~1’ which means ‘latest non-breaking 1 release’.
We run our development App Services using ‘~1’ however, so we can determine periodically to update our production runtime to the latest known good runtime version so we don’t lag too far behind the good work the Functions team is doing.
Our other main issue during this revision was that our App Plan stopped being able to read the x509 cert store for some reason and our KeyVault client started failing. I never got to the bottom of it, but it was fixed through deploying to a new Plan.
Talking Functions
I was lucky enough to get to talk a bit about the solution we’d been building in early February.
The video quality varies, but here it is if you’re interested.
Hello Compiled Functions… oooooh Application Insights! (Rev 4!)
Compiled Functions? Sure, why not?!
The benefit here is the reduced startup times produced by the pre-compiled nature of the solution. The only downside we’ve found is deployments can cause transient thread-abort exceptions in some libraries, but these are not substantial and we deal with them gracefully in our solution.
As an example, we had previously seen Function cold start times some times up to 15 seconds. We now rarely see any impacts, some of which is the result of lessons we’ve learned along the way, coupled with the compiled codebase and the hard work the Functions team is doing to continuously improve their platform.
Early on in the life of our solution we had looked at Application Insights as a way to track our Function App performance and trace logging. We abandoned it because we ended up sometimes writing more App Insights code than app code! The newly introduced App Insights support, however, does away with all of this and it works a treat:
Unfortunately as part of the move here we also dropped Visual Studio 2015 support which meant we dropped the ‘funproj’ project layout. As Compiled Functions now give us local design time benefits we didn’t get before, it’s a good trade-off for my mind.
So… revision 5 anyone?!
We’re not here yet, though once the Visual Studio 2017 tooling hits GA we’ll probably look seriously at moving to it.
Ideally we’ll also move to a more fully testable solution and one that supports VSTS Release Management which we use for many other aspects of our overall solution.
Gee… sounds like a lot of work!
This might sound like a lot of busy work, but to be honest, the microservices we’re running using Functions are so small as to be easy to port from one revision to the next. We make conscious decisions around the right time to move, when we feel that the benefits to be realised are worth the small hit to our productivity.
Ultimately our goal is a shown below (minus the flames of course!)
That’s a wrap!
The Functions team is open in their work – ask them questions on Stack Overflow http://stackoverflow.com/questions/tagged/azure-functions
Raise your issues and submit your PRs here: https://github.com/Azure/Azure-Functions
Happy Days 🙂
