Deploy Python-based Azure Functions using Azure DevOps

Important note: this blog post covers in-preview services that are subject to change before launch. As such I’ll update this post as / when required to reflect official guidance from the Functions team on the best approach to deployments. Additionally, for the purpose of this post I am not deploying App Settings as there are existing methods that continue to work.

Know your versions

As this is under-development software it is important to provide context on versions that were used to produce this blog post. The key version being that of the Azure Functions Core Tools. The version installed and working here is 2.3.199-1. Python support currently sits at Python 3.6 with the underlying Functions tools needing .Net Core 2.1.502.

Foundation

Before working through this post it’s important that you are familiar with the Azure Functions (version 2) Python support. At the time of writing the best place to go for this is the Microsoft Docs site. Once you undestand the development and lifecycle of them then you’re ready for the rest of this post.

You will need to make sure you have a target Linux Function App in Azure ready for deployment as per the “Create Linux function app in Azure” section of the documentation.

When you have a working Python Function check it into an Azure Repos Git repository.

Define the Azure Pipelines Build

As a starting point ensure you have a Service Principal-based connection into the Azure Subscription you are going to deploy your Python Function to. This avoids anyone having to know usernames and passwords which is 👍.

The easiest way to setup your Build is to import the following YAML build definition and make the changes to the three locations (**yourfoldername**, **yourfunctionname**, **your azure subscription connection**). If you can’t see the YAML build definition embedded below then you can view it on GitHub.

If you are setting up the build manually, make sure to select a Linux-based build agent otherwise the build will fail.

My approach assumes that your Python Functions are stored as sub-folders inside Git. This allows you to store multiple unrelated Functions in a single repository which works in my scenerio as I am manually triggering builds because I don’t need to deploy all Functions at once.

You might be wondering why all this inline code is inside an Azure CLI task – the answer is that the actual deployment to Azure requires an active Azure CLI session.

I could probably break some of the earlier steps out into one or more Bash script task steps if I wanted, but for ease of maintenance I just made them a single script. I could also make the inline script an artefact in the repository and maintain it that way.

A lot of the work involved in the inline script (a copy of which is at the bottom of this post) is the result of a few items:

  • The requirement for the Azure Functions Core Tools to be installed
  • .Net Core is required by the Azure Functions Core Tools
  • Python 3.6 is required for Azure Functions Python support. The age of the Ubuntu distribution running the Azure DevOps hosted build agent means it doesn’t have Python 3.6 installed and Ubuntu don’t provide it (only Python 3.5). When these hosts are updated by Microsoft to a more modern Ubuntu release then this script will become much simpler
  • The need to re-initialise the Functions environment after check-out from Git with ‘init’ which overwrites our existing requirements.txt file
  • The need to manually restore any Extensions used in the Function.

My approach is functional, but there are clearly some sharp edges to it 🙂

There is also an implicit requirement (no pun intended!) on developers to maintain an up-to-date requirements.txt file, particularly as the Functions core Python and Functions Python runtime are both listed.

If you can’t (or don’t want to) use the XAML build definition above the visual definition setup is very simple – as shown below.

Build Definition

… and the inline script for you to use …

Depending on the complexity of your Python Functions and the location of your build Agent versus your Function App you should see a deployment complete within a few minutes (I’ve averaged around 7 minutes for a Python Function App with three Functions and a deployment package of 48MB).

Happy Days 😎

17 thoughts on “Deploy Python-based Azure Functions using Azure DevOps

  1. Hello Simon! I’m new to Azure pipelines, what is the input azureSubscription and how I get it? when is it used?

    1. Hi Pablo – the ‘azureSubscription’ value in the YAML definition is the display name of the Azure Subscription or Service Connection you will be deploying with.

      In my case I have an existing Service Principal connection called “MS FTE MSDN” and that value is what is contained in the YAML.

      The easiest way to find this value is to look at Project Settings > Pipelines > Service Connections in the Azure DevOps portal.

      When you define a designer-based build you will find this under the ‘Azure Subscription’ property on many Build Steps.

      Build step definition

      This information is required as it is the service connection into Azure for your Pipeline. If you didn’t have this you would be unable to deploy to Azure.

      Let me know if this doesn’t make sense!

      1. Thanks Simon! after i little lookup i found the option for creating a service principal under Project Settings and from there on it was intuitive.

        Thanks a lot!

    1. Hi Mohanish – sometimes on certain devices embedded code snippets from GitHub don’t display so it could be that’s your situation. Regardless, here are the embedded links for you. Hope this helps.

      – The YAML file: https://gist.githubusercontent.com/sjwaight/b60047b27e6c93bc285f0b5166850229/raw/bf4f64d33f5d4a18a484ade2b51ab52c72d7c150/build-deploy-python-func.yaml
      – The Shell script: https://gist.githubusercontent.com/sjwaight/e0047a235790b2d7489428ea5c750eb7/raw/ddda1a105dae019c8ccedddcc4618bb68e19d3b2/build-deploy-python-func.sh

  2. Hi Simon,
    This is a very informative post and very few in DevOps category.
    I tried running the above inline code but its giving errors as below seems it doent have any command line interpreters

    ‘wget’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4683068Z operable program or batch file.
    2019-07-29T10:28:20.4683678Z
    2019-07-29T10:28:20.4683998Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo dpkg -i packages-microsoft-prod.deb
    2019-07-29T10:28:20.4703047Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4703304Z operable program or batch file.
    2019-07-29T10:28:20.4704061Z
    2019-07-29T10:28:20.4704324Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo add-apt-repository ppa:deadsnakes/ppa
    2019-07-29T10:28:20.4719494Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4719849Z operable program or batch file.
    2019-07-29T10:28:20.4721710Z
    2019-07-29T10:28:20.4721953Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo apt-get update
    2019-07-29T10:28:20.4742046Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4742242Z operable program or batch file.
    2019-07-29T10:28:20.4744960Z
    2019-07-29T10:28:20.4745788Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo apt-get install azure-functions-core-tools
    2019-07-29T10:28:20.4764992Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4765157Z operable program or batch file.
    2019-07-29T10:28:20.4767746Z
    2019-07-29T10:28:20.4767947Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo apt-get install python3.6
    2019-07-29T10:28:20.4782745Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4782900Z operable program or batch file.
    2019-07-29T10:28:20.4784523Z
    2019-07-29T10:28:20.4784811Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>sudo apt-get install python3.6-venv
    2019-07-29T10:28:20.4799451Z ‘sudo’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4799603Z operable program or batch file.
    2019-07-29T10:28:20.4801486Z
    2019-07-29T10:28:20.4801732Z F:\Build\BP-MAPS-ZNE-USW0-N-IDM-Agent01\_work\r23\a>python3.6 -m venv .env
    2019-07-29T10:28:20.4827569Z ‘python3.6’ is not recognized as an internal or external command,
    2019-07-29T10:28:20.4827760Z operable program or batch file.
    2019-07-29T10:28:20.4830197Z

    1. Hi Qudir – while I don’t explicitly mention it you must choose to use a Linux-based build agent. Your build is failing because it is running on Windows. If you are using Microsoft managed build pipelines then you should select the “Hosted Ubuntu 1604” agent for your build. I’ll update the post to indicate that – thanks for helping improve my writing!

      1. Also i want to know for the below 2 variables:
        FunctionFolder and FunctionAppName
        It is just the name i have to give or the full tfs path with name. Say function FunctionFolder/function app name is FunFolder1.
        So weather i have to give
        FunctionFolder =’$ProjectPath/FunFolder1′
        or
        FunctionFolder = ‘FunFolder1’
        FunctionAppName = name of the target Azure Function that will receive the deployment.

      2. Hi Simon,
        Now i have sorted out folder issue. Now i am geting some requirements.txt missing issue even though i have those files within my tfs path

        2019-07-29T16:10:40.3417297Z (Reading database … 100%
        2019-07-29T16:10:40.3417360Z (Reading database … 214368 files and directories currently installed.)
        2019-07-29T16:10:40.3536261Z Preparing to unpack …/python3.6-venv_3.6.9-1+xenial1_amd64.deb …
        2019-07-29T16:10:40.3689824Z Unpacking python3.6-venv (3.6.9-1+xenial1) …
        2019-07-29T16:10:40.5744247Z Processing triggers for man-db (2.7.5-1) …
        2019-07-29T16:10:41.0953951Z Setting up python3.6-venv (3.6.9-1+xenial1) …
        2019-07-29T16:10:44.8086187Z mv: cannot stat ‘$/Upstream_Tf_SubsurfaceReferenceLibrary/Dev/BP.USL/SRL.PipeLineFApp/requirements.txt’: No such file or directory
        2019-07-29T16:10:45.4439388Z Writing .funcignore
        2019-07-29T16:10:45.7099687Z Writing .gitignore
        2019-07-29T16:10:45.7303320Z Writing host.json
        2019-07-29T16:10:45.7411664Z Writing local.settings.json
        2019-07-29T16:10:45.7535648Z Writing /home/vsts/work/r1/a/$/Upstream_Tf_SubsurfaceReferenceLibrary/Dev/BP.USL/SRL.PipeLineFApp/.vscode/extensions.json
        2019-07-29T16:10:45.7695997Z mv: cannot stat ‘$/Upstream_Tf_SubsurfaceReferenceLibrary/Dev/BP.USL/SRL.PipeLineFApp/requirements.orig.txt’: No such file or directory
        2019-07-29T16:10:46.3922959Z No action performed because no functions in your app require extensions.
        2019-07-29T16:10:51.8723771Z Getting site publishing info…
        2019-07-29T16:10:51.8726654Z Assuming –nozip (do not run from package) for publishing to Linux dedicated plan.
        2019-07-29T16:10:51.8858274Z Creating archive for current directory…
        2019-07-29T16:10:52.8954520Z Creating archive for current directory…
        2019-07-29T16:10:53.8961001Z Creating archive for current directory…
        2019-07-29T16:10:53.9020302Z requirements.txt is not found. requirements.txt is required for python function apps. Please make sure to generate one before publishing.
        2019-07-29T16:10:53.9074899Z /home/vsts/work/_temp/azureclitaskscript1564416551220.sh: line 42: addSpnToEnvironment:: command not found

      3. On the machine you are developing your Python Functions on you should run this command in the root folder for your Functions projects:

        pip freeze > requirements.txt

        Then check the file into source control. If you have Python 2 and 3 installed you might need to do ‘pip3’ instead of just ‘pip’.

      4. Hi Simon,

        Thanks again for your quick response.
        I ran the command which you gave. But still i am getting the below error. What to do now?

        2019-07-30T14:54:43.2659036Z Could not open requirements file: [Errno 13] Permission denied: ‘/root/FunctionFolder/requirements.txt’
        2019-07-30T14:54:43.3729442Z You are using pip version 18.1, however version 19.2.1 is available.
        2019-07-30T14:54:43.3730009Z You should consider upgrading via the ‘pip install –upgrade pip’ command.
        2019-07-30T14:54:43.4066460Z mv: cannot stat ‘xxxxxxxxxxxxxxxxxxxx/USLPipeLineFuncApp/requirements.txt’: No such file or directory
        2019-07-30T14:54:43.8502768Z Writing .funcignore
        2019-07-30T14:54:44.0894808Z Writing .gitignore
        2019-07-30T14:54:44.1136050Z Writing host.json
        2019-07-30T14:54:44.1277766Z Writing local.settings.json
        2019-07-30T14:54:44.1400655Z Writing /home/vsts/work/r1/a/xxxxxxxxxxxxxxxxx/USLPipeLineFuncApp/.vscode/extensions.json
        2019-07-30T14:54:44.1591781Z mv: cannot stat ‘$xxxxxxxxxxxxxxxxx/USLPipeLineFuncApp/requirements.orig.txt’: No such file or directory
        2019-07-30T14:54:44.7974995Z No action performed because no functions in your app require extensions.
        2019-07-30T14:54:50.3729149Z Getting site publishing info…
        2019-07-30T14:54:50.3730573Z Assuming –nozip (do not run from package) for publishing to Linux dedicated plan.
        2019-07-30T14:54:50.3867729Z Creating archive for current directory…
        2019-07-30T14:54:51.3963812Z Creating archive for current directory…
        2019-07-30T14:54:52.3969931Z Creating archive for current directory…
        2019-07-30T14:54:52.4081904Z requirements.txt is not found. requirements.txt is required for python function apps. Please make sure to generate one before publishing.

      5. It looks like you haven’t checked the requirements.txt file you generated into source control, or the file is in a different location to what is expected by the script from the blog.

      6. Hi Simon,

        I have checked the correct requirements.txt. One more thing when i generate dthe function app the requiremnets.txt was generate dand its blank is that causing the issue?

      7. The requirements.txt file should not be blank – this means you are likely running the pip freeze command in the wrong folder. Ensure you run the command at a prompt in the same folder as the host.json file.

  3. Hi Simon,
    Thanks for such a quick and swift reply.
    I got to know before reading your above comment that i need Ubuntu agent. However i am not able to see any function getting created in my function app even my build was successful.

    See my Buils logs
    .
    2019-07-29T12:57:29.2625598Z Setting up python3.6-venv (3.6.9-1+xenial1) …
    2019-07-29T12:57:32.9653932Z mv: cannot stat ‘$/ProjectPathxxxxxxxxx/xxxFunctionAppName/requirements.txt’: No such file or directory
    2019-07-29T12:57:33.5730981Z Writing .funcignore
    2019-07-29T12:57:33.8488710Z Writing .gitignore
    2019-07-29T12:57:33.8647940Z Writing host.json
    2019-07-29T12:57:33.8776051Z Writing local.settings.json
    2019-07-29T12:57:33.8899082Z Writing /home/vsts/work/r1/a/$/ProjectPathxxxxxxxxx/xxxFunctionAppName/.vscode/extensions.json
    2019-07-29T12:57:33.9067964Z mv: cannot stat ‘$/ProjectPathxxxxxxxxx/xxxFunctionAppName/requirements.orig.txt’: No such file or directory
    2019-07-29T12:57:34.4505595Z No action performed because no functions in your app require extensions.
    2019-07-29T12:57:39.0243152Z Can’t find app with name “$/ProjectPathxxxxxxxxx/xxxFunctionAppName/FunctionName”
    2019-07-29T12:57:39.6038602Z Writing .funcignore
    2019-07-29T12:57:39.9961593Z Writing .gitignore
    2019-07-29T12:57:40.7882305Z Writing host.json
    2019-07-29T12:57:40.7882990Z Writing local.settings.json
    2019-07-29T12:57:40.7883875Z Writing /home/vsts/work/r1/a/$/ProjectPathxxxxxxxxx/xxxFunctionAppName/$/$/ProjectPathxxxxxxxxx/xxxFunctionAppName/FunctionName/.vscode/extensions.json
    2019-07-29T12:57:40.7919693Z [command]/usr/bin/az account clear
    2019-07-29T12:57:40.7920550Z ##[section]Finishing: Azure CLI Build an ddeploy azure python function

  4. Thanks for this, i am able to deploy my python function now using this, could you also provide yaml code for createing “App service plan” and “function app”, currently we are creating it manually

    1. Hi Ashok – we don’t use YAML for infrastructure deployments in Azure (YAML is used only for build / release definitions in Azure DevOps).

      You have a few choices to automate the creation of an App Service Plan and Function App:

      – Use Azure Resource Manager (ARM) Template: see: https://docs.microsoft.com/en-us/azure/azure-functions/functions-infrastructure-as-code
      – Use the Azure Command Line Interface (CLI): https://docs.microsoft.com/en-us/azure/app-service/scripts/cli-deploy-staging-environment
      – Use the Azure PowerShell Commandlets: https://docs.microsoft.com/en-us/azure/app-service/scripts/powershell-deploy-staging-environment

      Note that you might have to modify the above samples slightly as you don’t need to wire up the Git repository to deploy to the service.

      Once you have your script or ARM template you can invoke it in Azure DevOps using the appropriate task in Azure DevOps:

      – ARM Templates: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops
      – Azure CLI Task: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-cli?view=azure-devops
      – Azure PowerShell Task: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-powershell?view=azure-devops

      Hope this helps.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s