Quickstart: Create a MAKANA Python Function App

MAKANA Python Functions (MPF) are used to execute long-running Python scripts using Azure's serverless infrastructure. You can use the MPF to execute any Python script without having to setup any of the necessary computer power required for its execution. Just save your Python script on an Azure storage account and use an HTTP Post interface to trigger it.

This document walks you through setting up your first MPF implementation.

Note

If you already have an Azure subscription you can click here to add the MAKANA Python Function to your subscription.

Get an Azure subscription

If you don't have an Azure subscription, create a free account before you begin.

Adding a MAKANA Python Function App to your Azure tenant

Important

The following resource providers need to be registered in your subscription to enable MAKANA Python Function:

Microsoft.ContainerInstance
Microsoft.ContainerRegistry

Registering resource providers is free and can be done by opening up the Subscription in the azure portal and selecting Resource roviders from the left navigation bar

Select the Resource provider you want to register and click Register. Once the resource providers are registered, you can proceed with the below steps.

Select Create a resource from the top menu of the Azure portal. If Create a resource is not in the list, click on the More services arrow at the end of the menu list.
In the search box, type in MAKANA Python Function and press Enter.
Click on the Create button to initialize the creation wizard.
Provide the Basics information requested in the form.
- Resource group: Select Create new, type the name of the resource group (myResourceGroup), and select OK.
- Region: Select the region in which you want to create the MPF App. For better performance, choose a region that is the same or as close to your data region as possible.
- Application Name: enter the name of the MPF application (myApplication) and click on Review + create.
Complete the Review + create form:
- Check the I agree to the terms and conditions above. checkbox after you have reviewed the terms and the pricing information.
- Click on the Create button.
Your deployment is underway will be display. This will take a few minutes to complete:

Once the status is Created, you can go to the Resource group by clicking on Go to resource.
Your application (myApplication) should be listed in the Resource Group you created (myResourceGroup) with the type value equal to Managed Application.
Click on the application name and you should see the Welcome to your Managed Application screen:
Click on the Managed resource group name in the top right side of the screen.
This will open the managed resource group. You can check the status of the resources by looking at the Status tag:

The Status Tag will first be Creating Resources and will take a few minutes to an hour to finish creating all the MPF resources. Once all resources have been created, the Status Tag will change to Ready to Use, and an HTTP Endpoint tag will be displayed. This HTTP Endpoint URL will be used to execute the Python script.

Executing a Python script using MPF

For this part, we will use the sample Python script below - you can copy paste the code below into a file called myscript.py and changed the values for PythonScriptStorageAccountName and StorageAccountKey:

# This script will create a BLOB container in the storage account below
# It takes two arguments:
# arg 1: the BLOB container suffix
# arg 2: the time to wait before creating the container to simulate a long running script
# some imports are not needed, but used to demonstrate the creation of a Python environment
# replace <PythonScriptStorageAccountName> and <StorageAccountKey> with your values for the location of the BLOB container

import time
import sys
import os, uuid
from sklearn import datasets
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient

connect_str="DefaultEndpointsProtocol=https;AccountName=<PythonScriptStorageAccountName>;AccountKey=<StorageAccountKey>;EndpointSuffix=core.windows.net"
blob_service_client = BlobServiceClient.from_connection_string(connect_str)

# Create a unique name for the container
container_name = "testmpf" + str(uuid.uuid4()) + "-" + sys.argv[1]

time.sleep(int(sys.argv[2]))

# Create the container
container_client = blob_service_client.create_container(container_name)

The above script will use a Python environment composed of the following libraries:

scikit-learn
azure.storage.blob

You can test this script by running the following command on your local Python environment:

python myScript.py abc 120

The following steps will instruct you on how to execute this script using MPF.

Sign in to the Azure portal using your account.
Create an Azure storage account or use an existing storage account.
Click on Files shares in the storage account and create a new file share by clicking on + File share.
Open the new file share by clicking on it and create a directory on the file share called Python by clicking on + Add directory, then upload your python script to the Python directory.
Under the storage account Settings, choose Access keys.
Copy the Storage account name and one of the Key values, you will need those later:
- Storage account name: mystorageaccount
- key value: Vj/IbL1Hol/qHwYS82BQIboH4EEf+n7iaSY4VdnaBmJfFnu5MgDAoldwrL9JvTYc9qu8v2OY8fhkkiqmjKXyNA==
To use MPF, you will need the following information:
- The HTTP Endpoint URL for your MPF App
- The resource group name of the storage account that has the Python script
- The name of the storage account that has the Python script
- The storage account key for the storage account that has the Python script
- The share name where the Python script is on the storage account
- The name of the Python script file that is stored in the share on the storage account
- The value of any arguments or parameters that the Python script requires
- The relative path to the script file on the storage account share
- The packages list that are needed to build the Python environment for the script.

Once you have gather all the information above you can perform an HTTP POST with the following information:

curl --location --request POST 'https://fiocaingomqw72.azurewebsites.net/api/RunPython?code=9kEkAx2CslGMHBwnQmbeSEXrvQz47UeJtRIvSprH5z9GIkW/87jJzg==' \
--header 'Content-Type: application/json' \
--data-raw '{
    "ResourceGroupName":"<storageAccountResourceGroup>",
    "StorageAccountName":"mystorageaccount",
    "StorageAccountKey":"Vj/IbL1Hol/qHwYS82BQIboH4EEf+n7iaSY4VdnaBmJfFnu5MgDAoldwrL9JvTYc9qu8v2OY8fhkkiqmjKXyNA==",
    "ShareName":"myshare",
    "ScriptFileName":"myscript.py abc 120",
    "RelativePathToScriptFile":"//Python",
    "PackagesList":["scikit-learn","azure.storage.blob"]
}'

The response will be a JSON string with of the following format:

{
    "id": "e4f1418782ef4a278dd2d971aa09e1e2",
    "statusQueryGetUri": "https://fac3fps5k276zu.azurewebsites.net/runtime/webhooks/durabletask/instances/e4f1418782ef4a278dd2d971aa09e1e2?taskHub=fac3fps5k276zu&connection=Storage&code=/U5/K8h83PhYLlKsUtze5B7G4Nfbdak67wkoYzxoO10jn2cvq4jH8w==",
    "sendEventPostUri": "https://fac3fps5k276zu.azurewebsites.net/runtime/webhooks/durabletask/instances/e4f1418782ef4a278dd2d971aa09e1e2/raiseEvent/{eventName}?taskHub=fac3fps5k276zu&connection=Storage&code=/U5/K8h83PhYLlKsUtze5B7G4Nfbdak67wkoYzxoO10jn2cvq4jH8w==",
    "terminatePostUri": "https://fac3fps5k276zu.azurewebsites.net/runtime/webhooks/durabletask/instances/e4f1418782ef4a278dd2d971aa09e1e2/terminate?reason={text}&taskHub=fac3fps5k276zu&connection=Storage&code=/U5/K8h83PhYLlKsUtze5B7G4Nfbdak67wkoYzxoO10jn2cvq4jH8w==",
    "rewindPostUri": "https://fac3fps5k276zu.azurewebsites.net/runtime/webhooks/durabletask/instances/e4f1418782ef4a278dd2d971aa09e1e2/rewind?reason={text}&taskHub=fac3fps5k276zu&connection=Storage&code=/U5/K8h83PhYLlKsUtze5B7G4Nfbdak67wkoYzxoO10jn2cvq4jH8w==",
    "purgeHistoryDeleteUri": "https://fac3fps5k276zu.azurewebsites.net/runtime/webhooks/durabletask/instances/e4f1418782ef4a278dd2d971aa09e1e2?taskHub=fac3fps5k276zu&connection=Storage&code=/U5/K8h83PhYLlKsUtze5B7G4Nfbdak67wkoYzxoO10jn2cvq4jH8w=="
}

You can use the statusQueryGetUri URL in a GET HTTP to check the status of the Python script job. The status would be displayed in the runtimeStatus JSON field of the statusQueryGetUri GET. While the script is running the value is Running; when the script completes its execution, the value will change to Completed.

{
    "name": "MachineLearningOrchestrator",
    "instanceId": "e4f1418782ef4a278dd2d971aa09e1e2",
    "runtimeStatus": "Running",
    "input": {
        "RequestId": "149e53da-a959-4af3-83c8-353f5e586345",
        "InstanceId": null,
        "ImageName": "",
        "StorageAccountName": "mystorageaccount",
        "StorageAccountKey": "Vj/IbL1Hol/qHwYS82BQIboH4EEf+n7iaSY4VdnaBmJfFnu5MgDAoldwrL9JvTYc9qu8v2OY8fhkkiqmjKXyNA==",
        "ShareName": "myfileshare",
        "VolumeName": null,
        "ScriptFileName": "myscript.py abc 120",
        "RelativePathToScriptFile": "//Python",
        "Status": "",
        "RequestMessage": "",
        "Language": "Python",
        "Packages": null,
        "PackagesList": [
            "scikit-learn",
            "azure.storage.blob"
        ],
        "ContainerImage": null,
        "RequestStatus": 0,
        "LanguageType": 0,
        "PartitionKey": null,
        "RowKey": null,
        "Timestamp": "0001-01-01T00:00:00+00:00",
        "ETag": null
    },
    "customStatus": "Executing Script",
    "output": null,
    "createdTime": "2020-04-01T17:35:07Z",
    "lastUpdatedTime": "2020-04-01T17:38:33Z"
}