Navigate back to the homepage

Pull Assets To Local Storage With Python

Jason Hanshaw
January 5th, 2020 · 4 min read

In my experience developing in Salesforce Marketing Cloud, one of the more frustrating aspects has been a lack of real version-control for assets in the platform. Most solutions involve either complicated integrations or hacky workarounds to prevent unwanted edits, deletions or path changes. Thankfully, the Content Builder REST API provides us with everything we need to create this functionality ourselves.

I’ll be going into the complete method of creating your own git-like wrapper for Content Builder assets in a future post but, for now, let’s take a look at a piece of that functionality that has broad applications for how we manage our content.

Pulling To Local Storage

In this post, we’ll be looking at a python script that will take input from a user and, based on that input, will download Content Builder assets and put them in a folder specified by the input. Before we can look at the script, we’ll need to configure our system.

Prerequisites

  • Python 3.8.1
  • Installed Package set up in your SFMC Account

If you don’t currently have python installed on your local system, here is a guide on how to get set up:

Python 3 Installation Guide

If you are unsure of how to create and install a package in your SFMC account, please refer to the following documentation for more information:

Create and Install Packages

Getting Started

Now that we have both Python and an SFMC package created and installed, we’ll start configuring project-specific items. We need to set up our project folder, add libraries and set some environment variables that we can use to make requests to SFMC.

First, let’s create a folder on our desktop called python-pull-assets. This will be our project folder and will house our files. After that, let’s create a new file in that directory called .ENV. Here, we’ll set some environment variables for our API calls so that we can authenticate and access the assets within Content Builder. You’ll need to replace the values in this file to map to the information present in the installed package you setup in SFMC.

/python-pull-assets/.ENV

1clientID = YOUR CLIENT ID
2clientSecret = YOUR CLIENT SECRET
3authURL = YOUR AUTH URL
4baseURL = YOUR BASE REST URL

Now that we have our project directory and environment variables created, let’s add some libraries that the script will rely on. First, we’ll add the requests library. This will make it easier for us to form our REST calls to SFMC. To do this, let’s open a command prompt and navigate to our project folder:

cd Desktop
cd python-pull-assets

Then, we’ll add the requests library using pip:

pip install requests

Let’s also add the decouple library. This will allow us to import the variables from our .ENV file into our script:

pip install python-decouple

That’s all we need for our initial configuration! Now, let’s take a look at the actual scripting necessary to pull Content Builder assets to local storage.

Adding Python Script

We’ll break down the script into smaller blocks and analyze each one but, first, let’s take a look at the full script (in case you want to skip this section and just dive in):

/python-pull-assets/downloadAssets.py

1from decouple import config
2import requests, json, os, io, shutil
3
4# set req vars
5clientID = config("clientID")
6clientSecret = config("clientSecret")
7authURL = config("authURL")
8baseURL = config("baseURL")
9
10# prepare the target directory
11if 'assets' not in os.listdir(os.getcwd()):
12 os.makedirs('assets')
13print("\n/**** SFMC ASSET PULL ****/\n")
14# get asset id for api call
15assetID = int(input("\nENTER ASSET ID: "))
16# get folder name in order to build path
17folderName = input("\nENTER FOLDER NAME: ")
18filePath = "assets/" + folderName + "/"
19if filePath not in os.listdir(os.getcwd()):
20 os.makedirs(filePath)
21#get access token for api call
22def getToken():
23 payload = {
24 "clientSecret": clientSecret,
25 "clientId":clientID
26 }
27 print("\nAuthenticating...\n")
28 res = requests.post(authURL, data=payload)
29 res.raise_for_status()
30 return json.loads(res.text)["accessToken"]
31 print("Done.")
32
33accessToken = getToken()
34headers = {
35 'content-type': 'application/json',
36 'Authorization': 'Bearer ' + accessToken
37}
38# content request payload
39contentRequest = {
40 "query": {
41 "property": "assetType.id",
42 "simpleOperator": "equals",
43 "valueType": "int",
44 "value": assetID
45 },
46 "page": {
47 "pageSize": 500
48 }
49}
50print("Getting assets...\n")
51# get asset data from Content Builder rest api
52res = requests.post(baseURL + 'asset/v1/content/assets/query', data=json.dumps(contentRequest, separators=(',', ':')), headers=headers)
53res.raise_for_status()
54print("Done\n")
55
56assetsJSON = res.text
57assetLoad = []
58numOfAssetsFound = 0
59print("Downloading Library\n")
60assetsList = json.loads(assetsJSON)["items"]
61for asset in assetsList:
62 # append html extension if asset id greater than 194
63 fileIDExt = filePath + asset["name"].replace("/","_")
64 if assetID > 194:
65 fileID = fileIDExt + ".html"
66 else:
67 fileID = fileIDExt
68 targetFile = io.open(fileID, 'w+', encoding="utf-8")
69 # if template-based, html or text email
70 if assetID == "207" or assetID == "208" or assetID == "209":
71 if "content" in asset:
72 # write asset to local storage
73 targetFile.write(str(asset["views"]["html"]["content"]))
74 # if not in the criteria above but asset id is of type "block"
75 elif (assetID != "207" and assetID != "208") and (assetID > 194):
76 if "content" in asset:
77 # write asset to local storage
78 targetFile.write(str(asset["content"]))
79 else:
80 assetURL = str(asset["fileProperties"]["publishedURL"])
81 # retrieve asset from the web and download to local storage
82 response = requests.get(assetURL, stream=True)
83 with open(fileID, 'wb') as out_file:
84 shutil.copyfileobj(response.raw, out_file)
85 del response
86 targetFile.close()
87 numOfAssetsFound += 1
88print(str(numOfAssetsFound) + " assets found\n")
89print("Data successfully pulled to local storage")
90exit()

For the first portion of our script, we are importing the necessary libraries we’ll need along with the environment variables for our SFMC API calls.

1from decouple import config
2import requests, json, os, io, shutil
3
4# set req vars
5clientID = config("clientID")
6clientSecret = config("clientSecret")
7authURL = config("authURL")
8baseURL = config("baseURL")

Next, we’ll check to see if an assets folder already exists and, if it doesn’t, we’ll go ahead and create that directory. Then, we’ll prompt the user to input both the Id of the Asset Type they want to pull along with the name of the folder they wish to create and store these assets in. We can then use this data to create both the file path variable and the necessary folder structure in which to store our assets.

Note: You can find a list of Asset Types and their corresponding Id’s here.

1# prepare the target directory
2if 'assets' not in os.listdir(os.getcwd()):
3 os.makedirs('assets')
4print("\n/**** SFMC ASSET PULL ****/\n")
5# get asset id for api call
6assetID = int(input("\nENTER ASSET ID: "))
7# get folder name in order to build path
8folderName = input("\nENTER FOLDER NAME: ")
9filePath = "assets/" + folderName + "/"
10if filePath not in os.listdir(os.getcwd()):
11 os.makedirs(filePath)

With our folder structure in place, and our user input captured, we can make an API call to SFMC to generate an access token for authenticating into the platform. In our example below, we are using the legacy endpoint/authentication to generate our access token. Newer packages using the v2/token endpoint will require a different payload and token retrieval path. For more information on this please refer to the documentation on Server-to-Server Integrations with Client Credentials Grant Type.

1#get access token for api call
2def getToken():
3 payload = {
4 "clientSecret": clientSecret,
5 "clientId":clientID
6 }
7 print("\nAuthenticating...\n")
8 res = requests.post(authURL, data=payload)
9 res.raise_for_status()
10 return json.loads(res.text)["accessToken"]
11 print("Done.")
12
13accessToken = getToken()

Let’s take the access token generated in our last snippet and now make a request to the SFMC Content Builder API to return all assets that match the Asset Type Id that the user defined in their input. With our contentRequest object declaration, we are specifying the advanced query logic that will filter the data we return from SFMC. This query can include nested AND/OR logic based on the asset parameters returned in a 200 response.

Also, we can set values for the number of pages and page size of the data that we wish to return. Pagination of the data is not included in this example but, unless the amount of assets for a given type is exceedingly large, you should be able to get by with setting the pageSize value to a large upper limit to return all results in one page.

1headers = {
2 'content-type': 'application/json',
3 'Authorization': 'Bearer ' + accessToken
4}
5# content request payload
6contentRequest = {
7 "query": {
8 "property": "assetType.id",
9 "simpleOperator": "equals",
10 "valueType": "int",
11 "value": assetID
12 },
13 "page": {
14 "pageSize": 500
15 }
16}
17print("Getting assets...\n")
18# get asset data from Content Builder rest api
19res = requests.post(baseURL + 'asset/v1/content/assets/query', data=json.dumps(contentRequest, separators=(',', ':')), headers=headers)
20res.raise_for_status()
21print("Done\n")

Now that we have taken our user’s input, generated an access token and made our request to SFMC for the specified Asset type, it’s time to parse the response from SFMC and save our assets to our local folder.

First, we’ll take the response and find the nested level containing the information we want to retrieve using json.loads(assetsJSON)["items"]. Then, we’ll iterate through the asset list and push the content from each file to local storage.

Since the structure of the data differs depending on the type of asset that you are pulling, we’ll need to use some simple if/else statements to let our script know exactly where and how to pull the correct information from the JSON. In the snippet below, you can see that the nested data structure for Id’s 207, 208 and 209 (template-based, html and text emails respectively) isn’t the same as that of the standard “block” types in SFMC.

Even different still is our method of parsing and saving non-HTML type content (i.e. images, documents, etc…). For this, we’ll need to grab the URL of the asset within Content Builder and make a separate request to retrieve that object and write it’s output to storage.

1assetsJSON = res.text
2assetLoad = []
3numOfAssetsFound = 0
4print("Downloading Library\n")
5assetsList = json.loads(assetsJSON)["items"]
6for asset in assetsList:
7 # append html extension to file if asset id greater than 194 (type "block")
8 fileIDExt = filePath + asset["name"].replace("/","_")
9 if assetID > 194:
10 fileID = fileIDExt + ".html"
11 else:
12 fileID = fileIDExt
13 targetFile = io.open(fileID, 'w+', encoding="utf-8")
14 # if template-based, html or text email
15 if assetID == "207" or assetID == "208" or assetID == "209":
16 if "content" in asset:
17 # write asset to local storage
18 targetFile.write(str(asset["views"]["html"]["content"]))
19 # if not in the criteria above but asset id is of type "block"
20 elif (assetID != "207" and assetID != "208") and (assetID > 194):
21 if "content" in asset:
22 # write asset to local storage
23 targetFile.write(str(asset["content"]))
24 else:
25 assetURL = str(asset["fileProperties"]["publishedURL"])
26 # retrieve asset from the web and download to local storage
27 response = requests.get(assetURL, stream=True)
28 with open(fileID, 'wb') as out_file:
29 shutil.copyfileobj(response.raw, out_file)
30 del response
31 targetFile.close()
32 numOfAssetsFound += 1
33print(str(numOfAssetsFound) + " assets found\n")
34print("Data successfully pulled to local storage")
35exit()

All that’s left to do is save the full script inside of our python-pull-assets folder with a file name of downloadAssets.py. Once that is saved, we can go back to the command prompt and simply run the following:

py downloadAssets.py

You should be presented with a prompt for the Asset ID and folder name. After successful input and script execution, navigate to assets/YOUR_FOLDER_NAME in your project directory to view the assets that you’ve retrieved from Content Builder.

Conclusion

Utilizing this approach should allow you to maintain consistent backups of your Content Builder content as well as giving you the ability to construct a version-control system that resembles a more modern development workflow. With some simple modifications to the above script, you could also pull journey assets or expand this into a full-fledged application for managing and updating your content.

In addition to this blog post, you can find the code for this example in this github repository.

More articles from InvokeCreate

CloudPage Deep Linking

A dynamic deep linking solution that routes users based on device type

January 1st, 2020 · 3 min read

Search A Data Extension From Sales Cloud

Create a lightning component to search a Send Logging data extension in Marketing Cloud.

February 14th, 2020 · 7 min read
© 2019 InvokeCreate
Link to $https://github.com/invokecreate/Link to $https://www.linkedin.com/in/jason-hanshaw-developer/Link to $https://salesforce.stackexchange.com/users/50722/jason-hanshaw