Well, I have wrapped up from Sitecore Symposium last week.  Not fully recovered, but I wanted to blog about the topic for my session at Sitecore Symposium 2022 in Chicago.  Now the topic that I did, was on Developing for the Engagement Cloud, but one of the topics I covered that I want to talk about/recap here today, is the PoC (Proof of Concept) that I had built for this presentation for CDP/Personalize.  This PoC was meant to show a new and I will admit better way of building things in your tenant, using the tool that every developer is familiar with, which is Source Control.

## Where Did This Idea Come From

So a couple of months ago, I was told about some APIs that could be used for migration.  Now after exploring these APIs I realized they only covered a few specific use cases and that there was a greater need.  At the time, I knew that customers were delivered a UAT and a Production tenant, but everything was manual to move items between those environments.  However, the APIs that had been delivered was only for Decision Models, and not for what I considered developer tasks (I think this is a larger discussion for another blog post in the future).  So I started to dig in and research if there were any more APIs that could be used to solve my challenges.  This is when I stumbled upon a set of APIs that are being used by the frontend application of the tenant, but these same APIs are not currently documented anywhere.  These APIs are very powerful, and open up the door for a bunch of different use cases:

- Using Source Control
- Migration
- One-Off Accelerator Deployments

## What are these APIs?

As I just mentioned they are not currently in the official Sitecore documentation.  I’ve worked to try to determine why that is, and it’s simply because the APIs aren’t officially supported by Sitecore currently.  But does that mean you should avoid them? I don’t think that’s the case, and in fact, they open up and allow for automation scenarios that are too great to ignore.  So how do you find out about these APIs?  Well, it’s quite simple, if you log in to a CDP/Personalize tenant, open up `Chrome Tools` and view the network tab, you’ll notice as you interact with the interface, there are a lot of interactions taking place asynchronously, including with all of the APIs that I’m describing today.  Now I’ve spent some time documenting what I’ve found here and created a very rough draft Swagger document of these APIs: [https://dylanyoung-dev.github.io/sitecore-cdp-personalize-api-spec/](https://dylanyoung-dev.github.io/sitecore-cdp-personalize-api-spec/).  Just keep in mind that these are unofficial and something that I’ve personally spent time creating.  These undocumented APIs are listed under `Tenant Management`, this isn’t an official name for these, but these are the ones that are taking place in the browser.

## How to Start Using these APIs?

Okay, so now you know about the `hidden` APIs, but what do you do with them and how do you start using them?  Well, the first major hurdle you’ll deal with when using these APIs is authentication.  It’s different from the Batch, Stream, and Interactive APIs that you would typically use to ingest data to CDP/Personalize.  These APIs use OAuth to handle authentication, and there are two core ways to authenticate and get back the access token that you’ll use as a bearer token in the rest of the API requests.  There is the `password` grant type, which is where you would use your username and password for the tenant, to authenticate to use these APIs.  The problem with this approach is that if you have Multi-Factor Authentication (MFA) enabled, which is likely the case in a production environment, then you’ll need to pass in your MFA code as well.  This presents issues when you are trying to have a system run an automated process, such is the case during CI/CD DevOps deployments.  The second method is the preferred method of using `client credentials` grant type, which will use the `Client Key` and `API Token` from the API access panel in your tenant:

![](https://s3.us-west-2.amazonaws.com/secure.notion-static.com/2654d617-7be6-4274-86d0-7d1d7deee1e3/API-Access-panel.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20221028%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221028T174406Z&X-Amz-Expires=3600&X-Amz-Signature=615258348bbed9cbb22f0ee8aeec7c0b6a32e798ec47e5b177155af88a07ef8f&X-Amz-SignedHeaders=host&x-id=GetObject)

## Proof of Concept

So let’s talk a little more about the Proof of Concept (PoC) that I worked on that utilizes the APIs above.  I had this crazy idea in my head, maybe 6 months ago, that source control could be used instead to create and control the flow of various assets to a specific tenant.  Something like the diagram below was what I was envisioning.  At the same time that I was dreaming up this new approach, the product team was talking about how it would be nice to test these further, but in my mind, how do you just test a tenant’s assets directly?  I didn’t think the use case of randomly testing what was there, really made sense, and I thought it would make much more sense in a DevOps flow, where before you deploy to a tenant, you run unit tests and confirm they pass before you deploy, and then Automation tests could be done post-deploy to confirm if the changes were deployed successfully.

![](https://github.com/dylanyoung-dev/cdp-personalize-examples/raw/main/docs/images/Using-Source-Control.jpg)

Now as of this writing, I have only some simple examples of Unit Testing, and no examples of how to handle automation testing, but rest assured, that is the next part of what I’ll be working on next.  I now have two repositories that I’ll describe and link to below, but the serializer application needs to be broken apart into the SDK parts for calling the APIs, and then I should be able to share the Automated Testing scenarios utilizing that SDK.  So more on that to come.

But before I get into the projects, just wanted to describe the illustration above.  Simply put, a developer will no longer develop their changes directly in the tenant and instead will set up a repository with the serialized assets that they would build locally, and then there would be a process to deploy those changes to their tenant.  That’s where we would get into the repositories and their use cases.  Today all the examples use GitHub and Mocha for Unit testing, but my intention is for this PoC to be tech agnostic, so if you wanted to deploy changes, you could use Azure DevOps or GitLab, or any other Source Control Management system.  I plan to create a lot more content coming up on different scenarios that could be done for this PoC.  But with my PoC it is meant to be a community project, and thus all the repositories below are open source and I suspect will get way more traction as the community builds them out and other scenarios that these APIs can handle.

## Serializer Repository

This is the brains of the operation and where all the magic happens.  It takes everything that is in physical form (in a source control repository) and moves that (via a `deploy` command) using a Node CLI and moves that to a CDP/Personalize tenant.  Currently, the PoC that was originally created only includes examples for Decision, Audience, and Web Experience templates, but the plan is that it really should support anything that you would create in your tenant directly.  I always like to build things like this, to be unopinionated as possible, so those teams that have specific use cases should be able to handle their specific use cases using the same project.  So for example, if your organization is structured in a way where everything is done by developers and directed by a marketing team then your developers can configure as much as they desire in Source Control and have those assets deployed to their tenants.  Another use case, which I would assume is likely more common is that only certain assets are considered `developer` assets, such as templates, and then only those assets are in source control, while everything else falls outside of that and can be freely created in Source Control.

GitHub Repository: [https://github.com/dylanyoung-dev/sitecore-cdp-serializer](https://github.com/dylanyoung-dev/sitecore-cdp-serializer)

Available on NPM: [https://www.npmjs.com/package/sitecore-cdp-serializer](https://www.npmjs.com/package/sitecore-cdp-serializer)

## Examples Repository

This is the repository that shows how all this comes together to achieve the diagram above.  Now it’s opinionated because it uses a specific set of technology, such as Mocha and Super test for Unit/Automated tests and it also uses VS Code and GitHub Workflow for CI/CD deployments.  But it’s really easy to either create a template from this project and start creating it based on your own scenarios, or it’s also configured as a `template` repository, so when you land on the link below, you can click on `Use template` and you can start building based on this repository for your scenarios today. 

GitHub Repository: [https://github.com/dylanyoung-dev/cdp-personalize-examples](https://github.com/dylanyoung-dev/cdp-personalize-examples)

## Extra Credit

So all of this is great for customers but this opens up a new world for the community or partners that want to build out starter kits or accelerators for a specific industry.  For example, partner A builds experiences all the time for specific verticals like energy or health care, and could now use this strategy to either build a bunch of items for a specific industry so that those customers can get off the line faster.  Now I don’t have any specific examples of this today, but I challenge the community to contribute to these projects and start to create out these use cases.

Developer Experience

Introducing Sitecore CDP/Personalize Serializer/CLI Tool

Welcome back friends, today I’m happy to announce a new feature in Sitecore Personalize, which is a new capability of creating access tokens (JWT Tokens) for specific purposes.  To learn more about this release, check out this link here from the Developer Portal Change Log: [https://developers.sitecore.com/changelog/personalize/developers-can-now-create-api-keys-to-access-personalize-rest-apis](https://developers.sitecore.com/changelog/personalize/developers-can-now-create-api-keys-to-access-personalize-rest-apis).  Today’s article is going to spend some time reviewing exactly what this change means for developers.  Also, the new documentation pages for Sitecore Personalize include specific use cases for these new user-generated tokens: [https://doc.sitecore.com/personalize/en/developers/api/index-en.html](https://doc.sitecore.com/personalize/en/developers/api/index-en.html#UUID-ac90ff74-4b6e-95e6-3e15-aeba2d72db73).

## What are User Generated Tokens

So before we get into what they are, let’s talk a little about why they were needed.  Those familiar with the early days of the Sitecore acquisition of Boxever and the existing Boxever product are aware of the concept of a Key to rule them all.  This was a single key you would use for almost anything you desired.  Unfortunately this key also, with its broad scope, created many security concerns, because once you had the key, it was very difficult to reset it, because you only had one.  So if you reset the key, all your services that relied on that key would break at the same time.  Furthermore, the key was seen by practically everyone and that created issues when you had partners or developers working on your tenant, but when they left, they still had access to that token.  There were many issues with this approach and so that’s why there needed to be a better way of handling this.

This is why there is now a self-service interface to create tokens for specific use cases.  What is great about this approach, is that no one can see the key except for the developer/admin that created the key.  Furthermore, if that developer leaves the company or the partner that the developer works for leaves the organization, another developer/admin can create a new key, update the key for the automated process and then remove the old key with no issues with downtime during that process.  Also, these keys are no longer configured to do anything/everything with the product but are limited to specific use cases.  So if you are a developer and you want to configure automated unit tests (which I’ll get to a little bit more about in a second) you can now create a specific key for this specific purpose, which then locks that key from accessing `Guest Context` or `Templates` since the token that you’ve created is limited to a very specific use case.

Sounds great, so how do I use it?

## Create Your First Token

Before we get ahead of ourselves, there are some limitations, first off, this change specifically relates to Sitecore Personalize and only tenants on the cloud portal.  If you are an existing Boxever tenant, don’t worry, likely your tenant will be migrated to the Cloud Portal soon and then you can enjoy these new benefits of the product.

To get to create a new token, the steps are quite simple, you need to log in to your new tenant, which is straight forward because you can log in directly via the cloud portal: [https://portal.sitecorecloud.io/](https://portal.sitecorecloud.io/).  Log in using your cloud portal login credentials.  Once you are logged in, navigate to your Personalize tenant:

![](https://cdn.sanity.io/images/lanua4su/production/ea1ab7067018cd36b35bc57661a6c93d90640b72-1188x405.jpg)

Once you are in your tenant, it’s quite easy to start managing your API keys, you just need to make sure that your access role is set to either `Developer` or `Admin` roles.  To get to the new API keys section, you’ll navigate to the header part of the webpage (top portion of the web page) and you’ll hover over and click on `Developer Center` and select `API Keys` to navigate to this new page.  You should now see a list of keys generated by anyone on your tenant.  Keep in mind, you only have the ability to delete these keys, and once you create the key, that will be your only opportunity to copy the key to use for your purposes.  Let’s jump into the next section where I’ll cover the latest release of the Sitecore CDP (Personalize) Serializer version 1.8.0 and how you can create a key for the Serializer specifically.

## Sitecore Personalize Serializer

Those that are familiar with the content of this blog, are aware that last year at Sitecore Symposium I announced a Node-based Serialization tool for Sitecore Personalize, which uses hidden/lesser-known APIs that would allow a developer to store in a repository all their Sitecore Personalize assets (templates, connections, etc) and then build and deploy those changes using a CI/CD pipeline into Sitecore Personalize.  This announcement directly relates to this tool, because it now allows a user of Sitecore Personalize to create a token with the type of “Multiple Resources”, which will cover this specific use case (among others).

When you create your API key, make sure you give it a descriptive name, so that others managing the set of active API Keys have a better understanding of their purposes.

![](https://cdn.sanity.io/images/lanua4su/production/03ba6c5ad42c3486485cb8925608d444d7e4fe8c-613x454.jpg)

Also make sure to select the `Multiple resources` above, so that you can manage many different types of developer elements in the product.  Specifically, Multiple Resources include CRUD operations on Connections, Templates, Decision Models, Flows (Experiences and Experiments), Conditions, and JS Modules.  The only thing at the time of this writing that you can’t control with this access token is the ability to create Offer Templates, but hopefully, in the future, this will be improved/added as well.

Once you have your key, which consists of the Client Key and Secret, you’ll copy these values for future use.  You should never share these keys with anyone else, and where ever you plan to use them, such as a GitHub Secret or Azure Key Vault, you should move them directly to that location, so that the risk of someone getting access to those keys is minimal.

Recently I released a new version of the serializer, currently called `Sitecore CDP Serializer` however I’m considering changing it to the Sitecore Personalize Serializer since the serializer mostly deals with Personalize assets rather than CDP assets.  The latest version 0.2.1 includes a new parameter during the `auth login` where you would pass `-c` if you want to authenticate with Cloud Portal.  The authentication method with these new API Keys is completely different than the traditional [api.boxever.com](http://api.boxever.com) authentication.

```bash
> npx sitecore-cdp-serializer auth login -id "<Your Client Key>" -s "<Your Client Secret>" -l "US" -c
```

The new authentication uses Client Credentials instead of the password grant type moving forward.  So if you are building directly with the APIs keep that in mind.  With that change came another improvement within the product.  Prior to this release, if you made client credential changes, inside the Personalize interface you would see `Unknown` if you created elements that made it tough to determine who was making those changes, or really which API keys made those changes.  With this new release, however, it’ll now show the key’s name in the listing of elements when you make changes with a key:

![](https://cdn.sanity.io/images/lanua4su/production/5e3cabd9afb320d2cbd5c77e3b47fa500a0737e7-1454x266.jpg)

## Additional Use Cases

In addition to using it with my very own tool, the serializer, there are actually quite a lot of useful use cases that could make sense here.  I wanted to provide a few details about those below, with the hopes that I’ll create further content on these in the future.

### Unit Testing / Automated Integration Testing

This is actually the use case I had originally planned to create a video to talk about but haven’t been able to prepare that video by the time of writing this content.  Sitecore Personalize provides a great way to create tests of various decision models, detailed in this guide here: [https://sitecore.cdpknowledgehub.com/docs/decision-models-automated-testing-guide](https://sitecore.cdpknowledgehub.com/docs/decision-models-automated-testing-guide).  That guide is great although is even greater now if you create a new API key with the Feature of `Decision model testing` selected.  Now you’ll be able to create a key specifically for your use case.  I would recommend only selecting that option if it’s the only thing you need to do.  If you plan to deploy your decision models to a tenant and then test, then you’ll likely want to select `Multiple resources` because otherwise you’ll need two keys and need to authenticate twice.

### Other Automated Use Cases

Another great use case for these new REST APIs and the API keys is to use them to create use cases on the fly.  If you have an external system, that you’d like to use to create an `Experience` or a `Decision Model` based on something in your system, this would be now a great way to achieve this.  One use case I think would be extremely valuable at some point, is to create a Generative AI example, of creating experiences/experimentation on the fly.  Definitely hope to cover this use case in the future.

Well that sums this up, if you haven’t already, you should plan to check this feature out today and write a blog about your specific use case!

Sitecore Personalize: New Developer Focused API Keys

In 2022, after just joining Sitecore as a Developer Advocate, I went and took my learnings into Sitecore Engagement Cloud and how to develop using the products within the cloud.  The presentation focused on the work I had done to change the DevEx for the product, with the introduction of the Serializer.

Sitecore Symposium 2022 - Developing with Engagement Cloud

Related Content

Introducing Sitecore CDP/Personalize Serializer/CLI Tool

Sitecore Personalize: New Developer Focused API Keys