Better A/B Testing with EdgeWorkers + EdgeKV
A/B testing is a way to compare two versions of a web experience to determine which one performs better. But did you know that, in practice, this testing process can actually negatively impact both page performance and user experience?
Akamai EdgeWorkers and Akamai EdgeKV can help mitigate these challenges. EdgeWorkers enables developers to create and deploy microservices on the largest distributed serverless network, while EdgeKV helps you build data-driven EdgeWorkers applications that involve fast, frequent reads and infrequent writes.
Read on to learn the challenges of running A/B tests, how you can use Akamai EdgeWorkers to solve those challenges, and ways to further enhance your A/B testing with EdgeKV.
Today’s challenges to effective A/B testing
A/B testing is beneficial for testing new features and measuring user engagement. But this process often negatively impacts page performance and the user experience. This is because A/B tests prevent content from being served from your Akamai cache (content is served on the same URL, but there are different versions of the content sharing the same cache key at Akamai). The logic for determining which piece of content to respond with runs on the origin web server, reducing offload and adding latency — and creating a significantly worse user experience.
One approach for combating these challenges is to move A/B logic into the browser. In doing so, you ensure the page is still cacheable and can be served from the Akamai Intelligent Edge Platform. However, this method also causes flickering content and slower page loads, which lead to a poor user experience. This is because the page is updated with JavaScript while the A/B test logic runs client-side after the browser receives the initial HTML response.
So, how do we solve these problems without impacting the user experience?
Better A/B testing at the edge
By moving the A/B test logic to the Akamai Intelligent Edge, we can cache multiple variants of the same page close to the user to decide which variant to serve the user — without the need to make the long round trip to the origin web server or rely on client-side code.
To implement A/B testing at the edge, we can use EdgeWorkers, deploying JavaScript code to the edge that determines which page variant to display. This JavaScript code can also update the cache key so each variant is cached at the edge, while also adding headers to the request to enable origin to signal with which variant it should be delivered.
Using EdgeWorkers to run our A/B test, we can enhance our solution and take advantage of the flexibility of running code on the Akamai Intelligent Edge. This extra flexibility helps manage changes made to the A/B testing logic over time. For example, we may want to start a test with 1% of users receiving alternative content, then increase that percentage as we gain confidence that the proposed change won’t introduce any issues that stand in the way of our goals.
Further challenges
How can we easily toggle tests on and off? How can we run a suite of A/B tests?
One approach is to place the logic in our Edgeworker instance, then update the EdgeWorkers package each time the test changes. However, this requires modifying and deploying code for each change, which takes considerable time.
Instead of hardcoding this data within the code, we can store configuration data in EdgeKV, then simply pull it into EdgeWorkers. We can store our test configurations in EdgeKV, which includes the weighting to determine the percentage of users bucketed to each variant. We then include a flag to control if the test logic should be run at all, as well as updated JavaScript code for running a suite of tests.
The end result is a test suite managed in EdgeKV with the flexibility to update and trigger tests via one simple data update. When data is updated in EdgeKV, the new value will be seen globally within 10 seconds — all without the need for any code updates!
A/B testing code
Here’s what the final A/B testing code looks like. Read on for a breakdown of how it works.
import { Cookies, SetCookie } from 'cookies';
import { logger } from 'log';
import { EdgeKV } from './edgekv.js';
// Initialize EdgeKV library
const edgeKv_abpath = new EdgeKV({namespace: "default", group: "ab-data"});
export async function onClientRequest(request) {
// Read config data from EdgeKV
let abConfig = await edgeKv_abpath.getJson({ item: "ab-config"});
// Check the ab-variant cookie to see if user has an existing variant assigned.
let cookies = new Cookies(request.getHeader('Cookie'));
let abVariant = cookies.get('ab-variant');
// If there is no variant assigned, choose a variant at random,
// based on the configuration data in EdgeKV
if (!abVariant) {
logger.log('choosing random variant');
abVariant = getRandomVariant(abConfig)
}
// Add forward header to communicate the variant to the origin server
request.addHeader(abConfig.forwardHeaderName, abVariant)
// Add variant name to the cache key, enabling caching of multiple variants.
request.setVariable('PMUSER_AB_VARIANT', abVariant);
request.cacheKey.includeVariable('PMUSER_AB_VARIANT');
// Log variant to debug headers for debugging purposes
logger.log('Variant: %s', abVariant);
}
export function onClientResponse(request, response) {
// Create or extend ab-variant cookie
let variantId = request.getVariable('PMUSER_AB_VARIANT');
if (variantId) {
let expDate = new Date();
expDate.setDate(expDate.getDate() + 7);
let setBucketCookie = new SetCookie({name: "ab-variant", value: variantId, expires: expDate});
response.addHeader('Set-Cookie', setBucketCookie.toHeader());
}
}
// Select random variante, using a weighted selection based on A/B config data.
function getRandomVariant(abConfig){
let variantsArr = [];
let cumulativeWeight = 0;
for (let variantId in abConfig.variants) {
let variant = abConfig.variants[variantId];
cumulativeWeight += variant.weight;
variantsArr.push({id: variantId, cumulativeWeight: cumulativeWeight});
}
var random = Math.random() * cumulativeWeight;
let chosenVariant;
for (let weightedVariant of variantsArr) {
if (random < weightedVariant.cumulativeWeight) {
chosenVariant = weightedVariant.id;
break;
}
}
return chosenVariant;
}
Example EdgeKV data:
{
"forwardHeaderName": "X-AB-Variant",
"variants": {
"a": {
"weight":25
},
"b": {
"weight": 25
},
"c": {
"weight": 50
},
"enabled": "true"
}
}
A/B testing code, explained
Step 1
import { Cookies, SetCookie } from 'cookies';
import { logger } from 'log';
import { EdgeKV } from './edgekv.js';
First, we import the JavaScript modules we will use. For reference, you can see all the available built-in modules supported by EdgeWorkers here.
Take special note of the imported edgekv.js file. This is a utility library published by Akamai that makes working with EdgeKV easier, and abstracts away complexity. Click here for the full helper library documentation.
Step 2
// Initialize EdgeKV library
const edgeKv_abpath = new EdgeKV({namespace: "default", group: "ab-data"});
Next, we initialize the EdgeKV database. For this example, we’re using the default namespace which is suitable for testing purposes. For more information on how to initialize EdgeKV, click here.
Steps 3 through 7 occur throughout the client request stage. You can learn more about the EdgeWorkers event model here.
Step 3
// Read config data from EdgeKV
let abConfig = await edgeKv_abpath.getJson({ item: "ab-config"});
This code sample reads the A/B test configuration data from our EdgeKV database as JSON data, but it could also be read as text. Both getter functions are documented here.
Step 4
// Check the ab-variant cookie to see if user has an existing variant assigned.
let cookies = new Cookies(request.getHeader('Cookie'));
let abVariant = cookies.get('ab-variant');
// If there is no variant assigned, choose a variant at random,
// based on the configuration data in EdgeKV
if (!abVariant) {
logger.log('choosing random variant');
abVariant = getRandomVariant(abConfig)
}
For our A/B test scenario, we use a cookie in the browser to maintain the variant a user is enrolled in. Our code first obtains the cookie's object from the client request, then gets the specific cookie named "ab-variant" used for this A/B test. If no cookie exists, we randomly assign a variant based on our test configuration stored in EdgeKV. Read more about cookies support in EdgeWorkers.
Step 5
// Add forward header to communicate the variant to the origin server
request.addHeader(abConfig.forwardHeaderName, abVariant)
Now, we add a header to the request. This tells the origin web server which HTML variant to return when one does not already exist in the Akamai cache. Read the tech docs for more details about this process.
Step 6
// Add variant name to the cache key, enabling caching of multiple variants.
request.setVariable('PMUSER_AB_VARIANT', abVariant);
request.cacheKey.includeVariable('PMUSER_AB_VARIANT');
To cache the different variants at the edge, we need to modify the cache key, which makes each variant unique. Per the above code sample, we simply add the variable ‘PMUSER_AB_VARIANT’, which is set and then added to the cache key. This functionality is described in more detail in the document.
Step 7
// Log variant to debug headers for debugging purposes
logger.log('Variant: %s', abVariant);
The final request event step is to log helpful debug data. This code should be removed in production builds. Read about EdgeWorkers documented JavaScript logging and standard debug headers.
Steps 8 through 9 happen during the client response stage. Learn more about the EdgeWorkers event model.
Step 8
let variantId = request.getVariable('PMUSER_AB_VARIANT');
The A/B test logic has already been executed in the client request and corresponding response events, so now we respond with the content corresponding to the user’s bucketed variant. To do so, simply read the previously set property manager variable.
Step 9
let expDate = new Date();
expDate.setDate(expDate.getDate() + 7);
let setBucketCookie = new SetCookie({name: "ab-variant", value: variantId, expires: expDate});
response.addHeader('Set-Cookie', setBucketCookie.toHeader());
Finally, we generate a cookie to track the user’s bucketed variant. This cookie gets added to the response so that for subsequent user requests, we keep them sticky to the A/B test variant. The cookie’s time to live is set to seven days, but you can use any value that makes sense for your A/B test. For more about this process, read the cookie documentation.
Note that no logic is required to send the HTML response, since it happens naturally during the standard EdgeWorkers response/request flow. If the HTML is in the Akamai cache, we serve the response from the Akamai Intelligent Edge. But if we have a cache miss, we instead fetch the content from the origin during the first request, then serve it to the browser and cache it so subsequent requests receive a cache hit.
Improved user experiences start with EdgeKV and EdgeWorkers
By setting the cache key in our code, we’re able to create the functionality needed to place different cached content samples in the same URL, which reduces load times to greatly improve the overall user experience. That’s the power of EdgeKV with EdgeWorkers!
Learn more
If you’re ready to improve your A/B testing processes to offer users the best experiences possible, read the EdgeKV and EdgeWorkers documentation to learn more about what Akamai EdgeKV and Akamai EdgeWorkers can do for your business.
Once you’re up and running with EdgeKV and EdgeWorkers, each code sample discussed above can be packaged and run by following the steps outlined in EdgeWorkers Management application.
You can find the additional step required to load the A/B test scenario into EdgeKV in this CLI documentation.
