Need cloud computing? Get started now

Dark background with blue code overlay

Blog

Visualize and Analyze Bots With Real User Monitoring

Hitoshi Kaneko

Written by

Hitoshi Kaneko

May 12, 2021

Hitoshi Kaneko

Written by

Hitoshi Kaneko

Hitoshi Kaneko is a Senior Technical Solutions Architect within the Compute and Delivery Division at Akamai. His primary focus is helping customers develop computing strategies and improve website performance.

You interact with more than just customers on your website. Bots make up a disproportionate amount of traffic, which can skew human behavior analysis data — and cause you to make the wrong business decisions. In fact, one Akamai customer with 30% of total traffic from bots noticed a huge gap in business metrics like bounce and conversion rates. Since human and bot behaviors are unique, you need to analyze website performance to see how both impact your organization separately.

With deep expertise in bot traffic, Akamai Bot Manager can visualize 485 million bot requests per hour and 280 million bot logins a day. Akamai mPulse, a real user monitoring solution, can take advantage of Bot Manager technology to improve your bot detection and classification capabilities. With mPulse, your website performance and business traffic can be easily segmented into humans and bots, so you can monitor the impact of bots on real user and business metrics. 

Getting started with mPulse

In the Beacons tab of the mPulse app configuration, there is a checkbox to enable the collection of bot beacons (Include Bot Beacons). If you want to collect synthetic monitoring beacons, also check the Include Synthetic Monitoring Beacons option.

mPulse

Even with these checkboxes enabled, you may not see some beacons on your dashboard. That’s because mPulse beacons can only be collected from clients that run the mPulse JavaScript API correctly.

Human, bot, and combined insights

The mPulse data shown below is separated into humans and bots. You can see a 75% bounce rate for humans compared with a 54% bounce rate for bots. However, if we only look at the combined data, we see a 66% bounce rate. When it comes to the largest contentful paint, an important user-centric metric for measuring perceived load speed, the bots experienced load speeds that were 5.5 seconds slower than the humans’ load speeds. From this example, it’s easy to see that bots skew human data.

Human, bot, and combined insights

Advanced bot monitoring

Users without Bot Manager can separate human traffic with mPulse, as described above. For users with Bot Manager, mPulse integration can provide even more powerful information, including detailed bot types and expanded bot categories.

Akamai Bot Manager Bot Type
Analysis

Data specialists are here to help

With the help of a data specialist, one mPulse customer was able to look at the top five URLs visited by bots and figure out why its digital commerce site saw a higher conversion (cart completed) rate with bots. As it turns out, bots were trying to purchase products with limited availability in order to resell them at a much higher price.

  URL Page Views
1 https://www.example.net/products/AKAMAI0031 12031
2 https://www.example.net/products/AKAMAI0078 574
3 https://www.example.net/products/AKAMAI0128 438
4 https://www.example.net/products/AKAMAI0005 201
5 https://www.example.net/products/AKAMAI0280 98

Understand real human behavior on your website

Bots are everywhere. They might crawl your website to get information. They might look for security vulnerabilities on your website. Or they might want to buy your products to resell them. Find out how mPulse can visualize human and bot behaviors separately to help you better understand who’s coming to your website and what they’re trying to do. Start your free trial today



Hitoshi Kaneko

Written by

Hitoshi Kaneko

May 12, 2021

Hitoshi Kaneko

Written by

Hitoshi Kaneko

Hitoshi Kaneko is a Senior Technical Solutions Architect within the Compute and Delivery Division at Akamai. His primary focus is helping customers develop computing strategies and improve website performance.