Managing Hype Events to Delight Customers, Not Bot Operators
Online commercial events and flash sales have existed for years, but it seems that with more and more business transactions taking place online, especially since the beginning of the COVID-19 pandemic, those events have become more common and intense.
Five to ten years ago, big sales events were happening in brick-and-mortar shops where people would queue for hours to get a chance to lay their hands on the latest fashionable items. For years now, shoe companies have run these hype events successfully online.
And as this year’s New York Fashion Week kicks off — followed by Fashion Weeks in London, Milan, and Paris through February and into March — expect to see the fashion and luxury brands turn their moment in the sun into hype events.
Online sales events mania and scalping
Sneakers resellers closely follow hype events. They have mastered the art of scalping, which consists of grabbing as much of the inventory as possible and reselling it at a profit. Based on this success, some are branching out their businesses to electronics and other items in limited availability.
Due to shortages in the supply chain, in part caused by the COVID-19 pandemic, some items like the latest Sony PlayStation or GPUs used by avid gamers or crypto miners have run in short supply.
Low supplies are an opportunity for scalpers to score a high-profit margin. These professional resellers continuously keep an eye on evolving trends to figure out which items are in high demand, monitor product resupplies on retailer sites, buy the stock when available, and resell them on popular marketplaces like Amazon or eBay (Figure 1).
This practice, which generally involves the use of bots to scale the operation, is typically against most retailer websites' acceptable usage policy but is technically legal. This obviously can significantly impact the price of goods, and the retailer’s reputation, which digital commerce sites need to protect against.
Fig. 1: The art of scalping
This dramatic change in the market dynamics significantly increases the competition where legitimate consumers compete with resellers. Crafty software developers found a new opportunity within this new dynamic by offering bot software to help scalpers acquire their products and help regular consumers compete with scalpers.
In this article, we’ll describe in detail the bot market, what these sales events look like, and what it takes to prepare for a successful event.
The retailer botnet market
Bot software developers who traditionally focused on a handful of sneaker retailers have branched out to offer their services to support other fashion brands, electronic retailers, and even home improvement stores.
Several bot vendors on the market sell software designed to look for specific items on various websites, add them to a cart, and automatically go through the full checkout process (Figure 2).
Fig. 2: Bot vendors sell software that can automatically complete the entire sale
The big fish
There are at least a dozen bot products available on the market designed for scalpers and hype sales events. Their prices vary based on their known success rate in checking out products. The two examples below come up on a regular basis and illustrate the products’ sophistication.
Bot creator “Most Advanced Bot” offers various bot solutions for some of the top retail brands on the internet for prices between $79.99 and $99,999. Sneaker bots, which require more complex software to defeat the advanced bot management products that protect those sites, usually cost the most.
This solution does not require any complex infrastructure to run the botnet since it comes as a Google Chrome extension. Once installed, all the user has to do is provide a description of the item they are looking for; enter their login, payment, and shipping information; enable the bot; and let it do the work. The bot will check for the availability of the items at regular intervals and automatically check them out once found.
AIOBot.com offers a subscription to their software, which is compatible with many retailer websites, for $299 per year. The subscription includes regular software updates to stay effective and keep up with the evolution of bot management products. AIOBot supports proxies to load balance the traffic through multiple IP addresses to reduce the risk that bot management solutions will block the traffic.
It also supports CAPTCHA solving through integration with popular third-party solver offerings, like 2Captcha, Death by Captcha, and Anti-Captcha. The AIOBot site offers plenty of tutorials and advice on how to configure the bot to buy a product successfully and how to become a reseller (scalper).
The little fish
The botnet services mentioned above are the most mainstream and established, but there are plenty of other offerings from independent freelance developers who are ready to develop custom bot solutions. A list of those offerings can be found on fiverr.com.
From the point of view of the defenders, this shows the extent of the adversarial challenge that bot manager product vendors face: Many professional developers produce easy-to-use software designed to defeat detection and provide the best service to their users.
The only way to combat such a threat is to partner with a vendor with many years of experience that has an acute understanding of the internet, the bot problem, and the ability to look at multiple attack vectors at scale to ensure only legitimate users physically engaged on the site are allowed to check out products and put the bot software vendors out of business.
Anatomy of a hype event
A hype event, also sometimes referred to as a high-heat event, can be characterized by a massive change in traffic volume within a short period. The traffic ramps up quickly to at least four times the normal volume for 30 minutes or less. Some of the most intense events may peak at more than 1,000 times the normal traffic volume and ramp up within only a few seconds.
These events get promoted for a specific time and catch the interest of large numbers of people, and some (like scalpers) may leverage bots to get a better chance to grab some of the inventory.
Because the event lasts such a short time, it is challenging to manage from the defender's point of view and requires some preparation. It’s very important for the bot management product to be able to detect the typical attack vectors by default and have an adequate response strategy applied.
During the event, there is no time for a security analyst to review the specificity of the attack, define a custom signature or adjust the detection setting, and deploy the changes to production. These steps take some time even for a seasoned security analyst, and by the time the changes are in place, the event would likely be over.
Sample event: shoe company
Let’s look at a simulated sample event to illustrate its intensity, timeline, and how it affects the traffic pattern on various endpoints. This example reflects the typical bot and human traffic patterns observed before and during a hype real event.
Sneakerheads can’t get enough of the recurring shoe drops from retailers of popular shoe brands to enhance their collections. This is a great opportunity for scalpers to make a big profit when reselling the shoes. Shoe companies and footwear retailers may run several events a week.
Users who wish to acquire the limited-edition item during a launch event must go through a specific workflow. The journey usually starts with the item being announced on social media and on a dedicated page on the site. Once the item becomes available, the user can add it to their cart and complete the checkout process, which may involve logging in, entering the payment information and shipping address, and eventually completing the purchase.
For these hype events, the user must have a valid login. An increase in activity on the login endpoint is typically observed within the hour preceding the event. This corresponds to legitimate users getting ready for the event, and also to some bot operators configuring their system.
Shortly after the limited-edition items are made available for sale, we’ll generally see a spike in login activity — mostly from humans, but also some bots (Figure 3).
Fig. 3: A spike in login activity appears shortly after the limited-edition items go on sale
Some users may have trouble remembering their credentials during login, which could translate to an increase in password reset prior to and during the event.
Once logged in, the user will enter their payment details. The activity coming from bots is particularly important and is a clear indication that legitimate users are also taking advantage of bot products to get a better chance at checking out the items and waiting until the last minute to configure their bot. Scalpers would typically have this step already configured in their system.
The bot traffic represented in red sometimes starts earlier and lasts a while longer than the event itself. Figure 4 shows a distinct increase in activity preceding the event.
Fig. 4: Increases in bot configuration activity preceding the events, as seen on the payment details endpoint
Finally, as soon as the event starts, we see a significant increase in activity on the checkout endpoint (Figure 5). The activity at the checkout from bots is generally significant. All users may have to go through a waiting-room process, and only users who have successfully completed the workflow while there is still some inventory available will access the final checkout step. The “wind down” period may vary depending on many factors, including how quickly the inventory sells out. In some cases, the inventory sells out within minutes.
Fig. 5: A significant increase in activity on the checkout endpoint
The bot activity often goes beyond the event itself. Many attackers create new accounts in advance when retailers require users to have a valid login to buy the limited-edition item. This commonly translates to recurring excessive bot activity on the account creation endpoint. Figure 6 shows an excessive bot activity that subsides before the event. Attackers may also create these accounts days or weeks before the events, let them age, and enroll them into exclusive groups as needed to get a better chance to successfully check out an item during the event.
Fig 6: Excessive bot activity that subsides before the event
Protecting each step of the workflow is essential to have as many opportunities as possible to screen the traffic to differentiate bots from humans. Bots detected at each step should be mitigated in the most efficient manner. For example, on the page where the announcement is made, bot traffic can be simply denied — if they can’t see the item, they can’t buy it. Preventing bots from creating new accounts will also deny them the right to participate in the event.
At the login stage, additional verifications of the user identity may help. Although at this stage, it is fair to assume that bot operators are more likely to be already logged in to access the checkout stage quicker.
The recipe for a successful user experience
Just like any major sales event at a regular store, online events must be prepared properly. If the event were to occur in a regular store, the manager would typically increase its sales and security staff to ensure that there is no trouble and every consumer gets a fair chance at buying the item on sale. Extra security staff or police may also monitor the line that forms in front of the store before the store opens.
Online events should be managed the same way — only the tools and methods are different.
5 steps to properly prepare for your event
Work with your professional services team
Protect each step of the workflow with a bot manager
Adopt a strong response strategy
Adopt the challenge response strategy
Ensure your origin is able to support the load
Work with your professional services team
Get in touch with your team ahead of the event to ensure your bot managing configuration is tuned to get the best detection accuracy. A security professional should also watch the activity during the event and do an analysis once it’s over to assess the effectiveness of the solution in place.
Bot attack methods and bot detection evolve continuously. The lessons learned from previous events targeting the digital commerce industry, in general, can help with tuning the setup to improve your defense strategy and ensure the success of future events.
Protect each step of the workflow with a bot manager
This should include at least the endpoints that handle account creation, login, password reset, add-to-card, and checkout. The page where the item for sale will be announced should also be protected. Bot operators may create multiple new accounts ahead of time and use them during the event.
It is especially important to protect the account creation endpoint if the event is only available to exclusive club members. Preventing the mass creation of new accounts will reduce the risk of attackers infiltrating your premium user channel and disturbing the event.
Adopt a strong response strategy
Work with your team to ensure adequate response strategies are in place to defeat the bot activity. When bots are detected, don’t let the traffic unmitigated or unchallenged. An inadequate response strategy will give bots an unfair advantage compared with legitimate users.
Adopt the challenge response strategy
A crypto or CAPTCHA challenge will alter the workflow for suspicious traffic and provide a second opportunity to evaluate traffic that appears suspicious. As described earlier, some bots run on browsers as an add-on, so enforcing a CAPTCHA in one of the steps of the workflow will break the automation workflow and require the user to be present to complete the transaction.
Because some of the bot solutions do support the ability to offload solving of the CAPTCHA puzzle to third parties, you may also consider applying the crypto challenge to one of the steps of the workflow for diversity. Even if the bot is able to solve one type of challenge, it is unlikely to be able to solve both, thus significantly affecting the effectiveness of the botnet.
It is preferable to enable the challenge only minutes before the event starts to catch the bot operators off guard and turn it off immediately after the event is over to deny them the opportunity to learn and update their bot. Remember that only bots will be challenged; legitimate users will not be affected.
Ensure your origin is able to support the load
The traffic generated by these events can be overwhelming for the origin infrastructure. Be sure to add enough capacity to your infrastructure to support the additional load.
Akamai is working on a new solution that combines the power of the Visitor Prioritization Cloudlet and Bot Manager Premier to ensure the traffic can be fully managed and contained at the edge and only requests from legitimate users reach the origin until the inventory has sold out. Stay tuned for further announcements on this topic later in 2023.
Conclusion
Running a successful hype sales event online can be just as challenging as running similar events in a regular store. It requires preparation and finesse to keep unwanted visitors at bay and allow your most loyal customers to access the premium content.
The advanced features of Akamai Bot Manager and Account Protector products can help make your event a success. Call your Akamai representative to discuss how to make sure you are prepared for your next hype event.
