Bots Compose 42% of Overall Web Traffic; Nearly Two-Thirds Are Malicious

Akamai Technologies, Inc. (NASDAQ: AKAM), the cloud company that powers and protects life online, today released a new State of the Internet (SOTI) report that details the security and business threats that organizations face with the proliferation of web scraping bots. Scraping Away Your Bottom Line: How Web Scrapers Impact Ecommerce finds that bots compose 42% of overall web traffic, and 65% of these bots are malicious.

With its reliance on revenue-generating web applications, the ecommerce sector has been most affected by high-risk bot traffic. Although some bots are beneficial to business, web scraper bots are being used for competitive intelligence and espionage, inventory hoarding, imposter site creation, and other schemes that have a negative impact on both the bottom line and the customer experience. There are no existing laws that prohibit the use of scraper bots, and they are hard to detect due to the rise of artificial intelligence (AI) botnets, but there are some things companies can do to mitigate them.

“Bots continue to present massive challenges resulting in multiple pain points for app and API owners,” said Patrick Sullivan, CTO, Security Strategy at Akamai.  “This includes scraping that can steal web data and produce brand impersonation sites. The scraper landscape is also changing due to advancements like headless browser technology, requiring organizations to take an approach to managing this type of bot activity that is more sophisticated than other JavaScript-based mitigations.”

Key findings from the report include:

• AI botnets have the ability to discover and scrape unstructured data and content that is in a less consistent format or location. Additionally, they can use actual business intelligence to enhance the decision-making process through collecting, extracting, and then processing data.
• Scraper bots can be leveraged to generate more sophisticated phishing campaigns by grabbing product images, descriptions, and pricing information to create counterfeit storefronts or phishing sites aimed at stealing credentials or credit card information.
• Bots can be used to facilitate new account opening abuse — which, according to recent research, composes up to 50% of fraud losses.
• Technical impacts that organizations face as a result of being scraped, whether the scraping was done with malicious or beneficial intentions, include website performance degradation, site metric pollution, compromised credentials attacks from phishing sites, increased compute costs, and more.

The Scraping Away Your Bottom Line research report offers mitigation strategies against scraper bots and features a case study that shows how websites operate much faster and efficiently once defenses against these bots are put into place. In addition, the research addresses compliance considerations that must be taken into account in light of these increasing attacks.

This year marks the 10th anniversary of Akamai’s State of the Internet (SOTI) reports. The SOTI series provides expert insights on the cybersecurity and web performance landscapes, based on data gathered from Akamai Connected Cloud.