Global AI Platform Elevates User Experience with Arkose Labs Collaboration

Summary

A global AI research and deployment company was facing an unprecedented volume and range of cyberattacks, including LLM platform abuse, SMS toll fraud, account takeover (ATO), new account fraud and advanced phishing attacks. These attacks were completely exhausting its processing capacity, costing the company tens of millions of dollars each month and leaving good users unable to leverage its service while bots ran rampant. Additionally, the company needed a way to ensure its services weren’t accessible in countries where the platform was prohibited. The company partnered with Arkose Labs to detect, mitigate and deter these automated attacks, without requiring additional internal resources. This led to a substantial reduction in malicious activity, freeing up processing capacity, protecting the tech trailblazer’s operational costs, and improving the experience of legitimate users.

The Business Problem

As one of the leaders in the burgeoning LLM platform market, the company faces complex, unique and ongoing challenges. With millions and millions of daily prompts created on its platform, the estimated 4-5% percent of that traffic that was suspicious posed significant financial risks and disrupted the legitimate user experience. In an attack type coined “LLM platform abuse,” bad actors were proxying the company’s premier model and, profiting from this data, circumventing API fees completely – all while putting a massive strain on platform resources, to the point where global processing units (GPUs) were overrun in the midst of a global GPU shortage.

In addition, attackers were selling subscriptions to consumers, proxying the service out to other users, using fake credit cards, and selling market servicesin countries where the company is not allowed to operate. The incumbent attack-prevention solution lacked automated mitigation, resulting in the company investing significant time and effort to tackle these issues.

In response to this threat, the AI company sought a true partner with around-the-clock resources that not only thwarted automated abuse but also upheld a positive user experience for its genuine, paying customers.

The Arkose Labs Solution

Arkose Labs, which includes the Arkose Cyber Threat Intelligence Research unit (ACTIR) and Security Operations Center (SOC), brings a full-fledged approach to attack mitigation, all the while smoothing the path for good user throughput. Arkose Labs collaborated with the global AI company’s security engineering, infrastructure management and fraud prevention teams to tailor the Arkose Bot Manager solution, which includes capabilities like Arkose GPT Protect, to safeguard the company’s platform against multiple attack vectors. The Arkose Labs team configured its technology to align seamlessly with the company’s requirements, while proactively researching attacker GitHub repositories and the company’s Discord channels to anticipate real-time countermeasures.

The initial focus was on the registration flow, where Arkose Bot Manager rapidly curtailed fake account sign-ups and reduced the impacts of SMS toll fraud (aka IRSF, SMS pumping, etc.). As soon as that flow became more difficult for attackers to target, they doubled their efforts against the company’s premier chat prompts via existing accounts.

During this time, Arkose Labs was being deployed and quickly implemented mitigation defenses against these sophisticated attacks, including Arkose Labs’ token enforcement and Arkose MatchKey challenges, a robust CAPTCHA-based defense. Bots faced difficulties with Arkose MatchKey challenges due to the increased cost to attack them, prompting attackers to resort to human fraud farms.

As the cost to conduct attacks on this flow increased, the attacks shifted to other parts of the company’s business. The company and Arkose Labs teams then pivoted to safeguarding additional flows – including login, forgot password, a developer portal and profile update – in a combined effort to thwart attackers in real-time. Given the extremely valuable nature of the company’s services, further advanced features available from Arkose Labs were implemented, including Arkose Email Intelligence, which evaluates the risk associated with an email address and provides a risk score, suggests actions to be taken, and indicates additional signals to combat fraud.

Arkose Labs also worked with the company to detect and shut down advanced phishing attacks. In these attacks, different sites emulate the company’s real chat login page, with the only noticeable difference typically being the URL, which often goes unnoticed by legitimate users. When telltales indicated traffic passing through phishing sites, mitigation consisted of a customized message that warned users about the site’s authenticity.

This approach proved to be so effective against the cybercriminals that the bad actors gave up and shut down their repositories within weeks. Meanwhile, legitimate users experienced no friction whatsoever. In addition, Arkose Labs has partnered closely with the company to understand and evaluate the capabilities of large language models (LLMs), especially multimodal models and their potential use to evade detection and solve CAPTCHA-type challenges. This ongoing process, which establishes guardrails to ensure that attackers are not able to use the LLMs to evade or solve Arkose MatchKey challenges, includes:

• Evaluating the cognitive capabilities of multimodal LLMs to understand and describe Arkose Labs challenges.
• Analyzing and understanding the general behaviors and approaches that multimodal LLMs take to solve image questions.
• Incorporating the learnings from various experiments into the Arkose Labs CAPTCHA design process.

Demonstrated Results

The AI platform company saw a decrease in platform and prompt abuse, advanced phishing attacks, fake accounts, SMS toll fraud, account takeovers and other attacks – protecting the company’s income and their good customer experience. The Arkose Lab solution’s effectiveness was clearly evident when the company attempted to assess its impact by temporarily deactivating it. Volumetric attacks overwhelmed the company less than 24 hours after deactivation, and the solution was quickly reinstated.