Dating back to the early days of the Internet, firewalls were part of the foundation upon which modern web app security was built. Web application firewalls (WAFs) are at the center of many organizations’ security infrastructures because of their capabilities in blocking malicious HTTP traffic.
There’s just one problem: when it comes to bot traffic in particular, WAFs just aren’t sufficient. The sophisticated attack techniques of bad bots have far outpaced any incremental improvements in WAF bot management technology. Cracks are starting to show, and organizations have a choice: they can either incrementally patch the holes and hope bots don’t get through, or adopt a purpose-built, modern solution that is up to the task.
What WAF Bot Management Can and Can’t Do
First, let’s back up and talk about how WAFs work. A WAF is a type of reverse-proxy server that acts as a shield between a web application and the Internet. User requests must pass through the WAF before hitting your server. Using preset policies, the WAF filters out malicious traffic from legitimate traffic, preventing successful attacks against your site.
WAFs are good at protecting you from familiar threats, such as cross-site scripting, SQL injection, buffer overflow and DDoS attacks. Where they falter is in recognizing unknown bot threats in real-time. They cannot recognize bots that piggyback on the identities of real humans and mimic their behavior. Nor can they identify botnets that rotate through thousands of different IP addresses to bypass IP-based rules.
Today’s bots are highly distributed, don’t carry attack signatures and target flaws in your site logic rather than known vulnerabilities. They are extremely complex, continuously evolving and re-tooling to get around WAF policies. WAF bot management capabilities are merely patches on top of a larger security system, and WAFs don’t learn in real-time. Bots, on the other hand, are evolving in real time to evade security systems. Binary, pre-set rules just can’t keep up.
Where WAFs, CAPTCHAs and MFA Fall Short
WAFs were built to answer one simple question: is this traffic on the blocklist or not? The question of whether it’s a malicious bot or not goes unanswered. However, knowing whether or not traffic is bot-driven is actually quite relevant to today’s online businesses. And that disconnect is at the core of why WAF bot management isn’t enough.
Even if you were to supplement your WAF bot management with a CAPTCHA test, questions regarding accuracy remain. Bots have gotten quite good at solving CAPTCHAs, rendering them ineffective at weeding out bad actors. Furthermore, these challenges have become harder for humans to solve. They add friction to the user experience and decrease conversion rates.
Similarly, multi-factor authentication (MFA) also adds frustration to the user experience, and many organizations do not require it for that reason. While MFA may reduce the likelihood of an account breach, it doesn't actually block bot traffic. Bots still flood login and authentication pages, raising your risk of a breach and increasing security costs. In addition, MFA leads to locked accounts, causing customers to frequently contact support.
This top regional bank experienced this firsthand. The company discovered that its WAF and MFA solutions could not keep up with the influx of automated bot attacks. “Almost 70% of the requests to our login and authentication pages were coming from malicious bots,” the CISO stated.
The bank implemented a purpose-built bot management solution. Since then, it has been “able to precisely detect and block even sophisticated bots that emulated human behavior, bringing the false positive rate below 0.01%.”
Shifting the Burden of Proof
WAFs are a critical part of any security infrastructure, but they can’t fight bots on their own. Today’s digital businesses need a tool that will help them accurately answer the fundamental question: is this a bad bot or not? When it comes to identifying bots, behavioral analysis with machine learning is the best method — and only purpose-built bot management solutions can deliver.
Whereas WAFs, CAPTCHAs and MFA are dependent on user input to execute on pre-programmed commands, modern bot management solutions leverage machine learning and behavioral analysis to detect automated threats in real time. According to Gartner, “[With a bot management solution], the burden to ‘prove humanity’ is removed from the customer and placed where it belongs: in the domain of the technology intended to support the integrity of a web or mobile property.”
And, not surprisingly, I’ve got a system in mind. HUMAN Bot Defender offers exceptional accuracy, platform agnostic architecture and a customer-centric approach to bot management that accurately detects bots on your web and mobile applications and API endpoints in real time. Laser-focused on the user journey, the solution offers an alternative captcha solution that provides a great customer experience.
Bot Defender integrates with your WAF, MFA, CDN and edge infrastructure, so you will see immediate impact without having to rip and replace your existing security infrastructure. The solution continuously learns and evolves using a combination of intelligent fingerprinting, behavioral signals and predictive analytics to protect your business and consumers from automated bot attacks in real time.
