Incident
Since yesterday there is an incident with google recaptcha, it is sending an api request from the browser to a relative url at /recaptcha/api2/clr instead of the normal google url and returns some kind of binary encrypted data and a 404 error may occur.
This behavior is caught by mod_security with OWASP3 920420 rule and causes firewalls like csf to block legitimate users trying to use their websites.
While there was an incident at:
https://status.cloud.google.com/incidents/MzARofVtutSd2HB5vmkT
It was for enterprise only, while all tiers of recaptcha service were affected.
My guess is that a global update to their framework regarding their new pricing policies (goodby free recaptcha) messed up everything. Nice work google!
Mitigation
For users with apache servers and mod_security you just have to add
<LocationMatch "/recaptcha/api2/clr">
SecRuleRemoveById 949110 920420
</LocationMatch>
To your global apache conf so these to rules do not fire up for this bogus url.
Restart apache and check your firewall if it still blocks users but it shouldn't.
For users with CSF firewall, find the ips and manually unblock them after making the above change.
For example, I used an one-liner to find IP addresses from Greece where my clients are located and its result was redirected to a file lets call it gr_mods_falsep_ips.txt
cat /etc/csf.deny | grep GR | awk '{print $9}' | grep -v more > /etc/csf/gr_mods_falsep_ips.txt
Then with a small bash script I unblocked them:
#!/bin/bash
if [ $# -eq 0 ]
then
echo "No filename provided"
exit 1
fi
ips="$1"
while read -r line; do
/sbin/csf -dr $line
sleep 3
done < "$ips"
