Avoiding Google Captcha and Geolocation Issues
Zscaler has built a multi-tenant global security cloud that provides secure Internet services to our customers, who have globally distributed workforce and offices. The Zscaler platform enables our customers to avoid the difficulties involved in deploying and managing security hardware, while providing secure access Internet access.
Zscaler also provides SSL inspection, which allows customers to scan encrypted content within SSL enabled sites.
Google search is the most preferred search engine in the world with more than 4 billion web searches performed every day. Google also provides a suite of productivity apps, ‘Google Apps for Business’ along with tons of other consumer tools, such as Maps and Gmail. By default, the Google search home page and other Google Enterprise services are accessed over an SSL connection.
Sporadically, Zscaler customers have reported the following issues while accessing Google Search services through the Zscaler platform:
- Captcha on Google.com: When a user types a search string in Google.com, instead of displaying search results, users are subject to a Captcha screen that they must resolve before the search results are displayed.
- Geolocation error: The google.com home page and search results displayed do not match with user’s location. For example: A user in Miami may be shown search results in Spanish and the Google home page is https://google.com.mx
What is Zscaler doing about it?
Zscaler has taken multiple steps to mitigate the aforementioned issues:
- Google and Zscaler are in constant communication regarding Zscaler’s multi-tenant data center footprint and IP address ranges.
- The Zscaler Operations team has automated the monitoring for Google Captcha and geolocation issues.
- Zscaler has an agreement with Google to use XFF headers to isolate offending Zscaler clients.
Google Captcha Issues
Google displays a Captcha screen when it needs to verify that a search query is being performed by an actual user and not by an automated bot machine or programmatically with malicious intent. Google has built a proprietary algorithm to detect such behavior. While Zscaler forbids the use of its proxy services for automated testing as part of its contract, some customers doing automated queries on Google trigger the search engine to respond with a Captcha. This can affect other users sending traffic through the same Zscaler IP address.
Zscaler recommends that customers enable SSL inspection, so Zscaler can insert an XFF header in each request. Inserting an XFF header allows Google to isolate the offending client, without affecting the traffic of other clients.
In addition to working with Google, Zscaler has also added monitoring capabilities to detect Google Captcha on its ZEN dynamically where a signature pattern has been added to detect Google Captcha incidents and the Zscaler Operations team is alerted in real-time when a Captcha incident happens. When a Captcha is detected, Zscaler may add IP addresses to the affected ZEN or rotate IP addresses to reduce the likelihood of this issue.
Geolocation errors may occur when the Zscaler data center IP address ranges are incorrectly interpreted by Google’s Geo-IP mapping system. This may lead to an IP address being incorrectly mapped to a different region. For example, an IP address in Miami may map to an IP address in Mexico City.
To resolve this issue, Zscaler has shared its IP ranges with Google and barring a rare corruption of the Geo-IP system, this issue is fixed.
Also, in certain regions, geolocation errors may be attributed to the fact that Zscaler customers are sending data to Zscaler from countries where Zscaler doesn't have a data center. In this case, Google returns content based on the IP address of the Zscaler data center.
To resolve this issue, Zscaler recommends that customers turn on SSL and enable XFF forwarding, as stated earlier. This allows Google to read a customer’s actual IP address along with the Zscaler data center IP address and return geolocalized content. Customers who cannot enable SSL and XFF forwarding may run into this issue.
What can I do to resolve these issues?
Zscaler recommends enabling SSL inspection, so Zscaler can insert XFF headers in each request sent to Google servers, to mitigate Captcha as well as geolocation errors.
Additionally, to avoid as geolocation errors, you may ask your users to use Google’s NCR plugin for the Chrome browser to bookmark google.com/ncr to redirect your traffic to google.com instead of a local, geolocalized Google page.
NOTE: Please note that suggestions in this tech note reduce the likelihood that these issues will occur, but they may not completely resolve them. Zscaler will continue to work with Google to resolve these issues permanently and will update this tech note accordingly.