302 Redirect when using Googlebot Agent

[csdougliss]

http://www.redirect-checker.org/index.php

Checking our site as a user, no 302 re-direct
Checking our site using GOOGLEBOT, 302 re-direct.

The only option we have found is to set Validate HTTP_USER_AGENT" to "No" which is a security related feature of Magento we would rather not switch off.

This prevents the 302 re-direct.

Any suggestions?

Same issue here: #881

[csdougliss]

Can confirm this issue does not appear when varnish/turpentine is not used on a separate magento site.

[csdougliss]

Here is my work around if anyone else needs it:

app/code/local/Mage/Core/Model/Session/Abstract/Varien.php (copied from app/code/core) around line 473:

protected function _validate()
        if ($this->useValidateHttpUserAgent()
            && $sessionData[self::VALIDATOR_HTTP_USER_AGENT_KEY] !=                                     
           $validatorData[self::VALIDATOR_HTTP_USER_AGENT_KEY]
        ) {

// don't verify if crawler agent
            // e.g. Googlebot/2.1 (+http://www.googlebot.com/bot.html)
            $turpentineCrawlerAgents = Mage::helper('turpentine/data')->cleanExplode(',',
                    Mage::getStoreConfig(
                        'turpentine_vcl/backend/crawler_user_agents' ));
            if($turpentineCrawlerAgents && is_array($turpentineCrawlerAgents)) {
                foreach ($turpentineCrawlerAgents as $agent) {
                    if (preg_match('/' . $agent . '/iu', $validatorData[self::VALIDATOR_HTTP_USER_AGENT_KEY])) {
                        return true;
                    }
                }
            }

[axmue]

We have the same issue in one of our shops.
Environment:
nginx (https) -> varnish (4.1) -> apache2 (php5-fpm) Magento 1.9.x, Turpenine 0.7.4

Setting "HTTP_USER_AGENT" to "NO" doesn't help, neither does adding the above code to Varien.php.

Still searching.

[knambiar]

I'm still facing this issue on my magento 2 site https://rootedobjects.com. Has anyone found a solution?

[ChrisOrlando]

@knambiar your issue is unrelated to Turpentine if you are having it on a Magento 2 site. The Turpentine module does not work with Magento 2.