Forum Discussion

JB_41040's avatar
JB_41040
Icon for Nimbostratus rankNimbostratus
Sep 02, 2011

WebAccelerator causes problem for crawlers?

Has anyone else encountered this problem? we launched the BigIP WAM 3900 system in Mid july and have had significant drop in traffic - almost 30-40% and google webmaster tools indicates drop in google crawler activity as well. Would the IBR/WAM cause an issue for crawlers while helping browser based request?

 

 

any thoughts/ideas would be appreciated..

 

 

Thanks

 

JB

 

application delivery
performance
wa
  • Johnny_Schmidt_'s avatar
    Johnny_Schmidt_
    Historic F5 Account
    Dec 29, 2011
    You ought to be able to see if those crawlers are hitting your site at all by user agent string. That's where I'd start looking first for an answer. Create a request-logging profile (for v11) or enable logging in WAM (v10) to start collecting that data.

     

     

    Cheers,

     

    -dude