Forum Discussion

Anwar_csengg_25's avatar
Anwar_csengg_25
Icon for Nimbostratus rankNimbostratus
Mar 20, 2016

fix needed for ASM Web scraping attack

Dear Folks,

 

Here I am explaining the scenario what we are facing in our environment.

 

When user perform any activity in the applications, then Request is not going to the server. Requests has been caught in F5 and shows the Web scrapping attack.

 

Example: In F5, Requests attacks occuring here ->> /teamworks/ajaxCoach.

 

As this is local development environment, We need to customize the F5 filter for this issue. Please help to fine tune the rule for this kind of issues in order to get the response back without attacks.

 

Thanks, Anwar

 

ASM Advanced WAF
devops
security

3 Replies

Sort By
  • John_Buchanan's avatar
    John_Buchanan
    Icon for Nimbostratus rankNimbostratus
    Mar 21, 2016

    Does the URI follow a pattern of /teamworks/*? Have you added that as an allowed URL? Also could you share the details of the violation event? What does the User-Agent string look like? Are these requests coming in programmatically as the result of automated testing? Have you adjusted the thresholds for Web-Scraping detection, increased values for Bot Detection, Session Opening, and Session Transactions Anomaly?

     

  • Anwar_csengg_25's avatar
    Anwar_csengg_25
    Icon for Nimbostratus rankNimbostratus
    Mar 22, 2016

    Hi John buchanan,

     

    Thanks for your answer.

     

    I'm very new to this task

     

    Please find my answer marked with bold for your questions

     

    Does the URI follow a pattern of /teamworks/*? Have you added that as an allowed URL?

     

    No (is this the way to allow ?Please suggest me i will do accordingly)

     

    Also could you share the details of the violation event?

     

    Yes. Please see the attachment

     

    What does the User-Agent string look like?

     

    Are these requests coming in pro grammatically as the result of automated testing?

     

    Request are coming through the development application

     

    Have you adjusted the thresholds for Web-Scraping detection, increased values for Bot Detection, Session Opening, and Session Transactions Anomaly?

     

    No.These all item values are set by default values.And Please advise me what i have to adjust or Increase the value for Bot Detection, Session Opening, and Session Transactions Anomaly

     

    Thanks, Anwar

     

  • Khay_164420's avatar
    Khay_164420
    Historic F5 Account
    Mar 22, 2016

    Hi Anwar,

     

    I would strongly suggest you to first read this SOL: Web Scraping in 12.0 just to understand what Web scraping does and how the different features (bot detection, session opening, etc.) work.

     

    Then, there are several ways to deal with these false positives:

     

    • Adjust the thresholds in the Web Scraping configuration
    • Whitelist the IP addresses in the Web Scraping configuration if there are well known IP addresses
    • Remove the blocking flag for this feature (the security policy will be weakened then)