Forum Discussion

SivaYenduri's avatar
SivaYenduri
Icon for Cirrus rankCirrus
Feb 28, 2020
Solved

Troubleshooting traffic spike on loadbalancer

Hello All,

 

Can someone help me how can i troubleshoot this issue?

 

Customers are accessing a VIP:8080 and there are around 2k connections on the VIP.

But at 10:00AM CET everyday, connections are going form 2k to 5k+ for 10mins and then going to back to normal.

Traffic flow is as below

Intranet > DC WAN router > Firewall > router > Loadbalancer > Backend servers.

 

Please help me how to identify this sudden spike in traffic.

 

application delivery
BIG-IP
LTM
  • rafaelbn's avatar
    rafaelbn
    Feb 28, 2020

    Hello Siva!

    Do you happen to know the source IP of all those connections?

    I would start by looking at the connection table during that spike. Something like this...

    tmsh show sys connection cs-server-addr <VIP> cs-server-port 8080

    If everything comes from the same IP it could possibly narrow your tshoot down to a specific client.

    Best of luck!

3 Replies

Sort By
  • rafaelbn's avatar
    rafaelbn
    Icon for Cirrostratus rankCirrostratus
    Feb 28, 2020

    Hello Siva!

    Do you happen to know the source IP of all those connections?

    I would start by looking at the connection table during that spike. Something like this...

    tmsh show sys connection cs-server-addr <VIP> cs-server-port 8080

    If everything comes from the same IP it could possibly narrow your tshoot down to a specific client.

    Best of luck!

    • SivaYenduri's avatar
      SivaYenduri
      Icon for Cirrus rankCirrus
      Mar 01, 2020

      Thank you for the reply. Unfortunately source is entire corp network.. But this command is helpful.

  • F5SJ_'s avatar
    F5SJ_
    Icon for Altocumulus rankAltocumulus
    Feb 28, 2020

    Please run bigtop -n from bash, it will show you VIPs with highest number of connections. Then please check the timeout value of tcp profile assigned to the VIPs. I faced the same issue and I reduced the timeout value of tcp profile and I saw that all long running connections deleted from the connection table.