Forum Discussion

CarloS1's avatar
CarloS1
Icon for Nimbostratus rankNimbostratus
Oct 05, 2018

0% space on /var partition, a lots of files: storage[numbers].zip

Hi, I have a problem with storage on my partition /var. I got a info from my log that there isn't free space. After my investigation I found that there is a lot of file in directory /var/config/rest name storage[numbers].zip:

 

[root@:Active:In Sync] rest ls -alt | more

 

total 1926164

 

-rw-r--r--. 1 root root 1592770 2018-10-03 06:29 storage1538540986.zip

 

-rw-r--r--. 1 root root 1596945 2018-10-03 06:27 storage1538540866.zip

 

-rw-r--r--. 1 root root 1590046 2018-10-03 06:25 storage1538540746.zip

 

-rw-r--r--. 1 root root 1596580 2018-10-03 06:23 storage1538540626.zip

 

.....

 

I cant found information about this files. What they are for? Do I need them? Can I delete them? What processs is responsilbe for create them? Is something crushed? How can i fix it?

 

I will be very gracefull for help.

 

6 Replies

  • I have checked my own system but I do not have that. What modules are you running on your system? As you say, google searches come up short regarding this. But it seems that it is related to iControl REST. Are you using REST today?

     

    If you do not find the answers here at DevCentral I'd open up a case with the F5 Support. They will surely know the answer to it.

     

  • Carlos,

     

    Our system is experiencing the problem as you described right now as well. (you are not alone) :)

     

    With /var at 100%, we are being spammed with the following message:

     

    2018 Oct 18 11:01:11 slot1/F5-Lab logger: Re-starting fslogd

     

    S/W ver: 13.1.1 Build 0.23.4 Engineering Hotfix Platform: VIPRION 2400 (4 2250 blades) Modules: LTM, APM, AVR, AFM, SWG (all Nominal), FPGA f/w enabled

     

    I also couldn't find out any information on DevCentral. I'll open a ticket with F5 to see if there is a configuration issue/bug/unknown/fix.

     

    All the best!

     

  • First of all I want to say I use viprion technology with vCMP and HA architecture. In the beginning I deleted all files storage. After that I restarted vGuest from CLI. Despite this some of storege files was created after restart. Because this is production environment and I haven't response yet from TAC f5. I couldn't wait. I switched activity HA from active to standby. After failover the issue in active slot was not notice. On standby (active before switch where was issue) no files storage* created.

     

    Now the suggestion f5 is to switch activity to slot where was the issue and check if the problem is still exist. If still exist, f5 sugested to do this steps:

     

    " If both devices are doing the same thing, but only one is failing we would suggest trying the following (partially already done) :

     

    1. bigstart stop restjavad
    2. rm -rf /var/config/rest/storage*.zip
    3. rm -rf /var/config/rest/*.tmp
    4. bigstart start restjavad
    5. Monitor to see if the .zip files are being created now
    6. if the files are not being created ( should not if we understand correctly the current state ), try failing over to see if under traffic they are being generated "

    Other questions from f5: "May I ask if your devices are managed by BIG-IQ?"

     

    My answer: no

     

    "Have you introduced recently some REST commands/AVR config to look at the objects in : /mgmt/tm/analytics/dos-vis-common/ /mgmt/tm/analytics/dos-vis-attacks/"

     

    My answer: how should I know this?

     

  • Solution for this issue is to delete files and reboot system or process.

     

    f5 suggest restart process:

     

    1. bigstart stop restjavad
    2. rm -rf /var/config/rest/storage*.zip
    3. rm -rf /var/config/rest/*.tmp
    4. bigstart start restjavad
    5. Monitor to see if the .zip files are being created now
    6. if the files are not being created ( should not if we understand correctly the current state ), try failing over to see if under traffic they are being generated

    I rebooted the system:

     

    1. rm -rf /var/config/rest/storage*.zip

       

    2. rm -rf /var/config/rest/*.tmp

       

    3. reboot - command from CLI

       

    In my case reboot helped. I didn't test f5 procedure.

     

    I don't know what caused this issue.