f5 BIG-IP LTM VE v11.4.0 ESXi VM allocated with 200 gb hdd , 16 gb ram System > File Management > Data Group File List > Import > point to 3M-record data-file on disk ( 216 MB ) receive error : " File upload failed, please check log file for details " I review all logs but don't see any entries related to data-file load. System > File Management > iFile List > Import > point to the same 3M-record file I receive the same error. Examining /var/log/ltm I see : err diskmonitor: 011d0004: Disk partition _root_ has only 0% free but the timestamp of this error does not coincide with my attempted file uploads. I assume _root_ refers to the Linux disk partition -- why would I receive an error showing 0% free ? In /var/log i see tmm , tmm1 , tmm2 , tmm3

Three million records is a pretty decent number of rows. What kind of content is it? How large is the actual file? What is it you're trying to accomplish with this data once it's uploaded? With a little bit more information we may be able to more accurately assist. ;) Colin

Also, what error is listed in /var/log/ltm when the import fails? And how much RAM does the VE have allocated? And how many TMMs are running? Thanks, Aaron

Thanks for the response Colin. This is a string type file of key-value pairs of url redirects/rewrites; size is about 220 MB.

According to the TMOS concepts manuals, up to v11.3, the maximum size when importing an iFile is 4MB. Strangely, the line has been removed from the v11.4 manual, but there is no mention of a change in the release notes. I know that doesn't help much. Perhaps you could log a support request and ask them what the maximum size is for 11.4.

11.4.0 unable to import 3M-record external data-file

29 Replies

Colin_Walker_12
Historic F5 Account
Aug 13, 2013
Three million records is a pretty decent number of rows. What kind of content is it? How large is the actual file? What is it you're trying to accomplish with this data once it's uploaded? With a little bit more information we may be able to more accurately assist. ;) Colin
hooleylist
Cirrostratus
Aug 14, 2013
Also, what error is listed in /var/log/ltm when the import fails?

And how much RAM does the VE have allocated?

And how many TMMs are running?

Thanks, Aaron
BaltoStar_12467
Aug 14, 2013
Thanks for the response Colin. This is a string type file of key-value pairs of url redirects/rewrites; size is about 220 MB.
uni
Altostratus
Aug 14, 2013
According to the TMOS concepts manuals, up to v11.3, the maximum size when importing an iFile is 4MB. Strangely, the line has been removed from the v11.4 manual, but there is no mention of a change in the release notes.

I know that doesn't help much. Perhaps you could log a support request and ask them what the maximum size is for 11.4.
- BaltoStar_12467
  Aug 14, 2013
  Ok but I have successfully imported Data Group External File of size 120MB. Does anyone know how an iFile differs internally from a Data Group External File ?
- uni
  Altostratus
  Aug 14, 2013
  I don't know about data group files - I have only imported very small ones. On the import screen it say "Free space on disk". What does this say for you? On one I looked at, this corresponded to the amount of space free on the root (linux) partition, even though the file is stored in the /config partition, and which had very much more space available.
- BaltoStar_12467
  Aug 14, 2013
  iFiles are meant for web pages and other markup/script -- they allow one to avoid the TCL encodings necessary when the code is stored directly in an iRule. For me it also lists "Free space on disk" as a very small value, such as ~170MB. But why would the Linux `_root_` partition be so small ?
uni_87886
Cirrostratus
Aug 14, 2013
According to the TMOS concepts manuals, up to v11.3, the maximum size when importing an iFile is 4MB. Strangely, the line has been removed from the v11.4 manual, but there is no mention of a change in the release notes.

I know that doesn't help much. Perhaps you could log a support request and ask them what the maximum size is for 11.4.
- BaltoStar_12467
  Aug 14, 2013
  Ok but I have successfully imported Data Group External File of size 120MB. Does anyone know how an iFile differs internally from a Data Group External File ?
- uni_87886
  Cirrostratus
  Aug 14, 2013
  I don't know about data group files - I have only imported very small ones. On the import screen it say "Free space on disk". What does this say for you? On one I looked at, this corresponded to the amount of space free on the root (linux) partition, even though the file is stored in the /config partition, and which had very much more space available.
- BaltoStar_12467
  Aug 14, 2013
  iFiles are meant for web pages and other markup/script -- they allow one to avoid the TCL encodings necessary when the code is stored directly in an iRule. For me it also lists "Free space on disk" as a very small value, such as ~170MB. But why would the Linux `_root_` partition be so small ?
uni
Altostratus
Aug 14, 2013
I think I have it. When you import a file through the Configuration Utility, the file is uploaded to /tmp (this seems to be a function of the Apache web server), which is in the "/" (root) partition. Thus the size of the upload is limited to the amount of free space in the root partition. Once the file is imported to the local file system, it is moved to the /config partition, which may have more space available.

Curiously, if you use tmsh to load a file (tmsh create sys file ifile source-path ), the temporary file goes to /var/tmp, which should have a lot of space available.

I suggest you try the tmsh method
- BaltoStar_12467
  Aug 14, 2013
  Very interesting stuff uni. But shouldn't it be possible to expand the size of the "/" (root) partition ? How to do this ?
- hooleylist
  Cirrostratus
  Aug 15, 2013
  That looks like the best option Uni. Thanks for suggesting it. Balto, it would be great if you could raise this issue with F5 Support as I think the GUI operation should be fixed to support very large data group files. Thanks, Aaron
- uni
  Altostratus
  Aug 15, 2013
  I wouldn't pin my hopes on expanding partition sizes. The BigIP is an appliance not a server. If you start messing with the partition sizes, you will have to do it every time you update or apply a hotfix. In my opinion, F5 should link /tmp to somewhere in a larger partition, eg to /shared/tmp, or change where the httpd temporary files go
uni_87886
Cirrostratus
Aug 14, 2013
I think I have it. When you import a file through the Configuration Utility, the file is uploaded to /tmp (this seems to be a function of the Apache web server), which is in the "/" (root) partition. Thus the size of the upload is limited to the amount of free space in the root partition. Once the file is imported to the local file system, it is moved to the /config partition, which may have more space available.

Curiously, if you use tmsh to load a file (tmsh create sys file ifile source-path ), the temporary file goes to /var/tmp, which should have a lot of space available.

I suggest you try the tmsh method
- BaltoStar_12467
  Aug 14, 2013
  Very interesting stuff uni. But shouldn't it be possible to expand the size of the "/" (root) partition ? How to do this ?
- hooleylist
  Cirrostratus
  Aug 15, 2013
  That looks like the best option Uni. Thanks for suggesting it. Balto, it would be great if you could raise this issue with F5 Support as I think the GUI operation should be fixed to support very large data group files. Thanks, Aaron
- uni_87886
  Cirrostratus
  Aug 15, 2013
  I wouldn't pin my hopes on expanding partition sizes. The BigIP is an appliance not a server. If you start messing with the partition sizes, you will have to do it every time you update or apply a hotfix. In my opinion, F5 should link /tmp to somewhere in a larger partition, eg to /shared/tmp, or change where the httpd temporary files go
uni_87886
Cirrostratus
Aug 15, 2013
Having said don't mess with the file system, I just did, and the following worked. The Configuration Utility tells me there is as much space available as in the /shared partition. I doubt F5 would approve of this though.

mkdir /shared/tmp/tmp chgrp root /shared/tmp/tmp chmod 1777 /shared/tmp/tmp mv /tmp /tmp.save ln -s /shared/tmp/tmp /tmp

You should probably do a bigstart restart (or at least bigstart restart tomcat).
- uni_87886
  Cirrostratus
  Aug 15, 2013
  Oops. I left out where I save the old tmp directory before the ln line: mv /tmp /tmp.save
- BaltoStar_12467
  Aug 15, 2013
  Now I'm running out of space in the BIG-IP Common partition ( "/config" ) as I add more data-group-files. Does f5 provide a way to expand the Common partition ?
uni
Altostratus
Aug 15, 2013
Having said don't mess with the file system, I just did, and the following worked. The Configuration Utility tells me there is as much space available as in the /shared partition. I doubt F5 would approve of this though.

mkdir /shared/tmp/tmp chgrp root /shared/tmp/tmp chmod 1777 /shared/tmp/tmp mv /tmp /tmp.save ln -s /shared/tmp/tmp /tmp

You should probably do a bigstart restart (or at least bigstart restart tomcat).
- uni
  Altostratus
  Aug 15, 2013
  Oops. I left out where I save the old tmp directory before the ln line: mv /tmp /tmp.save
- BaltoStar_12467
  Aug 15, 2013
  Now I'm running out of space in the BIG-IP Common partition ( "/config" ) as I add more data-group-files. Does f5 provide a way to expand the Common partition ?