Recreate external data groups on a test LTM to test another LTMs

You can actually SCP the external data group files off the production system. The files themselves are stored in a BIG-IP filestore in a location (path) that uses the following naming convention:

/config/filestore/files_d/_d/data_group_d/

For example, if the external data group is defined in the Common partition, it can be found in /config/filestore/files_d/Common_d/data_group_d. Just list the directory and you should see all the external data groups defined there. The naming convention for a data group is "::__version" - where "" is a randomly generated number and "version" is the version number of the data group (only changes if you replace it with a new version). For example, I created an external data group named "test" in the Common partition, and it's full path and name is as follows:

/config/filestore/files_d/Common_d/data_group_d/:Common:test_62208_1

From there, it's a simple matter to just download it off the prod system and then import it to the test system at System > File Management > Import.

If I try copying the external data group files from the prod LTM The filenames look like this

%3ACommon%3Adatagroup_file_110335_4

when importing I get the error

0107169a:3: The data group external file (/config/filestore/.stage_d/24_d/Common_d/data_group_d/:Common:datagroup_file_20082919_1) has an invalid format, line: 2.

Then I tried editing the files leaving just the first one line - that seems to work

but then if I try loading the prod LTM config ....

01070712:3: Values (/Common/datagroup_file) specified for value list (/Common/datagroup): foreign key index (file_FK) do not point at an item that exists in the database. Unexpected Error: Loading configuration process failed.

still can't get the full config to test - only if I comment out the data group file line will the config load successfully, but with broken external data groups .... maybe I just have to delete them and recreate them, but I have to find all the references in iRules to deal with ...... not good.

} ltm data-group external /Common/datagroup {

external-file-name /Common/datagroup_file

type ip

}

The file will download with the strange file name. You can rename it (before importing) to eliminate the ":Common:" part as well as the random number and version suffix parts.

 

Also, did you edit the file at all between the time you download it from prod and imported it to test? The format of the file can be touchy, depending on the editor. (I use Notepad to avoid automatic insertion of all kinds of crazy coded information at the end of the line, as Word has a tendency to do.) There should be a comma at the end of each line and just a new-line character (\n) not a full CRLF (\r\n). There are some examples here. You can check for errant line feed characters in the file using:

 

cat | od -c

 

If you see both \r\n in the output, run this command to remove the "\r" characters:

 

tr -d "\r"

 

 

Lastly, you have to make sure you set up the File Management Data Group object name to be the same as the file name specified in the associated iRule Data Group object. The two tie together. Otherwise I expect the config won't load properly.