BIG-IQ Archive Error
Hello, We currently have a BIG-IQ that was successfully performing backups to the backup server until the server guys implemented some stricter SSH timeouts. From what I gather, the archive is failing because the BIG-IQ seems to open up the SCP connection before the backup finishes. Once the backup has finished it then proceeds to pass the username but the SSH session has timed out by that time. Does anyone know of a way to change the order in which the BIG-IQ does the backup, so change it from SSH -> Backup -> SSH credentials -> copy file to Backup -> SSH -> SSH credentials -> copy file to Backup?
F5 Backup procedure over SCP using iCall
Problem this snippet solves: Purpose: You should consider using this procedure under the following condition. * You want to transfer BIG-IP Backup to the remote SCP Server on a specific frequency, without entering the password each time to transfer the file. Prerequisites: You must meet the following prerequisite to use this procedure. * You have administrator access to the BIG-IP Configuration utility and Command Line Access. * You have a user account on SCP Server with file transfer privileges. Description BIG-IP offers feature to transfer files over a remote SCP Server. Secure Copy (SCP) is the preferred means of transferring files to or from an F5 device. SCP securely transfers files between hosts using the Secure Shell (SSH) protocol for authentication and encryption. Unlike FTP, SCP provides an option to preserve the original date stamp on the file during file transfers. You can use SCP to transfer files between an F5 device and a remote host using either command line SCP or Windows-based SCP. We can automate the authentication process by export the public key from BIG-IP to SCP Server. By doing this, SCP Server will establish trust relationship with BIG-IP, and will not prompt for the password every-time we transfer files from BIG-IP to SCP Server. To achieve this, we need to have a user account on SCP Server, which has file transfer privileges. In this document, we will be using Linux based SCP Server. Once the transfer of files through the SCP works successfully. We can prepare the script done in iCall (as written following) and put it to run in the required interval. Login to SCP Server 1.Create a user account with permission to accept files from remote location, we will be using f5_user as user account, or you can use root user account (default user account on every Linux system). 2.It is good to have organized directory structure to receive F5 Backup. We will be creating 2 directories on our SCP Server as following. /F5Backup is the directory to receive F5 backup on a configured frequency (weekly, monthly, yearly etc). /authorized_key is temporary directory where will be sending the public key from F5 to SCP. 3.To create a directory in any Linux machine, you can use following command, mkdir /root/…path Eg. mkdir /home/f5_user/tmp/F5Backup/ 4.Create another directory to copy F5's public RSA key. mkdir /home/f5_user/tmp/authorized_keys Login to BIG-IP CLI Login to F5 CLI Generate RSA key by executing following. ssh-keygen -t rsa The RSA key will authenticate BIG-IP when it communicates with SCP Server. It would ask you to enter name and password, ignore the prompt by pressing ENTER. It will generate the Public / Private key under the directory /root/.ssh/id_rsa Verify the generated key by executing the following command. cat /root/.ssh/id_rsa.pub This should show you the public key, you have generated using above command. Send this public key to your Linux SCP Server. (We will be using SCP command to transfer the file from F5 to Linux Server). scp id_rsa.pub f5_user@10.1.20.222:/tmp/authorized_keys It will prompt for the password, enter the password for the f5_user user. In case if you are using other user account than f5_user, which has file transfer permission granted, you can replace f5_user with that user account, and keep the rest of the command as it is. Back to SCP Server Check if the Public key sent from F5 is received successfully under /tmp/authorized_keys or not. cat /f5_user/tmp/authorized_keys Note, in case if you have used other user account than "f5_user", replace the "f5_user" with the username you are using. Copy the key to right location, in order to authorize SCP connection from F5. cat /f5_user/tmp/mykey >> /f5_user/.ssh/authorized_keys If "authorized_keys" directory isn't exist, create a one by using mkdir command as mentioned earlier in the document. Verify the key is successfully placed under /f5_user/.ssh/authorized_keys or not. cat /f5_user/.ssh/authorized_keys Once the key successfully placed to the right location, it is time to test the connectivity from F5 to SCP Server. Switch back to F5 F5's CLI, scp filetest f5_user@10.1.60.240:/home/f5_user/tmp/ filetest 100% 5 0.0KB/s 00:00 If you have notices, this time it won’t ask for the password. In case if it still prompts, means you haven’t place the RSA key to the right place. The objective to place the RSA key under the user directory/.ssh/authorized_key. Once the connectivity is tested successfully, we can try to send F5’s UCS file over SCP to the remote server by the same method. Here, we can use the following script to generate the BIG-IP Backup and send it to the remote server. F5 CLI, type the following to type the script as following. Create a script with the command "tmsh create sys icall script <script name>" and then edit with vi and insert the content below.</p> </script> How to use this snippet: sys icall script auto_backup { app-service none definition { #Delete backup files exec rm -f /shared/tmp/*.ucs #Set Current Date/Time for Filename set cdate [clock format [clock seconds] -format "%Y%m%d"] #Set source repository set localpath "/var/local/ucs/" #Set destination repository set destinationpath "/home/teste/f5_backups" #Set remote host set host "10.1.20.222" #set remote user set user "f5_user" #Set device hostname set hostname [exec uname -n | cut -d "." -f1] #Set source repository cd $localpath #Delete files created more than 45 Days. catch { exec find "/var/local/ucs/" -type f -mtime +45 | grep -v .conf | xargs rm -f {} ; } #Delete UCS file if it exists catch { tmsh::delete sys ucs $hostname } #Export UCS tmsh::save sys ucs $hostname #Set temporary path set tmpdir "/shared/tmp/" append filename $hostname "_" $cdate #Copy UCS to temporary path exec cp $localpath$hostname.ucs $tmpdir$filename.ucs #Set Remote path append destination $user "@" $host ":" $destinationpath #Set source path append source $tmpdir $hostname "_" $cdate ".ucs" #Send the files via SCP. Prerequisite: The public key of BIG-IP must be registered in the file "authorized_keys" of the remote server if { [catch { exec scp -c aes128-ctr $source $destination > /dev/null 2> aux }] } { exec logger -p local0.info "Backup upload failed." } else { exec logger -p local0.info "The backup has been successfully sent to $destination." } exec rm -f aux } description none events none `</pre> } **Create the iCall Handler** I run the backup once a day; however, the periodicity can be adjusted according to the need of each one. In this example, I set the first-occurrence, the interval (once a day), and the script to call: <pre>`sys icall handler periodic auto_backup { first-occurrence 2019-03-27:05:01:00 interval 86400 script auto_backup } I know there are other scripts available in the community (much more sophisticated), the idea is just to share a simple and functional model. Remember that each one must adapt in the best way to meet your need. Code : 92596 Tested this on version: 12.1
BigIP VE (LTM) crashes when uploading iso's by SSH/SCP
Hi all I have an odd problem. There is a virtual machine (BigIP VE) with version 13.1.3.x with enough ressources (RAM, CPU, disk) which crashes when I try to upload an ISO for upgrading using SCP/SSH (not the WebGUI!). When I start the upload it takes a moment (it ranges from 10 to 35% progress) the VE crashes either completly or starts to act strangely (not login possible anymore, but the traffic is still running). I have to power off the VM and then power it on again to actually remedy the issue. We will have an extensive testing with the ESX team and checking the VE itself, but preliminary checks don't show any issues with storage (ESX wise or internally VE, there is enough space) or ressources (like CPU, RAM, etc.). There is a second VM with the same statistics and same version, just another location and another ESX farm, but that one doesn't have an issue. Beside the fact we certainly need to check more in depth with ESX, anyone an idea what I could check on the F5 side what might cause this? Thanks a lot
F5 Auto backup script for SCP for V10 and 11.
Hello Guys, I have tried looking for the subjected topic within DevCentral, but unfortunately that didn't help. (I must be doing something wrong for sure). By the way I couldn't find any script for SCP which can be use for version 10. My requirement is to craft a script for SCP server which executes on daily basis. The customer is having 2 separate clusters, one is running on v10, another on 11. Is there any script which I can use for my requirement? Thank you,SCP not working anymore since upgrade to 12.1
Hi, I recently upgraded my Big-IP from 11.6 HF6 to 12.1 successfully. However, I am not able anymore to execute scp commands which I used to transfer my ucs backups remotely. Any attempt always end up with a "no hostkey alg" error. I tried many different things like regenerating rsa and dsa keys like I always do after an upgrade, but as of today have been unsuccessful... Any advice someone ? Thanks, Pascal.BIG-IP Backup Script with SCP
Problem this snippet solves: BIP-IP 10.2 backup script with SCP transfer option This script creates tar.bz2 archives which include a .ucs file and a .scf of the F5's configuration. It optionally stores them locally and/or transfers them to a remote system using SCP. Code : #!/bin/bash # f5backup.sh # Copyright (C) 2010 Colin Stubbs # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with this program. If not, see <> # # Additional files you'll need to create, example content: # # # /etc/cron.d/f5backup # SHELL=/bin/bash # PATH=/sbin:/bin:/usr/sbin:/usr/bin # MAILTO=user@somewhere # HOME=/var/tmp # 0 0 * * * root /bin/bash /var/local/bin/f5backup.sh 1>/var/tmp/f5backup.log 2>&1 # # EOF # # # /root/.ssh/f5archive_config # # Host * # User f5archive # PasswordAuthentication no # StrictHostKeyChecking yes # IdentityFile /root/.ssh/f5archive_dsa # Port 22 # Protocol 2 # Ciphers aes128-cbc,aes192-cbc,aes256-cbc # UserKnownHostsFile /root/.ssh/f5archive_host # # # EOF # # REQUIRES: # F5 BIG-IP 10.2.x # Debug ? Set to non-blank if so. DEBUG='' # Current date/time stamp DATETIME="`date +%Y%m%d%H%M%S`" # Base f5backup.sh working directory OUT_DIR='/var/tmp' # Unique temporary output location TEMP_DIR="`mktemp -d ${OUT_DIR}/f5backup.XXXXXXXXXX`" # Backup options # Export a UCS archive DO_UCS='x' # Export a single config file DO_SCF='x' # Export SCF with oneline bigpipe statements DO_SCF_ONELINE='x' # Use SCP with pubkey to export to remote system DO_SCP_EXPORT=1 # SCP options must be set if you want to use this SCP_OPTIONS="" # Destination list, can be a list of username:IPorHostname if you want to # transfer to multiple destinations. Same public key used for auth to all. # ** MAKE SURE YOU INCLUDE :<%DIRECTORY%> HERE SCP_DESTINATION="f5archive@192.0.2.10:" # All SCP options should be encapsulated in a special config file SCP_CONFIG="/root/.ssh/f5archive_config" # UCS output location UCS_FILE="${TEMP_DIR}/ucs.${DATETIME}.backup" # Encrypt UCS archive with passphrase # ** If blank this will not be used UCS_PASSPHRASE='' # SCF output location SCF_FILE="${TEMP_DIR}/scf.${DATETIME}.backup" # Local archive location # ** If this variable is blank, or the destination is not writable then # this script will not copy the backup to the local archive. LOCAL_ARCHIVE="/var/local/backups" # Remove all older local backups than this many days LOCAL_CLEANUP_DAYS=7 # Name of compressed backup archive to produce OUTPUT_FILE="f5backup-${HOSTNAME}-${DATETIME}.tar.bz2" if [ "${DEBUG}x" != "x" ] ; then for i in HOSTNAME DATETIME OUT_DIR TEMP_DIR DO_UCS DO_SCF DO_SCF_ONELINE DO_SCP_EXPORT SCP_DESTINATION SCP_OPTIONS SCP_CONFIG ASM_POLICY_LIST UCS_FILE UCS_PASSPHRASE SCF_FILE LOCAL_ARCHIVE OUTPUT_FILE ; do eval var=\$$i echo "${i} = $var" done fi function usage { echo "Usage: f5backup.sh" exit 0 } function export_scf() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi if [ "${DO_SCF}x" != "x" ] ; then if [ "${DO_SCF_ONELINE}x}" != "x" ] ; then bigpipe export oneline "${SCF_FILE}" else bigpipe export "${SCF_FILE}" fi fi } function export_ucs() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi if [ "${DO_UCS}x" != "x" ] ; then if [ "${UCS_PASSPHRASE}x" != "x" ] ; then bigpipe config save "${UCS_FILE}" passphrase "${UCS_PASSPHRASE}" else bigpipe config save "${UCS_FILE}" fi fi } function create_backup() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi tar -v -j -c -f "${OUT_DIR}/${OUTPUT_FILE}" -C "${TEMP_DIR}" . } # Transfer backup archive to offsite location/s function backup_remote() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi if [ -f "${OUT_DIR}/${OUTPUT_FILE}" ] ; then if [ "${DO_SCP_EXPORT}x" != "x" ] ; then echo "Copying to remote archive ${SCP_DESTINATION}${OUTPUT_FILE}" if [ "${DEBUG}x" != "x" ] ; then echo "Exec: /usr/bin/scp ${SCP_OPTIONS} -F ${SCP_CONFIG} ${OUT_DIR}/${OUTPUT_FILE} ${SCP_DESTINATION}" ; fi /usr/bin/scp ${SCP_OPTIONS} -F "${SCP_CONFIG}" "${OUT_DIR}/${OUTPUT_FILE}" "${SCP_DESTINATION}" || echo "Error: SCP ${OUT_DIR}/${OUTPUT_FILE} ${SCP_DESTINATION} failed!" fi else echo "Error: ${OUT_DIR}/${OUTPUT_FILE} doesn't exist, something has gone wrong!" fi } function backup_local() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi if [ "${LOCAL_ARCHIVE}x" != "x" ] && [ -d "${LOCAL_ARCHIVE}" ] ; then echo "Copying to local archive ${LOCAL_ARCHIVE}/${OUTPUT_FILE}" if [ ! -d "${LOCAL_ARCHIVE}" ] ; then mkdir "${LOCAL_ARCHIVE}" if [ ${?} -ne 0 ] ; then echo "Error: ${LOCAL_ARCHIVE} doesn't exist and I can't create it." fi fi mv -f "${OUT_DIR}/${OUTPUT_FILE}" "${LOCAL_ARCHIVE}"/ find "${LOCAL_ARCHIVE}" -type f -mtime +${LOCAL_CLEANUP_DAYS} -exec rm -v -f {} \; fi } # Cleanup what this script has done function cleanup() { if [ "${DEBUG}x" != "x" ] ; then echo "in ${FUNCNAME}(${*})" ; fi rm -rf ${TEMP_DIR} } # Sanity checking # 1. Must be run as root if [ "`id -u`x" != "0x" ] ; then echo "Error: You need to run this script as root." exit 1 fi # 2. Command line args # not applicable yet # 3. Temp dir must exist if [ ! -d "${TEMP_DIR}" ] ; then echo "Error: ${TEMP_DIR} was not created for some reason. Fix this issue and try again." exit 1 fi echo "${0} start `date +%Y%m%d%H%M%S`" export_ucs export_scf create_backup backup_remote backup_local cleanup echo "${0} finish `date +%Y%m%d%H%M%S`" # EOF Tested this on version: 10.2
copying certs in /config/filestore/files_d/Common_d/certificate_d/
After copying a bunch of certs that I upload and created chains and profiles to another ltm in the same directory, the certs do no appear when I check in System ›› File Management : SSL Certificate List file directory on the target LTM. I restarted apache and tomcat as well big3d but no luck the files that I scp'ed over do no appear when navigating to the section in the web utitly. They do appear in the /config/filestore/files_d/Common_d/certificate_d/ directory on the targeted machine. Is there a reason why the certs do not appear ?
