Forum Discussion
perl SOAP::Lite and file encoding
I'm having problems using System::ConfigSync::upload_file with SOAP Lite in perl.
In perl everything is working fine when I read in binary files with binmode set. When I read in text files, no matter what type encoding type I put in binmod (utf8 or byte) or if I try and read them in as text without binmode, the XML serialized data for the content has "\n" for every line and the file on the BIG-IP comes shows up as a binary file..all jacked up.
I know the API says char[], but how it is being serialized in SOAP::Lite and then written to the disk on the BIG-IP is not working for me. I jumped over to java and read in the files as byte arrays and, after trimming them up BTW..thanks for that,got everything working with binary and text files. So I'm sure this is a encoding/serialization issue in perl SOAP::Lite.
Anyone have this working in perl with some syntax?
Here is what does not work for text files... It's got a bunch of my module sub in it, but you will get the point.
sub copyFile
{
my ($self,$srcfile,$destfile) = @_;
writelog($self,"debug","Copy source file $srcfile to $destfile");
if( -e $srcfile ) {
my $request = getSoapRequestObject($self, "urn:iControl:System/ConfigSync");
my $buffer;
my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks) = stat($srcfile);
open INF, $srcfile or writelog($self,"err","Source file $srcfile can not be opened. $!"); This does work for text files. Tried :byte, :utf8. Tried checking file with if (-B ) and not encoding..doesn't matter
binmode INF, ":bytes";my $blocksize = 65535;
if($size < $blocksize) {
$blocksize = $size;
}
my $bytes_read = 0;
while ( read (INF, $buffer, $blocksize))
{
my $chain_type = "FILE_MIDDLE";
if($bytes_read < 1) {
$chain_type = "FILE_FIRST";
}
elsif (($bytes_read + length($buffer)) eq $size) {
$chain_type = "FILE_LAST";
}
writelog($self,"debug","Writing $destfile ($bytes_read/".length($buffer)."/$size) $chain_type");
my $FileTransferContext = {file_data => $buffer,
chain_type => $chain_type
};
my $response = $request->upload_file(
SOAP::Data->name(file_name => $destfile),
SOAP::Data->name(file_context => $FileTransferContext)
);
if(checkResponse($self,$response)){
$bytes_read = $bytes_read+length($buffer);
} else {
close INF;
return 0;
}
}
close INF;
return 1;
} else {
writelog($self,"err","Source file for copy, $srcfile, does not exist");
return 0;
}
}7 Replies
- Here's how I did it in the PerlConfigSync CodeShare entry (also as included in the SDK).
http://devcentral.f5.com/wiki/default.aspx/iControl/PerlConfigSync.html
I just tested it and it worked with both binary (.exe) and text files.sub uploadFile() { my ($localFile, $configName, $quiet) = (@_); $success = 0; if ( "" eq $configName ) { $configName = $localFile; } if ( "" eq $localFile ) { &usage("upload"); } $bContinue = 1; $chain_type = $FILE_FIRST; $preferred_chunk_size = $defaultChunkSize; $chunk_size = $defaultChunkSize; $total_bytes = 0; open(LOCAL_FILE, "<$localFile") or die("Can't open $localFile for input: $!"); binmode(LOCAL_FILE); while (1 == $bContinue ) { $file_data = ""; $bytes_read = read(LOCAL_FILE, $file_data, $chunk_size); if ( $preferred_chunk_size != $bytes_read ) { if ( $total_bytes == 0 ) { $chain_type = $FILE_FIRST_AND_LAST; } else { $chain_type = $FILE_LAST; } $bContinue = 0; } $total_bytes += $bytes_read; $FileTransferContext = { file_data => SOAP::Data->type(base64 => $file_data), chain_type => $chain_type }; $soap_response = $ConfigSync->upload_file ( SOAP::Data->name(file_name => $configName), SOAP::Data->name(file_context => $FileTransferContext) ); if ( $soap_response->fault ) { if ( 1 != $quiet ) { print $soap_response->faultcode, " ", $soap_response->faultstring, "\n"; } $success = 0; $bContinue = 0; } else { if ( 1 != $quiet ) { print "Uploaded $total_bytes bytes\n"; } $success = 1; } $chain_type = $FILE_MIDDLE; } print "\n"; close(LOCAL_FILE); return $success; }
-Joe - Teach me not to look... Sorry Joe. I know you work hard for a reason! RTM...
The SOAP::Data->type(base64 => $buffer) is the exact help I needed...
From the WSDL, how could I tell it required base64 encoding? Just wondering. struts in Java seems to understand this need with just byte[] array and do it automatically if that what is needed in the XML.
John - It's funny, this is the second time this exact issue came up this week and it's the only time I can remember in the 6 years or so since I wrote that first ConfigSync sample.
Anytime in iControl you see a datatype of "char []" that means it's a base64 encoded byte array. All the strongly typed toolkits take care of it for you, but it's the perl and python's in the world that suffer from having to manually specify the encoding type.
As far as I know there are only a couple of methods that use the "char []" type and those are in System.ConfigSync and Management.LicenseAdministration.
-Joe - That tells you where the zeitgeist for iControl is then...
Ironically.. SOAP::Lite doesn't require me to base64 decode content on download_file. When I DATA::dump it, it doesn't show base64 in the response XML, but clearly that's what it is. SOAP::Lite... confused little kit..
Thanks again Joe. 
L4L7_53191Nimbostratus
John: Really? That's extremely odd to me. I'd think that you'd need to explicitly decode the response - something must be going on there. Perl gets ASCII back - just like Python, which requires me to explicitly decode. So I'd expect it to duck type it as ASCII on the response side, which in terms of dynamic languages is the Right Thing to do. I'm extremely interested to see why this happens with SOAP::Lite.
I may have to keep my old buddy honest and test this out for myself :)
-Matt- Actually, the methods that use parameters with char[] data types will result in a SOAP type of base64Binary. Looking at a soap trace on the download_file method, the resulting data looks something like this:
< E:Envelope xmlns:E="http://schemas.xmlsoap.org/soap/envelope/" xmlns:A="http://schemas.xmlsoap.org/soap/encoding/" xmlns:s="http://www.w3.org/2001/XMLSchema-instance" xmlns:y="http://www.w3.org/2001/XMLSchema" xmlns:iControl="urn:iControl" E:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" > < E:Body > < m:download_fileResponse xmlns:m="urn:iControl:System/ConfigSync" > < return s:type="iControl:System.ConfigSync.FileTransferContext" > < file_data s:type="y:base64Binary" >YXV0aCBwYXJ0aXRpb24gQ29tbW9uIHsKICAgIGRlc2N...."< /file_data > < chain_type s:type="iControl:Common.FileChainType" >FILE_FIRST_AND_LAST< /chain_type > < /return > < file_offset s:type="y:long" >7656< /file_offset > < /m:download_fileResponse > < /E:Body > < /E:Envelope >
SOAP::Lite sees the "base64Binary" type and knows to decode it. Maybe I'm missing something...
-Joe - L4L7_53191Ah. That explains it on the return side then.
-MC
![\"F5](\"https://www.f5.com/content/dam/f5/f5-logo.svg\"){kind=logo}
