TripleDES in Perl/PHP/ColdFusion

Recently a problem arose regarding hooking up an API with a payment processor who were requesting a string to be encrypted to be used as a token, using the TripleDES standard. Our Applications run using ColdFusion, which has an Encrypt tag - that supports TripleDES - however the result we were getting back was not what the payment processor expected.

First of all, here is the resulting token the payment processor were expecting.

AYOF+kRtg239Mnyc8QIarw==

And below is the snippet of ColdFusion we were using, and the resulting string.

<!--- Coldfusion Crypt (here be monsters) --->
<cfset theKey="123412341234123412341234">
<cfset theString = "username=test123">
<cfset strEncodedEnc = Encrypt(theString, theKey, "DESEDE", "Base64")>
<!---
 resulting string(strEncodedEnc): tc/Jb7E9w+HpU2Yvn5dA7ILGmyNTQM0h
--->

As you can see, this was not returning the string we were hoping for. Seeking a solution, we ditched ColdFusion for this process and attempted to reproduce the token in PHP.

Now I'm aware that various languages implement encryption in different ways - for example in the past managing encryption between a C# application and PHP back-end, I've had to play about with padding in order to get the two to talk, but my experience has been that PHP generally behaves when it comes to encryption standards.

Anyway, on to the PHP source we tried, and the resulting string.

/* PHP Circus (here be Elephants) */
$theKey="123412341234123412341234";
$theString="username=test123";
$strEncodedEnc=base64_encode(mcrypt_ecb (MCRYPT_3DES, $theKey, $theString, MCRYPT_ENCRYPT));
/*
 resulting string(strEncodedEnc): sfiSu4mVggia8Ysw98x0uw==
*/

As you can plainly see, we've got another string that differs from both the string expected by the payment processor AND the one produced by ColdFusion. Cue head-against-wall integration techniques.

After many to-and-fro communications with the payment processor (lots and lots of reps stating 'we can't help with coding issues, you must be doing it incorrectly, read the manual') we were finally escalated to someone with more than a couple of brain-cells to rub together, who was able to step back and actually look at and diagnose the issue.

He agreed, our CF and PHP attempts were not resulting in the correct string. After a quick search, he also agreed that it was not neccesarily our source, but rather how the two languages implemented their vision of the TripleDES standard.

Coming into the office this morning, we were met by an email with a snippet of source code, in Perl. This is was the code they were directly using on their end to produce the expected token.

#!/usr/bin/perl
# Perl Crypt Calamity (here be...something)
use strict;
use CGI;
use MIME::Base64;
use Crypt::TripleDES;

my $cgi = CGI->new();
my $param = $cgi->Vars();

$param->{key} = "123412341234123412341234";
$param->{string} = "username=test123";
my $des = Crypt::TripleDES->new();

my $enc = $des->encrypt3($param->{string}, $param->{key});
$enc = encode_base64($enc);
$enc =~ s/\n//gs;

# resulting string (enc): AYOF+kRtg239Mnyc8QIarw==

So, there we have it. Three languages, three implementations of what they quote in the documentation as TripleDES Standard Encryption, and three totally different resulting strings.

My question is, from your experience o开发者_开发知识库f these three languages and their implementations of the TripleDES algorithm, have you been able to get any two of them to give the same response, and if so what tweaks to the code did you have to make in order to come to the result?

I understand this is a very drawn out question, but I wanted to give clear and precise setting for each stage of testing that we had to perform.

I'll also be performing some more investigatory work on this subject later, and will post any findings that I come up with to this question, so that others may avoid this headache.

The Perl's TripleDES should never be used. It does so many weird things and you are going to have fun.

Your first problem is that the keys in Perl are hex and you need to convert them into binary. Try this in PHP,

$theKey="123412341234123412341234";
$key = pack('H*', str_pad($theKey, 16*3, '0'));
$strEncodedEnc=base64_encode(mcrypt_ecb (MCRYPT_3DES, $key, $theString, MCRYPT_ENCRYPT));
echo $strEncodedEnc, "\n";

The result is,

AYOF+kRtg239Mnyc8QIarw==

Then you have to pad it in a weird way. I forgot the details. You are lucky with this sample (it's 16 chars).

The Coldfusion Answer:

The first problem is that your key length is not correct for Triple DES. ZZ Coder correctly deduced that it needs to be padded to the correct length with 0's.

The next step is that the key needs to be converted to hex. To do this in CF, we have:

<cfset theKey="123412341234123412341234000000000000000000000000">
<cfset encodedKey = ToBase64(BinaryDecode(theKey, "HEX"))>

The final step is that the result is not being padded either, so we need to specify this in the encryption algorithm in CF:

<cfset strEncodedEnc = Encrypt(theString, encodedKey, "DESEDE/ECB/NoPadding", "Base64")>

The resulting complete code:

<cfset theKey="123412341234123412341234000000000000000000000000">
<cfset encodedKey = ToBase64(BinaryDecode(theKey, "HEX"))>
<cfset theString = "username=test123">
<cfset strEncodedEnc = Encrypt(theString, encodedKey, "DESEDE/ECB/NoPadding", "Base64")>
<cfdump var="#strEncodedEnc#"><br>

results in:

AYOF+kRtg239Mnyc8QIarw==

I'll include the code below for anyone that happens to be working on CCBill upgrade (which sounds like the company referred to in the original post). The PHP functions below will match the output from CCBill's 3DES/TripleDES internal encryption as described in the documentation here: http://www.ccbill.com/cs/manuals/CCBill_Subscription_Upgrade_Users_Guide.pdf

//Encrypt String using 3DES Key
function encrypt($str,$key){
    $hex_key = hexmod($key);
    $bin_hex_key = pack('H*', str_pad($hex_key, 16*3, '0'));
    //Pad string length to exact multiple of 8
    $str = $str. str_repeat(' ',8-(strlen($str)%8) );   
    $out = base64_encode( mcrypt_ecb(MCRYPT_3DES, $bin_hex_key, $str, MCRYPT_ENCRYPT) );
    //print_r('Key/Hex/Str: '.$key.' -> '.$hex_key.' -> '.$str.' -> '.$out,1);
    return $out;
}

//Hex Modulus: Converts G-Z/g-z to 0-f (See @Jinyo's Post)
//Necessary to match CCBill's Encryption
function hexmod($str){
    //Convert G-Z & g-z to 0-f
    $ascii_in  = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
    $ascii_out = '0123456789ABCDEF0123456789ABCDEF0123abcdef0123456789abcdef0123';
    $hex_out = str_replace(str_split($ascii_in),str_split($ascii_out),$str);
    return $hex_out;
}

$triple_des_key = 'ABCDEFGHIJKLMNOPQRSTUVWX'; // <!-- 24char 3DES Key
$username_string = 'username=<username here>'; // Encrypt this string
$encrypted_username = encrypt($username_string,$triple_des_key); // <-- Output

Oh, this is fun!

> hex clear_text
0000  75 73 65 72 6e 61 6d 65  3d 74 65 73 74 31 32 33  username =test123

> openssl des3 -in clear_text -out crypt_text
enter des-ede3-cbc encryption password: 123412341234123412341234
Verifying - enter des-ede3-cbc encryption password: 123412341234123412341234

> hex crypt_text
0000  53 61 6c 74 65 64 5f 5f  d7 1b 37 a6 e0 c4 99 d1  Salted__ ..7.....
0010  ce 39 7f 87 5e 8b e8 8a  27 ca 39 41 58 01 38 16  .9..^... '.9AX.8.
0020  a5 2b c8 14 ed da b7 d5                           .+......

> base64 crypt_text
U2FsdGVkX1/XGzem4MSZ0c45f4dei+iKJ8o5QVgBOBalK8gU7dq31Q==

> openssl version
OpenSSL 0.9.8k 25 Mar 2009

> base64 --version | head -n 1
base64 (GNU coreutils) 7.1

You should talk to a crypto expert, try perhaps the mailing lists openssl-users or dev-tech-crypto@mozilla unless someone useful shows up here.

ZZ Coder was nearly there. There's just a few more caveats to why the Perl and PHP codes returned different encryptions.

Firstly, whenever there are invalid hex letters (letters after F), replace them according to the following rule:

• G->0
• H->1
• I->2
• J->3
• ...
• P->9
• Q->A
• R->B
• ...
• V->F
• W->0
• ...
• Z->3

Using this method, the key for AZ98AZ98AZ98AZ98AZ98AZ98 is A398A398A398A398A398A398000000000000000000000000 (after padding with zeroes).

Secondly, the text to be encrypted should be padded with whitespaces so that the number of characters is divisible by 8. In this example, username=test123 is divisible by 8 so it doesn't need to be padded. But, if it were username=test12, then it needs one whitespace at the end.

The following PHP code returns an encryption that matches the perl encryption

$theKey="A398A398A398A398A398A398000000000000000000000000";
 $key = pack("H*", $theKey);
$input = "username=test123";

$strEncodedEnc=mcrypt_ecb (MCRYPT_3DES, $key, $input, MCRYPT_ENCRYPT);
$strEncodedEnc64=base64_encode($strEncodedEnc);
echo $strEncodedEnc . "<br />";
echo $strEncodedEnc64 . "<br />";

Took me most of an evening, but this is how @Eric Kigathi's solution looks in ruby

def encoding(key, val)
  require "openssl"
  des = OpenSSL::Cipher::Cipher.new('des-ede3')
  des.encrypt
  des.key = convert_key_to_hex_bin key

  #ENCRYPTION
  des.padding = 0 #Tell Openssl not to pad
  val += " " until val.bytesize % 8 == 0 #Pad with zeros
  edata = des.update(val) + des.final 
  b64data = Base64.encode64(edata).gsub(/\n/,'')
end

def convert_key_to_hex_bin(str)
  decoder_ring = Hash['0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz+/'.split(//).zip('0123456789ABCDEF0123456789ABCDEF0123ABCDEF0123456789ABCDEF012345'.split(//))]
  str.gsub!(/./, decoder_ring)
  [str.ljust(16*3, '0')].pack("H*")
end

Do be careful, though. I'm not quite sure what the + and / convert to at the end. I guessed at 4 and 5, but I can't tell you if that's true.

Hat tip to http://opensourcetester.co.uk/2012/11/29/zeros-padding-3des-ruby-openssl/ the encryption code and commentary.

The ColdFusion Answer is missing modifying the ccbill key to work (like in Eric's Answer)... I have modified Eric's Answer to Lucee Code. It shouldn't take much work to take it back to ACF compatible Code (changing the structure in ReplaceNoCase with individual ones).

public function ccbillupgrade(string key = "XXXXXXXXXXXXXXXXXXXXXXXX", string username){

    var remote_user = padUserName("username=#arguments.username#");
    var padded_key = 
        Ucase(
            Replace(
                LJustify(
                    hexmod(arguments.key)
                , 48), // Pad key to 48 bytes (hex) 
                " ", '0', 'all'
            )
        );

    var encodedKey = ToBase64(BinaryDecode(padded_key, "HEX"));

    return Encrypt(remote_user, encodedKey, "DESEDE/ECB/NoPadding", "Base64");
}

private string function hexmod(string input) {
    return ReplaceNoCase( arguments.input,
        {
            'G' = '0', 'H' = '1',
            'I' = '2', 'J' = '3',
            'K' = '4', 'L' = '5',
            'M' = '6', 'N' = '7',
            'O' = '8', 'P' = '9',
            'Q' = 'A', 'R' = 'B',
            'S' = 'C', 'T' = 'D',
            'U' = 'E', 'V' = 'F',
            'W' = '0', 'X' = '1',
            'Y' = '2', 'Z' = '3'

        }
    );
}
private string function padUserName(string username) {
    var neededLength = Len(arguments.username) + ( 8 - Len(username) % 8 );
    return LJustify(arguments.username, neededLength);
}

There are two problems (or not) with Crypt::TripleDES:

1. The fact that keys for Crypt::TripleDES are HEX (explained earlier by ZZ Coder). You can hex your key by either using unpack or by using ord/sprintf or a bunch of other methods:

   • $pass = unpack("H*", "YOUR PASSPHRASE"); #pack/unpack version

   • $pass = join('', map { sprintf("%x",$)} map { ord($) } split(//, "YOUR PASS"));

   Crypt::TripleDES pads the pass-phrase with spaces (which was ok for me)

2. Crypt::TripleDES does whitespace padding only of the plain text. There are numerous padding methods which are used on Java or PHP mcrypt_encrypt:

   • (ie. PKCS5, PKCS7, CMS) - pad with bytes of the same value indicating the number of bytes padded eg: "andrei" -> hex: 61 6e 64 72 65 69 -> padded: 61 6e 64 72 65 69 02 02
   • pad with null characters eg: 61 6e 64 72 65 69 00 00
   • pad with spaces (Crypt::TripleDES already does this)
   • pad with zeros (null chars) except for the last byte which will be the number of padded bytes eg: 61 6e 64 72 65 69 00 02
   • pad with 0x80 followed by null chars eg: 61 6e 64 72 65 69 80 00

Pay attention to your cipher-text, if it matches up until some point but the ending is different then you have a plain-text padding problem. Otherwise you might have a pass-phrase problem, a cipher block mode problem (EBC,CBC,..) http://www.tools4noobs.com/online_tools/encrypt/help_modes.php or an algorithm problem.

So what I did in Perl to be able to match the cipher-text from Java (which used null chars padding):

my $pass = unpack("H*", "MY PASS");
my $text = "bla bla bla";
my $pad = 8 - (length $text % 8);
$pad = 0 if ( $pad > 7 );
$text .= chr(00) x $pad;

my $des = new Crypt::TripleDES;
my $cipher = $des->encrypt3( $text, $pass );

Hope this helps