AspEncrypt fails to verify signatures generated by other packages

Problem Description

A singature created by a Java-based application using the class MD5withRSA (which implements the signature algorithm of MD5 with RSA as defined in PKCS#1) always fails when verified by AspEncrypt's VerifySignature method of the CryptoHash object.

Solution

The Microsoft CryptoAPI (and hence AspEncrypt) implements the PKCS#1 format with bytes of data in a reverse order. Therefore, signatures generated by CryptoAPI and AspEncrypt may be incompatible with cryptographic implementations that strictly conform to the PKCS#1 standard (including Java).

If you are trying to verify a signature generated by a non-Microsoft implementation, you must reverse the bytes in the signature before passing it to the method VerifySignature.

The following code snipped demonstrates this technique:

' Obtain public key from signer certificate
Set context = CM.OpenContext("MyContainer", True)
Set cert = CM.ImportCertFromFile("d:\verification.cer")
Set Key = context.ImportKeyFromCert(cert)

' import the signature value into a blob object.
' in this sample, the signature is in the Base64 format
Set blob = CM.CreateBlob
blob.Base64 = "HHnojfbhAdduF4fFipVNn.....rest not shown here"
N = Len(blob.Hex)

' reverse bytes in the signature using Hex format
For i = 1 To N - 1 Step 2
  s = Mid(blob.Hex, i, 2) & s
Next

' Blob2 will contain signature in correct PKCS#1 format
Set blob2 = CM.CreateBlob
blob2.Hex = s

' calculate hash of text we are verifying against
Set hash = context.CreateHash(calgMD5)
hash.AddText "Was it a cat I saw?"

' verify signature
If hash.VerifySignature(blob2, Key) Then
  MsgBox "Verified."
Else
  MsgBox "Not verified."
End If