c - Understanding engine initialization in OpenSSL

Question

I'm trying to set up a basic test of HMAC-SHA-256 hashing but I'm having problems with the engine setup. Ideally I would like to set up only the HMAC-SHA-algorithm but so far I haven't even gotten the general case where load all the algorithms to work. Currently I'm getting segfaults on the row where I try to set the default digests.

Also, I'm regularly a Java guy, so don't hesitate to point out any mistakes in the code.

#include <openssl/hmac.h>
#include <openssl/evp.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main() {
  unsigned char* key = (unsigned char*) "0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b";
  unsigned char* data = (unsigned char*) "4869205468657265";
  unsigned char* expected = (unsigned char*) "b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7";
  unsigned char* result;
  HMAC_CTX* ctx;
  ENGINE* e;

  ENGINE_load_builtin_engines();
  ENGINE_register_all_complete();
  ENGINE_set_default_digests(e);

  HMAC_CTX_init(ctx);
  HMAC_Init_ex(ctx, key, 40, EVP_sha256(), e);
  result = HMAC(NULL, NULL, 40, data, 16, NULL, NULL);
  HMAC_CTX_cleanup(ctx);

  ENGINE_finish(e);
  ENGINE_free(e);

  if (strcmp((char*) result, (char*) expected) == 0) {
    printf("Test ok
");
  } else {
    printf("Got %s instead of %s
", result, expected);
  }
}

EDIT: The program has now evolved to the following, but I'm still segfaulting at HMAC_Init_ex:

unsigned char* key = (unsigned char*) "0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b";
unsigned char* data = (unsigned char*) "4869205468657265";
unsigned char* expected = (unsigned char*) "b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7";
unsigned char* result;
unsigned int result_len = 64;
HMAC_CTX ctx;
ENGINE* e;

result = (unsigned char*) malloc(sizeof(char) * result_len);
e = (ENGINE*) ENGINE_new();

ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
ENGINE_set_default_digests(e);

HMAC_CTX_init(&ctx);
HMAC_Init_ex(&ctx, key, 16, EVP_sha256(), e); 
HMAC_Update(&ctx, data, 40);
HMAC_Final(&ctx, result, &result_len);
HMAC_CTX_cleanup(&ctx);

ENGINE_finish(e);
ENGINE_free(e);

Answer 1

The problem with your original suggestion is, as Martin said, that you need to initialise the ENGINE. The problem with your edited code was that you were doing ENGINE_new, which is getting you a completely new ENGINE of your own, which you then need to provide with cipher methods, digest methods, etc. In fact, for what you want (and what almost everybody wants), just completely ignoring all of the ENGINE stuff is the right choice.

Some subsidiary problems:

• your strings were hex, but you needed a x per character to actually get that hex byte at that position in the string, which I suspect was what you wanted.
• you were trying to hash 40 bytes from "data", which wasn't that long (actual effect: you'd end up partly hashing your result string)
• your expected result was (as far as I can tell) incorrect
• you would print out random characters to the terminal, since the HMAC function will produce 32 bytes of random binary data, not printable stuff.

The following code compiles, works and passes the test. It's a bit different to the example code you found (since it still uses the individual HMAC_* functions - useful if you want to do your hashing bit by bit using HMAC_Update):

#include <openssl/engine.h>
#include <openssl/hmac.h>
#include <openssl/evp.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>


int main(void)
{
        unsigned char* key = (unsigned char*) "x0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0bx0b";
        unsigned char* data = (unsigned char*) "x48x69x20x54x68x65x72x65";
        unsigned char* expected = (unsigned char*) "x49x2cxe0x20xfex25x34xa5x78x9dxc3x84x88x06xc7x8fx4fx67x11x39x7fx08xe7xe7xa1x2cxa5xa4x48x3cx8axa6";
        unsigned char* result;
        unsigned int result_len = 32;
        int i;
        HMAC_CTX ctx;

        result = (unsigned char*) malloc(sizeof(char) * result_len);

        ENGINE_load_builtin_engines();
        ENGINE_register_all_complete();

        HMAC_CTX_init(&ctx);
        HMAC_Init_ex(&ctx, key, 16, EVP_sha256(), NULL);
        HMAC_Update(&ctx, data, 8);
        HMAC_Final(&ctx, result, &result_len);
        HMAC_CTX_cleanup(&ctx);

        for (i=0; i!=result_len; i++)
        {
                if (expected[i]!=result[i])
                {
                        printf("Got %02X instead of %02X at byte %d!
", result[i], expected[i], i);
                        break;
                }
        }
        if (i==result_len)
        {
                printf("Test ok!
");
        }
        return 0;
}

Of course, it doesn't answer your original question about how to initialise ENGINEs, but there's really no right answer to that without having more context, which context turns out not to be relevant in your situation...