Information Hiding Methods used by Flexlm Targets An Explanation of the Flexlm Seed Hiding System.	Not Assigned
8-Oct-1999	by Nolan Blender
	Courtesy of Fravia's page of reverse engineering	slightly edited by tsehp
fra_00xx 98xxxx handle 1100 NA PC
	There is a crack, a crack in everything That's how the light gets in
Rating	( )Beginner (X)Intermediate ( )Advanced ( )Expert

Flexlm has undergone a gradual evolution.  Due to pressure from crackers
the programmers who have written this code have improved the quality of the
protection it provides.  At the current stage of product evolution, it is
no longer possible to simply extract the seeds from the lc_init call, even
if in reality it is called from the lc_new_job call.  This essay will 
explain
the new methods used by Flexlm to hide relevant information needed to build
licenses.

For a full understanding of of how lm_new_job() works, I suggest that you
read the essay by Dan, as it will provide you with a complete understanding
of how the hiding is used in flexlm licensed targets.  This essay provides
auxillary information that will clarify exactly how the random data in the
job structure is combined with the vendor code data.


The current version of Flexlm uses special techniques to hide the encryption
seeds required to generate licenses.  Earlier versions simply stored the
information in global data, then passed the information into lc_init().  No
attempt was made to disguise the information, so of course examining the
initialized global area would reveal the seeds and the vendor codes.  From
that point, it was a simple matter to extract vendorcode5 and xor it with
data[1] and data[2], which would reveal the original seeds.  Later versions
hid the information using lm_new.c.

The process for storing the encrypted data involves storing one byte chunks
of data in the global space, and then reassembling the data in the routine
pointed to by l_n36_buf.  There is a lot of code which does not get 
executed,
however the references to the symbols is enough to keep the filler globals 
from
being compiled out.   The vendorname and the vendor keys that are returned
are correct.  The encryption seeds are not set to the real encryption seeds,
or for that matter, the encryption seeds xored with vendor key5, but the
encryption seeds xored with a random value, and then salted with values from
the vendor name and the first two vendor keys. The other task that this 
program
performs is to set l_n36_buff (note: two 'f's) to the decrypting function.
The routine takes two arguments, buf and v, where buf is a char * pointer 
that
will have the vendor name in it after the function is called.  v is a 
pointer
to a vendorcode structure that will be filled in, but the encryption seeds 
will
have been xored wih a random value.

The decrypting routine takes in 3 values; the job structure (a shared data
structure which is passed around between routines), the vendor_id, and
a pointer to the vendorcode structure, which contains the randomized seeds
and the correct vendor keys.

There are two modes of operation for the decrypting routine.  If a null
pointer for job is passed into the routine, the routine extracts the
correct values for the encryption seeds and stores them in the vendorcode
structure.  If a valid pointer is passed in for job, time based randomized
data is stored in the 12 bytes starting at v+8, then the returned seeds
xored with a select set of those 12 bytes.  That way the clear seeds are
not available until a routine that knows how to xor the correct 4 bytes
with the vendor keys is called.

A careful examination of the decryption routine in lm_new.c shows many
operations, and the seeds are xored with many values.  The main point
of interest from a seed extraction process is the selection of the
bytes from the job structure, since knowing that will enable us to get the
correct seeds from the structure.

        key->data[0] ^=
                (((((long)sig[0] << 0)|
                    ((long)sig[1] << 2) |
                    ((long)sig[2] << 3) |
                    ((long)sig[3] << 1))
                ^ ((long)(t->a[5]) << 0)
                ^ ((long)(t->a[0]) << 8)
                ^ x
                ^ ((long)(t->a[1]) << 16)
                ^ ((long)(t->a[4]) << 24)
                ^ key->keys[1]
                ^ key->keys[0]) & 0xffffffff) ;

sig, which is a value generated from the vendor_id, is shifted and ored with
encryption seed, as well as the randomized value x.  This really doesn't
matter, since this is accounted for already in the n_36_buf routine.  The
indicies used for the t->a[%d] values are what is of interest here.

The routine uniqcode in lmrand2.o was examined, and the the code which
generates the decrypting program was partly reversed.  It turns out that
the first character of the vendor name is extracted, then that value mod 20
is used to select the bytes used to decrypt the seeds.  Rather than reverse
the entire routine, a program to generate vendor codes was written based
on earlier reversing techniques, and an lm_new.c was generated for each of
the 20 different possibilities.  The values were then extracted, and a table
containing these values was generated.  I'm certain that the table could 
have
been extracted from the lmrand2.o file, but it was easier to get the data
from runs of lmrand2.  You can use Softice or DDE or adb to see what is
being done, the selection code happens right at the beginning of uniqcode.

Link this demonstration program with lm_new.obj, and run.  This program
has been tested on HPUX using the ANSI compiler, and under Windows NT
using Microsoft Visual C++ 5.0.

#include


/*
*
* Module: extr.c v1.0.0.0
*
* Description: Demonstration program to show interaction
*              between security code in lm_new.c and main
*              program.
*
* blendern
* 08-oct-1999
*
* Last modified: 08-oct-1999
*/

/*
* Decoding table type
*/

typedef struct decode_table_s {
        int offsets[4];
} decode_table_t;

/*
* really vendorcode5, but renamed to avoid conflict
* if Globetrotter headers are included.
*
* The encryption seeds are stored in the data part of this structure.
*
* the VENDOR_CODES are stored in the keys part of this structure.
*/

typedef struct my_vendorcode5 {
                            short type;    /* Type of structure */
                            unsigned long data[2]; /* 64-bit code */
                            unsigned long keys[4]; /*- [0]: Product features 
*/
                                                   /*- [1]: platforms */
                                                   /*- [2]: platforms */
                                                   /*- [3]: Expiration date/ 
*/
                                                   /*-      Key check */
                            short flexlm_version;
                            short flexlm_revision;
                            char flexlm_patch[2];
#define LM_MAX_BEH_VER 4
                            char behavior_ver[LM_MAX_BEH_VER + 1];
                          } MY_VENDORCODE5;

/*
* decryption table values.
*/

decode_table_t decode_table[] = {
        {3, 5, 4, 11}, /* index 0 */
        {9, 8, 3, 1}, /* index 1 */
        {8, 1, 2, 5}, /* index 2 */
        {2, 1, 10, 5}, /* index 3 */
        {3, 0, 1, 7}, /* index 4 */
        {1, 10, 3, 7}, /* index 5 */
        {7, 3, 5, 11}, /* index 6 */
        {0, 1, 9, 4}, /* index 7 */
        {0, 4, 1, 10}, /* index 8 */
        {11, 8, 1, 3}, /* index 9 */
        {8, 4, 2, 5}, /* index 10 */
        {6, 1, 0, 9}, /* index 11 */
        {4, 3, 8, 9}, /* index 12 */
        {0, 4, 2, 10}, /* index 13 */
        {3, 10, 8, 7}, /* index 14 */
        {1, 11, 0, 3}, /* index 15 */
        {6, 5, 1, 0}, /* index 16 */
        {0, 2, 4, 8}, /* index 17 */
        {5, 0, 1, 4}, /* index 18 */
        {10, 3, 5, 1} /* index 19 */
};


/* function that we get from lm_new.c */
extern int (*l_n36_buf)();

/* function pointer that will be set later. */
void (*l_n36_buff)();

/************************************************************
*
* extr: demostrate calls to lm_new
*
*/

void main(int argc, char *argv)
{
        int retcode;
        int i;
        int key_index;
        int tabindex1;
        int tabindex2;
        int tabindex3;
        int tabindex4;
        int decrypt_xor_value;

        MY_VENDORCODE5 teststruct;
        char myvendorname[1024];  /* This is the vendor_id */

        /* Job structure */
        struct s_tmp
        {
                int i;
                char *cp;
                unsigned char a[12];
        } myjob;

        /* Initialize job */
        myjob.i = 66;
        myjob.cp = 0;

        /*
         * ensure that l_n36_buff is null, since lm_new tests
         * for this before setting it.
         */
        l_n36_buff = 0;

        /* Get vendorname with clear keys but encrypted seeds */
        retcode = (*l_n36_buf)(myvendorname, &teststruct);
        if (retcode != 1)
        {
                printf ("Problem with call to lm_new initializer.\n");
                printf ("retcode = %d\n", retcode);
                return;
        }

        /* DEBUG see what was returned */
        printf ("After call to l_n36_buf routine.\n");
        printf ("myvendorname: %s\n", myvendorname);
        for (i = 0; i < 2; i++)
        {
                printf ("data[%d] = %08x\n", i, teststruct.data[i]);
        }
        for (i = 0; i < 4; i++)
        {
                printf ("keys[%d] = %08x\n", i, teststruct.keys[i]);
        }

        (*l_n36_buff)(&myjob, myvendorname, &teststruct);

        /*
         * extract the xoring value for the job now.
         *
         * the index into the decoding table is the first
         * character of the vendor name mod 20.
         */

        key_index = (int)(myvendorname[0] & 0xff) % 20;

        tabindex1 = decode_table[key_index].offsets[0];

        tabindex2 = decode_table[key_index].offsets[1];

        tabindex3 = decode_table[key_index].offsets[2];

        tabindex4 = decode_table[key_index].offsets[3];


        /*
         * use values to reverse xor which occurs if
         * valid job given.
         */
        decrypt_xor_value = ((long)(myjob.a[tabindex1]) << 0)
                | ((long)(myjob.a[tabindex2]) << 8)
                | ((long)(myjob.a[tabindex3]) << 16)
                | ((long)(myjob.a[tabindex4]) << 24);

        printf ("Decrypt xor value: %08x\n", decrypt_xor_value);
        printf ("seed1: %08x\n", teststruct.data[0] ^ decrypt_xor_value);
        printf ("seed2: %08x\n", teststruct.data[1] ^ decrypt_xor_value);

        return;
}

Here is one run of the program; you will get different results for
the decrypt xor value for each run, however the seeds will be the
same each time.



After call to l_n36_buf routine.
myvendorname: blenderd
data[0] = 6c116a9b

data[1] = adf8a253

keys[0] = c450f9f4

keys[1] = 4d12be88

keys[2] = f52bcf4d

keys[3] = 3309994c

Decrypt xor value: 37a2cfa5
seed1: ae37b151
seed2: 6fde7999


As Dan has mentioned in his essay, the important thing to observe is that
the decrypting function reveals the seeds which are encoded in the l_n36_buf
function.


The hiding methods that are used are quite good, but there are some 
weaknesses
that can be exploited.   Both seed1 and seed2 are xored with the same 
values,
so it may be possible to launch a brute force attack using 2**32 different
values.   If we are given seed1^secretval and seed2^secretval we can cycle
through the 2**32 values of secretval in search of the key.  It is generally
easier to simply dig the seeds out of the target.

An interesting point is that the vendor name isn't used at all in the
key generation process - if the license data and encyption seeds are left
unchanged, but the vendor name in the license and the vendor keys are 
changed,
the same license key results.  This agrees with Dan's findings.   I have 
tried
using quite different vendor keys, and the generated keys are the same, and
only depend on feature/hostid information and the encryption seeds.

Final 
Notes

This pretty much sums up the new protection.  Later versions
of FLEXlm never call the lm_new decoding routine with a null
pointer, so you never see plaintext seeds from this routine.

homepage links search_forms +ORC how to protect academy database
reality cracking how to search javascript wars
tools anonymity academy cocktails antismut CGI-scripts mail_Fravia
Is reverse engineering legal?