Forwarding data in a DLL

I need to forward a set of symbols from one DLL to another (to support some versioning scheme, PEP 384 if you wonder). It works fine for functions; I write a module definition file, saying

LIBRARY "python3"
EXPORTS
  PyArg_Parse=python32.PyArg_Parse
  PyArg_ParseTuple=python32.PyArg_ParseTuple
  PyArg_ParseTupleAndKeywords=python32.PyArg_ParseTupleAndKeywords
[...]

However, it fails for data. If I say

PyBaseObject_Type=python32.PyBaseObject_Type

then the linker complains that PyBaseObject_Type is an unresolved symbol, even though it actually is exported from python32.dll. Looking at the import library, I notice that, for data, there is only the _imp__ symbol, so I tried

PyBaseObject_Type=python32._imp__PyBaseObject_Type

The linker does actually create a DLL now, however, in this DLL, the forwarding goes to the _imp__ symbol, which then cannot be resolved at runtime. I also tried putting DATA into the line (with or without the _imp__); this doesn't make a difference.

IIUC, forwarding data should work fine, since the data is declared as __declspec(dllimport) for any importer of the DLL, so the compiler should interpret the reference correctly.

So: How can I generate a DLL 开发者_高级运维that does data forwarding?

---

It seems to me that the solution is not to use DATA during exporting of data from the main DLL (DLL which hold the data).

To reproduce what I mean you can create a project having DllDataForward.c:

#include <Windows.h>

EXTERN_C __declspec(dllexport) int myData = 5;

#ifndef _DEBUG
EXTERN_C BOOL WINAPI _DllMainCRTStartup (HINSTANCE hinstDLL, DWORD fdwReason,
                                         LPVOID lpvReserved)
#else
BOOL WINAPI DllMain (HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
#endif
{
    if (fdwReason == DLL_PROCESS_ATTACH)
        DisableThreadLibraryCalls(hinstDLL);

    return TRUE;
    UNREFERENCED_PARAMETER (lpvReserved);
}

EXTERN_C __declspec(dllexport) BOOL WINAPI MyFunc()
{
    return TRUE;
}

and DllDataForward.def:

LIBRARY "DllDataForward"
EXPORTS
    myData
    MyFunc

Typically one will be use "myData DATA" instead of "myData".

Then you can create a ForwardingDll.c:

#include <Windows.h>

#ifndef _DEBUG
EXTERN_C BOOL WINAPI _DllMainCRTStartup (HINSTANCE hinstDLL, DWORD fdwReason,
                                         LPVOID lpvReserved)
#else
BOOL WINAPI DllMain (HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
#endif
{
    if (fdwReason == DLL_PROCESS_ATTACH)
        DisableThreadLibraryCalls(hinstDLL);

    return TRUE;
    UNREFERENCED_PARAMETER (lpvReserved);
}

with ForwardingDll.def:

LIBRARY "ForwardingDll" 
EXPORTS
    myNewData=DllDataForward.myData DATA
    MyNewFunc=DllDataForward.MyFunc

You should include import library DllDataForward.lib created during compiling of the DllDataForward as input for the linker during building of the ForwardingDll.dll. Such import library do can be used successfully and you will receive ForwardingDll.dll.

dumpbin.exe ForwardingDll.dll /EXPORTS

produce as the output

...
    ordinal hint RVA      name

          1    0          MyNewFunc (forwarded to DllDataForward.MyFunc)
          2    1          myNewData (forwarded to DllDataForward.myData)
...

A simple test application build using DllDataForward.lib only having the source test.c:

#include <Windows.h>
#include <stdio.h>
#include <tchar.h>

EXTERN_C __declspec(dllimport) int myNewData;
EXTERN_C __declspec(dllimport) BOOL WINAPI MyNewFunc();

int main()
{
    BOOL isSuccess = MyNewFunc();
    int i=myNewData;
    _tprintf (TEXT("i=%d\nisSuccess=%s\n"),
              i, isSuccess? TEXT("TRUE"): TEXT("FALSE"));
}

produce as the output

i=5
isSuccess=TRUE

UPDATED: I want to add a little more information why the usage of "myData DATA" instead of "myData" in the DEF file do a trick and how to use the trick which I suggest with an existing DLL like python32.dll without any changes in python32.dll and without recompiling it. I will show that the original python32.lib miss the exports of all data variables like PyBaseObject_Type. I will show how you can create an additional python32.lib which do has symbols with the data which we need.

First of all I want to clear want changes we have in the import library after the changing from "myData DATA" to "myData" in the DEF file. First let us we compile DllDataForward.dll with the DEF file having "myData DATA" and we look inside of the import library DllDataForward.LIB:

dumpbin.exe DllDataForward.lib /all >%TEMP%\DllDataForward-lib.txt
notepad %TEMP%\DllDataForward-lib.txt

We will see that the lib has 6 public symbols:

  224 __IMPORT_DESCRIPTOR_DllDataForward
  46A __NULL_IMPORT_DESCRIPTOR
  5A8 DllDataForward_NULL_THUNK_DATA
  776 __imp__myData
  708 _MyFunc@0
  708 __imp__MyFunc@0

Next change DEF file from "myData DATA" to "myData", create dll and the import library and look inside it one more time. We will see that now the import library has 7 (!!!) instead of 6 public symbols:

  23A __IMPORT_DESCRIPTOR_DllDataForward
  480 __NULL_IMPORT_DESCRIPTOR
  5BE DllDataForward_NULL_THUNK_DATA
  78C __imp__myData
  78C _myData
  71E _MyFunc@0
  71E __imp__MyFunc@0

So we have problem with the usage of DEF file having "myData DATA" because the created import library not contain the public symbol _myData.

We can stay with the correct DLL having "myData DATA" and create an additional second import library which exports _myData manually. We will make no changes in the DllDataForward.dll, just make and additional lib manually.

To do this we dump the exports of the DllDataForward.dll with respect of dumpbin.exe DllDataForward.dll /exports. We will see:

...
    ordinal hint RVA      name

          1    0 00001020 MyFunc = _MyFunc@0
          2    1 00003000 myData = _myData
...

Now we create the new DllDataForward.def file in another directory based only on the output of dumpbin.exe DllDataForward.dll /exports:

LIBRARY "DllDataForward"
EXPORTS
    myData = _myData

Next using the command

lib.exe /DEF:DllDataForward.def /OUT:DllDataForward.lib /MACHINE:X86

we create the second DllDataForward.lib (in another directory an the original DllDataForward.lib). Now we can compile ForwardingDll.dll using two DllDataForward.lib and receive the DLL which we need. Test.exe will show that the approatch work.

Exactly in the same way we examine python32.lib from the current version 3.2a3:

dumpbin.exe "C:\Program Files\Python32\libs\python32.lib" /all >python32-lib.txt
notepad python32-lib.txt

we will find out following lines (about at the beginning of the file)

1957 public symbols
…
1BCCC _PyArg_Parse
1BCCC __imp__PyArg_Parse
…
1BFF6 __imp__PyBaseObject_Type
…

We can also verify with

dumpbin C:\Windows\system32\python32.dll /exports >%TEMP%\python32-exports.txt
notepad %TEMP%\python32-exports.txt

that the symbol PyBaseObject_Type will be exported as

 14    D 001DD5D0 PyBaseObject_Type

So we can create additional python32.lib from the python32.def file

LIBRARY "python32"
EXPORTS
    PyBaseObject_Type

using

lib /DEF:python32.def /OUT:python32.lib /MACHINE:X86

Now you can define the DEF of your dll

LIBRARY "python3"
EXPORTS
  PyArg_Parse=python32.PyArg_Parse
  PyArg_ParseTuple=python32.PyArg_ParseTuple
  PyArg_ParseTupleAndKeywords=python32.PyArg_ParseTupleAndKeywords
  PyBaseObject_Type=python32.PyBaseObject_Type DATA

exactly like you want originally, but we will use both "C:\Program Files\Python32\libs\python32.lib" and the small second python32.lib which we created during linking.

I don't use python myself and don't know the size of PyBaseObject_Type, but if I declare it as int

EXTERN_C __declspec(dllimport) int PyBaseObject_Type;

I can verify that the first part of PyBaseObject_Type is 1. It work!

Sorry for the long answer and thanks for all who read all my answer till this place.

---

I played around with this a bit and was not able to find the correct combination that worked with a .def file. However, I was able to forward both functions and data via the following pragmas in the source of the forwarding DLL:

#pragma comment(linker,"/export:_data=org.data,DATA")
#pragma comment(linker,"/export:_func=org.func")

Note I had to use the decorated data and function names.

Below are the files for a complete example:

org.c

int data = 5;
int func(int a)
{
    return a * 2;
}

org.def

EXPORTS
    data DATA
    func

fwd.c

#pragma comment(linker,"/export:_data=org.data,DATA")
#pragma comment(linker,"/export:_func=org.func")

int func2(int a)
{
    return a + 2;
}

fwd.def

EXPORTS
    func2

example.c

#include <stdio.h>

__declspec(dllimport) int data;
__declspec(dllimport) int func(int a);
__declspec(dllimport) int func2(int a);

int main()
{
    printf("data=%d func(5)=%d func2(5)=%d\n",data,func(5),func2(5));
    return 0;
}

makefile

all: example.exe org.dll

example.exe: example.c fwd.dll
    cl /W4 example.c /link fwd.lib

org.dll: org.c
    cl /LD /W4 org.c org.def

fwd.dll: fwd.c
    cl /LD /W4 fwd.c fwd.def

clean:
    del *.exe *.dll *.obj *.exp *.lib

---

This answer is slightly different, with no .def files and a forwarded __stdcall function, which is more similar to the situation. Note that to export the stdcall name, the decorated name was needed on both sides of the = in the /export. It also has a fancier #include "fwd.h" so it can be reused in the DLL and the client EXE.

org.c

#define ORGAPI __declspec(dllexport)

ORGAPI int data = 5;
ORGAPI int __stdcall func(int a)
{
    return a * 2;
}

fwd.c

#define FWDEXPORTS
#include "fwd.h"

int func2(int a)
{
    return a + 2;
}

fwd.h

#pragma once

#ifdef FWDEXPORTS
    #define FWDAPI __declspec(dllexport)
    // forwarded APIs exported here
    #pragma comment(linker,"/export:_func@4=org._func@4")
    #pragma comment(linker,"/export:_data=org.data")
#else
    #define FWDAPI __declspec(dllimport)
    // forwarded APIs imported here
    FWDAPI int __stdcall func(int a);
    FWDAPI int data;
#endif

// APIs unique to forwarding DLL here
FWDAPI int func2(int a);

example.c

#include <stdio.h>
#include "fwd.h"

int main()
{
    printf("data=%d func(5)=%d func2(5)=%d\n",data,func(5),func2(5));
    return 0;
}

makefile

all: example.exe org.dll

example.exe: example.c fwd.h fwd.dll
   cl /nologo /W4 example.c /link /nologo fwd.lib

org.dll: org.c
    cl /nologo /LD /W4 org.c /link /nologo

fwd.dll: fwd.c fwd.h
    cl /nologo /LD /W4 fwd.c /link /nologo

clean:
    del *.exe *.dll *.obj *.exp *.lib

---

This is actually a bug in the MS linker. But there are 2 workarounds.

Background: The import library for a dll contains small chunks of data that have symbols like imp__FooBar, where FooBar is the name of the exported function of data. When the importing module has declared the data or function using __declspec(dllimport), the compiler will automatically reference these __imp* symbols and create indirect function calls (call dword ptr [imp__FooBar]) or memory references. If a function was not declared with dllimport, it still works, because for function exports the library contains additionall symbols without the _imp prefix. These symbols belong to small code stubs that consists of an indirect jump instruction ("FooBar: jmp dword ptr [_imp__FooBar]"). On x86 the symbols would have a leading underscore prefix and possibly a stdcall decoration, but that is not important.

Now to the linker bug: The linker requires that the symbol that you want to forward to is available. This makes sense, since you need to link the import library of the target dll when building the forwarding dll and it prevents wrong exports to be generated. But in fact the linker takes a shortcut and doesn't check if the target symbol is in the right dll, but it only checks if the symbol is available. The symbol does not get linked into the dll. Now the real bug is that the linker checks for FooBar rather than for _imp_FooBar, which it should, because the latter is the actual import symbol, the former only a convenience "addon" for function exports.

Now to the possible 2 solutions. One was already mentioned: create a new import library that pretends to export a function instead of data. The only difference is that the generated import library creates an additional indirect jmp instruction stub that jumps to the exported data. This instruction does not have any purpose except providing the symbol that the linker needs. As mentioned before this stub does not get linked into the dll.

The second solution doesn't even require to create an import library and might be simpler in certain cases. You simply add the symbol to your forwarding dll. It can be any kind of symbol. "char FooBar;" is good enough. When your export file contains "FooBar=otherdll.FooBar DATA" the dll will have an export FooBar, which redirects to otherdll.dll, the symbol in the forwardng dll will not be used.