Thread overview
Exported Classes and DLLs. Working Exceptions from Dlls
Nov 17, 2005
Venix
Nov 17, 2005
pragma
Nov 18, 2005
Venix
Nov 18, 2005
Venix
Nov 19, 2005
Venix
November 17, 2005
After a little bit of time and effort spent working with DLLs, D, exceptions, and classes.  I believe I understand the basics to the problems that exist.

Exceptions crash.
Annoying. One solution was to wrap all dll bodies with try/catch, which would be going back to the C method. So i'm going to present a solution that works.

The problem here is that Exceptions are static. Meaning both your DLL and program get a copy. So if your using the posted method of syncing your garbage collection, this is trivial to do. Just add a void*[] argument to you Load function. Then when you call it. Call
  Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
Now within Load. You have to copy the supplied vtbl over the internal Exception vtbl.
    for (int i=0; i<vtbl.length;++i)
        Exception.classinfo.vtbl[i] = vtbl[i];

Now exceptions should work like a charm

http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)


Minor issues(Can be worked around with a simple D parser):
1: new cannot be used on an imported class.
2: Classes/Interfaces cannot be inherited from a DLL

Both those stem from the class name not be exported. Quoting D's specs. "A class can be exported, which means its name and all its non-private members are exposed externally to the DLL or EXE." which i have found not to be true with the current compiler.

Solution:
I compiled the D front end. Modified func.c FunctionDeclaration::isExport to return true if the class is exported. Also i added isExport to AggregateDeclaration.  This allowed me to export all exported symbols. Class/Interfaces/etc need to be doctored with _Class_ _interface, and the _ from functions need to be removed.

That allows me to inherit and theoretically call new on functions. Then I run into access violations.

Major Issues(I didn't find a solution):
1: Inheritance doesn't work. Access violation on method calls.
2: new causes access violation.

After a few hours of fiddling. I came to the following conclusions.
If I make a static library. It imports 2 more symbols for every class _init_ & _vtbl_. So I thought if I exported em it may work.
Wrong. The linker/compiler never uses those for exported classes.

In fact I believe all access violations, with dlls, are caused by that issue.  Upon inspecting the imported class within its dll. ClassInfo resolves into something useful. Outside the class, ClassInfo is garbage, just try to print baseclass.classinfo.name.  I have tried to fix this improper Classinfo structure by copying over the real with the bad, but i got nothing but access violations. So I'll assume that this ClassInfo structure is not initialized and I'm just writing to bad memory.

I don't know how that structure is made. I would guess maybe by _init_ or _vtbl_. But I hope this information may help fix the issues.

I'm going to play around with the GDC compiler in hopes that I can get a working solution with that. I believe once DLL classes are supported phobos/ares can be can be compiled as a dll and then the exception issue will resolve itself. Along with the garbage collection. Allowing Dlls to be used without a load function.
November 17, 2005
If it helps, I documented these and a few other drawbacks to DLL's in D a while back:

http://www.prowiki.org/wiki4d/wiki.cgi?BestPractices/DLL

It prompted me to start this:

http://trac.dsource.org/projects/ddl

Which is an outright replacement to operating with DDLs by borrowing a lot from other (existing) models of dynamic module usage.  While the project is still alpha/beta, it should be pretty clear what's been accomplished so far without needing to hack the compiler or use a modified frontend.

If you're interested, the current proof-of-concept for dynamic linking is described here:

http://www.dsource.org/forums/viewtopic.php?p=6538#6419

- EricAnderton at yahoo


In article <dlievm$1lne$1@digitaldaemon.com>, Venix says...
>
>After a little bit of time and effort spent working with DLLs, D, exceptions, and classes.  I believe I understand the basics to the problems that exist.
>
>Exceptions crash.
>Annoying. One solution was to wrap all dll bodies with try/catch, which
>would be going back to the C method. So i'm going to present a solution
>that works.
>
>The problem here is that Exceptions are static. Meaning both your DLL and program get a copy. So if your using the posted method of syncing your garbage collection, this is trivial to do. Just add a void*[] argument to you Load function. Then when you call it. Call
>   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
>Now within Load. You have to copy the supplied vtbl over the internal Exception vtbl.
>     for (int i=0; i<vtbl.length;++i)
>         Exception.classinfo.vtbl[i] = vtbl[i];
>
>Now exceptions should work like a charm
>
>http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)
>
>
>Minor issues(Can be worked around with a simple D parser):
>1: new cannot be used on an imported class.
>2: Classes/Interfaces cannot be inherited from a DLL
>
>Both those stem from the class name not be exported. Quoting D's specs. "A class can be exported, which means its name and all its non-private members are exposed externally to the DLL or EXE." which i have found not to be true with the current compiler.
>
>Solution:
>I compiled the D front end. Modified func.c
>FunctionDeclaration::isExport to return true if the class is exported.
>Also i added isExport to AggregateDeclaration.  This allowed me to
>export all exported symbols. Class/Interfaces/etc need to be doctored
>with _Class_ _interface, and the _ from functions need to be removed.
>
>That allows me to inherit and theoretically call new on functions. Then I run into access violations.
>
>Major Issues(I didn't find a solution):
>1: Inheritance doesn't work. Access violation on method calls.
>2: new causes access violation.
>
>After a few hours of fiddling. I came to the following conclusions.
>If I make a static library. It imports 2 more symbols for every class
>_init_ & _vtbl_. So I thought if I exported em it may work.
>Wrong. The linker/compiler never uses those for exported classes.
>
>In fact I believe all access violations, with dlls, are caused by that issue.  Upon inspecting the imported class within its dll. ClassInfo resolves into something useful. Outside the class, ClassInfo is garbage, just try to print baseclass.classinfo.name.  I have tried to fix this improper Classinfo structure by copying over the real with the bad, but i got nothing but access violations. So I'll assume that this ClassInfo structure is not initialized and I'm just writing to bad memory.
>
>I don't know how that structure is made. I would guess maybe by _init_ or _vtbl_. But I hope this information may help fix the issues.
>
>I'm going to play around with the GDC compiler in hopes that I can get a working solution with that. I believe once DLL classes are supported phobos/ares can be can be compiled as a dll and then the exception issue will resolve itself. Along with the garbage collection. Allowing Dlls to be used without a load function.


November 18, 2005
After playing around with GDC.  I finally got it compiled with mingw and msys.  With a few simple modifications GDC can have full support for windows DLLs.  Inheritance and calling.  I haven't tested this extensively. My simple test case I had for DMD compiled and ran with GDC.  I will be playing with them more in the upcoming weeks, so any subtle errors will most likely show themselves.

One rather minor issue, is my exception fix no longer works. As I believe GDC makes vtbl's readonly. Once again making phobos a dll would fix that.  I'll try the a2dll tool that comes with mingw and see how that works.

Venix wrote:
> After a little bit of time and effort spent working with DLLs, D, exceptions, and classes.  I believe I understand the basics to the problems that exist.
> 
> Exceptions crash.
> Annoying. One solution was to wrap all dll bodies with try/catch, which would be going back to the C method. So i'm going to present a solution that works.
> 
> The problem here is that Exceptions are static. Meaning both your DLL and program get a copy. So if your using the posted method of syncing your garbage collection, this is trivial to do. Just add a void*[] argument to you Load function. Then when you call it. Call
>   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
> Now within Load. You have to copy the supplied vtbl over the internal Exception vtbl.
>     for (int i=0; i<vtbl.length;++i)
>         Exception.classinfo.vtbl[i] = vtbl[i];
> 
> Now exceptions should work like a charm
> 
> http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)
> 
> 
> Minor issues(Can be worked around with a simple D parser):
> 1: new cannot be used on an imported class.
> 2: Classes/Interfaces cannot be inherited from a DLL
> 
> Both those stem from the class name not be exported. Quoting D's specs. "A class can be exported, which means its name and all its non-private members are exposed externally to the DLL or EXE." which i have found not to be true with the current compiler.
> 
> Solution:
> I compiled the D front end. Modified func.c FunctionDeclaration::isExport to return true if the class is exported. Also i added isExport to AggregateDeclaration.  This allowed me to export all exported symbols. Class/Interfaces/etc need to be doctored with _Class_ _interface, and the _ from functions need to be removed.
> 
> That allows me to inherit and theoretically call new on functions. Then I run into access violations.
> 
> Major Issues(I didn't find a solution):
> 1: Inheritance doesn't work. Access violation on method calls.
> 2: new causes access violation.
> 
> After a few hours of fiddling. I came to the following conclusions.
> If I make a static library. It imports 2 more symbols for every class _init_ & _vtbl_. So I thought if I exported em it may work.
> Wrong. The linker/compiler never uses those for exported classes.
> 
> In fact I believe all access violations, with dlls, are caused by that issue.  Upon inspecting the imported class within its dll. ClassInfo resolves into something useful. Outside the class, ClassInfo is garbage, just try to print baseclass.classinfo.name.  I have tried to fix this improper Classinfo structure by copying over the real with the bad, but i got nothing but access violations. So I'll assume that this ClassInfo structure is not initialized and I'm just writing to bad memory.
> 
> I don't know how that structure is made. I would guess maybe by _init_ or _vtbl_. But I hope this information may help fix the issues.
> 
> I'm going to play around with the GDC compiler in hopes that I can get a working solution with that. I believe once DLL classes are supported phobos/ares can be can be compiled as a dll and then the exception issue will resolve itself. Along with the garbage collection. Allowing Dlls to be used without a load function.
November 18, 2005
Phobos DLL
a2dll failed. So i did the old fashioned trick of exploding the library and remaking it a dll. Tha gave me link errors. So I then decided to move all the problemsome functions into a folder called lib.  Once i got a clean compile. I had a Phobos dll.

Disclaimer:
For the record. I have little internal knowledge on dlls or even libraries for that matter. Just lots of accumlated random knowledge. So I can't guarntee that the dll produced will always work for every situation.  I can't even guaranteee it fixes the GC DLL issue.  All I can say is that it didn't crash and handles the Exceptions problem.

With that said, Here's the procedure: It does require the modifed GDC to suport full DLLs.

1. Create a directory called gphobos. Enter.
2. Execute "ar x libgphobos.a". That will extrain all the obj files.
3. Create a lib directory.
4. move cmain.o com.o cstream.o dgccmain2.o file.o iunknown.o loader.o mmfile.o moduleinit.o recls.o recls_api.o recls_api_win32.o rundmain.o socket.o socketstream.o stream.o zip.o zlib.o Into lib
5. In the lib directory type "ar r libg.a *.o". Return to gphobos.
6. Execute "gcc -shared -o gphobos.dll *.o  lib/libg.a -Wl,--out-implib=libgphobos.a"
7. Re-enter lib. type "ar x ../libgphobos.a". Once again this explodes the library.We do it to get the function stubs.
8. Execute "ar r libgphobos.a *.o". Now copy over your orignal libgphobos in your gdc/lib directory.

Notes:
Multiple files may produce duplicate symbols. moduleinit.o will conflict with D00XXX.o when you first try to make your program. I do not know how exactly to determine this before creating the library. So when it tells you a name. Delete the dxxxx.o file and redo step 8.

Once again I guarantee none of this. I can try to help if you want to follow my footsteps.

If someone can tell me how to use patch. I can release the gdc dll changes. If someone can tell me how to determine which obj files will conflict before linking. I can try to create a script that will auto convert phobos to a dll.

Venix wrote:
> After playing around with GDC.  I finally got it compiled with mingw and msys.  With a few simple modifications GDC can have full support for windows DLLs.  Inheritance and calling.  I haven't tested this extensively. My simple test case I had for DMD compiled and ran with GDC.  I will be playing with them more in the upcoming weeks, so any subtle errors will most likely show themselves.
> 
> One rather minor issue, is my exception fix no longer works. As I believe GDC makes vtbl's readonly. Once again making phobos a dll would fix that.  I'll try the a2dll tool that comes with mingw and see how that works.
> 
> Venix wrote:
> 
>> After a little bit of time and effort spent working with DLLs, D, exceptions, and classes.  I believe I understand the basics to the problems that exist.
>>
>> Exceptions crash.
>> Annoying. One solution was to wrap all dll bodies with try/catch, which would be going back to the C method. So i'm going to present a solution that works.
>>
>> The problem here is that Exceptions are static. Meaning both your DLL and program get a copy. So if your using the posted method of syncing your garbage collection, this is trivial to do. Just add a void*[] argument to you Load function. Then when you call it. Call
>>   Load(std.gc.getGCHandle(), Exception.classinfo.vtbl);
>> Now within Load. You have to copy the supplied vtbl over the internal Exception vtbl.
>>     for (int i=0; i<vtbl.length;++i)
>>         Exception.classinfo.vtbl[i] = vtbl[i];
>>
>> Now exceptions should work like a charm
>>
>> http://www.digitalmars.com/d/dll.html Dll Info(garbage collection sync)
>>
>>
>> Minor issues(Can be worked around with a simple D parser):
>> 1: new cannot be used on an imported class.
>> 2: Classes/Interfaces cannot be inherited from a DLL
>>
>> Both those stem from the class name not be exported. Quoting D's specs. "A class can be exported, which means its name and all its non-private members are exposed externally to the DLL or EXE." which i have found not to be true with the current compiler.
>>
>> Solution:
>> I compiled the D front end. Modified func.c FunctionDeclaration::isExport to return true if the class is exported. Also i added isExport to AggregateDeclaration.  This allowed me to export all exported symbols. Class/Interfaces/etc need to be doctored with _Class_ _interface, and the _ from functions need to be removed.
>>
>> That allows me to inherit and theoretically call new on functions. Then I run into access violations.
>>
>> Major Issues(I didn't find a solution):
>> 1: Inheritance doesn't work. Access violation on method calls.
>> 2: new causes access violation.
>>
>> After a few hours of fiddling. I came to the following conclusions.
>> If I make a static library. It imports 2 more symbols for every class _init_ & _vtbl_. So I thought if I exported em it may work.
>> Wrong. The linker/compiler never uses those for exported classes.
>>
>> In fact I believe all access violations, with dlls, are caused by that issue.  Upon inspecting the imported class within its dll. ClassInfo resolves into something useful. Outside the class, ClassInfo is garbage, just try to print baseclass.classinfo.name.  I have tried to fix this improper Classinfo structure by copying over the real with the bad, but i got nothing but access violations. So I'll assume that this ClassInfo structure is not initialized and I'm just writing to bad memory.
>>
>> I don't know how that structure is made. I would guess maybe by _init_ or _vtbl_. But I hope this information may help fix the issues.
>>
>> I'm going to play around with the GDC compiler in hopes that I can get a working solution with that. I believe once DLL classes are supported phobos/ares can be can be compiled as a dll and then the exception issue will resolve itself. Along with the garbage collection. Allowing Dlls to be used without a load function.
November 19, 2005
Due to some issues with the first method I described. I've generated a
script which will do all the work for you.  One oddity in the script tho
is I let moduleinit.o get included twice.  This is because I need to
static link it. Yet it's required by the majority of the code. The
reason for the static link is that _moduleDtor crashes on program exit.
   Statically linking it fixes it.

Script should be ran with msys.

Create a directory called gphobos.
Put the script in it. Put libgphobos.a in it.
Run the script. gphobos.dll can be found in the root. libgphobos.a can
be found in the ./lib/

Just copy the new libgphobos.a over the old. And make sure you include gphobos.dll with the binaries.