Hello all,

I have been trying out the "Win32 DLLs in D" example found at
http://www.digitalmars.com/d/dll.html (the "D code calling D code in DLLs"
section), and now I have many questions :)
I am trying to figure out how a plugin system could be written in D.

Heh, the first thing that strikes me about the example is that there are quite a few things I don't understand. Like what is "HINSTANCE g_hInst;" and what does it mean?

What does
"extern (C)
{
	void gc_init();
	void gc_term();
	void _minit();
	void _moduleCtor();
	void _moduleUnitTests();
}"
do and why is it contained within extern (C)? Isn't this D code interfacing
with D code?

What does
fp = GetProcAddress(h, "D5mydll16MyDLL_InitializeFPvZv");
do, and why is the second argument a mangled name?

Also I noticed that if I copied the code of mydll.d into a second file and named it "mydll2.d", test.d would fail to run it after I compiled it to "mydll2.dll". It would fail "MyDLL_Initialize()" ("error loading symbol MyDLL_Initialize()"). But if the source file is named mydll.d, compiled and then simply renamed to mydll2.dll, test.d will load it without problem. Why is that? It is not very versatile of a program only to accept a plugin compiled under a certain name - I would like a more general solution. Is that possible, possible but complicated or impossible?

I realize by now that it probably would be good idea to read up on how DLLs work in general. Does anyone have any tips on where (websites, books) I could find an introduction to this?


Many thanks!

Ok, let's see what I can help with...

Nox / Lux wrote:
> Hello all,
> 
> I have been trying out the "Win32 DLLs in D" example found at
> http://www.digitalmars.com/d/dll.html (the "D code calling D code in DLLs"
> section), and now I have many questions :)
> I am trying to figure out how a plugin system could be written in D.

To be perfectly honest, DDL is probably better suited for this.  It was designed specifically for runtime loading of D code, and avoids many problems that can crop up when using Windows DLLs.

http://dsource.org/projects/ddl

> Heh, the first thing that strikes me about the example is that there are quite a few things I don't understand. Like what is "HINSTANCE g_hInst;" and what does it mean?

I believe HINSTANCE is a handle to an instance of an application; a running process.  Without more context, I can't be 100% sure.

It's a Windows API type.  If you don't know what it is, you should probably go and read up on the Windows API.

> What does
> "extern (C)
> {
> 	void gc_init();
> 	void gc_term();
> 	void _minit();
> 	void _moduleCtor();
> 	void _moduleUnitTests();
> }"
> do and why is it contained within extern (C)? Isn't this D code interfacing
> with D code?

The first two functions are for starting and stopping D's garbage collector.  The next three are for running module startup code, unit tests, etc.

These functions are normally not directly used; the "main" function that starts the program takes care of setting all this up before calling your "main" function.  However, when you're making a Windows application, the built-in main isn't used; WinMain is called instead.  As such, you need to manually kick off the GC, initialize the modules, etc.

The reason they use "extern(C)" is that they are compiled with the C calling convention, and they're kinda-sorta externs to the current module.  This is done because these methods are called as part of bootstrapping the application, and the C calling convention is about the easiest way of doing this.  It also avoids name mangling problems, which is where

> What does
> fp = GetProcAddress(h, "D5mydll16MyDLL_InitializeFPvZv");
> do, and why is the second argument a mangled name?

comes in to play.  GetProcAddress is a Windows API call.  Windows doesn't know ANYTHING about D's name mangling.  It was designed for C libraries, which don't use name mangling (apart from putting in an initial "_" before the symbol's name).  As a result, in order to load a D symbol from a DLL, you need to use the fully mangled name.

Ugly, eh? :P

> Also I noticed that if I copied the code of mydll.d into a second file and named it "mydll2.d", test.d would fail to run it after I compiled it to "mydll2.dll". It would fail "MyDLL_Initialize()" ("error loading symbol MyDLL_Initialize()"). But if the source file is named mydll.d, compiled and then simply renamed to mydll2.dll, test.d will load it without problem. Why is that? It is not very versatile of a program only to accept a plugin compiled under a certain name - I would like a more general solution. Is that possible, possible but complicated or impossible?

This is because a module's filename is the same as the module's name. If we de-mangle the above name, the symbol's full name is:

  mydll.MyDLL_Initialize

When you rename the source file to mydll2, what happens is that the symbol's full name becomes:

  mydll2.MyDLL_Initialize

You're basically changing what module the function is in, which is why the application can't load the symbol; it exists under a different name!

You can avoid this by requiring that a plugin's initialize function is named after the .DLL file itself.  So a plugin stored in "myplugin.dll" would have an initializer function called "myplugin.Initialize".

> I realize by now that it probably would be good idea to read up on how DLLs work in general. Does anyone have any tips on where (websites, books) I could find an introduction to this?

Not especially.  Your best bet is just to google for information on the subject, or maybe try MSDN.

> Many thanks!

Like I said above, if you want to write a D program that uses plugins, you *need* to take a look at DDL.  It's absolutely brilliant.

Hope this helps,

	-- Daniel

-- 
Unlike Knuth, I have neither proven or tried the above; it may not even make sense.

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/

Nox / Lux wrote:
> Hello all,
> 
> I have been trying out the "Win32 DLLs in D" example found at
> http://www.digitalmars.com/d/dll.html (the "D code calling D code in DLLs"
> section), and now I have many questions :)
> I am trying to figure out how a plugin system could be written in D.
> 
> Heh, the first thing that strikes me about the example is that there are quite
> a few things I don't understand. Like what is "HINSTANCE g_hInst;" and what
> does it mean?

HINSTANCE is a type definition from the Win32 API, which means essentially "instance handle". Win32 is a C API that hides the actual implementation of data structures behind HANDLE types. Every application that runs on Windows is assigned an instance handle of type HINSTANCE. Some Win32 API calls require this handle to be passed as an argument. If you don't need to call Win32 or setup DirectX or anything, then you don't need to worry about it.

> 
> What does
> "extern (C)
> {
> 	void gc_init();
> 	void gc_term();
> 	void _minit();
> 	void _moduleCtor();
> 	void _moduleUnitTests();
> }"
> do and why is it contained within extern (C)? Isn't this D code interfacing
> with D code?

These functions are part of the bootstrap code that launches every D application. It is all implemented in C, not in D. Every D application must call these functions in order to operate properly. When using a main() method, this is done automatically for you (look in dmd/src/phobos/dmain2.d for the implementation -- this is the real app entrypoint function that is executed from C code before your D main function is ever called). When using WinMain, you are creating the acutal entry point and bypassing the default bootstrap code, so you are responsible for calling all of the C bootstrap stuff yourself.

> 
> What does
> fp = GetProcAddress(h, "D5mydll16MyDLL_InitializeFPvZv");
> do, and why is the second argument a mangled name?

GetProcAddress is a Win32 API function that returns a pointer to a function in a DLL. Normally, when linking with an import library on Windows the DLL is imported into the application's memory space for you automatically. But an API also exists for you to do the same thing manually, and that is what is happening here.

The mangled name is the name of the function as it appears in the DLL. If you were to declare a D function as "extern C", it would use a different form of mangling. In this case, MyDLL_Initialize has D linkage (i.e. the default extern(D)) and therefore uses D mangling. When fetching a function pointer from a DLL using the GetProcAddress, you must know the mangled function name. It is possible to create aliases for DLL function names via a DEF file.

> 
> Also I noticed that if I copied the code of mydll.d into a second file and
> named it "mydll2.d", test.d would fail to run it after I compiled it to
> "mydll2.dll". It would fail "MyDLL_Initialize()" ("error loading symbol
> MyDLL_Initialize()"). But if the source file is named mydll.d, compiled and
> then simply renamed to mydll2.dll, test.d will load it without problem. Why is
> that? It is not very versatile of a program only to accept a plugin compiled
> under a certain name - I would like a more general solution. Is that possible,
> possible but complicated or impossible?

It's because of D's name mangling. The module in which a method appears is included in the mangled name. Notice the first part of the mangled function name: D5mydll. See the mydll there? The source file 'mydll.d' is the module 'mydll'. The source file 'mydll2.d' is the module 'mydll2'. So when you compiled mydll2.d, the mangled name in the DLL changed ('D5mydll' would be replaced by 'D5mydll2'). You didn't change the name of the function you were trying to load, so the call to GetProcAddress failed.

The inclusion of module names in mangled function names makes it impossible to implement a generic plug-in system by loading mangled functions. I've never used D DLLs before, but I've used them in C a lot. I expect that if you research .def files and function name aliases, you should be able to cobble together a solution.

If you want a premade solution, you might want to check out the DDL (D Dynamic Libraries) project at dsource.org.

> 
> I realize by now that it probably would be good idea to read up on how DLLs
> work in general. Does anyone have any tips on where (websites, books) I could
> find an introduction to this?

You might start at MSDN (Microsoft Developer Network) and the Knowledge Base there: http://msdn2.microsoft.com/en-us/default.aspx. Also, Charles Petzold's book "Programming Windows, 5th Edition" has a section on DLLs. That's a big book to buy just for the one section though, so I'd only recommend it if you are interested in learning Win32 programming in general. It's an excellent source for doing so. Otherwise, I can't think of any online DLL resources off the top of my head. See what Google tells you.

Daniel Keep wrote:
...
> 
> Hope this helps,
> 
> 	-- Daniel
> 

I'm a really slow typist!

Daniel Keep: Like I said above, if you want to write a D program that uses plugins, you *need* to take a look at DDL.  It's absolutely brilliant.

Mike Parker: If you want a premade solution, you might want to check out the DDL (D Dynamic Libraries) project at dsource.org.

I actually checked it out first thing when I got this into my head. It looks just like what the doctor ordered, but unfortunately I found it a bit problematic at the moment. There are a few tickets outlining some issues that needs to be worked through before a stable solution can be built on it, I believe. But I WILL end up using DDL eventually. Portability is just one of the reasons. But I figure DLL:s will be good enough for a proof of concept of what I intend to do. DDL is due to hit 1.0 before the end of the year if the developers keep at it. I am very excited about DDL, it looks promising.

Thank you both for your answers. I believe I understand a bit more of what is going on here now.

Mike Parker wrote:
> [lots]

You might be slower, but at least you're clearer than I am :3

	-- Daniel

-- 
Unlike Knuth, I have neither proven or tried the above; it may not even make sense.

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/