I'm just wondering how friendly D is with DLL.
I haven't tryed yet but I'm aware you could write a D DLL.

My wonder is would it be a standart D feature. i.e. would phobos be shipped
as a DLL/.so ?
because, if not, even if I could write a D DLL, multiple D program will all
ship multiple copies of statically linked phobos library.

not only it's a waste of memory (when runned) but it also end with multiple GC which do not play nicely all together...

Lloyd Dupont wrote:
> I'm just wondering how friendly D is with DLL.
> I haven't tryed yet but I'm aware you could write a D DLL.
> 
> My wonder is would it be a standart D feature. i.e. would phobos be shipped
> as a DLL/.so ?
> because, if not, even if I could write a D DLL, multiple D program will all
> ship multiple copies of statically linked phobos library.
> 
> not only it's a waste of memory (when runned) but it also end with multiple
> GC which do not play nicely all together...

There is some basic information on creating DLL's on the Win32 page of Walter's D site:

http://www.digitalmars.com/d/windows.html

I would hazard the guess that Phobos would be available as a DLL as well as a static lib. This would parallel C/C++.

In modern Windows, a DLL is loaded into the address space of each calling program. So we are comparing a single static lib vs. a single DLL, the memory usage difference is minimal. The disk space differs certainly, but disk space in the modern age is cheap, far cheaper than memory. DLL's introduce their own issues -- mostly dependencies, but often times you need to do some tweaking to have them load the way you want them to vs. the default.

Most of the GC's that I've seen have problems when used with DLL's. For instance, if you have a DLL that is allocating memory for many different processes, you now need a GC that has to know about all the separate processes... more of an OS-level GC than an application-level GC. This is for a DLL that even knows about GC and uses the same GC as the calling app. You could also have the DLL rely on calling back into the calling app to use the app GC to alloc memory. Some clever system engineering can enable many different options. And then when calling functions in DLL's that are written in non-GC languages, it becomes even more of a puzzle.

Moreover, when two applications are built with different versions of D, you want to make sure that each is built and tested with all the versions of runtime libs that it is expected to use. Once you get a body  of code out there and the language evolves, there will be many runtime DLL's each with its own version of the GC, hopefully each runtime DLL will have a unique name.

Theoretically with static linking, a smart linker could leave out everything that isn't used resulting in very small runtime image sizes. However, in practice this seems difficult to achieve. At Borland, I built a version of the C RTL as individual object modules (not a LIB) to see if TLink would do a better job with it. Well, it was smaller but not that much smaller. D uses OPTLink and I'm not familiar with how smart it is about not linking in unreferenced obj's/modules. I do remember, fuzzily, that OPTLink was a pretty good linker, so I hopeful. The sieve.d example works out to about 90K, but I don't know what's in there. I need to run TDump and take a peek.

I don't know what will happen with modern Windows and DLL's as we know them today. Microsoft is dying to deprecate all the old styles of API's and go with .NET. However, much of the "OS" is still in COM -- DirectX for instance. And Microsoft got a lot of feedback on their attempt to back burner C++ (and replace it with C#) and is putting some more resources on properly bringing C++ into the .NET age.

And please forgive my DLL ramble. I've spent far too much time with Windows DLL's.

--ms

I will try to clarify my assumption to clarify my questions.

lets assume you have a stack/array/pool/set/something which hold the garbage collected values.

let's imaging that there is an internal function like this to access it:

Set getGCSet();


let's imaging you have a D executable called a.exe wich, of course have a
statically linked phobos library withe the folowing function:
Set getGCSet() in its own private address space.

let's imaging that tis a.exe program use a DLL which happened to be a D DLL. As it's also statically linked this DLL has it own copy of Set getGCSet(); which also run its own address space.

now, there is th duplicate 'Set getGCSet()', why not ? but it also mean you have to take particular care when passing an object from the exe to the DLL and the other way round, because you should assume (rightfully) thatthe other exe/dll doesn't know how to handle you reference. and this is so sad....

Where as if phobos would be a DLL both this exe and this DLL would share the same Set getGCSet() function and would shared object reference without minding too much.

I had to admit there is the problem that D could link against C or D exe/DLL
and that a C program could use a D DLL, so you have to take care anyway.
But maybe there is a way to solve this problem ?
Or did I get it completly wrong ?

Michael Slater wrote:

> Lloyd Dupont wrote:
>> I'm just wondering how friendly D is with DLL.
>> I haven't tryed yet but I'm aware you could write a D DLL.
>> 
>> My wonder is would it be a standart D feature. i.e. would phobos be
>> shipped as a DLL/.so ?
>> because, if not, even if I could write a D DLL, multiple D program will
>> all ship multiple copies of statically linked phobos library.
>> 
>> not only it's a waste of memory (when runned) but it also end with multiple GC which do not play nicely all together...
> 
> There is some basic information on creating DLL's on the Win32 page of Walter's D site:
> 
> http://www.digitalmars.com/d/windows.html
> 
> I would hazard the guess that Phobos would be available as a DLL as well as a static lib. This would parallel C/C++.
> 
> In modern Windows, a DLL is loaded into the address space of each calling program. So we are comparing a single static lib vs. a single DLL, the memory usage difference is minimal. The disk space differs certainly, but disk space in the modern age is cheap, far cheaper than memory. DLL's introduce their own issues -- mostly dependencies, but often times you need to do some tweaking to have them load the way you want them to vs. the default.
> 
> Most of the GC's that I've seen have problems when used with DLL's. For instance, if you have a DLL that is allocating memory for many different processes, you now need a GC that has to know about all the separate processes... more of an OS-level GC than an application-level GC. This is for a DLL that even knows about GC and uses the same GC as the calling app. You could also have the DLL rely on calling back into the calling app to use the app GC to alloc memory. Some clever system engineering can enable many different options. And then when calling functions in DLL's that are written in non-GC languages, it becomes even more of a puzzle.
> 
> Moreover, when two applications are built with different versions of D,
> you want to make sure that each is built and tested with all the
> versions of runtime libs that it is expected to use. Once you get a body
>   of code out there and the language evolves, there will be many runtime
> DLL's each with its own version of the GC, hopefully each runtime DLL
> will have a unique name.
> 
> Theoretically with static linking, a smart linker could leave out everything that isn't used resulting in very small runtime image sizes. However, in practice this seems difficult to achieve. At Borland, I built a version of the C RTL as individual object modules (not a LIB) to see if TLink would do a better job with it. Well, it was smaller but not that much smaller. D uses OPTLink and I'm not familiar with how smart it is about not linking in unreferenced obj's/modules. I do remember, fuzzily, that OPTLink was a pretty good linker, so I hopeful. The sieve.d example works out to about 90K, but I don't know what's in there. I need to run TDump and take a peek.
> 
> I don't know what will happen with modern Windows and DLL's as we know them today. Microsoft is dying to deprecate all the old styles of API's and go with .NET. However, much of the "OS" is still in COM -- DirectX for instance. And Microsoft got a lot of feedback on their attempt to back burner C++ (and replace it with C#) and is putting some more resources on properly bringing C++ into the .NET age.
> 
> And please forgive my DLL ramble. I've spent far too much time with Windows DLL's.
> 
> --ms

A few comments.

Wonder why a consrvative GC for C is always a DLL?
There are a number of resons for this:
 - because a DLL gets notified of thread events, and can keep track of them without need for special code generation;
 - because a DLL has no limitations acessing the main programme's memory. All dlls run in the same space as the main programme.

DLL mutable sections are loaded into each application's space. However, immutable sections, such as code, remain shared. Thus a DLL works as if multiple copies were made of it. But since there are means of inter-process communication, a DLL which knows itself very well could also use it to provide functionality, where one application should depend on another. "Came first- served first" and similar things.

Another thing I remember, when designing a DLL with Delphi, it was often requiered to include a special "memory manager DLL", which handled strings and something. I'm not really sure how.

If loaded only once at a time, a static lib certainly takes less space because:
 - it only contains the functions which are really in use;
 - and besides, it doesn't even need to be loaded completely to run. "Real" OSs load executables by parts.

However, when an appliation gets started and closed very often, it makes sense to place much of its functionality into a DLL, since DLLs are retained in memory after all programs using them are closed, and only unloaded when OS decides. (e.g. when it's gotten short of memory.)

A comment of how good a linker eliminates unused functions is really irrelevant. Its job is simple: if a function is referenced somewhere in the code, it gets retained. If not, it's removed. BUT referenced does not necesarily mean called. An example of this are classes. Every class carries referencies to functions in its VTable, so that they cannot be removed, even if you never call them. Maybe a compiler could have some way to identify which functions are "really" used? I'm not very clear about that, but anyway the linker is not to be blamed. AFAIK, linker simply gets a list of "symbols" associated with "something", such as code and data, it starts inserting "something" into the executable. Now, whenever this "something" references a "symbol", another "something" associated with a symbol is placed into the executable, and the final adress of it lands onto the place of that original reference. This way, all unreferenced stuff gets ignored.

BTW, that's where some way to add functionality to already created object instances could be useful. For example, a usual GUI element such as a button may support tons of features, like it can accept leaves falling on it, track weather and somesuch. :) But usually it's not requiered, and would only bloat an executable. Wasn't there some discussion on mix-ins? Like extension on interfaces?

DLLs have always been and shall continue to be. They are a basis of not only w1ndow$, but also of any OS.

-i.

Lloyd Dupont wrote:
> Where as if phobos would be a DLL both this exe and this DLL would share the same Set getGCSet() function and would shared object reference without minding too much.
> 
> I had to admit there is the problem that D could link against C or D exe/DLL and that a C program could use a D DLL, so you have to take care anyway. But maybe there is a way to solve this problem ?
> Or did I get it completly wrong ?

Most if not all modern C/C++ compilers provide for both static and dynamic linking with the runtime library. I believe D will be the same.

The default behavior under Windows is for a shared memory management DLL
(such as a runtime library with non-GC malloc(), garbage collector with GC malloc(), etc.) is to manage memory on a per-process basis.

For each process that calls a particular DLL, Windows maps that DLL's data segment into the address space of the calling process. Thus by default, no data is shared between DLL's.

If a developer wants to share some information amongst various instances of a DLL, the DLL's own data segment can be marked as shared. This data segment must contain initialized data for this to work properly. And I believe if a DLL has a shared data segment, only one instance of a particular calling application can be run.

So if we have:

A.EXE (PHOBOS100.LIB)
   KOOLGFX1.DLL (PHOBOS100.DLL)

Yes, we are going to have two GC's going. A doesn't know anything else other than to use the statically linked runtime lib. KOOLGFX1 of course was compiled to use the runtime DLL. I would not be surprised if you got runtime errors in this "mixed model" configuration. I don't remember the exact "why" of this, a downside to not having a Matt Pietrek to converse with.

To facilitate the sharing that you desire, you'd want to build 'everything' to use the runtime library DLL:

A.EXE (PHOBOS100.DLL)
   KOOLGFX1.DLL (PHOBOS100.DLL)

How memory management usually works is for a runtime DLL to use VirtualAlloc(). This function reserves memory in the virtual address space of the calling process. This memory is easy to work with.

For specific Windows-related tasks, it is important that a systems programming language provide a way to handle memory blocks that are outside of the usual runtime library functions and definitely outside the scope of the usual GC.

When we need to share memory between our application process and DDE, the Clipboard, or to use OLE/COM uniform data transfer operations we have to allocate memory for this using GlobalAlloc() which allocates memory off the Windows heap. Note this memory cannot be directly shared between processes.

To create a more complex system where memory is shared between processes, a couple things come to mind. One method would be to create a memory-mapped file that is shared by all the running processes.

One could also use VirtualAllocEx() along with ReadProcessMemory() and WriteProcessMemory()... going into the land of functions that a debugger usually uses. This would be an entirely kludgy system, though. Everything in the modern programming world assumes each process has its own heap and that this heap is more or less sacrosanct.

Is what I'm writing going in any helpful direction or are we wandering around in the weeds?

--ms