Hi, I'm trying to hot reload a DLL while the program is running to allow for rapid iteration in a game engine I am working on. I started reading up on how DLLs work between D code here: https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs

I followed that example, however when I try to do the following I crash:
1. Load DLL
2. Unload DLL
3. Load DLL again

The problem I am getting is a Bad File Descriptor exception right after step 2, on my very next writeln to be exact.

Here is the exception I am getting: http://imgur.com/a/b9XBm
I spent a couple of hours trying to debug it, googling all sorts of combinations of "dlang unloadlibrary bad file descriptor", until I got the idea to try "dlang freelibrary bad file descriptor" and found this old bug report from 2004:

http://forum.dlang.org/post/cg5hpf$kpv$1@digitaldaemon.com

This sounds absolutely spot on to the problems I am having. I tried to save stdout and reset it after unloading my library but I still crashed on the next writeln, does anyone have an idea for a workaround on this?

I'm hoping that someone like Walter or Andrei will see this and decide that it's time for this 2004 issue to finally die. :P

On 02/09/2017 10:38 PM, Markus Pursche wrote:
> Hi, I'm trying to hot reload a DLL while the program is running to allow for rapid iteration in a game engine I am working on. I started reading up on how DLLs work between D code here: https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs
> 
> I followed that example, however when I try to do the following I crash:
> 1. Load DLL
> 2. Unload DLL
> 3. Load DLL again
> 
> The problem I am getting is a Bad File Descriptor exception right after step 2, on my very next writeln to be exact.
> 
> Here is the exception I am getting: http://imgur.com/a/b9XBm
> I spent a couple of hours trying to debug it, googling all sorts of combinations of "dlang unloadlibrary bad file descriptor", until I got the idea to try "dlang freelibrary bad file descriptor" and found this old bug report from 2004:
> 
> http://forum.dlang.org/post/cg5hpf$kpv$1@digitaldaemon.com
> 
> This sounds absolutely spot on to the problems I am having. I tried to save stdout and reset it after unloading my library but I still crashed on the next writeln, does anyone have an idea for a workaround on this?
> 
> I'm hoping that someone like Walter or Andrei will see this and decide that it's time for this 2004 issue to finally die. :P

*sigh* 2018H1 plan I am listing shared library support.
Even better still, I will argue it as the ONLY language/druntime feature we need in it!

Seriously, this is the biggest thing holding back D atm.

On 02.09.2017 23:38, Markus Pursche wrote:
> Hi, I'm trying to hot reload a DLL while the program is running to allow for rapid iteration in a game engine I am working on. I started reading up on how DLLs work between D code here: https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs

Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section.

Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html

In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface

> 
> I followed that example, however when I try to do the following I crash:
> 1. Load DLL
> 2. Unload DLL
> 3. Load DLL again
> 
> The problem I am getting is a Bad File Descriptor exception right after step 2, on my very next writeln to be exact.
> 
> Here is the exception I am getting: http://imgur.com/a/b9XBm
> I spent a couple of hours trying to debug it, googling all sorts of combinations of "dlang unloadlibrary bad file descriptor", until I got the idea to try "dlang freelibrary bad file descriptor" and found this old bug report from 2004:
> 
> http://forum.dlang.org/post/cg5hpf$kpv$1@digitaldaemon.com
> 
> This sounds absolutely spot on to the problems I am having. I tried to save stdout and reset it after unloading my library but I still crashed on the next writeln, does anyone have an idea for a workaround on this?
> 
> I'm hoping that someone like Walter or Andrei will see this and decide that it's time for this 2004 issue to finally die. :P

You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;"
somewhere in your DllMain.

On Sunday, 3 September 2017 at 13:41:33 UTC, Rainer Schuetze wrote:
>
>
> On 02.09.2017 23:38, Markus Pursche wrote:
>> [...]
>
> Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section.
>
> Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html
>
> In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface
>
>> [...]
>
> You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;"
> somewhere in your DllMain.

This worked! So, is there any chance that you could tell me why this isn't the default behavior from the start? Like is there a reason for it not to be there automatically, like a performance loss or a leak or something?

On 03.09.2017 20:21, Markus Pursche wrote:
> On Sunday, 3 September 2017 at 13:41:33 UTC, Rainer Schuetze wrote:
>>
>>
>> On 02.09.2017 23:38, Markus Pursche wrote:
>>> [...]
>>
>> Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section.
>>
>> Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html
>>
>> In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface
>>
>>> [...]
>>
>> You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;"
>> somewhere in your DllMain.
> 
> This worked! So, is there any chance that you could tell me why this isn't the default behavior from the start? Like is there a reason for it not to be there automatically, like a performance loss or a leak or something?

This workaround has side effects, i.e. it doesn't automatically close any file still open by the DLLs' instance of the C runtime, so it might cause incomplete files if someone relies on these being automatically flushed and closed.

It's a bug in the dmc runtime. Only Walter can fix it as it is closed source.

On 9/3/2017 1:07 PM, Rainer Schuetze via Digitalmars-d wrote:
>
> This workaround has side effects, i.e. it doesn't automatically close any file still open by the DLLs' instance of the C runtime, so it might cause incomplete files if someone relies on these being automatically flushed and closed.
>
> It's a bug in the dmc runtime. Only Walter can fix it as it is closed source.

The source is on github, but it's in a private repo.  I'm sure Walter would be happy to give access to it should someone who wants to fix it step forward and ask him.

On 9/4/17 2:20 AM, Brad Roberts via Digitalmars-d wrote:
> On 9/3/2017 1:07 PM, Rainer Schuetze via Digitalmars-d wrote:
>>
>> This workaround has side effects, i.e. it doesn't automatically close any file still open by the DLLs' instance of the C runtime, so it might cause incomplete files if someone relies on these being automatically flushed and closed.
>>
>> It's a bug in the dmc runtime. Only Walter can fix it as it is closed source.
> 
> The source is on github, but it's in a private repo.  I'm sure Walter would be happy to give access to it should someone who wants to fix it step forward and ask him.

To that point, he did give me access, and I submitted a PR to fix an issue with pipes and std.process a few years back.

However, I was never able to completely build DMC's library from source, so in order to test, I had to replace the affected .obj file in the library with my modified one. I was missing a MASM specific version.

-Steve