The following causes the DLL to crash on Windows:

Input: immutable(short)* data (immutable because in separate thread).
// Later
core.stdc.stdlib.free(cast(short *)data);

(short* data is provided by the C library, where the memory is allocated)

On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?

On Wed, 12 Nov 2014 12:40:30 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:

> The following causes the DLL to crash on Windows:
> 
> Input: immutable(short)* data (immutable because in separate
> thread).
> // Later
> core.stdc.stdlib.free(cast(short *)data);
> 
> (short* data is provided by the C library, where the memory is allocated)
> 
> On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?
seems that you are using two different allocators here. one is that comes with DLL and other that comes with D. i bet they either using different runtimes, or C runtime is doesn't know about another C runtime in DLL.

on GNU/Linux there is only one runtime (most of the time), so there is no problem with different allocators.

On Wed, 12 Nov 2014 12:40:30 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:

> The following causes the DLL to crash on Windows:
> 
> Input: immutable(short)* data (immutable because in separate
> thread).
> // Later
> core.stdc.stdlib.free(cast(short *)data);
> 
> (short* data is provided by the C library, where the memory is allocated)
> 
> On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?
p.s. i mean "different C runtimes".

On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via Digitalmars-d-learn wrote:
> On Wed, 12 Nov 2014 12:40:30 +0000
> Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>
>> The following causes the DLL to crash on Windows:
>> 
>> Input: immutable(short)* data (immutable because in separate thread).
>> // Later
>> core.stdc.stdlib.free(cast(short *)data);
>> 
>> (short* data is provided by the C library, where the memory is allocated)
>> 
>> On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?
> seems that you are using two different allocators here. one is that
> comes with DLL and other that comes with D. i bet they either using
> different runtimes, or C runtime is doesn't know about another C
> runtime in DLL.
>
> on GNU/Linux there is only one runtime (most of the time), so there
> is no problem with different allocators.

That makes a lot of sense. Hm. How can I work around this problem then? What's involved are a C-dll and a D-dll that uses the C-dll and is loaded into a Python program. To complicate things further short* data is passed to C by D. short* data is then allocated and populated in C.

On Wed, 12 Nov 2014 14:11:35 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:

> On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via Digitalmars-d-learn wrote:
> > On Wed, 12 Nov 2014 12:40:30 +0000
> > Chris via Digitalmars-d-learn
> > <digitalmars-d-learn@puremagic.com> wrote:
> >
> >> The following causes the DLL to crash on Windows:
> >> 
> >> Input: immutable(short)* data (immutable because in separate
> >> thread).
> >> // Later
> >> core.stdc.stdlib.free(cast(short *)data);
> >> 
> >> (short* data is provided by the C library, where the memory is allocated)
> >> 
> >> On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?
> > seems that you are using two different allocators here. one is
> > that
> > comes with DLL and other that comes with D. i bet they either
> > using
> > different runtimes, or C runtime is doesn't know about another C
> > runtime in DLL.
> >
> > on GNU/Linux there is only one runtime (most of the time), so
> > there
> > is no problem with different allocators.
> 
> That makes a lot of sense. Hm. How can I work around this problem then? What's involved are a C-dll and a D-dll that uses the C-dll and is loaded into a Python program. To complicate things further short* data is passed to C by D. short* data is then allocated and populated in C.
if you can extend C DLL, just add wrapper for `free()` there. so you will not call `free()` from D, but call C DLL function which will free the memory. it's a good practice anyway, 'cause it's recommended to free memory in the same library where you allocated it.

On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via Digitalmars-d-learn wrote:
> On Wed, 12 Nov 2014 14:11:35 +0000
> Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>
>> On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via Digitalmars-d-learn wrote:
>> > On Wed, 12 Nov 2014 12:40:30 +0000
>> > Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>> >
>> >> The following causes the DLL to crash on Windows:
>> >> 
>> >> Input: immutable(short)* data (immutable because in separate thread).
>> >> // Later
>> >> core.stdc.stdlib.free(cast(short *)data);
>> >> 
>> >> (short* data is provided by the C library, where the memory is allocated)
>> >> 
>> >> On Linux it works fine and never crashes, in the Windows DLL it randomly causes an access violation in memory (both read and write). Note that it doesn't crash immediately, it goes on for a while, but sooner or later it crashes. If I comment out this line, everything works fine. However, if I don't free the memory, I'll have a memory leak. Any hints/advice/guesses?
>> > seems that you are using two different allocators here. one is that
>> > comes with DLL and other that comes with D. i bet they either using
>> > different runtimes, or C runtime is doesn't know about another C
>> > runtime in DLL.
>> >
>> > on GNU/Linux there is only one runtime (most of the time), so there
>> > is no problem with different allocators.
>> 
>> That makes a lot of sense. Hm. How can I work around this problem then? What's involved are a C-dll and a D-dll that uses the C-dll and is loaded into a Python program. To complicate things further short* data is passed to C by D. short* data is then allocated and populated in C.
> if you can extend C DLL, just add wrapper for `free()` there. so you
> will not call `free()` from D, but call C DLL function which will free
> the memory. it's a good practice anyway, 'cause it's recommended to
> free memory in the same library where you allocated it.

I initially had an implementation that did exactly that (I usually do that), but for some reason it didn't work properly in this particular case and caused all sorts of undefined behavior. But I'll have a look at it again.

On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
> On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
>> if you can extend C DLL, just add wrapper for `free()` there. so you
>> will not call `free()` from D, but call C DLL function which will free
>> the memory. it's a good practice anyway, 'cause it's recommended to
>> free memory in the same library where you allocated it.
>
> I initially had an implementation that did exactly that (I usually do that), but for some reason it didn't work properly in this particular case and caused all sorts of undefined behavior. But I'll have a look at it again.

I've changed the code so that the memory is freed in C. Although it works "better" it crashes too every now and then

(WindowsError : exception : access violation writing 0x0310A1B4)

Will look into it.

On Wed, 12 Nov 2014 16:03:08 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:

> On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
> > On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
> >> if you can extend C DLL, just add wrapper for `free()` there.
> >> so you
> >> will not call `free()` from D, but call C DLL function which
> >> will free
> >> the memory. it's a good practice anyway, 'cause it's
> >> recommended to
> >> free memory in the same library where you allocated it.
> >
> > I initially had an implementation that did exactly that (I usually do that), but for some reason it didn't work properly in this particular case and caused all sorts of undefined behavior. But I'll have a look at it again.
> 
> I've changed the code so that the memory is freed in C. Although it works "better" it crashes too every now and then
> 
> (WindowsError : exception : access violation writing 0x0310A1B4)
> 
> Will look into it.
this also can happen due to allocators conflict somehow. or due to other code which stores the pointer somewhere and then accesses the memory. i think that it will be hard to trace without debugger.

On Wednesday, 12 November 2014 at 16:10:34 UTC, ketmar via Digitalmars-d-learn wrote:
> On Wed, 12 Nov 2014 16:03:08 +0000
> Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:
>
>> On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
>> > On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
>> >> if you can extend C DLL, just add wrapper for `free()` there. so you
>> >> will not call `free()` from D, but call C DLL function which will free
>> >> the memory. it's a good practice anyway, 'cause it's recommended to
>> >> free memory in the same library where you allocated it.
>> >
>> > I initially had an implementation that did exactly that (I usually do that), but for some reason it didn't work properly in this particular case and caused all sorts of undefined behavior. But I'll have a look at it again.
>> 
>> I've changed the code so that the memory is freed in C. Although it works "better" it crashes too every now and then
>> 
>> (WindowsError : exception : access violation writing 0x0310A1B4)
>> 
>> Will look into it.
> this also can happen due to allocators conflict somehow. or due to
> other code which stores the pointer somewhere and then accesses the
> memory. i think that it will be hard to trace without debugger.

Thanks a million! Just checked it this morning. It was the latter. I kept a reference to short* data in a class variable in D and didn't clear that reference when freeing the memory in the C library.

Interesting though that it never crashes on Linux, only on Windows did this cause problems.

It is also interesting that core.stdc.stdlib.free() and free() in the C library produced a slightly different crash behavior. But that might be down to the fact that the two happen in different places in the program.

On Thu, 13 Nov 2014 10:08:47 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn@puremagic.com> wrote:

> Interesting though that it never crashes on Linux, only on Windows did this cause problems.
seems that libc allocator is not marking free pages as "unreadable", and windows libc allocator does something like this. i got alot of such "use after free" things in my C code, but valgrind is brilliant to track that down. i miss valgrind on windows. ;-)

now i prefer to write code in GNU/Linux if possible and use valgrind to track down memory issues, and only after it works on linux and valgrind is happy, i'm starting to port it to windows. this, of course, hard to do with winapi-tied code, but wine/winelib can help there too (to some extent).