On 9/5/12 4:59 PM, Benjamin Thaut wrote:
> Still, comparing two type info objects will result in one or multiple
> allocations most of the time.

Could you please submit a patch for that? Thanks!

Andrei

P.S. Very nice work. Congrats!

Iain Buclaw:

Most of the array allocation cases we are talking about are like:

void main() {
  int[3] a = [1, 2, 3]; // fixed size array
}


That currently produces, with DMD:

__Dmain:
L0:     sub ESP, 010h
        mov EAX, offset FLAT:_D12TypeInfo_xAi6__initZ
        push EBX
        push 0Ch
        push 3
        push EAX
        call near ptr __d_arrayliteralTX
        add ESP, 8
        mov EBX, EAX
        mov dword ptr [EAX], 1
        mov ECX, EBX
        push EBX
        lea EDX, 010h[ESP]
        mov dword ptr 4[EBX], 2
        mov dword ptr 8[EBX], 3
        push EDX
        call near ptr _memcpy
        add ESP, 0Ch
        xor EAX, EAX
        pop EBX
        add ESP, 010h
        ret



There is also the case for dynamic arrays:

void main() {
  int[] a = [1, 2, 3];
  // use a here
}

But this is a harder problem, to leave for later.


> this infact caused many strange SEGV's in quite
> a few of my programs  (most are parsers / interpreters, so things that
> go down *heavy* nested into itself, and it was under these
> circumstances that array literals on the stack would go corrupt in one
> way or another causing *huge* errors in perfectly sound code).

Do you know the cause of such corruptions? maybe they are caused by other compiler bugs...

And what to do regarding those exceptions in constructors? :-)

Bye,
bearophile

On 5 September 2012 16:31, bearophile <bearophileHUGS@lycos.com> wrote:
> Iain Buclaw:
>
> Most of the array allocation cases we are talking about are like:
>
> void main() {
>   int[3] a = [1, 2, 3]; // fixed size array
> }
>
>
> That currently produces, with DMD:
>
> __Dmain:
> L0:     sub ESP, 010h
>         mov EAX, offset FLAT:_D12TypeInfo_xAi6__initZ
>         push EBX
>         push 0Ch
>         push 3
>         push EAX
>         call near ptr __d_arrayliteralTX
>         add ESP, 8
>         mov EBX, EAX
>         mov dword ptr [EAX], 1
>         mov ECX, EBX
>         push EBX
>         lea EDX, 010h[ESP]
>         mov dword ptr 4[EBX], 2
>         mov dword ptr 8[EBX], 3
>         push EDX
>         call near ptr _memcpy
>         add ESP, 0Ch
>         xor EAX, EAX
>         pop EBX
>         add ESP, 010h
>         ret
>
>
>
> There is also the case for dynamic arrays:
>
> void main() {
>   int[] a = [1, 2, 3];
>   // use a here
> }
>
> But this is a harder problem, to leave for later.
>
>
>
>> this infact caused many strange SEGV's in quite
>> a few of my programs  (most are parsers / interpreters, so things that
>> go down *heavy* nested into itself, and it was under these
>> circumstances that array literals on the stack would go corrupt in one
>> way or another causing *huge* errors in perfectly sound code).
>
>
> Do you know the cause of such corruptions? maybe they are caused by other compiler bugs...
>
> And what to do regarding those exceptions in constructors? :-)
>

I think it was mostly due to that you can't tell the difference between array literals that are to be assigned to either dynamic or static arrays (as far as I can tell).   I do believe that the issues surrounded dynamic arrays causing SEGVs, and not static  (I don't recall ever needing the use of a static array :-).


Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

Iain Buclaw:

> I think it was mostly due to that you can't tell the difference
> between array literals that are to be assigned to either dynamic or static arrays (as far as I can tell).

I see.


> I do believe that the issues
> surrounded dynamic arrays causing SEGVs, and not static  (I don't recall ever needing the use of a static array :-).

I use fixed size arrays all the time in D. Heap-allocated arrays are overused in D. They produce garbage and in lot of cases they are not needed. Using them a lot is sometimes a bad habit (if you are writing script-like programs they are OK), that's also encouraged by making them almost second-class citizens in Phobos (and druntime, using them as AA keys causes performance troubles).

If you take a look at Ada language you see how much static/stack-allocated arrays are used. In high performance code they help, and I'd like D programmers and Phobos devs to give them a little more consideration.

Bye,
bearophile

Am 05.09.2012 16:57, schrieb bearophile:
> Benjamin Thaut:
>
>> http://3d.benjamin-thaut.de/?p=20#more-20
>
> Regardind your issues list, most of them are fixable, like the one
> regarding array literals, and even the one regarding the invariant handler.
>
> But I didn't know about this, and I don't know how and if this is fixable:
>
>> The new statement will not free any memory if the constructor throws a
>> exception.<
>
> Insights welcome.
>
> Bye,
> bearophile

Well, as overloading new and delete is deprecated, and the new which is part of the language only works together with a GC I don't think that anything will be done about this.

Its not a big problem in D because you can't create arrays of objects so that multiple constructors will be called at the same time. (Which is the biggest issue in c++ with exceptions and constructors). Also doe to memory pre initialization the object will always be in a meaningfull state, which helps with exception handling too. My replacement just calls the constructor, and if a exception is thrown, the destructor is called and the memory is freed, then the new statement returns null. Works flawlessley so far.

Kind Regards
Benjamin Thaut

Am 05.09.2012 16:07, schrieb anonymous:
>
> It was interesting to read it. What about GDC MMM?
>

The GDC druntime does have a different folder structure, which makes it a lot more time consuming to add in the changes. Also it is not possible to rebuild phobos or druntime with the binary release of GDC Mingw. You need the complete build setup for GDC mingw to do that. As this is not documented very well and quite some work I didn't go through that additional effort.

Kind Regards
Benjamin Thaut

> If you take a look at Ada language you see how much static/stack-allocated arrays are used. In high performance code they help, and I'd like D programmers and Phobos devs to give them a little more consideration.

Also, the lack of variable length stack allocated arrays in D forces you to over-allocate, wasting stack space, or forces you to use alloca() that is bug-prone and makes things not easy if you need a multi dimensional array.

Bye,
bearophile

My "standard" library is now aviable on github:

https://github.com/Ingrater/thBase

Kind Regards
Benjamin Thaut

Am Wed, 05 Sep 2012 13:03:37 +0200
schrieb Benjamin Thaut <code@benjamin-thaut.de>:

> I rewrote a 3d game I created during my studies with D 2.0 to manual memory mangement. If I'm not studying I'm working in the 3d Engine deparement of Havok. As I needed to pratice manual memory management and did want to get rid of the GC in D for quite some time, I did go through all this effort to create a GC free version of my game.
> 
> The results are:
> 
>      DMD GC Version: 71 FPS, 14.0 ms frametime
>      GDC GC Version: 128.6 FPS, 7.72 ms frametime
>      DMD MMM Version: 142.8 FPS, 7.02 ms frametime
> 
> GC collection times:
> 
>      DMD GC Version: 8.9 ms
>      GDC GC Version: 4.1 ms
> 
> As you see the manual managed version is twice as fast as the garbage collected one. Even the highly optimized version created with GDC is still slower the the manual memory management.
> 
> You can find the full article at:
> 
> http://3d.benjamin-thaut.de/?p=20#more-20
> 
> 
> Feedback is welcome.

Would be great if some of the code could be merged into phobos, especially the memory tracker. But also things like memory or object pools would be great in phobos, an emplace wrapper which accepts a custom alloc function to replace new (and something similar for delete), etc. We really need a module for manual memory management (std.mmm?). And functions which currently use the GC to allocate should get overloads which take buffers (Or better support custom allocators, but that needs an allocator design first).

Am 05.09.2012 19:31, schrieb Johannes Pfau:
>
> Would be great if some of the code could be merged into phobos,
> especially the memory tracker. But also things like memory or object
> pools would be great in phobos, an emplace wrapper which accepts a
> custom alloc function to replace new (and something similar for delete),
> etc. We really need a module for manual memory management (std.mmm?).
> And functions which currently use the GC to allocate should get
> overloads which take buffers (Or better support custom allocators, but
> that needs an allocator design first).
>

I personally really like my composite template, which allows for direct composition of one class instance into another. It does not introduce additional indirections and the compiler will remind you, if you forgett to initialize it.

https://github.com/Ingrater/druntime/blob/master/src/core/allocator.d#L670

Kind Regards
Benjamin Thaut