On Wednesday, 14 January 2015 at 08:42:55 UTC, Iain Buclaw via D.gnu wrote:
> On 14 January 2015 at 04:00, Mike via D.gnu <d.gnu@puremagic.com> wrote:
>> On Tuesday, 13 January 2015 at 14:36:15 UTC, Dicebot wrote:
>>>
>>>
>>> I remember speaking about it with Martin and Daniel during DConf 2014 and
>>> I think it was Daniel who mentioned that by default TypeInfo/ModuleInfo is
>>> emitted in some weird packed way. When LDC announced using --gc-sections by
>>> default it was mentioned they had to change ModuleInfo emitting to make it
>>> actually work.
>>>
>>> Can it be the same issue?
>>
>>
>> Thanks, Dicebot, for bringing this to my attention.  That would
>> explain what I'm seeing.
>>
>> Is this something unique to GDC, or is it an artifact inherited
>> from DMD?
>>
>> Mike
>
> It's an artifact inherited from DMD.
>
> ModuleInfo is of a dynamic size, depending on what is implemented in the module.
>
> See: https://github.com/D-Programming-Language/druntime/blob/081591237ee7d666ffd81463dac1b7f38e7d9798/src/object_.d#L1589
>
> However it's size is correctly recorded before being sent to be written.
>
> The ModuleInfo symbols themselves aren't put into any particular
> section, they  also can't go in rodata because of how the D runtime
> start-up works, so they end up in the same section as __gshared data.
>
> The same is also true with TypeInfo_Class (alias ClassInfo) where
> interface vtables are written packed immediately after the data
> structure ends.  Again, it's size is treated as dynamic and is
> correctly recorded before being written, and again it cannot be in
> rodata because the __monitor field is directly written to.
>

Ok, but I have a mess of classes generated by templates (and I love it).  Their `name` properties [1] should go in .rodata, right?  But why aren't they being put in their own sections when compiling with -fdata-sections?

[1] - https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/object_.d#L81

Mike

On Wednesday, 14 January 2015 at 09:04:50 UTC, Mike wrote:
>
> Ok, but I have a mess of classes generated by templates (and I love it).  Their `name` properties [1] should go in .rodata, right?  But why aren't they being put in their own sections when compiling with -fdata-sections?
>
> [1] - https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/object_.d#L81
>

Well, I was working a reduced test case and found that it has
something to do with my trace.d file here:
https://github.com/JinShil/stm32f42_discovery_demo/blob/master/source/stm32f42/trace.d

If I add a trace.writeLine("x") in my program, then the binary
goes from 2K to 130K.  Anyway, it appears it has nothing to do
with TypeInfo.  I'll continue to try to reduce.

Thanks for the help and useful information.

Mike

On 14 January 2015 at 13:32, Mike via D.gnu <d.gnu@puremagic.com> wrote:
> On Wednesday, 14 January 2015 at 09:04:50 UTC, Mike wrote:
>>
>>
>> Ok, but I have a mess of classes generated by templates (and I love it). Their `name` properties [1] should go in .rodata, right?  But why aren't they being put in their own sections when compiling with -fdata-sections?
>>
>> [1] - https://github.com/D-Programming-GDC/GDC/blob/master/libphobos/libdruntime/object_.d#L81
>>
>
> Well, I was working a reduced test case and found that it has
> something to do with my trace.d file here:
> https://github.com/JinShil/stm32f42_discovery_demo/blob/master/source/stm32f42/trace.d
>
> If I add a trace.writeLine("x") in my program, then the binary goes from 2K to 130K.  Anyway, it appears it has nothing to do with TypeInfo.  I'll continue to try to reduce.
>

Could it be that the entire module trace takes up 128K?  Probably not very likely.  And I doubt the writeLine/write templates contribute to much.

On Wednesday, 14 January 2015 at 13:32:53 UTC, Mike wrote:

>
> Well, I was working a reduced test case and found that it has
> something to do with my trace.d file here:
> https://github.com/JinShil/stm32f42_discovery_demo/blob/master/source/stm32f42/trace.d
>
> If I add a trace.writeLine("x") in my program, then the binary
> goes from 2K to 130K.  Anyway, it appears it has nothing to do
> with TypeInfo.  I'll continue to try to reduce.
>
> Thanks for the help and useful information.
>
> Mike

Final update:

The problem was with this function here:

void write(A...)(in A a)
{
     foreach(t; a)
     {
         write(t);
     }
}

I think that since this is an open-ended template and I could
potentially pass any type to it, the compiler thinks it should
remember the TypeInfo.name values for every type in my program.

However, since I only ever used write("x") in my program, I
expect the linker to be able to see that those TypeInfo.name
values are never used, and strip them out when compiled with
-fdata-sections and -Wl,--gc-sections.  Perhaps it couldn't
because of the way the data is packed.

Anyway, I guess I'll see about modifying my code to be less
flexible in order to reign this data in.

Suggestions are welcome.

Mike

On Wednesday, 14 January 2015 at 14:20:50 UTC, Iain Buclaw via
D.gnu wrote:

>
> Could it be that the entire module trace takes up 128K?  Probably not
> very likely.  And I doubt the writeLine/write templates contribute to
> much.

Sorry, I didn't see this reply before I submitted my last post.

The problem was my .rodata was filling up with the names of every
single type in my program.  The .text section was quite small
(2K), so I don't think it was the executable code generated from
the trace module.

I'll try to make a reduced test case on Linux that others can run
and see.

Mike

On Wednesday, 14 January 2015 at 14:34:48 UTC, Mike wrote:

> I'll try to make a reduced test case on Linux that others can run
> and see.

Ok, here's my reduced test case that runs on Linux 64-bit.  I don't know if I really needed to implement the syscalls, but I just wanted to get rid of everything I could so the important stuff would stand out.

test.d
***************************************
void sys_exit(long arg1) nothrow
{
    asm
    {
        "syscall"
        :
        : "a" 60,
        "D" arg1,
        : "memory", "cc", "rcx", "r11";
    }
}

long sys_write(long arg1, in void* arg2, long arg3) nothrow
{
    long result;

    asm
    {
        "syscall"
        : "=a" result
        : "a" 1,
        "D" arg1,
        "S" arg2,
        "m" arg2,
        "d" arg3
        : "memory", "cc", "rcx", "r11";
    }

    return result;
}

void write(in string text) nothrow
{
    sys_write(2, text.ptr, text.length);
}

void write(A...)(in A a) nothrow
{
    foreach(t; a)
    {
        write(t);
    }
}

final abstract class TestClass1 { }

// final abstract class TestClass2 { }
// final abstract class TestClass3 { }
// final abstract class TestClass4 { }
// final abstract class TestClass5 { }
// final abstract class TestClass6 { }
// final abstract class TestClass7 { }
// final abstract class TestClass8 { }
// final abstract class TestClass9 { }

extern(C) void main()
{
    write("x");
    sys_exit(0);
}
***************************************

compile with:
gdc -static -frelease -fno-emit-moduleinfo -nophoboslib -nostdlib test.d --entry=main -ffunction-sections -fdata-sections -Wl,--gc-sections -o test

show .rodata:
***************************************
objdump -s -j .rodata test

Contents of section .rodata:
 4001c4 74657374 2e546573 74436c61 73733100  test.TestClass1.
 4001d4 78                                   x

See the "test.TestClass1" string there?  That's the problem.  Now uncomment the other `TestClass`s.


show .rodata with more types:
***************************************
objdump -s -j .rodata test

Contents of section .rodata:
 4001c4 74657374 2e546573 74436c61 73733100  test.TestClass1.
 4001d4 74657374 2e546573 74436c61 73733200  test.TestClass2.
 4001e4 74657374 2e546573 74436c61 73733300  test.TestClass3.
 4001f4 74657374 2e546573 74436c61 73733400  test.TestClass4.
 400204 74657374 2e546573 74436c61 73733500  test.TestClass5.
 400214 74657374 2e546573 74436c61 73733600  test.TestClass6.
 400224 74657374 2e546573 74436c61 73733700  test.TestClass7.
 400234 74657374 2e546573 74436c61 73733800  test.TestClass8.
 400244 74657374 2e546573 74436c61 73733900  test.TestClass9.
 400254 78                                   x

Now imagine I have a highly templated library generating 100s of these little classes.

Now, comment out the `write("x")`.

show .rodata `write("x")` commented out:
***************************************
objdump -s -j .rodata test

objdump: section '.rodata' mentioned in a -j option, but not found in any input file

No .rodata at all!!

IMO the toolchain should be able to recognize that the TypeInfo.name property is not being used and strip it out.

If you can explain the mechanics causing this, please enlighten me.  Bug? Enhancement? By design?

Thanks for the help,
Mike

On Thursday, 15 January 2015 at 11:04:37 UTC, Mike wrote:
> If you can explain the mechanics causing this, please enlighten me.  Bug? Enhancement? By design?

Random guess: can it possibly confuse template-based variadics with runtime variadics? Latter require RTTI to work. If something like that happens it is surely a bug.

I don't see any obvious legitimate reason for this behavior.

On Thursday, 15 January 2015 at 11:42:31 UTC, Dicebot wrote:
>
> Random guess: can it possibly confuse template-based variadics with runtime variadics? Latter require RTTI to work. If something like that happens it is surely a bug.
>

Thanks for the reply.  I have to apologize for the last post, as I didn't fully reduce the code, and posted prematurely.

Here's a further reduction without any templates or variadics, so I'm under the impression that neither templates nor variadics are contributing to the problem.

The following code won't execute, but it will reproduce the problem at hand:  Inflating .rodata with TypeInfo.name.

test.d
**************************************
void write(in string text) nothrow {}

final abstract class TestClass1 { }

final abstract class TestClass2 { }
final abstract class TestClass3 { }
final abstract class TestClass4 { }
final abstract class TestClass5 { }
final abstract class TestClass6 { }
final abstract class TestClass7 { }
final abstract class TestClass8 { }
final abstract class TestClass9 { }

extern(C) void main()
{
    write("");
}

compile with:
gdc -static -frelease -fno-emit-moduleinfo -nophoboslib -nostdlib test.d --entry=main -ffunction-sections -fdata-sections -Wl,--gc-sections -o test


objdump -s -j .rodata test
*************************************
Contents of section .rodata:
 400152 74657374 2e546573 74436c61 73733100  test.TestClass1.
 400162 74657374 2e546573 74436c61 73733200  test.TestClass2.
 400172 74657374 2e546573 74436c61 73733300  test.TestClass3.
 400182 74657374 2e546573 74436c61 73733400  test.TestClass4.
 400192 74657374 2e546573 74436c61 73733500  test.TestClass5.
 4001a2 74657374 2e546573 74436c61 73733600  test.TestClass6.
 4001b2 74657374 2e546573 74436c61 73733700  test.TestClass7.
 4001c2 74657374 2e546573 74436c61 73733800  test.TestClass8.
 4001d2 74657374 2e546573 74436c61 73733900  test.TestClass9.



Interestingly, if I change the argument to `write` from a string to a char, all is good.

test.d
**************************************
void write(in char text) nothrow {}

final abstract class TestClass1 { }

final abstract class TestClass2 { }
final abstract class TestClass3 { }
final abstract class TestClass4 { }
final abstract class TestClass5 { }
final abstract class TestClass6 { }
final abstract class TestClass7 { }
final abstract class TestClass8 { }
final abstract class TestClass9 { }

extern(C) void main()
{
    write(' ');
}

objdump -s -j .rodata test
**************************************
objdump: section '.rodata' mentioned in a -j option, but not found in any input file

I guess all I'm really showing is how little I understand about this problem.  Again, I ask for help.

Mike

Am Thu, 15 Jan 2015 11:42:30 +0000
schrieb "Dicebot" <public@dicebot.lv>:

> On Thursday, 15 January 2015 at 11:04:37 UTC, Mike wrote:
> > If you can explain the mechanics causing this, please enlighten me.  Bug? Enhancement? By design?
> 
> Random guess: can it possibly confuse template-based variadics with runtime variadics? Latter require RTTI to work. If something like that happens it is surely a bug.
> 
> I don't see any obvious legitimate reason for this behavior.

That'd be quite weird.

My best guess is that the strings are always placed in rodata, never in separate sections. If you do write("x"), "x" is also in rodata, the rodata section can't be removed. If you delete the write call there's no reference to rodata and it's possible to remove the complete section.

After some google-fu:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=192
Considering this was filed in 2000 I'd say it's not very likely to get
fixed soon :-(

So the best option is probably to get rid of this problem by patching the compiler (@notypeinfo or -fnortti).

Am Thu, 15 Jan 2015 11:51:41 +0000
schrieb "Mike" <none@none.com>:

> On Thursday, 15 January 2015 at 11:42:31 UTC, Dicebot wrote:
> >
> > Random guess: can it possibly confuse template-based variadics with runtime variadics? Latter require RTTI to work. If something like that happens it is surely a bug.
> >
> 
> Thanks for the reply.  I have to apologize for the last post, as I didn't fully reduce the code, and posted prematurely.
> 
> Here's a further reduction without any templates or variadics, so I'm under the impression that neither templates nor variadics are contributing to the problem.
> 
> The following code won't execute, but it will reproduce the problem at hand:  Inflating .rodata with TypeInfo.name.
> 
> test.d
> **************************************
> void write(in string text) nothrow {}
> 
> final abstract class TestClass1 { }
> 
> final abstract class TestClass2 { }
> final abstract class TestClass3 { }
> final abstract class TestClass4 { }
> final abstract class TestClass5 { }
> final abstract class TestClass6 { }
> final abstract class TestClass7 { }
> final abstract class TestClass8 { }
> final abstract class TestClass9 { }
> 
> extern(C) void main()
> {
>      write("");
> }
> 
> compile with:
> gdc -static -frelease -fno-emit-moduleinfo -nophoboslib -nostdlib
> test.d --entry=main -ffunction-sections -fdata-sections
> -Wl,--gc-sections -o test
> 
> 
> objdump -s -j .rodata test
> *************************************
> Contents of section .rodata:
>   400152 74657374 2e546573 74436c61 73733100  test.TestClass1.
>   400162 74657374 2e546573 74436c61 73733200  test.TestClass2.
>   400172 74657374 2e546573 74436c61 73733300  test.TestClass3.
>   400182 74657374 2e546573 74436c61 73733400  test.TestClass4.
>   400192 74657374 2e546573 74436c61 73733500  test.TestClass5.
>   4001a2 74657374 2e546573 74436c61 73733600  test.TestClass6.
>   4001b2 74657374 2e546573 74436c61 73733700  test.TestClass7.
>   4001c2 74657374 2e546573 74436c61 73733800  test.TestClass8.
>   4001d2 74657374 2e546573 74436c61 73733900  test.TestClass9.
> 
> 
> 
> Interestingly, if I change the argument to `write` from a string to a char, all is good.
> 
> test.d
> **************************************
> void write(in char text) nothrow {}
> 
> final abstract class TestClass1 { }
> 
> final abstract class TestClass2 { }
> final abstract class TestClass3 { }
> final abstract class TestClass4 { }
> final abstract class TestClass5 { }
> final abstract class TestClass6 { }
> final abstract class TestClass7 { }
> final abstract class TestClass8 { }
> final abstract class TestClass9 { }
> 
> extern(C) void main()
> {
>      write(' ');
> }
> 
> objdump -s -j .rodata test
> **************************************
> objdump: section '.rodata' mentioned in a -j option, but not
> found in any input file
> 
> I guess all I'm really showing is how little I understand about this problem.  Again, I ask for help.
> 
> Mike

The char is probably not placed in rodata but reproduced in some other way (hardcoded instruction with literal or whatever). This matches the theory in my other reply.