On Monday, 10 June 2013 at 21:33:20 UTC, Walter Bright wrote:
> On 6/10/2013 2:28 PM, nazriel wrote:
>>> program was compiled with dmd (2.063) using the following flags: -g -debug
>>> -unittest
>>
>> I suspected it may be the problem with shared libraries.
>> Can you try compiling that hello world with static libphobos?
>> Or can you attach your segfaulting binary?
>
> Statically linking with libphobos2.a is the default.

Brandon's back-trace mentions libphobos2.so.0.63

OP's backtrace shows that SIGSEGV occurs in _d_dso_registry()

My guess would be to check that first.

On 6/10/2013 2:38 PM, nazriel wrote:
> On Monday, 10 June 2013 at 21:33:20 UTC, Walter Bright wrote:
>> On 6/10/2013 2:28 PM, nazriel wrote:
>>>> program was compiled with dmd (2.063) using the following flags: -g -debug
>>>> -unittest
>>>
>>> I suspected it may be the problem with shared libraries.
>>> Can you try compiling that hello world with static libphobos?
>>> Or can you attach your segfaulting binary?
>>
>> Statically linking with libphobos2.a is the default.
>
> Brandon's back-trace mentions libphobos2.so.0.63
>
> OP's backtrace shows that SIGSEGV occurs in _d_dso_registry()
>
> My guess would be to check that first.

linking with -g -debug -unittest will statically link, it will not link with the .so

On 6/10/2013 2:56 PM, Walter Bright wrote:
> On 6/10/2013 2:38 PM, nazriel wrote:
>> On Monday, 10 June 2013 at 21:33:20 UTC, Walter Bright wrote:
>>> On 6/10/2013 2:28 PM, nazriel wrote:
>>>>> program was compiled with dmd (2.063) using the following flags: -g -debug
>>>>> -unittest
>>>>
>>>> I suspected it may be the problem with shared libraries.
>>>> Can you try compiling that hello world with static libphobos?
>>>> Or can you attach your segfaulting binary?
>>>
>>> Statically linking with libphobos2.a is the default.
>>
>> Brandon's back-trace mentions libphobos2.so.0.63
>>
>> OP's backtrace shows that SIGSEGV occurs in _d_dso_registry()
>>
>> My guess would be to check that first.
>
> linking with -g -debug -unittest will statically link, it will not link with the
> .so

One way to be sure - delete all the libphobos2.so files. All of them, then try again.

Walter Bright <newshound2@digitalmars.com> writes:

> On 6/10/2013 2:56 PM, Walter Bright wrote:
>> On 6/10/2013 2:38 PM, nazriel wrote:
>>> On Monday, 10 June 2013 at 21:33:20 UTC, Walter Bright wrote:
>>>> On 6/10/2013 2:28 PM, nazriel wrote:
>>>>>> program was compiled with dmd (2.063) using the following flags: -g -debug
>>>>>> -unittest
>>>>>
>>>>> I suspected it may be the problem with shared libraries.
>>>>> Can you try compiling that hello world with static libphobos?
>>>>> Or can you attach your segfaulting binary?
>>>>
>>>> Statically linking with libphobos2.a is the default.
>>>
>>> Brandon's back-trace mentions libphobos2.so.0.63

His code appears to die after main() is in progress.

>>> OP's backtrace shows that SIGSEGV occurs in _d_dso_registry()
>>>
>>> My guess would be to check that first.
>>
>> linking with -g -debug -unittest will statically link, it will not link with the .so

Yes, strace on dmd shows that I'm linking with libphobos2.a.

...
[pid 23169] open("/home/jlquinn/dmd2/linux/bin64/../lib64/libphobos2.a", O_RDONLY|O_CLOEXEC) = 11

I just started having this problem as well, however it appears whether I'm using the latest DMD or older versions (I only tried as far back as 2.060).  Like others, the program compiled with 2.063 dies at _d_dso_registry().  The programs compiled with previous versions die at gc_init().

I've made no other relevant changes recently, so I'm rather perplexed.

On 6/9/2013 11:41 PM, Jerry wrote:
> LD_LIBRARY_PATH is empty.  I've now reproduced this segfault on a Debian
> testing machine as well as my Ubuntu one.  I'm pretty confused.

LD_LIBRARY_PATH only applies to shared libraries.

On 6/10/2013 9:52 AM, Jerry wrote:
>> 5. Does gdb reveal anything interesting?
>
> Unfortunately there's no debugging symbols in _d_dso_registry().  I
> assume the compiler is writing asm directly.

Use the 'disassemble' command starting at _d_dso_registry, and point out which instruction is faulting.

You might also try running the program under valgrind.

On 6/10/13 6:10 PM, Jerry wrote:
> [pid 23169] open("/home/jlquinn/dmd2/linux/bin64/../lib64/libphobos2.a", O_RDONLY|O_CLOEXEC) = 11

OK, so it seems this is the cause. I'm trying to repro it on a fresh Ubuntu 12.10 VM. So at least two bugs:

1. We link dynamically by default at least on some systems.

2. The dynamically linked programs crash again at least on some systems.


Andrei

Walter Bright <newshound2@digitalmars.com> writes:

> On 6/10/2013 9:52 AM, Jerry wrote:
>>> 5. Does gdb reveal anything interesting?
>>
>> Unfortunately there's no debugging symbols in _d_dso_registry().  I assume the compiler is writing asm directly.
>
> Use the 'disassemble' command starting at _d_dso_registry, and point out which instruction is faulting.
>
> You might also try running the program under valgrind.

Valgrind output followed by disassembly within gdb:

jlquinn@wyvern:~/re/test$ valgrind ./junk
==24371== Memcheck, a memory error detector
==24371== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==24371== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==24371== Command: ./junk
==24371==
==24371== Invalid write of size 8
==24371==    at 0x4187B0: _d_dso_registry (in /home/jlquinn/re/test/junk)
==24371==    by 0x4176B6: ??? (in /home/jlquinn/re/test/junk)
==24371==    by 0x4302BC: __libc_csu_init (in /home/jlquinn/re/test/junk)
==24371==    by 0x52786FF: (below main) (libc-start.c:185)
==24371==  Address 0x105616130 is not stack'd, malloc'd or (recently) free'd
==24371==
==24371==
==24371== Process terminating with default action of signal 11 (SIGSEGV)
==24371==  Access not within mapped region at address 0x105616130
==24371==    at 0x4187B0: _d_dso_registry (in /home/jlquinn/re/test/junk)
==24371==    by 0x4176B6: ??? (in /home/jlquinn/re/test/junk)
==24371==    by 0x4302BC: __libc_csu_init (in /home/jlquinn/re/test/junk)
==24371==    by 0x52786FF: (below main) (libc-start.c:185)
==24371==  If you believe this happened as a result of a stack
==24371==  overflow in your program's main thread (unlikely but
==24371==  possible), you can try to increase the size of the
==24371==  main thread stack using the --main-stacksize= flag.
==24371==  The main thread stack size used in this run was 8388608.
==24371==
==24371== HEAP SUMMARY:
==24371==     in use at exit: 120 bytes in 3 blocks
==24371==   total heap usage: 3 allocs, 0 frees, 120 bytes allocated
==24371==
==24371== LEAK SUMMARY:
==24371==    definitely lost: 8 bytes in 1 blocks
==24371==    indirectly lost: 0 bytes in 0 blocks
==24371==      possibly lost: 0 bytes in 0 blocks
==24371==    still reachable: 112 bytes in 2 blocks
==24371==         suppressed: 0 bytes in 0 blocks
==24371== Rerun with --leak-check=full to see details of leaked memory
==24371==
==24371== For counts of detected and suppressed errors, rerun with: -v
==24371== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)
Segmentation fault (core dumped)



Program received signal SIGSEGV, Segmentation fault.
0x00000000004187b0 in _d_dso_registry ()
(gdb) disassemble
Dump of assembler code for function _d_dso_registry:
   0x00000000004185b0 <+0>:	push   %rbp
   0x00000000004185b1 <+1>:	mov    %rsp,%rbp
   0x00000000004185b4 <+4>:	sub    $0x98,%rsp
   0x00000000004185bb <+11>:	push   %rbx
   0x00000000004185bc <+12>:	push   %r12
   0x00000000004185be <+14>:	push   %r13
   0x00000000004185c0 <+16>:	push   %r14
   0x00000000004185c2 <+18>:	push   %r15
   0x00000000004185c4 <+20>:	mov    %rdi,%r12
   0x00000000004185c7 <+23>:	movabs $0x1,%rax
   0x00000000004185d1 <+33>:	cmp    %rax,(%r12)
   0x00000000004185d5 <+37>:	jae    0x4185d8 <_d_dso_registry+40>
   0x00000000004185d7 <+39>:	hlt
   0x00000000004185d8 <+40>:	mov    0x8(%r12),%rcx
   0x00000000004185dd <+45>:	cmpq   $0x0,(%rcx)
   0x00000000004185e1 <+49>:	jne    0x4187b6 <_d_dso_registry+518>
   0x00000000004185e7 <+55>:	movabs $0x60,%rsi
   0x00000000004185f1 <+65>:	mov    %rax,%rdi
   0x00000000004185f4 <+68>:	callq  0x417220 <calloc@plt>
   0x00000000004185f9 <+73>:	mov    %rax,%r14
   0x00000000004185fc <+76>:	mov    0x8(%r12),%rdx
   0x0000000000418601 <+81>:	mov    %r14,(%rdx)
---Type <return> to continue, or q <return> to quit---
   0x0000000000418604 <+84>:	lea    -0x70(%rbp),%rax
   0x0000000000418608 <+88>:	xor    %rcx,%rcx
   0x000000000041860b <+91>:	mov    %rcx,(%rax)
   0x000000000041860e <+94>:	mov    %rcx,0x8(%rax)
   0x0000000000418612 <+98>:	mov    %rcx,0x10(%rax)
   0x0000000000418616 <+102>:	mov    %rcx,0x18(%rax)
   0x000000000041861a <+106>:	mov    %rcx,0x20(%rax)
   0x000000000041861e <+110>:	mov    %rcx,0x28(%rax)
   0x0000000000418622 <+114>:	mov    0x18(%r12),%rax
   0x0000000000418627 <+119>:	mov    0x10(%r12),%rbx
   0x000000000041862c <+124>:	sub    %rbx,%rax
   0x000000000041862f <+127>:	cqto
   0x0000000000418631 <+129>:	and    $0x7,%rdx
   0x0000000000418638 <+136>:	add    %rdx,%rax
   0x000000000041863b <+139>:	sar    $0x3,%rax
   0x000000000041863f <+143>:	mov    %rbx,%rdx
   0x0000000000418642 <+146>:	mov    %rax,-0x70(%rbp)
   0x0000000000418646 <+150>:	mov    %rdx,-0x68(%rbp)
   0x000000000041864a <+154>:	lea    -0x70(%rbp),%rsi
   0x000000000041864e <+158>:	lea    0x10(%r14),%rdi
   0x0000000000418652 <+162>:	movsq  %ds:(%rsi),%es:(%rdi)
   0x0000000000418654 <+164>:	movsq  %ds:(%rsi),%es:(%rdi)
   0x0000000000418656 <+166>:	movsq  %ds:(%rsi),%es:(%rdi)
---Type <return> to continue, or q <return> to quit---
   0x0000000000418658 <+168>:	movsq  %ds:(%rsi),%es:(%rdi)
   0x000000000041865a <+170>:	movsq  %ds:(%rsi),%es:(%rdi)
   0x000000000041865c <+172>:	movsq  %ds:(%rsi),%es:(%rdi)
   0x000000000041865e <+174>:	mov    0x28(%r12),%rcx
   0x0000000000418663 <+179>:	mov    0x20(%r12),%r13
   0x0000000000418668 <+184>:	sub    %r13,%rcx
   0x000000000041866b <+187>:	movabs $0xaaaaaaaaaaaaaaab,%rax
   0x0000000000418675 <+197>:	imul   %rcx
   0x0000000000418678 <+200>:	add    %rcx,%rdx
   0x000000000041867b <+203>:	mov    %rcx,%rax
   0x000000000041867e <+206>:	sar    $0x3f,%rax
   0x0000000000418682 <+210>:	sar    $0x4,%rdx
   0x0000000000418686 <+214>:	sub    %rax,%rdx
   0x0000000000418689 <+217>:	mov    %rdx,%rax
   0x000000000041868c <+220>:	mov    %r13,%rdx
   0x000000000041868f <+223>:	mov    %rax,(%r14)
   0x0000000000418692 <+226>:	mov    %rdx,0x8(%r14)
   0x0000000000418696 <+230>:	mov    0x8(%r12),%rsi
   0x000000000041869b <+235>:	lea    -0x40(%rbp),%rdi
   0x000000000041869f <+239>:	callq  0x418828 <_D2rt14sections_linux18findDSOInfoForAddrFNbxPvPS4core3sys5linux4link12dl_phdr_infoZb>
   0x00000000004186a4 <+244>:	test   %al,%al
   0x00000000004186a6 <+246>:	jne    0x4186a9 <_d_dso_registry+249>
---Type <return> to continue, or q <return> to quit---
   0x00000000004186a8 <+248>:	hlt
   0x00000000004186a9 <+249>:	mov    -0x28(%rbp),%ax
   0x00000000004186ad <+253>:	and    $0xffff,%eax
   0x00000000004186b2 <+258>:	mov    %eax,%eax
   0x00000000004186b4 <+260>:	mov    -0x30(%rbp),%rcx
   0x00000000004186b8 <+264>:	mov    %rax,-0x90(%rbp)
   0x00000000004186bf <+271>:	mov    %rcx,-0x88(%rbp)
   0x00000000004186c6 <+278>:	cmpq   $0x0,-0x90(%rbp)
   0x00000000004186ce <+286>:	je     0x41877a <_d_dso_registry+458>
   0x00000000004186d4 <+292>:	mov    -0x88(%rbp),%r12
   0x00000000004186db <+299>:	mov    -0x90(%rbp),%rdx
   0x00000000004186e2 <+306>:	imul   $0x38,%rdx,%rbx
   0x00000000004186e6 <+310>:	add    -0x88(%rbp),%rbx
   0x00000000004186ed <+317>:	mov    (%r12),%esi
   0x00000000004186f1 <+321>:	cmp    $0x1,%esi
   0x00000000004186f4 <+324>:	jne    0x418757 <_d_dso_registry+423>
   0x00000000004186f6 <+326>:	testl  $0x2,0x4(%r12)
   0x00000000004186ff <+335>:	je     0x418757 <_d_dso_registry+423>
   0x0000000000418701 <+337>:	mov    0x28(%r12),%rsi
   0x0000000000418706 <+342>:	mov    0x10(%r12),%rcx
   0x000000000041870b <+347>:	add    -0x40(%rbp),%rcx
   0x000000000041870f <+351>:	mov    %rsi,-0x80(%rbp)
   0x0000000000418713 <+355>:	mov    %rcx,-0x78(%rbp)
---Type <return> to continue, or q <return> to quit---
   0x0000000000418717 <+359>:	lea    0x40(%r14),%r13
   0x000000000041871b <+363>:	mov    0x0(%r13),%rsi
   0x000000000041871f <+367>:	mov    0x8(%r13),%r15
   0x0000000000418723 <+371>:	inc    %r15
   0x0000000000418726 <+374>:	mov    %r15,%rdi
   0x0000000000418729 <+377>:	shl    $0x4,%rdi
   0x000000000041872d <+381>:	callq  0x418b28 <_D2rt4util9container8xreallocFPvmZPv>
   0x0000000000418732 <+386>:	mov    %rax,0x0(%r13)
   0x0000000000418736 <+390>:	mov    %r15,0x8(%r13)
   0x000000000041873a <+394>:	mov    -0x78(%rbp),%rdx
   0x000000000041873e <+398>:	mov    -0x80(%rbp),%rax
   0x0000000000418742 <+402>:	mov    %r15,%r8
   0x0000000000418745 <+405>:	shl    $0x4,%r8
   0x0000000000418749 <+409>:	add    0x0(%r13),%r8
   0x000000000041874d <+413>:	mov    %rax,-0x10(%r8)
   0x0000000000418751 <+417>:	mov    %rdx,-0x8(%r8)
   0x0000000000418755 <+421>:	jmp    0x41876d <_d_dso_registry+445>
   0x0000000000418757 <+423>:	cmp    $0x7,%esi
   0x000000000041875a <+426>:	jne    0x41876d <_d_dso_registry+445>
   0x000000000041875c <+428>:	mov    -0x10(%rbp),%rdi
   0x0000000000418760 <+432>:	mov    %rdi,0x50(%r14)
   0x0000000000418764 <+436>:	mov    0x28(%r12),%rax
---Type <return> to continue, or q <return> to quit---
   0x0000000000418769 <+441>:	mov    %rax,0x58(%r14)
   0x000000000041876d <+445>:	add    $0x38,%r12
   0x0000000000418771 <+449>:	cmp    %rbx,%r12
   0x0000000000418774 <+452>:	jb     0x4186ed <_d_dso_registry+317>
   0x000000000041877a <+458>:	mov    0x2359f(%rip),%rsi        # 0x43bd20 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array>
   0x0000000000418781 <+465>:	mov    0x2359c(%rip),%rbx        # 0x43bd24 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array+4>
   0x0000000000418788 <+472>:	inc    %rbx
   0x000000000041878b <+475>:	mov    %rbx,%rdi
   0x000000000041878e <+478>:	shl    $0x3,%rdi
   0x0000000000418792 <+482>:	callq  0x418b28 <_D2rt4util9container8xreallocFPvmZPv>
   0x0000000000418797 <+487>:	mov    %rax,0x23582(%rip)        # 0x43bd20 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array>
   0x000000000041879e <+494>:	mov    %rbx,0x2357f(%rip)        # 0x43bd24 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array+4>
   0x00000000004187a5 <+501>:	mov    0x23574(%rip),%rcx        # 0x43bd20 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_l---Type <return> to continue, or q <return> to quit---
inux3DSOZ5Array>
   0x00000000004187ac <+508>:	lea    (%rcx,%rbx,8),%rax
=> 0x00000000004187b0 <+512>:	mov    %r14,-0x8(%rax)
   0x00000000004187b4 <+516>:	jmp    0x418819 <_d_dso_registry+617>
   0x00000000004187b6 <+518>:	mov    0x8(%r12),%rdx
   0x00000000004187bb <+523>:	mov    (%rdx),%rbx
   0x00000000004187be <+526>:	mov    0x2355b(%rip),%rsi        # 0x43bd20 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array>
   0x00000000004187c5 <+533>:	mov    0x23558(%rip),%r14        # 0x43bd24 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array+4>
   0x00000000004187cc <+540>:	dec    %r14
   0x00000000004187cf <+543>:	mov    %r14,%rdi
   0x00000000004187d2 <+546>:	shl    $0x3,%rdi
   0x00000000004187d6 <+550>:	callq  0x418b28 <_D2rt4util9container8xreallocFPvmZPv>
   0x00000000004187db <+555>:	mov    %rax,0x2353e(%rip)        # 0x43bd20 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array>
   0x00000000004187e2 <+562>:	mov    %r14,0x2353b(%rip)        # 0x43bd24 <_D2rt14sections_linux12_static_dsosS2rt4util9container36__T5ArrayTPS2rt14sections_linux3DSOZ5Array+4>
---Type <return> to continue, or q <return> to quit---
   0x00000000004187e9 <+569>:	mov    0x8(%r12),%rcx
   0x00000000004187ee <+574>:	movq   $0x0,(%rcx)
   0x00000000004187f5 <+581>:	lea    0x40(%rbx),%r13
   0x00000000004187f9 <+585>:	mov    0x0(%r13),%rsi
   0x00000000004187fd <+589>:	xor    %rdi,%rdi
   0x0000000000418800 <+592>:	callq  0x418b28 <_D2rt4util9container8xreallocFPvmZPv>
   0x0000000000418805 <+597>:	mov    %rax,0x0(%r13)
   0x0000000000418809 <+601>:	movq   $0x0,0x8(%r13)
   0x0000000000418811 <+609>:	mov    %rbx,%rdi
   0x0000000000418814 <+612>:	callq  0x417230 <free@plt>
   0x0000000000418819 <+617>:	pop    %r15
   0x000000000041881b <+619>:	pop    %r14
   0x000000000041881d <+621>:	pop    %r13
   0x000000000041881f <+623>:	pop    %r12
   0x0000000000418821 <+625>:	pop    %rbx
   0x0000000000418822 <+626>:	mov    %rbp,%rsp
   0x0000000000418825 <+629>:	pop    %rbp
   0x0000000000418826 <+630>:	retq
End of assembler dump.
(gdb)

On 6/11/13 12:22 AM, Andrei Alexandrescu wrote:
> On 6/10/13 6:10 PM, Jerry wrote:
>> [pid 23169]
>> open("/home/jlquinn/dmd2/linux/bin64/../lib64/libphobos2.a",
>> O_RDONLY|O_CLOEXEC) = 11
>
> OK, so it seems this is the cause. I'm trying to repro it on a fresh
> Ubuntu 12.10 VM. So at least two bugs:
>
> 1. We link dynamically by default at least on some systems.
>
> 2. The dynamically linked programs crash again at least on some systems.

Was unable to repro on a fresh installation of ubu 12.10/64 in a VirtualBox on OSX. (It's clear there is a problem somewhere though.)

Andrei