Hi all,

so there currently seems to be one remaining Travis build failure on the merge-2.064 branch, namely a linking issue in the release tests, and only on LLVM 3.4: https://travis-ci.org/ldc-developers/ldc/builds/25057239

The issue looks like a symbol emission problem where std.net.curl is getting pulled in randomly because it contained a template instance that is missing from whatever other module by chance. However, I can neither replicate the issue on x86_64 Linux locally nor on OS X.

Any ideas? We should really try to get the merge-* branches integrated into master ASAP.

Best,
David

On Sunday, 18 May 2014 at 02:04:52 UTC, David Nadlinger via digitalmars-d-ldc wrote:
> Hi all,
>
> so there currently seems to be one remaining Travis build failure on
> the merge-2.064 branch, namely a linking issue in the release tests,
> and only on LLVM 3.4:
> https://travis-ci.org/ldc-developers/ldc/builds/25057239
>
> The issue looks like a symbol emission problem where std.net.curl is
> getting pulled in randomly because it contained a template instance
> that is missing from whatever other module by chance. However, I can
> neither replicate the issue on x86_64 Linux locally nor on OS X.
>
> Any ideas? We should really try to get the merge-* branches integrated
> into master ASAP.
>
> Best,
> David

Hi David,

I think this is a problem with the multilib setup. This Travis build passes: https://travis-ci.org/ldc-developers/ldc/builds/25474830 - the only difference is that the 32bit libraries are not build (and some applications are not installed, e.g. gcc-multilib).

Regards,
Kai

It is also not reproducible on Ubuntu 13.10 with multilib creation.

Regards,
Kai

On Mon 19 May 2014 05:46:44 PM CEST, Kai Nacke via digitalmars-d-ldc wrote:
> It is also not reproducible on Ubuntu 13.10 with multilib creation.

Hi Kai,

On Mon, May 19, 2014 at 7:36 AM, Kai Nacke via digitalmars-d-ldc <digitalmars-d-ldc@puremagic.com> wrote:
> I think this is a problem with the multilib setup. This Travis build passes: https://travis-ci.org/ldc-developers/ldc/builds/25474830 - the only difference is that the 32bit libraries are not build (and some applications are not installed, e.g. gcc-multilib).

There is still be the possibility that this is a bug in LDC. If it's a case of template symbols not being emitted correctly, then the order in which the other object files in libphobos.a are tried might differ between different toolchain versions, and we might just get unlucky to hit curl.o (std.net) in the multilib case.

If you have a setup where you can actually reproduce this, it might be worth having a short look at what actually causes curl.o to be pulled in. Maybe GNU ld has an option similar to the OS X one to do this, but if you can't find it like me, you could adapt that hacky script I sent you once to track down which unresolved symbol causes the issue.

I agree that it is not worth postponing the release due to this, though, if we don't have concrete evidence that this also affects user code.

Best,
David

Hi David!

On Tuesday, 20 May 2014 at 10:16:46 UTC, David Nadlinger via digitalmars-d-ldc wrote:
> I agree that it is not worth postponing the release due to this,
> though, if we don't have concrete evidence that this also affects user
> code.

I changed my mind here. The problem is reproducible with the beta1 binaries if used with a different Linux distro. I have some time this weekend and will analyze this further.

Regards,
Kai

Hi David!

On Tuesday, 20 May 2014 at 10:16:46 UTC, David Nadlinger via digitalmars-d-ldc wrote:
> Hi Kai,
>
> On Mon, May 19, 2014 at 7:36 AM, Kai Nacke via digitalmars-d-ldc
> <digitalmars-d-ldc@puremagic.com> wrote:
>> I think this is a problem with the multilib setup. This Travis build passes:
>> https://travis-ci.org/ldc-developers/ldc/builds/25474830 - the only
>> difference is that the 32bit libraries are not build (and some applications
>> are not installed, e.g. gcc-multilib).

I am a bit stuck here.

I compiled stdiobase.d (one of the failing tests) with -unittest -main.
Then I extracted libphobos-ldc.a and resolved all dependencies by hand. This results in

gcc -o stdiobase ../stdiobase.o ../__main.o src_rt_dmain2.o std_stdio.o src_object_.o src_ldc_eh.o src_rt_monitor_.o src_rt_critical_.o src_rt_lifetime.o src_rt_tlsgc.o src_rt_aaA.o src_rt_cast_.o src_core_memory.o src_gc_gc.o src_gc_bits.o src_core_sync_mutex.o src_core_sync_exception.o src_core_exception.o src_core_thread.o src_gc_proxy.o src_core_time.o src_rt_adi.o src_rt_typeinfo_ti_*.o src_rt_util_console.o src_rt_sections_ldc.o src_rt_sections_linux.o std_string.o std_exception.o std_format.o src_core_sys_posix_netdb.o std_utf.o std_array.o std_conv.o std_typecons.o std_algorithm.o std_range.o std_typetuple.o std_traits.o std_ascii.o std_functional.o std_uni.o  src_rt_memory.o src_core_stdc_errno.o src_rt_util_hash.o src_gc_os.o src_rt_minfo.o src_core_bitop.o src_rt_util_string.o src_rt_util_utf.o src_rt_util_container.o src_rt_aApply.o src_core_runtime.o src_ldc_arrayinit.o src_rt_qsort.o std_math.o std_random.o std_bitmanip.o std_container.o std_internal_unicode_comp.o std_internal_unicode_tables.o src_core_demangle.o std_numeric.o std_complex.o src_rt_switch_.o errno.c.o  -lc -lpthread -lm -ldl -lrt

This creates the executable stdiobase without a link error and without using -lcurl. I don't understand this. Any ideas? (All on Ubuntu 12 64bit.)

Regards,
Kai

Hi Kai,

On 05/26/2014 07:56 AM, Kai Nacke via digitalmars-d-ldc wrote:
> This creates the executable stdiobase without a link error and without
> using -lcurl. I don't understand this. Any ideas? (All on Ubuntu 12 64bit.)

As I mentioned, I suspect that the issue is dependent on the order the linker searches the object files. Several object files might have the missing template symbol, curl.o just being one of them.

If you run the failing compile with -L-M, you should see a mention of why curl.o is pulled in near the top of the output. IIRC the output also includes information about for which specifc module the symbol was requested. Then, you'd need to debug into LDC to see why the symbol in question is not emitted to the module that needs it.

Best,
David

On Monday, 26 May 2014 at 09:03:29 UTC, David Nadlinger via digitalmars-d-ldc wrote:
> Hi Kai,
>
> On 05/26/2014 07:56 AM, Kai Nacke via digitalmars-d-ldc wrote:
>> This creates the executable stdiobase without a link error and without
>> using -lcurl. I don't understand this. Any ideas? (All on Ubuntu 12 64bit.)
>
> As I mentioned, I suspect that the issue is dependent on the order the linker searches the object files. Several object files might have the missing template symbol, curl.o just being one of them.
>
> If you run the failing compile with -L-M, you should see a mention of why curl.o is pulled in near the top of the output. IIRC the output also includes information about for which specifc module the symbol was requested. Then, you'd need to debug into LDC to see why the symbol in question is not emitted to the module that needs it.
>
> Best,
> David

This reveals:

/build/work/ldc/runtime/../lib/libphobos-ldc.a(std_net_curl.o)
                              /build/work/ldc/runtime/../lib/libphobos-ldc.a(std_stdio.o) (_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm)

The mentioned weak symbol is defined in several files:
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_uni.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_math.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_exception.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_conv.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_functional.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_string.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_typetuple.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_container.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_numeric.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_complex.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_array.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_algorithm.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_bitmanip.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_range.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_format.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_utf.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_traits.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_typecons.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_random.o
_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm in std_net_curl.o

but std.stdio is missing. This at least explains it... Thanks.

Regards,
Kai

> /build/work/ldc/runtime/../lib/libphobos-ldc.a(std_net_curl.o)
> 
> /build/work/ldc/runtime/../lib/libphobos-ldc.a(std_stdio.o)
> (_D6object15__T8capacityTaZ8capacityFNaNbNdAaZm)

You are saying std_stdio.o uses that symbol - and the bug is that the instantiation was not emitted into that object file?

The symptom is that it then uses the instantiation from std_net_curl.o, leading to a linker error because -lcurl wasn't passed?

My runtime/std/stdio.o also only has a
U _D6object15__T8capacityTaZ8capacityFNaNbNdAaZm
so I think I can reproduce it locally.

Regards,
Christian