Thread overview
[FreeBSD] 0x000000000061d8cd in rt.aaA.Impl.findSlotLookup(ulong, const(void*), const(TypeInfo)) inout ()
Sep 23, 2017
Mengu
Sep 23, 2017
Nicholas Wilson
Sep 23, 2017
Nicholas Wilson
Sep 23, 2017
Mengu
September 23, 2017
hello everyone

i have a small program that parses an xml file, holding a list with 13610 elements. after making the list, it iterates over the list (paralele), goes to a web site and grabs the related data for that element.

it works perfect for the first 1K element in the list. after that i get a very annoying segmentation fault. no exceptions, nothing. it just dumps a core file. below is the full stack trace. (also available at https://pastebin.com/PT1R5D7S)

i appreciate any help.

my dmd info:
DMD64 D Compiler v2.076.0-dirty
Copyright (c) 1999-2017 by Digital Mars written by Walter Bright

my os info:
FreeBSD metropol.com 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64



(gdb) bt full
#0  0x000000000061d8cd in rt.aaA.Impl.findSlotLookup(ulong, const(void*), const(TypeInfo)) inout ()
No symbol table info available.
#1  0x00000000005dcb21 in _aaInX ()
No symbol table info available.
#2  0x000000000067fef0 in std.regex.internal.ir.getMatcher(std.uni.InversionList!(std.uni.GcPolicy).InversionList) ()
No symbol table info available.
#3  0x0000000000677a26 in std.regex.internal.parser.CodeGen.charsetToIr(std.uni.InversionList!(std.uni.GcPolicy).InversionList) ()
No symbol table info available.
#4  0x0000000000655f47 in std.regex.internal.parser.Parser!(immutable(char)[], std.regex.internal.parser.CodeGen).Parser.parseEscape() ()
No symbol table info available.
#5  0x0000000000654680 in std.regex.internal.parser.Parser!(immutable(char)[], std.regex.internal.parser.CodeGen).Parser.parseAtom() ()
No symbol table info available.
#6  0x00000000006541e3 in std.regex.internal.parser.Parser!(immutable(char)[], std.regex.internal.parser.CodeGen).Parser.parseRegex() ()
No symbol table info available.
#7  0x00000000006570f0 in std.regex.internal.parser.Parser!(immutable(char)[], std.regex.internal.parser.CodeGen).Parser.this!(const(char)[]).this(immutable(char)[], const(char)[]) ()
No symbol table info available.
#8  0x0000000000652fb1 in std.regex.regexImpl!(immutable(char)[]).regexImpl(immutable(char)[], const(char)[]) ()
No symbol table info available.
#9  0x0000000000636af3 in std.functional.memoize!(std.regex.regexImpl!(immutable(char)[]).regexImpl(immutable(char)[], const(char)[]), 8).memoize(immutable(char)[], const(char)[]) ()
No symbol table info available.
#10 0x00000000005fd617 in std.regex.regex!(immutable(char)[]).regex(immutable(char)[][], const(char)[]) ()
No symbol table info available.
#11 0x00000000005fd472 in std.regex.regex!(immutable(char)[]).regex(immutable(char)[], const(char)[]) ()
No symbol table info available.
#12 0x00000000005f7bb7 in std.net.curl.HTTP.Impl.onReceiveHeader(void(const(char[]), const(char[])) delegate).__lambda2(const(char[])) ()
No symbol table info available.
#13 0x00000000005fb98a in std.net.curl.Curl.onReceiveHeader(void(const(char[])) delegate).__lambda2(const(char[])) ()
No symbol table info available.
#14 0x00000000005fbdce in std.net.curl.Curl._receiveHeaderCallback(const(char*), ulong, ulong, void*) ()
No symbol table info available.
#15 0x000000080681838b in ?? () from /usr/local/lib/libcurl.so
No symbol table info available.
#16 0x0000000806816c95 in ?? () from /usr/local/lib/libcurl.so
No symbol table info available.
#17 0x000000080683119c in ?? () from /usr/local/lib/libcurl.so
No symbol table info available.
#18 0x000000080683bd94 in ?? () from /usr/local/lib/libcurl.so
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#19 0x000000080683b56b in curl_multi_perform () from /usr/local/lib/libcurl.so
No symbol table info available.
#20 0x00000008068334b0 in curl_easy_perform () from /usr/local/lib/libcurl.so
No symbol table info available.
#21 0x00000000005fb77c in std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) ()
No symbol table info available.
#22 0x00000000005f8a20 in std.net.curl.HTTP.perform(std.typecons.Flag!("throwOnError").Flag) ()
No symbol table info available.
#23 0x00000000005cff60 in provider.SunHotels.getHotels(destination.Destination) (this=0x80097e000, destination=...) at source/provider.d:226
        __closptr = 0x800980800
        filePath = "/usr/home/search-master/search-api/ext/data/hotels/6593.xml"
        __dollar = 7
        hotelsPath = "http://some-domain.com"
        data = {ptr = 0x80097f240}
        http = {p = {_refCounted = {_store = 0x805a14000}}}
        postData = "some=postData"
        __flag = 2
        __EAX = 0x8002901170
        __exception_object = 0x0
        __EAX = 0x8002901170
        __EDX = -851895465
        __handler = 0
        __exception_object = 0x801f12540
#24 0x000000000059cdb2 in app.getDestinations().__foreachbody1(ref destination.Destination) (this=0x7fffffffe850, __applyArg0=...) at source/app.d:164
        destination = {destinationId = 6593, destinationCode = "GRR", destinationCode_1 = 0x0, destinationCode_2 = 0x0, destinationCode_3 = 0x0,
          destinationCode_4 = 0x0, destinationName = "Cedar (MI)", countryId = 43, countryName = "USA", countryCode = "US", timeZone = "-05:00"}
#25 0x000000000059e6bc in std.parallelism.ParallelForeach!(destination.Destination[]).ParallelForeach.opApply(scope int(ref destination.Destination) delegate).doIt() (this=0x7fffffffe7d0) at /home/search-master/dmd2/freebsd/bin64/../../src/phobos/std/parallelism.d-mixin-3824:3870
        myUnitIndex = 5
        start = 2130
        end = 2556
        __key3499 = 2306
        __limit3500 = 2556
        i = 2306
        __EAX = 0x8002901170
        __EDX = -851895465
        __handler = 0
        __exception_object = 0x802016500
#26 0x00000000005eadcc in std.parallelism.run!(void() delegate).run(void() delegate) ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#27 0x00000000005ea92c in std.parallelism.Task!(std.parallelism.run, void() delegate).Task.impl(void*) ()
No symbol table info available.
#28 0x00000000006371af in std.parallelism.AbstractTask.job() ()
No symbol table info available.
#29 0x00000000005e9670 in std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) ()
No symbol table info available.
#30 0x00000000005e97c6 in std.parallelism.TaskPool.executeWorkLoop() ()
No symbol table info available.
#31 0x00000000005e976e in std.parallelism.TaskPool.startWorkLoop() ()
No symbol table info available.
#32 0x000000000061177c in core.thread.Thread.run() ()
No symbol table info available.
#33 0x000000000065bade in thread_entryPoint ()
No symbol table info available.
#34 0x0000000800d6abc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#35 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfffe000
September 23, 2017
On Saturday, 23 September 2017 at 08:45:00 UTC, Mengu wrote:
> hello everyone
>
> i have a small program that parses an xml file, holding a list with 13610 elements. after making the list, it iterates over the list (paralele), goes to a web site and grabs the related data for that element.
>
> it works perfect for the first 1K element in the list. after that i get a very annoying segmentation fault. no exceptions, nothing. it just dumps a core file. below is the full stack trace. (also available at https://pastebin.com/PT1R5D7S)
>
> i appreciate any help.
>
> my dmd info:
> DMD64 D Compiler v2.076.0-dirty
> Copyright (c) 1999-2017 by Digital Mars written by Walter Bright
>
> my os info:
> FreeBSD metropol.com 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
>
>
>
> (gdb) bt full
> [...]

So it fails:
trying to find if an element exists in an AA
in a regex
invoked as a callback from curl
inside a parallel foreach.

Interesting that it just straight up core dumps, usually you'll get an exception.
see https://forum.dlang.org/thread/rrpmgzqqtkqgeicjdgps@forum.dlang.org for a recent discussion. What it the stack limit? 35 frames is a fair bit, could be a stack overflow. What is the return code? this will probably give you some info as to what happened.

Only other thing I can suggest is try linking against a debug phobos to see if you can get some more diagnostics.
September 23, 2017
On Saturday, 23 September 2017 at 11:23:26 UTC, Nicholas Wilson wrote:
> Only other thing I can suggest is try linking against a debug phobos to see if you can get some more diagnostics.

You might also try LDC's -fsanitize=address option for catching memory bugs.

September 23, 2017
On Saturday, 23 September 2017 at 11:23:26 UTC, Nicholas Wilson wrote:
> On Saturday, 23 September 2017 at 08:45:00 UTC, Mengu wrote:
>> [...]
>
> So it fails:
> trying to find if an element exists in an AA
> in a regex
> invoked as a callback from curl
> inside a parallel foreach.
>
> Interesting that it just straight up core dumps, usually you'll get an exception.
> see https://forum.dlang.org/thread/rrpmgzqqtkqgeicjdgps@forum.dlang.org for a recent discussion. What it the stack limit? 35 frames is a fair bit, could be a stack overflow. What is the return code? this will probably give you some info as to what happened.
>
> Only other thing I can suggest is try linking against a debug phobos to see if you can get some more diagnostics.

hi nicholas

the latter is what i did. i re-compiled phobos master and used it. this time everything worked as expected.

btw, regex match happens in HTTP.Impl.onReceiveHeader. i think it no longer had access to that header to parse.