Hello,

I am trying to create a lazy range that iterates and chunks data over an array of lazy ranges but the code seems to lead to segFaults. I have tried to reduce the issue to minimum possible code that reproduces the error. My hypothesis is I am doing something wrong leading to the joiner and chunkBy code to not play well with each other, when trying to create a lazy range. But the code does work when I load all the data into memory i.e skip doing it lazily. Here is the reduced code example, running the code on run.dlang.io also reproduces the error:

void main()
{
    import std.range;
    import std.stdio;
    import std.algorithm;
    import etc.linux.memoryerror;
    static if (is(typeof(registerMemoryErrorHandler)))
        registerMemoryErrorHandler();

    ForwardRange!(int)[] listOfRanges;
    listOfRanges ~= iota(1).map!(a => a).inputRangeObject;
    listOfRanges ~= iota(2).map!(a => a).inputRangeObject;
    auto lazyFlattenedRange = joiner(listOfRanges);
    auto d1 = lazyFlattenedRange
                .array // Everything is loaded, no longer lazy
                .chunkBy!(a => a).map!(a => a[0])
                .inputRangeObject;
    writeln("Non Lazy Chunked Data length: ", d1.walkLength);

    // The NEXT part will cause a SEGFAULT.
    lazyFlattenedRange = joiner(listOfRanges); // Reinitializing, since we already iterated it once
    auto d2 = lazyFlattenedRange
                .chunkBy!(a => a).map!(a => a[0])
                .inputRangeObject;
    writeln("Lazy Chunked Data length: ", d2.walkLength); // <--- SegFault
}

Using the handler I was able to get the stack trace and it seems that the segFault is caused by joiner trying to call .save on a null object leading to a NullPointerError. But I have not been able to debug it further. Mostly it seems that there is something wrong with my understanding of ranges or it could be a genuine bug in std.range.
Can anyone help me debug this piece of code?

Thanks,
Keivan Shah

On 7/1/21 10:56 AM, Keivan Shah wrote:

Thanks for the detailed instructions, very easy to reproduce.

However, chunkBy is a hot mess, which probably has a bug in it somewhere. I don't exactly know where this is, but know that using objects for forward ranges is bound to result in some failures, simply because they are one of the only forward ranges that requires calling .save, and much code exists that forgets to do that.

I spent about 20 minutes trying to find this and can't see how chunkBy actually works in this case. I don't have any more time to spend on it, sorry.

I narrowed your code down to the minimal case that I could find that segfaults:

    ForwardRange!int[] listOfRanges = [iota(1).inputRangeObject];
    auto lazyFlattenedRange = joiner(listOfRanges);
    auto d2 = lazyFlattenedRange.chunkBy!(a => a);
    while(!d2.empty)
        d2.popFront;

I hope this helps someone narrow it down.

-Steve