Better yet, here's a slightly different version of the code that outright segfaults:


---- CODE ----

import std.range;
import std.stdio;

struct TransientStringArray
{
	dstring[] src;
	dchar[256] buf;

	this(dstring[] _src) {
		src = _src;
	}

	@property auto front() {
		assert(src[0].length <= buf.length);
		foreach (i, ch; src[0]) {
			buf[i] = ch;
		}
		return buf[0 .. src[0].length];
	}
}

auto joiner(RoR)(RoR r)
{
    static struct Result
    {
    private:
        RoR _items;
        ElementType!RoR _current;
    public:
        this(RoR r)
        {
            _items = r;
            _current = _items.front;	// This produces heap corruption
            //_current = r.front;		// This doesn't???!!!
        }
        @property auto ref front()
        {
            return _current.front;
        }
    }
    return Result(r);
}

auto makeTransientString() {
	auto arr = [ "ab"d, "cd", "ef" ];
	auto tsa = TransientStringArray(arr);
	//auto tsa = TransientStringArray([ "ab", "cd", "ef" ]);
	return joiner(tsa);
}

void func4(T)(T t) {
	writeln("func4:front = ", t.front);
}

void func3(T)(T t) {
	writeln("front = ", t.front);
	// Segfaults here
	func4(t);
}

void func2(R)(R src) {
	func3(src);
}

void func1(S)(S src) {
	func2(src);
}

void main() {
	// This produces a heap corruption
	{
		auto result = makeTransientString();
		func1(result);
	}

	// For some weird reason, this doesn't (or at least, it doesn't show
	// up)
	{
		auto result = makeTransientString();
		func2(result);
	}
}

Can you tell more about what is the corruption ?

Are you getting garbage elements from front ? Can you use a debugger or add some writeln to show some pointer addresses ?

Is GC.collect() changing something (like triggering the problem earlier) ?

On Thu, Dec 20, 2012 at 04:37:25AM +0100, deadalnix wrote:
> Can you tell more about what is the corruption ?
> 
> Are you getting garbage elements from front ? Can you use a debugger or add some writeln to show some pointer addresses ?

Yeah it's garbage elements from front. I'll try a debugger, but this problem only happens in git dmd, not in gdc, and dmd executables are a bit of a pain to deal with in gdb.


> Is GC.collect() changing something (like triggering the problem
> earlier) ?

I don't think it's the GC, because this program is so tiny.


T

-- 
I am Ohm of Borg. Resistance is voltage over current.

On Thursday, 20 December 2012 at 04:15:31 UTC, H. S. Teoh wrote:
> On Thu, Dec 20, 2012 at 04:37:25AM +0100, deadalnix wrote:
>> Can you tell more about what is the corruption ?
>> 
>> Are you getting garbage elements from front ? Can you use a debugger
>> or add some writeln to show some pointer addresses ?
>
> Yeah it's garbage elements from front. I'll try a debugger, but this
> problem only happens in git dmd, not in gdc, and dmd executables are a
> bit of a pain to deal with in gdb.
>

Can you try to check if the pointer of the returned string is the right one (IE, buff) but happen to contain garbage or the pointer itself is garbage ?

>
>> Is GC.collect() changing something (like triggering the problem
>> earlier) ?
>
> I don't think it's the GC, because this program is so tiny.
>

I'm mentioning that because it does look like my GC issue.

On Thu, Dec 20, 2012 at 05:45:51AM +0100, deadalnix wrote:
> On Thursday, 20 December 2012 at 04:15:31 UTC, H. S. Teoh wrote:
> >On Thu, Dec 20, 2012 at 04:37:25AM +0100, deadalnix wrote:
> >>Can you tell more about what is the corruption ?
> >>
> >>Are you getting garbage elements from front ? Can you use a debugger or add some writeln to show some pointer addresses ?
> >
> >Yeah it's garbage elements from front. I'll try a debugger, but this problem only happens in git dmd, not in gdc, and dmd executables are a bit of a pain to deal with in gdb.
> >
> 
> Can you try to check if the pointer of the returned string is the right one (IE, buff) but happen to contain garbage or the pointer itself is garbage ?

I did some digging, and found that the pointer is correct but the target is garbage.

Also, I may have found the cause: if buf is changed from a static array to a dynamic one (with .length set to 256 in the ctor) then the corruption goes away. I'm wondering if maybe the size of the struct is causing it to overflow the stack?

It would explain why the corruption starts to happen at the 4th level of nested function calls, since on Linux the default stack size is approx. 4 KB, and the size of dchar[256] plus 1 or 2 other struct members makes the size of the struct a little over 1 KB, so 4 copies of them on the stack would cause an overflow.


> >>Is GC.collect() changing something (like triggering the problem
> >>earlier) ?
> >
> >I don't think it's the GC, because this program is so tiny.
> >
> 
> I'm mentioning that because it does look like my GC issue.

OK.


T

-- 
It is impossible to make anything foolproof because fools are so ingenious. -- Sammy

2012/12/20 H. S. Teoh <hsteoh@quickfur.ath.cx>

> On Thu, Dec 20, 2012 at 05:45:51AM +0100, deadalnix wrote:
> > On Thursday, 20 December 2012 at 04:15:31 UTC, H. S. Teoh wrote:
> > >On Thu, Dec 20, 2012 at 04:37:25AM +0100, deadalnix wrote:
> > >>Can you tell more about what is the corruption ?
> > >>
> > >>Are you getting garbage elements from front ? Can you use a debugger or add some writeln to show some pointer addresses ?
> > >
> > >Yeah it's garbage elements from front. I'll try a debugger, but this problem only happens in git dmd, not in gdc, and dmd executables are a bit of a pain to deal with in gdb.
> > >
> >
> > Can you try to check if the pointer of the returned string is the right one (IE, buff) but happen to contain garbage or the pointer itself is garbage ?
>
> I did some digging, and found that the pointer is correct but the target is garbage.
>
>
Also, I may have found the cause: if buf is changed from a static array
> to a dynamic one (with .length set to 256 in the ctor) then the corruption goes away. I'm wondering if maybe the size of the struct is causing it to overflow the stack?


This is a self-specific pointer (==slice) problem.

        auto makeTransientString() {
            auto tsa = TransientStringArray([ "ab", "cd", "ef" ]);
            //return joiner(tsa);
            auto x = joiner(tsa); return x; // jsut same as above by RVO
            // In here, x._current points tsa.buf,
            // but tsa.buf is allocated in the stack and
            // will be corrupted after returning makeTransientString()
        }

It would explain why the corruption starts to happen at the 4th level of
> nested function calls, since on Linux the default stack size is approx. 4 KB, and the size of dchar[256] plus 1 or 2 other struct members makes the size of the struct a little over 1 KB, so 4 copies of them on the stack would cause an overflow.


Calling func2(result); is accidentally works IMO.

The root cause is saving _items.front in joiner.Result.ctor, and using it
later.
If you calculate joiner.Result.front every time, problem will disappear.

        @property auto ref front()
        {
            //return _current.front;
            return _items.front.front;
        }

Kenji Hara