http://ftp.digitalmars.com/dmd1beta.zip
http://ftp.digitalmars.com/dmd2beta.zip

Includes fixes for all but

http://d.puremagic.com/issues/show_bug.cgi?id=7815

Anyone want to have a go at reducing this one?
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Tue, Apr 10, 2012 at 1:09 PM, Walter Bright <walter@digitalmars.com>wrote:

>
> http://d.puremagic.com/issues/**show_bug.cgi?id=7815<http://d.puremagic.com/issues/show_bug.cgi?id=7815>
>
> Anyone want to have a go at reducing this one?
>


I'm going to take another shot at reducing this tonight or tomorrow now
that I'm back at my apartment and have access to my main dev box and more
time.  I can't guarantee results in this timeframe, though, because this is
really the Bug From Hell.  The struct that is failing to compile is in the
expression template system, where everything is very tightly coupled (lots
of circular dependencies; it's hard to remove one thing at a time).  For
the first test case I submitted, I managed to just inspect the SciD code
and write code that reproduced the bug rather than systematically deleting
code to get a test case.  I haven't had such luck now that the bug is
partially fixed.

On 10 Apr 2012, at 19:09, Walter Bright wrote:
> Includes fixes for all […]

I'll leave you with the last word on this, but I'm still not quite happy 
with the idea that the result of 7868 [1] is to just accept that an 
expression silently yields a different result if it is somehow evaluated 
from a static if condition. The (semi-)regression took me quite some 
time to track down due to its elusiveness and you some hours to make 
sense of, and I'm afraid that it will hit other users as well (who 
probably have no idea about how to debug the frontend).

I am not quite sure about the best way to fix this particular problem 
– from trying to figure out a way to get the Thrift code working with 
current Git master it seems that DMD more or less no longer allows 
adding something to a type based on its current members. It is clear 
that the semantics of this must be carefully defined, but it is useful 
for certain »declarative« kinds of meta-programming (especially due to 
the lack of something like ADL allowing you to »associate« code with a 
type in another way), it worked in previous DMD releases, and now fails 
in confusing ways. Maybe we finally need to sit down and formalize our 
forward reference model (perhaps along the lines of the proposal that 
came up some time ago – was it by Don? –, for doing semantic 
analysis one layer of conditionals at a time).

The underlying problem is really that as a developer doing 
metaprogramming-heavy stuff, you have no idea what actually is supposed 
to work, and after you found out the hard way by testing what DMD 
accepts, you still don't know whether it will continue to work in future 
releases. And this is a much bigger problem than e.g. breaking standard 
library changes, because you can easily work around the latter.

Oh, and while trying to get Thrift to work, I hit another related crash 
regression, issue 7886 [2].

David


[1] http://d.puremagic.com/issues/show_bug.cgi?id=7868
[2] http://d.puremagic.com/issues/show_bug.cgi?id=7886
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Wednesday, April 11, 2012 02:03:55 David Nadlinger wrote:
> I am not quite sure about the best way to fix this particular problem
> – from trying to figure out a way to get the Thrift code working with
> current Git master it seems that DMD more or less no longer allows
> adding something to a type based on its current members.

I honestly would have never expected anything like that to work. You can't 
normally do anything with a type until it's been declared, and if you're doing 
anything that requires knowledge about more than the type's name, then it has 
to be defined before you can do that. Examining it as you're constructing it 
goes against that, though clearly you've found use cases where it would be 
useful.

> It is clear
> that the semantics of this must be carefully defined, but it is useful
> for certain »declarative« kinds of meta-programming (especially due to
> the lack of something like ADL allowing you to »associate« code with a
> type in another way), it worked in previous DMD releases, and now fails
> in confusing ways. Maybe we finally need to sit down and formalize our
> forward reference model (perhaps along the lines of the proposal that
> came up some time ago – was it by Don? –, for doing semantic
> analysis one layer of conditionals at a time).
> 
> The underlying problem is really that as a developer doing
> metaprogramming-heavy stuff, you have no idea what actually is supposed
> to work, and after you found out the hard way by testing what DMD
> accepts, you still don't know whether it will continue to work in future
> releases. And this is a much bigger problem than e.g. breaking standard
> library changes, because you can easily work around the latter.

I agree. The semantics need to be clearly defined. And this is particularly 
critical in metaprogramming-heavy code.

- Jonathan M Davis
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/10/2012 5:03 PM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> Includes fixes for all […]
>
> I'll leave you with the last word on this, but I'm still not quite happy with 
> the idea that the result of 7868 [1] is to just accept that an expression 
> silently yields a different result if it is somehow evaluated from a static if 
> condition. The (semi-)regression took me quite some time to track down due to 
> its elusiveness and you some hours to make sense of, and I'm afraid that it 
> will hit other users as well (who probably have no idea about how to debug the 
> frontend).
>
> I am not quite sure about the best way to fix this particular problem – from 
> trying to figure out a way to get the Thrift code working with current Git 
> master it seems that DMD more or less no longer allows adding something to a 
> type based on its current members. It is clear that the semantics of this must 
> be carefully defined, but it is useful for certain »declarative« kinds of 
> meta-programming (especially due to the lack of something like ADL allowing 
> you to »associate« code with a type in another way), it worked in previous DMD 
> releases, and now fails in confusing ways. Maybe we finally need to sit down 
> and formalize our forward reference model (perhaps along the lines of the 
> proposal that came up some time ago – was it by Don? –, for doing semantic 
> analysis one layer of conditionals at a time).
>
> The underlying problem is really that as a developer doing 
> metaprogramming-heavy stuff, you have no idea what actually is supposed to 
> work, and after you found out the hard way by testing what DMD accepts, you 
> still don't know whether it will continue to work in future releases. And this 
> is a much bigger problem than e.g. breaking standard library changes, because 
> you can easily work around the latter.
>

I mostly agree with you, and apologize for the trouble it caused you. The fact 
that it worked at all before was a fluke, caused by incomplete checking by the 
compiler, and that incomplete checking caused numerous other problems.

Allowing forward references in general is a very tricky problem in the situation 
of circular definitions. The strategy that dmd originally followed was to do 
things in lexical order, and dang the consequences. The more advanced strategy 
dmd has been migrating to is to allow circular references as long as there is 
enough information about the partially analyzed type to break the cycle. For 
example, sometimes only the size of a struct is required. Therefore, dmd 
requires only that the members of the struct that contribute to its size are 
analyzed - the other members need not be.

In your case, you had a static if turning on and adding members by checking a 
condition that must check all the members. This cannot possibly work. I spent a 
lot of type trying to figure out a way that the cycle could be legitimately 
broken, but could not.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Yeah, I agree that whatever forward reference model we'll end up with 
down the road, the conditional thing I was doing in Thrift will probably 
be illegal there. Fortunately, that one's easy to remove – it was just 
to avoid adding a »useless« empty field, the value of which is 
questionable anyway.

Also, I hope that I didn't come across too grumpy – if so, it was 
certainly not intended, as I know that I'm not exactly sticking to 
well-trodden ground in some parts of the Thrift implementation. It's 
just that I already ended up tracking down seven issues during what I 
hoped would be a bit of hassle-free pre-release testing, and currently, 
I'm working on reducing another strange, possibly related bug…

David


On 11 Apr 2012, at 2:35, Walter Bright wrote:
> I mostly agree with you, and apologize for the trouble it caused you. 
> The fact that it worked at all before was a fluke, caused by 
> incomplete checking by the compiler, and that incomplete checking 
> caused numerous other problems.
>
> Allowing forward references in general is a very tricky problem in the 
> situation of circular definitions. The strategy that dmd originally 
> followed was to do things in lexical order, and dang the consequences. 
> The more advanced strategy dmd has been migrating to is to allow 
> circular references as long as there is enough information about the 
> partially analyzed type to break the cycle. For example, sometimes 
> only the size of a struct is required. Therefore, dmd requires only 
> that the members of the struct that contribute to its size are 
> analyzed - the other members need not be.
>
> In your case, you had a static if turning on and adding members by 
> checking a condition that must check all the members. This cannot 
> possibly work. I spent a lot of type trying to figure out a way that 
> the cycle could be legitimately broken, but could not.
> _______________________________________________
> dmd-beta mailing list
> dmd-beta@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-beta
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/10/2012 6:12 PM, David Nadlinger wrote:
> Yeah, I agree that whatever forward reference model we'll end up with down the 
> road, the conditional thing I was doing in Thrift will probably be illegal 
> there. Fortunately, that one's easy to remove – it was just to avoid adding a 
> »useless« empty field, the value of which is questionable anyway.
>
> Also, I hope that I didn't come across too grumpy – if so, it was certainly 
> not intended, as I know that I'm not exactly sticking to well-trodden ground 
> in some parts of the Thrift implementation. It's just that I already ended up 
> tracking down seven issues during what I hoped would be a bit of hassle-free 
> pre-release testing, and currently, I'm working on reducing another strange, 
> possibly related bug…
>
> David
>

I admit that what you're doing in the Thrift implementation just makes my brain 
hurt :-)

Our goal here is to make it a hassle free release, and that means stomping out 
any new regressions. The only thing that trumps that, however, is fixing broken 
things in the implementation. Unfortunately, your code was inadvertently relying 
on implementation bugs. My goal is to make all non-circular forward references 
work by switching to a lazy semantic evaluation method, and making as many of 
the circular references as possible work.

I'm relieved you were able to find an easy workaround.

The bugs you (and the others here) have posted are a huge help to making 2.059 a 
better, more hassle-free release.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 10 Apr 2012, at 19:09, Walter Bright wrote:
> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>
> Anyone want to have a go at reducing this one?

The regression is caused by commit b1031a0 [1], reverting it allows the 
full SciD test case to build (also see the reduced test case I posted to 
Bugzilla). I am not sure how to fix the problem without breaking some 
parts of 4269 again – Don?

As for Thrift, after some changes to the way its codegen works, it 
passes the test suite on OS X now (will test on the other platforms 
shortly).

David


[1] https://github.com/D-Programming-Language/dmd/pull/729
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Thanks much for tracking down which commit did the nasty. It should help figure 
out where things went wrong.

On 4/11/2012 8:04 AM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>>
>> Anyone want to have a go at reducing this one?
>
> The regression is caused by commit b1031a0 [1], reverting it allows the full 
> SciD test case to build (also see the reduced test case I posted to Bugzilla). 
> I am not sure how to fix the problem without breaking some parts of 4269 again 
> – Don?
>
> As for Thrift, after some changes to the way its codegen works, it passes the 
> test suite on OS X now (will test on the other platforms shortly).
>
> David
>
>
> [1] https://github.com/D-Programming-Language/dmd/pull/729
>
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/11/2012 8:04 AM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>>
>> Anyone want to have a go at reducing this one?
>
> The regression is caused by commit b1031a0 [1], reverting it allows the full 
> SciD test case to build (also see the reduced test case I posted to Bugzilla). 
> I am not sure how to fix the problem without breaking some parts of 4269 again 
> – Don?
>

All though Don's patch exposed the problem, it is not a bug in Don's patch. The 
problem, as I explained in

http://d.puremagic.com/issues/show_bug.cgi?id=7815

is, once again, a circular eponymous template expansion, where the compiler 
cannot figure out if it is eponymous or not because the static if adds members, 
but the static if depends on the members already being there.

So, I think it is a problem with the example, not the compiler. Though the error 
message is *terrible*, I will try to fix that.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta