http://ftp.digitalmars.com/dmd1beta.zip
http://ftp.digitalmars.com/dmd2beta.zip

Includes fixes for all but

http://d.puremagic.com/issues/show_bug.cgi?id=7815

Anyone want to have a go at reducing this one?
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Tue, Apr 10, 2012 at 1:09 PM, Walter Bright <walter@digitalmars.com>wrote:

>
> http://d.puremagic.com/issues/**show_bug.cgi?id=7815<http://d.puremagic.com/issues/show_bug.cgi?id=7815>
>
> Anyone want to have a go at reducing this one?
>


I'm going to take another shot at reducing this tonight or tomorrow now that I'm back at my apartment and have access to my main dev box and more time.  I can't guarantee results in this timeframe, though, because this is really the Bug From Hell.  The struct that is failing to compile is in the expression template system, where everything is very tightly coupled (lots of circular dependencies; it's hard to remove one thing at a time).  For the first test case I submitted, I managed to just inspect the SciD code and write code that reproduced the bug rather than systematically deleting code to get a test case.  I haven't had such luck now that the bug is partially fixed.

On 10 Apr 2012, at 19:09, Walter Bright wrote:
> Includes fixes for all […]

I'll leave you with the last word on this, but I'm still not quite happy with the idea that the result of 7868 [1] is to just accept that an expression silently yields a different result if it is somehow evaluated from a static if condition. The (semi-)regression took me quite some time to track down due to its elusiveness and you some hours to make sense of, and I'm afraid that it will hit other users as well (who probably have no idea about how to debug the frontend).

I am not quite sure about the best way to fix this particular problem – from trying to figure out a way to get the Thrift code working with current Git master it seems that DMD more or less no longer allows adding something to a type based on its current members. It is clear that the semantics of this must be carefully defined, but it is useful for certain »declarative« kinds of meta-programming (especially due to the lack of something like ADL allowing you to »associate« code with a type in another way), it worked in previous DMD releases, and now fails in confusing ways. Maybe we finally need to sit down and formalize our forward reference model (perhaps along the lines of the proposal that came up some time ago – was it by Don? –, for doing semantic analysis one layer of conditionals at a time).

The underlying problem is really that as a developer doing metaprogramming-heavy stuff, you have no idea what actually is supposed to work, and after you found out the hard way by testing what DMD accepts, you still don't know whether it will continue to work in future releases. And this is a much bigger problem than e.g. breaking standard library changes, because you can easily work around the latter.

Oh, and while trying to get Thrift to work, I hit another related crash regression, issue 7886 [2].

David


[1] http://d.puremagic.com/issues/show_bug.cgi?id=7868
[2] http://d.puremagic.com/issues/show_bug.cgi?id=7886
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On Wednesday, April 11, 2012 02:03:55 David Nadlinger wrote:
> I am not quite sure about the best way to fix this particular problem – from trying to figure out a way to get the Thrift code working with current Git master it seems that DMD more or less no longer allows adding something to a type based on its current members.

I honestly would have never expected anything like that to work. You can't normally do anything with a type until it's been declared, and if you're doing anything that requires knowledge about more than the type's name, then it has to be defined before you can do that. Examining it as you're constructing it goes against that, though clearly you've found use cases where it would be useful.

> It is clear
> that the semantics of this must be carefully defined, but it is useful
> for certain »declarative« kinds of meta-programming (especially due to
> the lack of something like ADL allowing you to »associate« code with a
> type in another way), it worked in previous DMD releases, and now fails
> in confusing ways. Maybe we finally need to sit down and formalize our
> forward reference model (perhaps along the lines of the proposal that
> came up some time ago – was it by Don? –, for doing semantic
> analysis one layer of conditionals at a time).
> 
> The underlying problem is really that as a developer doing metaprogramming-heavy stuff, you have no idea what actually is supposed to work, and after you found out the hard way by testing what DMD accepts, you still don't know whether it will continue to work in future releases. And this is a much bigger problem than e.g. breaking standard library changes, because you can easily work around the latter.

I agree. The semantics need to be clearly defined. And this is particularly critical in metaprogramming-heavy code.

- Jonathan M Davis
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/10/2012 5:03 PM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> Includes fixes for all […]
>
> I'll leave you with the last word on this, but I'm still not quite happy with the idea that the result of 7868 [1] is to just accept that an expression silently yields a different result if it is somehow evaluated from a static if condition. The (semi-)regression took me quite some time to track down due to its elusiveness and you some hours to make sense of, and I'm afraid that it will hit other users as well (who probably have no idea about how to debug the frontend).
>
> I am not quite sure about the best way to fix this particular problem – from trying to figure out a way to get the Thrift code working with current Git master it seems that DMD more or less no longer allows adding something to a type based on its current members. It is clear that the semantics of this must be carefully defined, but it is useful for certain »declarative« kinds of meta-programming (especially due to the lack of something like ADL allowing you to »associate« code with a type in another way), it worked in previous DMD releases, and now fails in confusing ways. Maybe we finally need to sit down and formalize our forward reference model (perhaps along the lines of the proposal that came up some time ago – was it by Don? –, for doing semantic analysis one layer of conditionals at a time).
>
> The underlying problem is really that as a developer doing metaprogramming-heavy stuff, you have no idea what actually is supposed to work, and after you found out the hard way by testing what DMD accepts, you still don't know whether it will continue to work in future releases. And this is a much bigger problem than e.g. breaking standard library changes, because you can easily work around the latter.
>

I mostly agree with you, and apologize for the trouble it caused you. The fact that it worked at all before was a fluke, caused by incomplete checking by the compiler, and that incomplete checking caused numerous other problems.

Allowing forward references in general is a very tricky problem in the situation of circular definitions. The strategy that dmd originally followed was to do things in lexical order, and dang the consequences. The more advanced strategy dmd has been migrating to is to allow circular references as long as there is enough information about the partially analyzed type to break the cycle. For example, sometimes only the size of a struct is required. Therefore, dmd requires only that the members of the struct that contribute to its size are analyzed - the other members need not be.

In your case, you had a static if turning on and adding members by checking a condition that must check all the members. This cannot possibly work. I spent a lot of type trying to figure out a way that the cycle could be legitimately broken, but could not.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Yeah, I agree that whatever forward reference model we'll end up with down the road, the conditional thing I was doing in Thrift will probably be illegal there. Fortunately, that one's easy to remove – it was just to avoid adding a »useless« empty field, the value of which is questionable anyway.

Also, I hope that I didn't come across too grumpy – if so, it was certainly not intended, as I know that I'm not exactly sticking to well-trodden ground in some parts of the Thrift implementation. It's just that I already ended up tracking down seven issues during what I hoped would be a bit of hassle-free pre-release testing, and currently, I'm working on reducing another strange, possibly related bug…

David


On 11 Apr 2012, at 2:35, Walter Bright wrote:
> I mostly agree with you, and apologize for the trouble it caused you. The fact that it worked at all before was a fluke, caused by incomplete checking by the compiler, and that incomplete checking caused numerous other problems.
>
> Allowing forward references in general is a very tricky problem in the situation of circular definitions. The strategy that dmd originally followed was to do things in lexical order, and dang the consequences. The more advanced strategy dmd has been migrating to is to allow circular references as long as there is enough information about the partially analyzed type to break the cycle. For example, sometimes only the size of a struct is required. Therefore, dmd requires only that the members of the struct that contribute to its size are analyzed - the other members need not be.
>
> In your case, you had a static if turning on and adding members by checking a condition that must check all the members. This cannot possibly work. I spent a lot of type trying to figure out a way that the cycle could be legitimately broken, but could not.
> _______________________________________________
> dmd-beta mailing list
> dmd-beta@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-beta
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/10/2012 6:12 PM, David Nadlinger wrote:
> Yeah, I agree that whatever forward reference model we'll end up with down the road, the conditional thing I was doing in Thrift will probably be illegal there. Fortunately, that one's easy to remove – it was just to avoid adding a »useless« empty field, the value of which is questionable anyway.
>
> Also, I hope that I didn't come across too grumpy – if so, it was certainly not intended, as I know that I'm not exactly sticking to well-trodden ground in some parts of the Thrift implementation. It's just that I already ended up tracking down seven issues during what I hoped would be a bit of hassle-free pre-release testing, and currently, I'm working on reducing another strange, possibly related bug…
>
> David
>

I admit that what you're doing in the Thrift implementation just makes my brain hurt :-)

Our goal here is to make it a hassle free release, and that means stomping out any new regressions. The only thing that trumps that, however, is fixing broken things in the implementation. Unfortunately, your code was inadvertently relying on implementation bugs. My goal is to make all non-circular forward references work by switching to a lazy semantic evaluation method, and making as many of the circular references as possible work.

I'm relieved you were able to find an easy workaround.

The bugs you (and the others here) have posted are a huge help to making 2.059 a better, more hassle-free release.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 10 Apr 2012, at 19:09, Walter Bright wrote:
> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>
> Anyone want to have a go at reducing this one?

The regression is caused by commit b1031a0 [1], reverting it allows the full SciD test case to build (also see the reduced test case I posted to Bugzilla). I am not sure how to fix the problem without breaking some parts of 4269 again – Don?

As for Thrift, after some changes to the way its codegen works, it passes the test suite on OS X now (will test on the other platforms shortly).

David


[1] https://github.com/D-Programming-Language/dmd/pull/729
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

Thanks much for tracking down which commit did the nasty. It should help figure out where things went wrong.

On 4/11/2012 8:04 AM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>>
>> Anyone want to have a go at reducing this one?
>
> The regression is caused by commit b1031a0 [1], reverting it allows the full SciD test case to build (also see the reduced test case I posted to Bugzilla). I am not sure how to fix the problem without breaking some parts of 4269 again – Don?
>
> As for Thrift, after some changes to the way its codegen works, it passes the test suite on OS X now (will test on the other platforms shortly).
>
> David
>
>
> [1] https://github.com/D-Programming-Language/dmd/pull/729
>
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta

On 4/11/2012 8:04 AM, David Nadlinger wrote:
> On 10 Apr 2012, at 19:09, Walter Bright wrote:
>> http://d.puremagic.com/issues/show_bug.cgi?id=7815
>>
>> Anyone want to have a go at reducing this one?
>
> The regression is caused by commit b1031a0 [1], reverting it allows the full SciD test case to build (also see the reduced test case I posted to Bugzilla). I am not sure how to fix the problem without breaking some parts of 4269 again – Don?
>

All though Don's patch exposed the problem, it is not a bug in Don's patch. The problem, as I explained in

http://d.puremagic.com/issues/show_bug.cgi?id=7815

is, once again, a circular eponymous template expansion, where the compiler cannot figure out if it is eponymous or not because the static if adds members, but the static if depends on the members already being there.

So, I think it is a problem with the example, not the compiler. Though the error message is *terrible*, I will try to fix that.
_______________________________________________
dmd-beta mailing list
dmd-beta@puremagic.com
http://lists.puremagic.com/mailman/listinfo/dmd-beta