On 02/28/2013 02:11 AM, Ary Borenszweig wrote:
> On 2/27/13 9:37 PM, Andrei Alexandrescu wrote:
>> Hello,
>>
>>
>> Walter and I have had a long conversation about the next radical thing
>> to do to improve D's standing. Like others in this community, we believe
>> it's a good time to consider bootstrapping the compiler.
>
> If you do it, I think it's an excellent opportunity to rewrite the compiler *from scratch*, using features in D, and probably using a better design. It's probably easier to design the compiler now that all the features are more or less known. I also remember that DMD didn't have a visitor of sort for the semantic analysis.
>
+1

I think translating the frontend from C++ to D will yield very bad code. The AST Node types in dmd use struct with inheritance. This leads to the question whether or not to use structs or classes in the D frontend. If classes are used the GC will hit you from time to time, so class are not a good idea. Sure the GC could be disabled, but imho will lead to the same problems the frontend has now. Using struct will also require to use heap an pointers, also bad, but imho there are great benefits by from structs and there is a GC workaround for structs. Structs will make it easier to make the frontend into a library and use it from anything that knows C. This is a huge +1 imo. To get around the GC, the AST could be build using shared_ptr pointing to structs. Sure this will have a overhead, but I would rather see the compiler taking longer than telling me "no more memory available". I would also argue that a clean room D impl. with tons of unittests, testings all parse methodes, the lexer, the semantic analyzer etc, will yield a much better maintainable and bug free compiler.

Pro clean room with shared_ptr!(ASTNode):
 - Binding from C
 - fixes memory problem
 - more maintainable code

On Thursday, 28 February 2013 at 00:55:33 UTC, bearophile wrote:
> I think having the D front-end in D is a nice idea, but maybe it's better to use a back-end written by someone else, so most developing work will be spent on D itself.

Won't that hinder the gdc/gcc work?

On Thursday, 28 February 2013 at 00:37:50 UTC, Andrei Alexandrescu wrote:
> Hello,
>
>
> Walter and I have had a long conversation about the next radical thing to do to improve D's standing. Like others in this community, we believe it's a good time to consider bootstrapping the compiler. Having the D compiler written in D has quite a few advantages, among which taking advantages of D's features and having a large codebase that would be its own test harness.
>
> By this we'd like to initiate a dialog about how this large project can be initiated and driven through completion. Our initial basic ideas are:
>
> 1. Implement the dtoh standalone program that takes a D module and generates its corresponding C++ header.
>
> 2. Use dtoh to initiate and conduct an incremental port of the compiler. At given points throughout the code D code will coexist and link with C++ code.
>
> 3. At a point in the future the last C++ module will be replaced with a D module. Going forward there will be no more need for a C++ compiler to build the compiler (except as a bootstrapping test).
>
> It is essential that we get support from the larger community for this. This is a large project that should enjoy strong leadership apart from Walter himself (as he is busy with dynamic library support which is strategic) and robust participation from many of us.
>
> Please chime in with ideas on how to make this happen.
>
>
> Thanks,
>
> Andrei

I don't think that is a good idea.

That will impair GDC and LDC quite a lot. Especially GDC as GCC team accept C++ only recently. This mean no inclusion in the official GCC collection.

This mean that porting DMD to D and making it the default implementation imply to stick with DMD backend (and it isn't open source even if almost). Not being open source can really impair D's popularity.

It also prevent to do actual progress on D during the translation process. FInally, Denis's ddmd hasn't succeeded.

On 2/27/2013 11:35 PM, Jacob Carlborg wrote:
> BTW, how are we going to bootstrap the compiler on any possibly new platforms?
> Cross compiling?

Cross compiling.

On 2/28/2013 2:03 AM, deadalnix wrote:
> That will impair GDC and LDC quite a lot. Especially GDC as GCC team accept C++
> only recently. This mean no inclusion in the official GCC collection.

Hmm. I had thought gccgo was written in go, but it is written in C++:

http://golang.org/doc/gccgo_contribute.html

On 2/27/2013 11:49 PM, Jacob Carlborg wrote:
> There are already several people doing clean room implementations. At least of
> the front end.

Why? The only point would be to change the license of the front end.

On 02/28/2013 01:37 AM, Andrei Alexandrescu wrote:
> Please chime in with ideas on how to make this happen.
>
Splitting off druntime's GC to also use it for dmd could be a great project.

On Thursday, 28 February 2013 at 00:37:50 UTC, Andrei
Alexandrescu wrote:
> Hello,
>
>
> Walter and I have had a long conversation about the next radical thing to do to improve D's standing. Like others in this community, we believe it's a good time to consider bootstrapping the compiler. Having the D compiler written in D has quite a few advantages, among which taking advantages of D's features and having a large codebase that would be its own test harness.

D contains hundred of files, a "little" file can contain 5K of
good old buggy C++ code. A big "file" can contain up to 10K of
lines. I didn't collect statistics. How do you plan to convert it?

> By this we'd like to initiate a dialog about how this large project can be initiated and driven through completion. Our initial basic ideas are:
>
> 1. Implement the dtoh standalone program that takes a D module and generates its corresponding C++ header.

With some kind of magical tool called 'dtoh'? Ok, it can
translate declarations. But what (maybe who) would rewrite code?

> 2. Use dtoh to initiate and conduct an incremental port of the compiler. At given points throughout the code D code will coexist and link with C++ code.

The fact that dmd outputs old object format at win32 would come
at help, wouldn't it? And how such code can coexists? From
straightforward POV, it is clear how D function can call C
function (forgetting about 64 C struct ABI problem), but how for
ex. D code can use C++ class and vice versa? And what about
runtime? Two runtimes: C++ and D? And how GC would treat C++
staff?

> 3. At a point in the future the last C++ module will be replaced with a D module. Going forward there will be no more need for a C++ compiler to build the compiler (except as a bootstrapping test).
>
> It is essential that we get support from the larger community for this. This is a large project that should enjoy strong leadership apart from Walter himself (as he is busy with dynamic library support which is strategic) and robust participation from many of us.

So, you both are asking community help? It is nice to hear, but I
consider that community was in some kind of discrimination
against you in the past except in trivial cases like fixing bugs
and asking something which was badly needed. The very single
example of when you both agreed that you are wrong (after long
insisting that you are right because you are right) is bugzilla
issue on class inheritance and preconditions - whether base class
invariant should be respected or not.

So, I see this idea (and I can be rude and biased here) as "we
haven't treated you seriously in the past, please rewtite 100K
from C++ to D for us, we are to high to do the dirty job
ourselves".

> Please chime in with ideas on how to make this happen.
>
>
> Thanks,
>
> Andrei

P.S. Latest passage is a sum of each small disappointment of how
D project is governed.

On Thursday, 28 February 2013 at 07:34:11 UTC, Jacob Carlborg wrote:
> Long term goal:
>
> When the translation is done we should refactor the compiler/front end to be a library, usable by other tools.

Something like Fabrice Bellard tcc/libtcc? (http://bellard.org/tcc/)
If you can call dmd using an api, you can write a new range of application. Every program could be a custom compiler. And can compile and patch itself.

I support the intention and agree that it needs to be done part-by-part with no re-factoring allowed to minimize regressions. Probably could have even added myself to volunteers, but not sure, it looks like a very time-intensive project.

But issues with LDC and GDC need to be settled first. If D front-end in D considerably harms any of those, it is a complete no-no, even if porting will be perfect. Just not worth the loss.