Interested in being abreast of the GSoC 2012 projects? Here's how (page 3)

May 23, 2012

Re: Interested in being abreast of the GSoC 2012 projects? Here's how

Posted by Andrew Wiley
in reply to Jacob Carlborg

Permalink

Andrew Wiley

Posted in reply to Jacob Carlborg

Attachments:

text/html part

Permalink

On Wed, May 23, 2012 at 1:38 AM, Jacob Carlborg <doob@me.com> wrote:

> On 2012-05-22 21:45, Andrew Wiley wrote:
>
>> On Tue, May 22, 2012 at 3:57 AM, Jacob Carlborg <doob@me.com <mailto:doob@me.com>> wrote:
>>
>>    On 2012-05-21 21:48, Andrew Wiley wrote:
>>
>>        Gee, thanks for your enthusiastic support for GSOC projects that
>>        will
>>        greatly forward the D ecosystem.
>>
>>        Ultimately, what's useful to the D community (for reasons
>>        discussed in
>>        these NGs many times over) is that we have working, mature,
>>        feature-rich
>>        IDEs. The languages they're implemented in are mostly
>>        irrelevant, and in
>>        MonoDevelop's case, trying to add language support via a plugin
>>        written
>>        in D to an IDE written in C# would be silly. Would you extend
>>        Eclipse in
>>        C++? It just doesn't make any sense at all.
>>
>>
>>    I see no reason why the compiler can't be implemented in D and have
>>    a C interface.
>>
>>
>> Certainly possible, but we'll need to keep a bootstrap compiler around.
>>
>
> You have the same problem with C and C++. Although that's way moare easier since that's what all systems use.
>

Yes, C/C++ are ubiquitous these days.

        What's more, building tools for D in languages other than D can be
>>        extremely useful. Every time a discussion for a D compiler
>>        written in D
>>        comes up, no one really likes to mention the benefits we've
>>        gotten from
>>        having a compiler written in C++:
>>
>>
>>    Again as above.
>>
>>
>> I never said a compiler couldn't be implemented in D. I said implementing it in C++ has given us advantages that no one generally considers.
>>
>
> Fair enough.
>
>           - there are no bootstrapping problems because C++ exists on
>>        basically
>>        every platform D would ever want to target
>>
>>
>>    Provide a C backend.
>>
>>
>> Not if you want good codegen. Implementing it in C would disable many high level optimizations and force a single implementation for low level concepts that should vary across implementations. I can pull some relevant discussions from the GCC mailing list if you're interested.
>>
>
> I don't understand this. Say you want port a hypothetical D compiler, DC, from Foo to Bar. You already have DC working on Foo and you already have a working C compiler on Bar. Then you just:
>
> 1. Compile DC with DC, outputting C code, on Foo
> 2. Take the C code to Bar and compile DC (now the C code) on Bar
> 3. Take the D code to Bar and compile DC (now the D code) on Bar using DC
> compiled from C code


Ultimately, it doesn't really change the number of steps required:
(Foo -> Bar means a compiler that runs on Foo and outputs binaries that run
on Bar)

Standard cross compiler sequence:
1. Compile DC (Foo -> Bar) on Foo using the existing DC (Foo -> Foo)
2. Compile DC (Bar -> Bar) on Foo using the newly built DC (Foo -> Bar)
Now you have Bar -> Bar, which is what you wanted, and we had to build DC
twice. DC also has to support codegen for both Foo and Bar.

What you seem to be wanting is this:
1. Compile DC ((C code) -> Bar) on Foo using the existing DC (Foo ->
{Foo,C})
2. Use the C compiler on Bar to turn (C code) -> Bar into Bar -> Bar (but
this build is slow because it used C as an intermediate form)
3. Use the slow Bar -> Bar to compile a fast Bar -> Bar
Again, we have Bar -> Bar after two builds of DC, but DC had to support
codegen for Foo, Bar, and C.

Targeting C doesn't really seem to make bootstrapping like this any more efficient.


 As for the more general discussion of building a compiler-as-a-library
>> in D, I agree that it would be tremendously useful, but I don't think
>> it's quite the holy grail it first appears to be. For tools written in
>> D, it could be tied right in, but on any VM platform, I question whether
>> using a D library directly is actually feasible. Building a VM<->Native
>> interoperability layer is simple enough when you're just calling the
>> library to perform simple tasks, but we're talking about a library that
>> would primarily be responsible for creating and updating large data
>> structures (namely, the AST). Tying that into a VM language would be
>> very difficult to do efficiently because you'd either make a lot of
>> VM->Native function calls or convert the entire AST back and forth from
>> native to VM-land. It gets even more fun because you'd have to maintain
>> a C-API on the native side as well as a JNI layer (or what Mono/CLI
>> uses) and all the wrapper code in the VM language.
>> Ultimately, it may be simpler just to port the library to the language
>> of the actual platform. I suppose even that would probably be an
>> improvement.
>>
>
> Ok, you have a point there. So what do you suggest:
>
> * Reinventing the wheel for every language that needs a D compiler
> * Don't integrate the compiler with languages that can't directly use C
> * Write and IDE using D can directly interface with the compiler library
>

Unfortunately, I don't really have a satisfying solution to this. At the moment, we're reinventing the wheel. The best alternative I see if a library like this were to exist would be to port it instead of reinventing it. Maybe using it from a VM wouldn't be as hard as I'm thinking, but it's hard to speculate.


> On the other hand if you have a compiler library you can build a tool based on the library that translates D code to Java, C# or perhaps their byte code equivalents. Then you can automatically translate the compiler library to whatever language you like and integrate it with VM-based IDE's.


I agree with Roman that automated translation to a VM language would probably be a difficult thing to attempt, although I think it could be doable. I don't think the effort/benefit ration is low enough. It's not often that someone needs a mechanical translation of a D library to another language.

On 2012-05-23 19:11, Andrew Wiley wrote:

> Ultimately, it doesn't really change the number of steps required:
> (Foo -> Bar means a compiler that runs on Foo and outputs binaries that
> run on Bar)
>
> Standard cross compiler sequence:
> 1. Compile DC (Foo -> Bar) on Foo using the existing DC (Foo -> Foo)
> 2. Compile DC (Bar -> Bar) on Foo using the newly built DC (Foo -> Bar)
> Now you have Bar -> Bar, which is what you wanted, and we had to build
> DC twice. DC also has to support codegen for both Foo and Bar.
>
> What you seem to be wanting is this:
> 1. Compile DC ((C code) -> Bar) on Foo using the existing DC (Foo ->
> {Foo,C})
> 2. Use the C compiler on Bar to turn (C code) -> Bar into Bar -> Bar
> (but this build is slow because it used C as an intermediate form)
> 3. Use the slow Bar -> Bar to compile a fast Bar -> Bar
> Again, we have Bar -> Bar after two builds of DC, but DC had to support
> codegen for Foo, Bar, and C.
>
> Targeting C doesn't really seem to make bootstrapping like this any more
> efficient.

Then what's the problem.

> Unfortunately, I don't really have a satisfying solution to this. At the
> moment, we're reinventing the wheel. The best alternative I see if a
> library like this were to exist would be to port it instead of
> reinventing it. Maybe using it from a VM wouldn't be as hard as I'm
> thinking, but it's hard to speculate.

I might be less of a problem when the language isn't changed that much.

>     On the other hand if you have a compiler library you can build a
>     tool based on the library that translates D code to Java, C# or
>     perhaps their byte code equivalents. Then you can automatically
>     translate the compiler library to whatever language you like and
>     integrate it with VM-based IDE's.
>
>
> I agree with Roman that automated translation to a VM language would
> probably be a difficult thing to attempt, although I think it could be
> doable. I don't think the effort/benefit ration is low enough. It's not
> often that someone needs a mechanical translation of a D library to
> another language.

Perhaps a tool for automatically creating bindings to the compiler library. But then I don't know how efficient it would be to move the necessary data across the VM boundaries.

-- 
/Jacob Carlborg

Forums