View mode: basic / threaded / horizontal-split · Log in · Help
May 24, 2010
Future of Descent and D Eclipse IDE
Hi.

I'm now in a position where I can dedicate a lot of free time for an 
open source project, and I'm seriously considering going back working on 
the D Eclipse IDE project. I worked on Mmrnmhrm a couple of years ago, 
as part of my thesis, which led to some restrictions on the kinds of 
tasks I should work on. Now I don't have that issue, I have (almost) 
complete freedom on what I can work on, and in particular I would like 
to unify the two current efforts for a D Eclipse IDE: Descent and 
Mmrnmhrm, as there is a lot of work being put into both (especially 
Descent ^^ )

Now, Ary has been inactive for quite a while, and he said he wasn't 
interested in working in Descent any more :( . This means I'm currently 
the only Eclipse developer, so I could just do things in the way I'm 
thinking of, but Ary, I would still like to get your input, because I 
hope one day you might be interested in coming back to Descent 
development. :)

There a few issues and ideas I would like to establish for the future D 
Eclipse IDE project. But for now I'll just mention one which I think is 
quite important:

* I think the DMD frontend Java port should be separated into its own 
plugin/bundle, and not be part of the descent.core plugin. The first 
reason is for abstraction and modularization: This project is quite 
distinct on its own, and separation would make it a bit easier for 
people to understand it, and work on it alone (especially since it would 
require almost no knowledge about Eclipse), or for it to be used in 
other projects (like it is now for Mmrnmhrm). Also this allows the 
plugin to not be a singleton, so there could be several plugin instances 
with different versions running at the same time.

Separation purely for modularization should be reason enough, but if you 
are not convinced, there is a more serious issue which regards licenses: 
descent.core source is (heavily) based on the JDT core, which means this 
derived code must be licensed under the EPL. Same for the other Descent 
plugins. But the DMD frontend is licensed under the Artistic License. So 
unless the frontend port could be relicensed as EPL as well, then it 
should be in its own plugin. This is so that each plugin has its own 
license which applies to all its underlying code, to make things clearer.

Ary, I would like to know if by any chance you are interested in doing 
that separation in the near future. If not, I can do it myself, but it 
might take a bit longer.


Cheers

-- 
Bruno Medeiros - Software Engineer
May 24, 2010
Re: Future of Descent and D Eclipse IDE
Hope you can update Descent! :)
I really like it.
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
> Hi.
>
> I'm now in a position where I can dedicate a lot of free time for an
> open source project, and I'm seriously considering going back working on
> the D Eclipse IDE project. I worked on Mmrnmhrm a couple of years ago,
> as part of my thesis, which led to some restrictions on the kinds of
> tasks I should work on. Now I don't have that issue, I have (almost)
> complete freedom on what I can work on, and in particular I would like
> to unify the two current efforts for a D Eclipse IDE: Descent and
> Mmrnmhrm, as there is a lot of work being put into both (especially
> Descent ^^ )
>
> Now, Ary has been inactive for quite a while, and he said he wasn't
> interested in working in Descent any more :(

I know I told you that, but now that I think of it, it's not that I'm 
not interested. The project has grown too big and in the last releases I 
added nice features without thinking much about the design and the 
flexibility of growth... so now I feel the project is kind of a mess and 
it's very hard to continue it. The problems I see are:

 * Porting DMD source to Java was done manually and it's a very boring 
and long task, and we need to find a way to automate it if we'd like to 
support really good integration with the language (I mean, real semantic 
value, and because D is not dynamic I think this is worth it).
 * The DMD source was modified a little bit for performance reasons and 
for integration with Descent so now it contains bugs that are very hard 
to fix.
 * The code is just too big because it has a lot of code from JDT, 
modified a little bit, and that makes it hard for other developers to join.

. This means I'm currently
> the only Eclipse developer, so I could just do things in the way I'm
> thinking of, but Ary, I would still like to get your input, because I
> hope one day you might be interested in coming back to Descent
> development. :)

I can program now and then if I have time, but what I'd really like is 
to plan how to start things almost from scratch and think of a plan of 
doing it well. Maybe Descent could be done with IMP, I don't think using 
JDT's source code will be good for the project. The only problem I see 
with IMP (or DLTK) is that they don't support some of the features 
Descent already provides... but as IMP gets better Descent will 
automatically get those updates.

>
> There a few issues and ideas I would like to establish for the future D
> Eclipse IDE project. But for now I'll just mention one which I think is
> quite important:
>
> * I think the DMD frontend Java port should be separated into its own
> plugin/bundle, and not be part of the descent.core plugin. The first
> reason is for abstraction and modularization: This project is quite
> distinct on its own, and separation would make it a bit easier for
> people to understand it, and work on it alone (especially since it would
> require almost no knowledge about Eclipse), or for it to be used in
> other projects (like it is now for Mmrnmhrm). Also this allows the
> plugin to not be a singleton, so there could be several plugin instances
> with different versions running at the same time.

Yes, I would like that! And I think it's very necessary. Then you can 
plug this module into any IDE implementation you want.

>
> Separation purely for modularization should be reason enough, but if you
> are not convinced, there is a more serious issue which regards licenses:
> descent.core source is (heavily) based on the JDT core, which means this
> derived code must be licensed under the EPL. Same for the other Descent
> plugins. But the DMD frontend is licensed under the Artistic License. So
> unless the frontend port could be relicensed as EPL as well, then it
> should be in its own plugin. This is so that each plugin has its own
> license which applies to all its underlying code, to make things clearer.

Ah, um, right. :-P

>
> Ary, I would like to know if by any chance you are interested in doing
> that separation in the near future. If not, I can do it myself, but it
> might take a bit longer.

Yes, I would like to do it, but first I think we need to think if we 
want to a) keep using the Java port and little by little upgrade it to 
the latest dmd versions, b) start another port but build a tool to more 
or less automate that process (Frank Benoit started something in that 
regard but I don't know what happened then) or c) make a less 
semantic-aware IDE like Mmrnmhrm (the good thing is that it might be 
simpler and faster, but the bad thing is that we might not get all the 
features we'd want).

Thanks for bringing this topic! A lot of effort has been put in Descent 
and it would maybe be sad if the project is abandoned... (but I really 
feel I'm stuck now and I don't know how to advance... or at least I know 
I would have to keep porting C++ code to Java, but...

http://www.explosm.net/comics/1804/

)

Cheers!
May 25, 2010
Re: Future of Descent and D Eclipse IDE
Am 25.05.2010 04:15, schrieb Ary Borenszweig:
> I can program now and then if I have time, but what I'd really like is
> to plan how to start things almost from scratch and think of a plan of
> doing it well.

you should try to bring some of your needed features directly into the 
official compiler suite - like the (far too "small" for your needs) json 
output - maybe there is a way to integrate your needs without stressing 
the code base (and walter) too much
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 2010-05-25 01.00, Trass3r wrote:
> Hope you can update Descent! :)
> I really like it.

Me too.

-- 
/Jacob Carlborg
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 2010-05-25 04.15, Ary Borenszweig wrote:
> On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
>> Hi.
>>
>> I'm now in a position where I can dedicate a lot of free time for an
>> open source project, and I'm seriously considering going back working on
>> the D Eclipse IDE project. I worked on Mmrnmhrm a couple of years ago,
>> as part of my thesis, which led to some restrictions on the kinds of
>> tasks I should work on. Now I don't have that issue, I have (almost)
>> complete freedom on what I can work on, and in particular I would like
>> to unify the two current efforts for a D Eclipse IDE: Descent and
>> Mmrnmhrm, as there is a lot of work being put into both (especially
>> Descent ^^ )
>>
>> Now, Ary has been inactive for quite a while, and he said he wasn't
>> interested in working in Descent any more :(
>
> I know I told you that, but now that I think of it, it's not that I'm
> not interested. The project has grown too big and in the last releases I
> added nice features without thinking much about the design and the
> flexibility of growth... so now I feel the project is kind of a mess and
> it's very hard to continue it. The problems I see are:
>
> * Porting DMD source to Java was done manually and it's a very boring
> and long task, and we need to find a way to automate it if we'd like to
> support really good integration with the language (I mean, real semantic
> value, and because D is not dynamic I think this is worth it).

I have two suggestions for this problem:

1. Could DMD be compiled to a dynamic library and then be used like a 
plugin, using JNI to interact between the compiler and the plugin.

2. Another suggestion is to use 
http://doc.trolltech.com/qtjambi-4.4/html/com/trolltech/qt/qtjambi-generator.html 
or http://www.gccxml.org/HTML/Index.html to either port DMD or create 
interfaces to it and then use it as the first suggestion.

> * The DMD source was modified a little bit for performance reasons and
> for integration with Descent so now it contains bugs that are very hard
> to fix.
> * The code is just too big because it has a lot of code from JDT,
> modified a little bit, and that makes it hard for other developers to join.
> . This means I'm currently
>> the only Eclipse developer, so I could just do things in the way I'm
>> thinking of, but Ary, I would still like to get your input, because I
>> hope one day you might be interested in coming back to Descent
>> development. :)
>
> I can program now and then if I have time, but what I'd really like is
> to plan how to start things almost from scratch and think of a plan of
> doing it well. Maybe Descent could be done with IMP, I don't think using
> JDT's source code will be good for the project. The only problem I see
> with IMP (or DLTK) is that they don't support some of the features
> Descent already provides... but as IMP gets better Descent will
> automatically get those updates.
>
>>
>> There a few issues and ideas I would like to establish for the future D
>> Eclipse IDE project. But for now I'll just mention one which I think is
>> quite important:
>>
>> * I think the DMD frontend Java port should be separated into its own
>> plugin/bundle, and not be part of the descent.core plugin. The first
>> reason is for abstraction and modularization: This project is quite
>> distinct on its own, and separation would make it a bit easier for
>> people to understand it, and work on it alone (especially since it would
>> require almost no knowledge about Eclipse), or for it to be used in
>> other projects (like it is now for Mmrnmhrm). Also this allows the
>> plugin to not be a singleton, so there could be several plugin instances
>> with different versions running at the same time.
>
> Yes, I would like that! And I think it's very necessary. Then you can
> plug this module into any IDE implementation you want.
>
>>
>> Separation purely for modularization should be reason enough, but if you
>> are not convinced, there is a more serious issue which regards licenses:
>> descent.core source is (heavily) based on the JDT core, which means this
>> derived code must be licensed under the EPL. Same for the other Descent
>> plugins. But the DMD frontend is licensed under the Artistic License. So
>> unless the frontend port could be relicensed as EPL as well, then it
>> should be in its own plugin. This is so that each plugin has its own
>> license which applies to all its underlying code, to make things clearer.
>
> Ah, um, right. :-P
>
>>
>> Ary, I would like to know if by any chance you are interested in doing
>> that separation in the near future. If not, I can do it myself, but it
>> might take a bit longer.
>
> Yes, I would like to do it, but first I think we need to think if we
> want to a) keep using the Java port and little by little upgrade it to
> the latest dmd versions, b) start another port but build a tool to more
> or less automate that process (Frank Benoit started something in that
> regard but I don't know what happened then) or c) make a less
> semantic-aware IDE like Mmrnmhrm (the good thing is that it might be
> simpler and faster, but the bad thing is that we might not get all the
> features we'd want).

If you refer to the project Tioport by Frank Benoit, it was a project to 
automate porting Java code to D, I don't see how it would help in this case.

> Thanks for bringing this topic! A lot of effort has been put in Descent
> and it would maybe be sad if the project is abandoned... (but I really
> feel I'm stuck now and I don't know how to advance... or at least I know
> I would have to keep porting C++ code to Java, but...
>
> http://www.explosm.net/comics/1804/
>
> )
>
> Cheers!


-- 
/Jacob Carlborg
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 2010-05-25 04.15, Ary Borenszweig wrote:
> On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
>> Hi.
>> ...

> * Porting DMD source to Java was done manually and it's a very boring
> and long task, and we need to find a way to automate it if we'd like to
> support really good integration with the language (I mean, real semantic
> value, and because D is not dynamic I think this is worth it).
> * The DMD source was modified a little bit for performance reasons and
> for integration with Descent so now it contains bugs that are very hard
> to fix.
> * The code is just too big because it has a lot of code from JDT,
> modified a little bit, and that makes it hard for other developers to join.

The best would be, for Descent's point of view, to do as Eclipse did: 
create a new incremental compiler completely written in Java. But this 
is easier said than done.


-- 
/Jacob Carlborg
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 25/05/2010 03:15, Ary Borenszweig wrote:
> On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
>
> * Porting DMD source to Java was done manually and it's a very boring
> and long task, and we need to find a way to automate it if we'd like to
> support really good integration with the language (I mean, real semantic
> value, and because D is not dynamic I think this is worth it).
> * The DMD source was modified a little bit for performance reasons and
> for integration with Descent so now it contains bugs that are very hard
> to fix.

That's actually exactly the concerns I had from the start with doing 
that approach. Even if we had an easy way to automate the port, I think 
we would still have a code that has issues of performance and 
scalability. And yet if we would try to modify it to address those 
issues, we would most likely introduce hard to correct bugs, and/or make 
the automated porting process (if we had one) much more complicated, if 
not impossible (aka, reverting to doing a lot of manual work).

That's why I think the approach of porting the full DMD frontend is not 
good, even it was more automated. So for me its either:

 * Port and use the DMD parser only, and implement semantic analysys 
features by scratch. This is Mmrnmhrm's current approach, and would mean 
the semantic capabilities would be much less than Descent: likely just 
resolving references (for code completion), but no semantic errors or 
CTFE debug.

 * Try and build the parser and semantic engine with the help of some 
other tool, like IMP for example.

I'll have to take a look at IMP again to think about this some more.

> * The code is just too big because it has a lot of code from JDT,
> modified a little bit, and that makes it hard for other developers to join.
>
> .. This means I'm currently
>> the only Eclipse developer, so I could just do things in the way I'm
>> thinking of, but Ary, I would still like to get your input, because I
>> hope one day you might be interested in coming back to Descent
>> development. :)
>
> I can program now and then if I have time, but what I'd really like is
> to plan how to start things almost from scratch and think of a plan of
> doing it well. Maybe Descent could be done with IMP, I don't think using
> JDT's source code will be good for the project. The only problem I see
> with IMP (or DLTK) is that they don't support some of the features
> Descent already provides... but as IMP gets better Descent will
> automatically get those updates.
>

Yes, this is the other issue is how to do all the IDE infrastructure 
stuff. I also think that porting JDT is less than ideal. Porting JDT has 
similar problems to the DMD approach (although to a much lesser degree). 
The code is not easy to understand, and we risk introducing bugs. And it 
is also hard to maintain and keep updated with new JDT versions.
If there was no project such as IMP or DLTK, I think it would still be 
worth to do the JDT porting. But since those projects exist and are 
currently actively developed, I think they are a much better choice.
In the cases where they don't provide the features that Descent already 
supports, I think that we might be able to do a little bit of 
mix-and-match, and use a some of JDT's ported code. But the feasibility 
of that will depend a lot on the particular feature, and whether IMP or 
DLTK is used.

As for which to use, I'm still inclined torwards DLTK. DLTK has more IDE 
infrastructure support and functionality than IMP, and might make it 
easier to mix-and-match with JDT porting, since a lot of DLTK itself is 
also based on JDT ported code. We might even submit contributions to 
DLTK itself. But it has no support for parsing or language semantic 
analysis, which IMP has. So that's an advantage for IMP.
But I'll have to take another look at IMP (an also the new DLTK version, 
since I'm not yet familiar with all the changes that have occurred in 
the last year or so).


>
>>
>> Ary, I would like to know if by any chance you are interested in doing
>> that separation in the near future. If not, I can do it myself, but it
>> might take a bit longer.
>
> Yes, I would like to do it, but first I think we need to think if we
> want to a) keep using the Java port and little by little upgrade it to
> the latest dmd versions, b) start another port but build a tool to more
> or less automate that process (Frank Benoit started something in that
> regard but I don't know what happened then) or c) make a less
> semantic-aware IDE like Mmrnmhrm (the good thing is that it might be
> simpler and faster, but the bad thing is that we might not get all the
> features we'd want).
>
> Thanks for bringing this topic! A lot of effort has been put in Descent
> and it would maybe be sad if the project is abandoned... (but I really
> feel I'm stuck now and I don't know how to advance... or at least I know
> I would have to keep porting C++ code to Java, but...
>
> http://www.explosm.net/comics/1804/
>
> )
>
> Cheers!

Cool to see you a bit more re-motivated. :)


So let me say what I want to do next: Mmrnmrhm at the moment has no 
support for D2 whatsoever (well its not completely unusable, one can 
open D2 files but it will treat them as D1 files). I want to take the 
current Descent parser, and update Mmrnmrhm's semantic engine to this 
new parser version, such that Mmrnmrhm's current semantic functionality 
(go to references, find references) works in D2 as well. While doing 
this, it should give a good idea of how complex it is to keep up the 
basic IDE functionality on par with language changes, with the approach 
of implementing the code by scratch. I'm confident this can be done 
without much effort, and thus it would validate itself as the best approach.


-- 
Bruno Medeiros - Software Engineer
May 25, 2010
Re: Future of Descent and D Eclipse IDE
On 25/05/2010 14:06, Jacob Carlborg wrote:
> On 2010-05-25 04.15, Ary Borenszweig wrote:
>> On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
>>> Hi.
>>>
>>> I'm now in a position where I can dedicate a lot of free time for an
>>> open source project, and I'm seriously considering going back working on
>>> the D Eclipse IDE project. I worked on Mmrnmhrm a couple of years ago,
>>> as part of my thesis, which led to some restrictions on the kinds of
>>> tasks I should work on. Now I don't have that issue, I have (almost)
>>> complete freedom on what I can work on, and in particular I would like
>>> to unify the two current efforts for a D Eclipse IDE: Descent and
>>> Mmrnmhrm, as there is a lot of work being put into both (especially
>>> Descent ^^ )
>>>
>>> Now, Ary has been inactive for quite a while, and he said he wasn't
>>> interested in working in Descent any more :(
>>
>> I know I told you that, but now that I think of it, it's not that I'm
>> not interested. The project has grown too big and in the last releases I
>> added nice features without thinking much about the design and the
>> flexibility of growth... so now I feel the project is kind of a mess and
>> it's very hard to continue it. The problems I see are:
>>
>> * Porting DMD source to Java was done manually and it's a very boring
>> and long task, and we need to find a way to automate it if we'd like to
>> support really good integration with the language (I mean, real semantic
>> value, and because D is not dynamic I think this is worth it).
>
> I have two suggestions for this problem:
>
> 1. Could DMD be compiled to a dynamic library and then be used like a
> plugin, using JNI to interact between the compiler and the plugin.
>

Nope, the compiler generates a big structure, an AST, which is composed 
of nodes from a complex hierarchy of classes. Transferring such 
sctructure across JNI would be incredibly hard do implement, not to 
mention probably inefficient. JNI (as with most any C interfacing) is 
good mostly just for calling C methods with simple parameters.

The other option would be for the compiler to expose just a thin API 
(without big data structures), and have the IDE query the semantic 
functionality directly to the compiler. But then the frontend itself 
would have to be extended a lot, which would me a lot of complicated 
coding in C... argh, no way...


-- 
Bruno Medeiros - Software Engineer
May 26, 2010
Re: Future of Descent and D Eclipse IDE
On 05/25/2010 03:31 PM, Bruno Medeiros wrote:
> On 25/05/2010 03:15, Ary Borenszweig wrote:
>> On 05/24/2010 08:38 AM, Bruno Medeiros wrote:
>>
>> * Porting DMD source to Java was done manually and it's a very boring
>> and long task, and we need to find a way to automate it if we'd like to
>> support really good integration with the language (I mean, real semantic
>> value, and because D is not dynamic I think this is worth it).
>> * The DMD source was modified a little bit for performance reasons and
>> for integration with Descent so now it contains bugs that are very hard
>> to fix.
>
> That's actually exactly the concerns I had from the start with doing
> that approach. Even if we had an easy way to automate the port, I think
> we would still have a code that has issues of performance and
> scalability. And yet if we would try to modify it to address those
> issues, we would most likely introduce hard to correct bugs, and/or make
> the automated porting process (if we had one) much more complicated, if
> not impossible (aka, reverting to doing a lot of manual work).
>
> That's why I think the approach of porting the full DMD frontend is not
> good, even it was more automated. So for me its either:
>
> * Port and use the DMD parser only, and implement semantic analysys
> features by scratch. This is Mmrnmhrm's current approach, and would mean
> the semantic capabilities would be much less than Descent: likely just
> resolving references (for code completion), but no semantic errors or
> CTFE debug.
>
> * Try and build the parser and semantic engine with the help of some
> other tool, like IMP for example.

If anyone cares, I have a functioning ANTLR grammar for d1 for which I 
someday intend to write semantic for. I'm also mostly done with an ANTLR 
grammar for d2.

They're a bit on the slow side, but I think they can successfully parse 
things that dmd currently can't.
« First   ‹ Prev
1 2 3 4
Top | Discussion index | About this forum | D home