View mode: basic / threaded / horizontal-split · Log in · Help
October 11, 2009
Re: dmd support for IDEs
On 10/11/09 11:56, Frank Benoit wrote:
> Walter Bright schrieb:
>> They say that the productivity gains of D's improvements are
>> overbalanced by the loss of productivity by moving away from an IDE. And
>> what is it about an IDE that is so productive?
>
> If you use Eclipse for Java, you have:
> - Debugger in place
> - Automatic builder, compile on save
> * Jump to line from error list
> * Jump to declaration
> - Show JavaDoc in tooltip, even in autocompletion
> - Show hierarchy tree
> - Autocompletion
> - Quick assist, e.g.
> 	- assign ctor parameter to new field
> 	- extract selected text into local variable
> 	- Mark source portion, extract to method, the IDE evaluates the needed
> parameters and return value
> 	- ...
> There is so much more. But the main thing is, you are not only able to
> use grep and friends on the pure text level. With an IDE you have
> semantic support. This makes refactoring your code so much easier. You
> can say "rename this method" and it works, all references to this method
> are also altered. "Move this inner class to a top level class in that
> package", "Derive from that class, yes add the needed ctors". There is
> even an API to automate refactorings.
>
> I think Descent is the right way. But here, a port of DMD is directly
> integrated into the plugin. To put more manpower in this project would
> be the best way imo.

I completely agree. I don't know if it would be better to have the parts 
of dmd that descent needs in a separate library to avoid porting half of 
dmd to java.
October 11, 2009
Re: dmd support for IDEs
Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:

> In my discussions with companies about adopting D, the major barrier
> that comes up over and over isn't Tango vs Phobos, dmd being GPL,
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> They say that the productivity gains of D's improvements are
> overbalanced by the loss of productivity by moving away from an IDE. And
> what is it about an IDE that is so productive? Intellisense (Microsoft's
> word for autocompletion).
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can
> help. dmd already puts out .doc and .di files. How about putting out an
> xml file giving all the information needed for an IDE to implement
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> The nice thing about an xml file is while D is relatively easy to parse,
> xml is trivial. Furthermore, an xml format would be fairly robust in the
> face of changes to D syntax.
> 
> What do you think?

Well since there is already a project working on an Eclipse plugin, I see 
little use for other IDEs at the moment. The D community is rather small 
and only a small amount of people are capable of developing and willing 
to donate their free time on free IDE development (commercial IDEs have 
small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio 
dominate the market). So why not concentrate on fixing the spec and 
fixing compiler bugs instead of building a modest IDE support no one will 
use?
October 11, 2009
Re: dmd support for IDEs
language_fan wrote:

> Sat, 10 Oct 2009 18:19:56 -0700, Walter Bright thusly wrote:
> 
>> In my discussions with companies about adopting D, the major barrier
>> that comes up over and over isn't Tango vs Phobos, dmd being GPL,
>> debugger support, libraries, bugs, etc., although those are important.
>> 
>> It's the IDE.
>> 
>> They say that the productivity gains of D's improvements are
>> overbalanced by the loss of productivity by moving away from an IDE. And
>> what is it about an IDE that is so productive? Intellisense (Microsoft's
>> word for autocompletion).
>> 
>> So, while I'm not going to be writing an IDE, I figure that dmd can
>> help. dmd already puts out .doc and .di files. How about putting out an
>> xml file giving all the information needed for an IDE to implement
>> autocompletion? There'd be one .xml file generated per .d source file.
>> 
>> The nice thing about an xml file is while D is relatively easy to parse,
>> xml is trivial. Furthermore, an xml format would be fairly robust in the
>> face of changes to D syntax.
>> 
>> What do you think?
> 
> Well since there is already a project working on an Eclipse plugin, I see
> little use for other IDEs at the moment. The D community is rather small
> and only a small amount of people are capable of developing and willing
> to donate their free time on free IDE development (commercial IDEs have
> small potential now that Netbeans/Eclipse/IntelliJ/KDevelop/Visual Studio
> dominate the market). So why not concentrate on fixing the spec and
> fixing compiler bugs instead of building a modest IDE support no one will
> use?

Because people do see the use of other IDEs (counting vim here too as an 
IDE) and xml / json output is useful for more than just IDEs.
October 11, 2009
Re: dmd support for IDEs
On 2009-10-11 01:57:08 -0400, Walter Bright <newshound1@digitalmars.com> said:

> You're certainly welcome to take the compiler front end and try and 
> make a dll out of it or integrate it directly into an IDE. But what I 
> suggested would probably get a lot of results for a minimal investment 
> in the front end and a minimal investment in existing IDEs.

And I've already done so in D for Xcode (with an old version of DMD).

I had to change the error handling to throw exceptions on errors (no 
call to exit in my IDE please!). I also added some data to tokens to 
get their exact range in the file allowing me to use the DMD lexer for 
syntax highlighting. The semantic also did preserve that information 
and could tell you in what class, template, or function your insertion 
point was on a per-character basis.

And then I stopped there. This is a pain to maintain when DMD gets 
updated, so I didn't. It's buggy because if the compiler crashes, the 
IDE crashes too (keep in mind that parsing incomplete code every few 
seconds has a tendency to cause more crashes than regular compilation).

And finally, Xcode 3 came with a much better syntax definition format 
and a complete revamp of syntax highlighting that obsoleted half the 
integration work I did. So the next version of D for Xcode will get rid 
of DMDFE as an internal component and use Xcode's built-in machinery.

It's not clear to me how much getting supplementary data from the 
compiler could help. If I only get what I can see through Ddoc, it's 
only half useful. I can already parse and get character ranges for the 
the high-level constructs (classes, tempaltes, functions, etc.). What 
will be harder is matching each symbol in function code to the correct 
definition because that depends on the context of the function and 
doing autocompletion for what you type depending on what's available in 
a given context.


> Experience also suggests that using fork/exec rather than a shared dll 
> approach is much more robust and easier to develop. The reason is that 
> the former uses separate processes, which cannot step on each other. 
> The latter puts everything in one process space, where you've got all 
> the lovely, time-consuming, hair-pulling concurrency problems. The 
> utter failure of the parse process also cannot bring down the IDE.

Indeed, you don't want the compiler to crash your IDE.

Also, can DMD accept D files from stdin? That way files wouldn't need 
to be saved on disk on each keystroke.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
October 11, 2009
Re: dmd support for IDEs
Jeremie Pelletier wrote:
> Walter Bright wrote:
>> But if you want to contribute, how about a JSON parser for phobos? 
>> You'll need one anyway for your IDE.
>>
>> BTW, JSON parsing comes for free with javascript. Why not incorporate 
>> dmdscript into your IDE as its extension language?
> 
> The official JSON website has tons of bindings, here's the C one:
> 
> http://fara.cs.uni-potsdam.de/~jsg/json_parser/
> 
> I'm gonna try and get it converted to D over the weekend.

Tango already has a good JSON parser, but I imagine its license (BSD) 
doesn't meet Walter's requirements.
October 11, 2009
Re: dmd support for IDEs
Walter Bright wrote:
> In my discussions with companies about adopting D, the major barrier 
> that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> They say that the productivity gains of D's improvements are 
> overbalanced by the loss of productivity by moving away from an IDE. And 
> what is it about an IDE that is so productive? Intellisense (Microsoft's 
> word for autocompletion).
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can 
> help. dmd already puts out .doc and .di files. How about putting out an 
> xml file giving all the information needed for an IDE to implement 
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> The nice thing about an xml file is while D is relatively easy to parse, 
> xml is trivial. Furthermore, an xml format would be fairly robust in the 
> face of changes to D syntax.
> 
> What do you think?

The huge things are:
 - code navigation (go to definition / find usages)
 - reformatting
 - refactoring
 - autocompletion

Code navigation alone is a huge help, and with reliable autocompletion 
would be sufficient for me to switch from vim. What you are suggesting 
would make both of those easier, though the IDE might need to duplicate 
D's symbol lookup.

I'm not sure whether what you are talking about will help at all with 
reformatting or refactoring, and I really have no idea what would be 
required for this.
October 11, 2009
Re: dmd support for IDEs
Walter Bright wrote:
> In my discussions with companies about adopting D, the major barrier 
> that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can 
> help. dmd already puts out .doc and .di files. How about putting out an 
> xml file giving all the information needed for an IDE to implement 
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> What do you think?

What I think is that even with an xml representing the parse tree (maybe 
with some semantic stuff resolved) it'll be still incomplete for a real 
IDE (the kind of thing users expect from an IDE). You can see this 
video, for example:

http://www.youtube.com/watch?v=KQbTT605ags

So you have:

---
module one;

class Foo(T) {
  static if (is(T == class)) {
    T property;
  } else {
    T someMethod() { return T.init; }
  }
  mixin(guessWhat!(T)());
}
---

You want to define an xml for that module that'll help IDEs. Can you 
think what it'll look like?

Now the user writes in another module:

class Bar {
}

void x() {
  auto foo = new Foo!(Bar)();
  foo. <-- what does the IDE do here?
}

Now, the correct thing for the IDE to do is to suggest the field "Bar 
property". How can the IDE do that just with an xml? It can't. It need 
to perform some kind of semantic anlysis to Foo's argument to see if 
it's a class, match the static if in the template, replace template 
parameters, etc. It also needs to evaluate the string mixin.

Of course you could say "Bah, just show all the declarations inside the 
template in the autocomplete", but that's wrong. That'll lead to files 
that don't compile. You could ommit supporting autocompletion or other 
nice features, but that's exactly the big features of D. If you don't 
support that then it's like using Java or C# from within the IDE: you 
could use the advanced features but the IDE won't help you. And in your 
discussions with companies adopting D, I'm sure they were talking about 
great IDEs like JDT Eclipse or Visual Studio, not just some tool that 
helps you a little but not anymore when things get interesting.

Oh, and you need to have some kind of semantic analysis to know the type 
of "auto foo". Again, maybe the IDE can be dummy and see "auto foo = new 
Foo!(Bar)" and say "ok, foo's type is Foo!(Bar)", but then you have:

auto b = foo.property;
b. <-- and here?
// remember "property" is templated and depends on static analysis
// or the IDE could need to resolve alias this or other things

So... my opinion (like some others, I see) is to either ask things to 
the compiler directly (but here the compiler lacks some info, like exact 
source range positions), or to have a compiler (not a full-blown one, 
just the front-end) built into the IDE, and that's what Descent is. 
Unfortunately Descent is sometimes slow, sometimes buggy, but that's 
normal: just a few people develop and maintain it (so I can see a 
similarity with dmd here, where each day I see two or three new bugs 
reported). If more people were into it, more unit tests were written 
into it and, most of all, more people would use it, it'll get better.

Another problem that people see in Descent (maybe also JDT Eclipse and 
Visual Studio0 is that it's huge, it consumes a lot of memory and they 
don't want to open a huge tool just to hack some lines. My answer is: 
memory performance can be improved (but not a lot), but since an IDE is 
a huge tool it requires a lof from the computer. And an IDE is not meant 
to be used to hack some lines, it's meant to help you write big project, 
huge projects without getting lost in the amount of code.

So my bet would be to start supporting an existing IDE that integrates a 
compiler into it. Updating it is easy: just port the diffs between DMD 
versions. It's a huge job for one or two people, but if a lot of people 
were involved it's not that much. Of course I'd recommend you to try 
Descent since I'm one of it's creators and I do believe it can get 
pretty good. :-)
October 11, 2009
Re: dmd support for IDEs
Lutger wrote:
> digited wrote:
> 
>> Walter Bright �����:
>>> The nice thing about an xml file is while D is relatively easy to parse,
>>> xml is trivial.
>> Why file? An IDE can call compiler process and get output with info from
>> stdout, that will be much faster, and if IDE will need to store the info,
>> it will, or will not, itself.
> 
> Yes please make it write to stdout!

That's a good idea, but traditionally Unix a program that outputs many 
files (such as a compiler) outputs them to named files, with "-" 
representing stdin/stdout.

Andrei
October 11, 2009
Re: dmd support for IDEs
Nick Sabalausky wrote:
> "Walter Bright" <newshound1@digitalmars.com> wrote in message 
> news:has92u$1vum$1@digitalmars.com...
>> Lutger wrote:
>>> What about file/line/column of the symbol? Is this much work / hard work 
>>> to add?
>> file/line of course, but I don't see a point to column.
> 
> So the IDE knows where it is and can actually do things with it, instead of 
> just knowing "Well, it's somewhere around here-ish." 
> 
> 

And what should the tab size be? >:o)

Andrei
October 11, 2009
Re: dmd support for IDEs
On Sun, 11 Oct 2009 05:19:56 +0400, Walter Bright  
<newshound1@digitalmars.com> wrote:

> In my discussions with companies about adopting D, the major barrier  
> that comes up over and over isn't Tango vs Phobos, dmd being GPL,  
> debugger support, libraries, bugs, etc., although those are important.
>
> It's the IDE.
>
> They say that the productivity gains of D's improvements are  
> overbalanced by the loss of productivity by moving away from an IDE. And  
> what is it about an IDE that is so productive? Intellisense (Microsoft's  
> word for autocompletion).
>
> So, while I'm not going to be writing an IDE, I figure that dmd can  
> help. dmd already puts out .doc and .di files. How about putting out an  
> xml file giving all the information needed for an IDE to implement  
> autocompletion? There'd be one .xml file generated per .d source file.
>
> The nice thing about an xml file is while D is relatively easy to parse,  
> xml is trivial. Furthermore, an xml format would be fairly robust in the  
> face of changes to D syntax.
>
> What do you think?

I believe it won't work. It will always be slow and incomplete.

Instead, I would make it easier to embed DMD into an IDE: separate DMDFE  
from DMDBE, fix memory leaks, remove all the static data (so that code  
would be re-intrable and it could work in different threads in parallel),  
move most of DMD code into a DLL so that an IDE could dynamically link  
with it and whatever it pleases with the source code.

In fact, that's what I do right now.

I'm writing my own D IDE in my spare time (in D, of course). I already  
made a lot of progress and now it's time to start implementing source code  
analysis.

First thing I did is I made complete D bindings for C++ code. It worked  
out quite well but it was leaking so badly that I dropped it.

Instead, I started porting DMD entirely to D (except the backend, of  
course), and I already got some great results. A few simple programs  
compile and produce byte-perfect binaries. It's still in its early stages  
and there is a lot of work to do, but it will be finished soon  
(hopefully). It could probably become a part of an official distribution,  
eventually. :)

If anyone is interested and is willing to test and/or help, I will gladly  
share my code.
1 2 3 4 5 6 7 8
Top | Discussion index | About this forum | D home