View mode: basic / threaded / horizontal-split · Log in · Help
October 11, 2009
dmd support for IDEs
In my discussions with companies about adopting D, the major barrier 
that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
debugger support, libraries, bugs, etc., although those are important.

It's the IDE.

They say that the productivity gains of D's improvements are 
overbalanced by the loss of productivity by moving away from an IDE. And 
what is it about an IDE that is so productive? Intellisense (Microsoft's 
word for autocompletion).

So, while I'm not going to be writing an IDE, I figure that dmd can 
help. dmd already puts out .doc and .di files. How about putting out an 
xml file giving all the information needed for an IDE to implement 
autocompletion? There'd be one .xml file generated per .d source file.

The nice thing about an xml file is while D is relatively easy to parse, 
xml is trivial. Furthermore, an xml format would be fairly robust in the 
face of changes to D syntax.

What do you think?
October 11, 2009
Re: dmd support for IDEs
Walter Bright ïèøåò:
> The nice thing about an xml file is while D is relatively easy to parse,
> xml is trivial.

Why file? An IDE can call compiler process and get output with info from stdout, that will be much faster, and if IDE will need to store the info, it will, or will not, itself.
October 11, 2009
Re: dmd support for IDEs
Walter Bright wrote:
> In my discussions with companies about adopting D, the major barrier 
> that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> They say that the productivity gains of D's improvements are 
> overbalanced by the loss of productivity by moving away from an IDE. And 
> what is it about an IDE that is so productive? Intellisense (Microsoft's 
> word for autocompletion).
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can 
> help. dmd already puts out .doc and .di files. How about putting out an 
> xml file giving all the information needed for an IDE to implement 
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> The nice thing about an xml file is while D is relatively easy to parse, 
> xml is trivial. Furthermore, an xml format would be fairly robust in the 
> face of changes to D syntax.
> 
> What do you think?

I think it would be great, but XML is only one format and a heavy one at 
that, JSON for example is much lighter and easier to parse. It shouldn't 
be hard to support both.

However I would make the file generation optional, as the IDE might just 
want to read from the standard output stream of dmd instead, this would 
also be useful for shell scripts.

Support to get the semantics information of multiple files at once would 
also be neat, just like dmd can generate one object file from multiple 
source files.

Would it even be possible to have the C api behind the xml/json 
frontends exported in a dll, so IDEs could just dynamically link to it 
and call that API directly instead of parsing an intermediary text format.

Jeremie
October 11, 2009
Re: dmd support for IDEs
Jeremie Pelletier wrote:
> I think it would be great, but XML is only one format and a heavy one at 
> that, JSON for example is much lighter and easier to parse. It shouldn't 
> be hard to support both.

I'd never heard of JSON, but looking at it, it makes sense. I don't see 
much point in supporting both.

> However I would make the file generation optional, as the IDE might just 
> want to read from the standard output stream of dmd instead, this would 
> also be useful for shell scripts.

Yes, writing it to stdout as an option is a good idea.


> Support to get the semantics information of multiple files at once would 
> also be neat, just like dmd can generate one object file from multiple 
> source files.

Yes.

> Would it even be possible to have the C api behind the xml/json 
> frontends exported in a dll, so IDEs could just dynamically link to it 
> and call that API directly instead of parsing an intermediary text format.

I did that with the C++ compiler, and it's a big pain to support. I 
don't think it would be onerous to fork/exec the compiler to do the 
work, capture the output, and parse it.
October 11, 2009
Re: dmd support for IDEs
Walter Bright wrote:
> In my discussions with companies about adopting D, the major barrier
> that comes up over and over isn't Tango vs Phobos, dmd being GPL,
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> They say that the productivity gains of D's improvements are
> overbalanced by the loss of productivity by moving away from an IDE. And
> what is it about an IDE that is so productive? Intellisense (Microsoft's
> word for autocompletion).
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can
> help. dmd already puts out .doc and .di files. How about putting out an
> xml file giving all the information needed for an IDE to implement
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> The nice thing about an xml file is while D is relatively easy to parse,
> xml is trivial. Furthermore, an xml format would be fairly robust in the
> face of changes to D syntax.
> 
> What do you think?

Well, that's a better solution than reimplementing semantic analysis in
the ide. If you make it, I will stop trying to do the latter.

In the xml, will we see ct stuff and other transformations that DMD
performs on the source expanded?

[very very minor] concerns:

standardized? DMD derivatives will have it, what about hypothetical
other D implementations?

If your ide can't see or doesn't have compiler, it won't be able to do
much (erm duh)

All in all, I think it would be the bomb. I'd even volunteer to help
implementing it if I thought my code contributions would do less harm
than good.
October 11, 2009
Re: dmd support for IDEs
Ellery Newcomer wrote:
> Well, that's a better solution than reimplementing semantic analysis in
> the ide. If you make it, I will stop trying to do the latter.

That's what I wanted to hear.

> In the xml, will we see ct stuff and other transformations that DMD
> performs on the source expanded?

ct stuff? What is that? Won't see dmd operations on the source expanded. 
What you'll see is basically what ddoc generates, but in a machine 
readable format. I.e. you'll see function names, with their types, line 
number, comment, etc. Essentially what intellisense would pop up.

> [very very minor] concerns:
> 
> standardized? DMD derivatives will have it, what about hypothetical
> other D implementations?

It would be part of the dmd front end, so all D compilers based on it 
would have it.

> If your ide can't see or doesn't have compiler, it won't be able to do
> much (erm duh)

Yeah, but without a compiler why edit D source? <g>

> All in all, I think it would be the bomb. I'd even volunteer to help
> implementing it if I thought my code contributions would do less harm
> than good.

I don't think it would be hard to implement.

But if you want to contribute, how about a JSON parser for phobos? 
You'll need one anyway for your IDE.

BTW, JSON parsing comes for free with javascript. Why not incorporate 
dmdscript into your IDE as its extension language?
October 11, 2009
Re: dmd support for IDEs
Walter Bright:

>In my discussions with companies about adopting D, the major barrier that comes up over and over isn't Tango vs Phobos, dmd being GPL, debugger support, libraries, bugs, etc., although those are important. It's the IDE. They say that the productivity gains of D's improvements are overbalanced by the loss of productivity by moving away from an IDE.<

Welcome to more modern times Walter :-)
You may have noticed that a small army of people in this newsgroup has told you the same things in the last years I've spent around here.
In practice modern statically typed languages aren't designed to be used alone, they are designed to be used with an IDE. So what in the past was a "programming language" today is a "programming language + IDE".
Probably there are ways to overdo this idea (like the idea of programming languages written in XML instead of nornal text), but D isn't risking to fall into this trap yet.


>And what is it about an IDE that is so productive? Intellisense (Microsoft's word for autocompletion).<

The productivity of modern IDEs is a complex thing, it comes from several tuned features. Intellisense is just one of those parts. Have you tried to program 2-3 days with dotnet C#? If you try, you will see several interesting things you don't know about.

Some form of reflection too helps IDEs, I think.
The file you talk about will help refracting tools, I guess.

In C# you have a syntax for sections of code that tells the IDE how to fold code, it's named:
#region
Some people don't like them, but they are common in C#:
http://www.codinghorror.com/blog/archives/001147.html

Attributes too are food for the IDEs, they add semantic information on the code, and IDEs love such information:
@someattribute(data1, datas2, ...)


> I'd never heard of JSON, but looking at it, it makes sense. I don't see 
> much point in supporting both.

XML is very common, so most tools already support it or have ways to support it. So probably big IDEs are able to read XML files. So supporting XML is good.

JSON is light, easy to parse, so if you want to write a simpler tool it can be good. I like JSON, it's quite common on the web and with dynamic languages. But JSON is less common in some situations, so some existing tools may not be already able to digest it.

That's why supporting both looks like a good idea. The good thing is that I don't think will be hard to generate one when DMD is able to generate the other.

If you really want to support only one of the two, then you have to look at the kind of generated data. XML is better for really complex structures, while JSON is enough if the data ha a low level of nesting. The tools designed to query very complex XML files are more advanced and common.

What kind of data do you want to put in such files?

The optional output to stdout of such data is good.

Bye,
bearophile
October 11, 2009
Re: dmd support for IDEs
Walter Bright Wrote:

> In my discussions with companies about adopting D, the major barrier 
> that comes up over and over isn't Tango vs Phobos, dmd being GPL, 
> debugger support, libraries, bugs, etc., although those are important.
> 
> It's the IDE.
> 
> They say that the productivity gains of D's improvements are 
> overbalanced by the loss of productivity by moving away from an IDE. And 
> what is it about an IDE that is so productive? Intellisense (Microsoft's 
> word for autocompletion).
> 
> So, while I'm not going to be writing an IDE, I figure that dmd can 
> help. dmd already puts out .doc and .di files. How about putting out an 
> xml file giving all the information needed for an IDE to implement 
> autocompletion? There'd be one .xml file generated per .d source file.
> 
> The nice thing about an xml file is while D is relatively easy to parse, 
> xml is trivial. Furthermore, an xml format would be fairly robust in the 
> face of changes to D syntax.
> 
> What do you think?

This is a great idea. If I every work on an IDE, I would use this. (I don't use IDEs. I like them, but I haven't found one that keeps out of my way enough.)

And this output isn't just useful for IDEs. Once I get time a couple months from now, I am going to finish a program that generates much better documentation files than Ddoc. So far, I have Ddoc generate custom output that I parse, but it still isn't very machine readable. Instead, I would use this provided it has all the information that Ddoc generates.
October 11, 2009
Re: dmd support for IDEs
Walter Bright:

> So, while I'm not going to be writing an IDE, I figure that dmd can 
> help. dmd already puts out .doc and .di files. How about putting out an 
> xml file giving all the information needed for an IDE to implement 
> autocompletion? There'd be one .xml file generated per .d source file.

An IDE is designed to not just let you explore and understand already written code, but to modify the code too. So such data changes as the programmer writes the code. So it may be positive for the IDE to have ways to query DMD and receive a smaller, faster and more focused amount of data regarding something. Otherwise generating all the data every few moments may slow down things.

Eventually a good thing is to go the route chosen by LLVM, splitting the compiler in parts, so the IDE can use some of those parts in a mode direct way.
Even LDC, that's based on LLVM, follows the monolithic design of GCC and DMD, but I am certain that today a loosely coupled design similar to LLVM is better.

Bye,
bearophile
October 11, 2009
Re: dmd support for IDEs
Walter Bright wrote:
> Ellery Newcomer wrote:
>> Well, that's a better solution than reimplementing semantic analysis in
>> the ide. If you make it, I will stop trying to do the latter.

I made it to the symbol table ..


> 
>> In the xml, will we see ct stuff and other transformations that DMD
>> performs on the source expanded?
> 
> ct stuff? What is that? Won't see dmd operations on the source expanded.
> What you'll see is basically what ddoc generates, but in a machine
> readable format. I.e. you'll see function names, with their types, line
> number, comment, etc. Essentially what intellisense would pop up.
> 

ctfe. compile time (weird connection?). what do string mixins evaluate
to? can I look at their result from the ide? what do templates expand
to? what does this here alias/typedef represent? what does this here
typeof expand to? what does this here c-style type normalize to (in
d-style)?

As for other transformations, it seemed like Ary had some neat tricks in
descent that showed things like int i; going to int i = 0; etc. maybe
wistful thinking.

while we're at it,

when I see a symbol, can I find its type? can I find every symbol that
would follow it in a dot list/exp?
when I see a symbol, can I find everywhere it's used?
when I see a scope, can I see every symbol that's in it?
when I see a module, can I find everywhere it's imported?
can I see exactly what symbols are pulled in? Can I perform analysis to
show me where those dang cyclic dependencies are?
when I see source code, can I perform a simple walk over the xml to
format it?

> It would be part of the dmd front end, so all D compilers based on it
> would have it.

How about the Intel D compiler? (It's going to happen. You know it will)

> 
> Yeah, but without a compiler why edit D source? <g>
> 

Weird users is the best answer I can offer. It happens.

Also, coming from the java IDEs, I'm feeling apprehensive about
integration on disparate platforms and whatnot. There's multiple ways
the compiler could not be there.

>> All in all, I think it would be the bomb. I'd even volunteer to help
>> implementing it if I thought my code contributions would do less harm
>> than good.
> 
> I don't think it would be hard to implement.
> 
> But if you want to contribute, how about a JSON parser for phobos?
> You'll need one anyway for your IDE.
> 
> BTW, JSON parsing comes for free with javascript. Why not incorporate
> dmdscript into your IDE as its extension language?

<g> sorry. my target is netbeans.

Although I could probably whip up something quick in ANTLR if I really
needed JSON in D.
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home