View mode: basic / threaded / horizontal-split · Log in · Help
February 04, 2012
dmd Lexer and Parser in D
Greetings! I am a rather new programmer and while this is my first post 
I wanted to say that I did some work on the ddmd project at dsource.org, 
which was kind of a big hairy mess. My fork of this project is at:

https://github.com/zachthemystic/ddmd-clean

The point is, I cleaned out the crappiness but I chucked the entire 
semantic and backend, so that you have left a port of the dmd lexer and 
parser in the D language now. The README there has more to say.

I might well "announce" this on D.announce but I'm too new to have a 
feel for the significance of it all.

Thanks for reading,

Zach
February 04, 2012
Re: dmd Lexer and Parser in D
Maybe there's some IDE that can make use of this.
"Unfortunately" VisualD already has its own ;)
February 04, 2012
Re: dmd Lexer and Parser in D
On Saturday, 4 February 2012 at 05:24:45 UTC, Zach Tollen wrote:
> Greetings! I am a rather new programmer and while this is my 
> first post I wanted to say that I did some work on the ddmd 
> project at dsource.org, which was kind of a big hairy mess. My 
> fork of this project is at:
>
> https://github.com/zachthemystic/ddmd-clean
>
> The point is, I cleaned out the crappiness but I chucked the 
> entire semantic and backend, so that you have left a port of 
> the dmd lexer and parser in the D language now. The README 
> there has more to say.
>
> I might well "announce" this on D.announce but I'm too new to 
> have a feel for the significance of it all.
>
> Thanks for reading,
>
> Zach

Very cool. I was talking with someone on the IRC about the 
possibility/difficulties of making DMD's parser/lexer/AST stay 
open in memory with protocols designed for IDE code-completion 
communication. It would be ideal to have an IDE's intellisense 
automatically update with DMD semantically.

Unfortunately the conclusion was that it would be to difficult an 
undertaking to be realistic, since DMD is designed to be 
run-and-done (also something about "Walter code" :-)). But maybe 
a rewrite/port of DMD, especially one written in D, might be able 
to be reworked with this goal in mind? How complete is DDMD?
February 04, 2012
Re: dmd Lexer and Parser in D
> Unfortunately the conclusion was that it would be to difficult an  
> undertaking to be realistic, since DMD is designed to be run-and-done  
> (also something about "Walter code" :-)). But maybe a rewrite/port of  
> DMD, especially one written in D, might be able to be reworked with this  
> goal in mind? How complete is DDMD?

It's stuck at 2.040. I doubt getting it up-to-date would be worth the  
effort.
Also there are still plenty of unimplemented functions.
February 04, 2012
Re: dmd Lexer and Parser in D
> It's stuck at 2.040. I doubt getting it up-to-date would be 
> worth the effort.
> Also there are still plenty of unimplemented functions.

Ah, oh well...
February 04, 2012
Re: dmd Lexer and Parser in D
On a related note, how much interest is there around here in having an 
official version of dmd written in D?

There are two ways I can imagine this actually happening:
1.
- Improve D's ability to link with C++
- Make D bindings out of the header files
- Port code to D incrementally

2.
- Dify the C++ source (no classes on the stack/embedded, no bitfields, etc)
- Fix all #ifdefs that break up expressions so they can be turned into 
versions
- Create a conversion program to turn it into D ('->' -> '.', (type) -> 
cast(type) etc)

Just something to think about for the distant future.
February 04, 2012
Re: dmd Lexer and Parser in D
"Daniel Murphy" <yebblies@nospamgmail.com> wrote in message 
news:jgj9mu$1q60$1@digitalmars.com...
> On a related note, how much interest is there around here in having an 
> official version of dmd written in D?
>

I'm interested in a D *API* for taking in D sources and spitting out the 
user's choice of either the parser results, or an AST with all the 
semantics/CTFE/etc already run. I get the impressiona lot of people are 
intrested in this.

As far as the actual *implementation* behind the D interface, I don't 
particularly care if it's C, C++, or D.

I suspect having it D might be a pain until a lot more issues get resolved. 
A bootstrapping compiler, I would imagine, would need a much more stable 
base than other types of software would need (though I don't have any 
experience with bootstrapping compilers, so I could be wrong).
February 04, 2012
Re: dmd Lexer and Parser in D
Well, I think, it might be easier to change the dmd implemention to use  
C only and then write language bindings to that. We all know the 
binding situation to C++ won't change sooner or later.

> "Daniel Murphy" <yebblies@nospamgmail.com> wrote in message
> news:jgj9mu$1q60$1@digitalmars.com...
>> On a related note, how much interest is there around here in having 
an
>> official version of dmd written in D?
>>
> 
> I'm interested in a D *API* for taking in D sources and spitting out 
the
> user's choice of either the parser results, or an AST with all the
> semantics/CTFE/etc already run. I get the impressiona lot of people 
are
> intrested in this.
> 
> As far as the actual *implementation* behind the D interface, I don't
> particularly care if it's C, C++, or D.
> 
> I suspect having it D might be a pain until a lot more issues get
> resolved. A bootstrapping compiler, I would imagine, would need a 
much
> more stable base than other types of software would need (though I 
don't
> have any experience with bootstrapping compilers, so I could be 
wrong).
February 04, 2012
Re: dmd Lexer and Parser in D
On 2/4/12 6:59 AM, F i L wrote:
> Very cool. I was talking with someone on the IRC about the
> possibility/difficulties of making DMD's parser/lexer/AST stay open in
> memory with protocols designed for IDE code-completion communication. It
> would be ideal to have an IDE's intellisense automatically update with
> DMD semantically.

This is my thinking too. One good thing about having cut the program is 
that it's a much lighter weight now, and I did it because I thought, 
well, maybe once it's paired down, I can actually steer it toward IDE 
functionality. For example, you could really cut out a lot of the 
members of the data structures which only point to backend functionality 
anyway.

Even if the whole project fails I won't regret doing it because I 
learned a lot about D in the process.

What I'm really wondering is if you wanted a program which helped you 
edit the syntax tree directly and only produced a text file for saving 
and running, what kind of data structure would you like to have 
representing the syntax tree? Without knowing anything else, I guessed 
that it would be nice to have something resembling the official D 
parse-tree.

> Unfortunately the conclusion was that it would be to difficult an
> undertaking to be realistic, since DMD is designed to be run-and-done
> (also something about "Walter code" :-)).

I was wondering if you couldn't take a parse-tree data structure and 
deparse (disparse?) it back to formatted program code so that you could 
see what you were editing? As unrealistic as that sounds, I'm 
sufficiently attracted to the idea that I'm investigating it with an 
open mind.

> But maybe a rewrite/port of DMD, especially one written in D, might
> be able to be reworked with this
> goal in mind? How complete is DDMD?

This is exactly what I'm aiming at. My basic hopes for its being 
possible are the comforting notion that the huge part of dmd is actually 
the stuff I threw out! The goal would be to construct the front end (of 
a front end) which was at least theoretically capable both of allowing 
code editing, and of translation to a more backend-friendly data 
structure. If that's not possible, then I'm stuck with this thought that 
you edit the tree, then the IDE reverses the parse back into ordinary 
code for saving and compiling.

If anybody can refer me to any examples and demonstrations of this type 
of code-editing, please do. As someone new to programming I'm really 
wondering why, if the program itself is understood by the computer as a 
tree, why do I have to edit a text file instead of a tree?

Zach
February 04, 2012
Re: dmd Lexer and Parser in D
"Nick Sabalausky" <a@a.a> wrote in message 
news:jgjngv$2je9$1@digitalmars.com...
> I'm interested in a D *API* for taking in D sources and spitting out the 
> user's choice of either the parser results, or an AST with all the 
> semantics/CTFE/etc already run. I get the impressiona lot of people are 
> intrested in this.
>

This is not that far off.  I've got a branch of dmd, with a di file for 
every h file, that is able to link to itself.  There are still some issues 
with vtables and static variables but hopefully I will sort them out in the 
near future.

What would the D api look like?  If D can link to c++ well enough to call 
into the dmd source, building an api on top of that wouldn't be that bad.

> I suspect having it D might be a pain until a lot more issues get 
> resolved. A bootstrapping compiler, I would imagine, would need a much 
> more stable base than other types of software would need (though I don't 
> have any experience with bootstrapping compilers, so I could be wrong).

Yeah.
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home