| Thread overview | |||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
July 09, 2010 What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 | Periodically, the term AST macros come up in the discussions here as a possible addition to the language - they even get discussed briefly in TDPL as a possible replacement for template mxins. However, I don't think that I've ever seen an actual explanation for what exactly is meant by the term AST macro. This raises two questions 1. What are AST macros and how do they differ from C-style macros? We obviously aren't going to be adding macros like that to D, since that would be dangerous. But what would a macro be then if it's not a textual replacement? The best I can think of would be that you'd have to indicate the macros to be replaced by surrounding them with macro() or something rather than letting any and all text that matches be replaced. So, I really don't know what AST macros are supposed to be other than they're macros of some kind and better than C-style macros. 2. What does AST stand for? The best that I can come up with for what it could stand for would be Abstract Syntax Tree, which is a nice, Computer Sciency, compiler-related term, but I haven't a clue how that would relate to macros. So, maybe an answer to the first question would answer this one as well, but an explanation would be nice. - Jonathan M Davis | |||
Permalink
ReplyJuly 09, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to Jonathan M Davis | "Jonathan M Davis" <jmdavisprog@gmail.com> wrote in message news:mailman.301.1278636124.24349.digitalmars-d@puremagic.com... > Periodically, the term AST macros come up in the discussions here as a > possible > addition to the language - they even get discussed briefly in TDPL as a > possible > replacement for template mxins. However, I don't think that I've ever seen > an > actual explanation for what exactly is meant by the term AST macro. This > raises > two questions > > 1. What are AST macros and how do they differ from C-style macros? We > obviously > aren't going to be adding macros like that to D, since that would be > dangerous. > But what would a macro be then if it's not a textual replacement? The best > I can > think of would be that you'd have to indicate the macros to be replaced by > surrounding them with macro() or something rather than letting any and all > text > that matches be replaced. So, I really don't know what AST macros are > supposed > to be other than they're macros of some kind and better than C-style > macros. > > 2. What does AST stand for? The best that I can come up with for what it > could > stand for would be Abstract Syntax Tree, which is a nice, Computer > Sciency, > compiler-related term, but I haven't a clue how that would relate to > macros. So, > maybe an answer to the first question would answer this one as well, but > an > explanation would be nice. > > - Jonathan M Davis Short answer: Reflection/introspection on steroids. Slightly longer answer: You're right, AST is "Abstract Syntax Tree". As for what that actually means: You know the HTML DOM? If you have the DOM for a specific HTML page, that DOM is a (mutable) representation of the HTML page's AST. If you manipulate the DOM (ie, DHTML), then that's basically AST macros. Except for one thing: HTML is merely markup and D is fully turing-complete. So imagine that the browser *also* has a DOM for the JavaScript itself, so the JavaScript can manipulate itself through a DOM...that's what we're talking about (although it'd be compile-time in our case). So yea...clearly powerful stuff. Purely hypothetical example: Suppose you have this code: module foo; void main(string[] args) { int i; i = 7 + 4 * 3; } The AST might look something roughly like this (I'm representing it here in XML, so I don't have to try to draw an ASCII-art graph): <module name="foo"> <function name="main" type="void"> <params> <param name="args" type="string[]" /> </params> <body> <declaration name="i" type="int" /> <assignment> <lvalue> <variable>i</variable> </lvalue> <rvalue> <add> <literal>7</literal> <multiply> <literal>4</literal> <literal>3</literal> </multiply> </add> </rvalue> </assignment> </body> </function> </module> (Or it might look totally different then that, depending on how the AST system is defined, but you get the idea.) Now imagine an API that exposes a mutable version of that tree (or something similar to it) at compile-time. Also, if you know what a parse tree is, an AST is similar to that, but generally a bit more human-understandable and semantics-oriented, whereas a parse tree is purely a literal representation of the formal syntax - no semantics all, and rather awkward to deal with. Another interesting thing: I've never really used LISP, but my understanding is that this sort of thing basically forms the heart of LISP and is the reason LISP is considered so powerful. (AIUI, LISP was invented purely as a hypothetical example of this exact sort of thing, and then somebody went and actually implemented a real interpreter for it.) | |||
July 09, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to Jonathan M Davis | On Thu, 08 Jul 2010 20:43:39 -0400, Jonathan M Davis <jmdavisprog@gmail.com> wrote: > Periodically, the term AST macros come up in the discussions here as a possible > addition to the language - they even get discussed briefly in TDPL as a possible > replacement for template mxins. However, I don't think that I've ever seen an > actual explanation for what exactly is meant by the term AST macro. This raises > two questions > > 1. What are AST macros and how do they differ from C-style macros? We obviously > aren't going to be adding macros like that to D, since that would be dangerous. > But what would a macro be then if it's not a textual replacement? The best I can > think of would be that you'd have to indicate the macros to be replaced by > surrounding them with macro() or something rather than letting any and all text > that matches be replaced. So, I really don't know what AST macros are supposed > to be other than they're macros of some kind and better than C-style macros. > > 2. What does AST stand for? The best that I can come up with for what it could > stand for would be Abstract Syntax Tree, which is a nice, Computer Sciency, > compiler-related term, but I haven't a clue how that would relate to macros. So, > maybe an answer to the first question would answer this one as well, but an > explanation would be nice. > > - Jonathan M Davis Check out Walter's slides and/or talk from the D conference. (http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.announce&article_id=12555) AST does stand for abstract syntax tree and they are much more like Lisp macros as opposed to the C preprocessor. | |||
July 09, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Nick Sabalausky | "Nick Sabalausky" <a@a.a> wrote in message news:i15uqd$1917$1@digitalmars.com... > > > You're right, AST is "Abstract Syntax Tree". As for what that actually Also, I'm very close to a new release of my Goldie language-processing project ( http://www.dsource.org/projects/goldie ). There are some tools in it ("Parse" and a customized version of "JsonViewer") that make it *very* easy to inspect the parse tree of a source file (although not D yet, sadly) and get a feel for how parse trees work. But it also has a fairly simple documentation-generating tool (really just an HTML pre-processor) that uses its own language, and when you use it, it can output the AST of the documentation source. Then you can use that customized "JsonViewer" to get a good feel for the AST, and how that works, and how it compares to the parse tree. You you might want to keep an eye out on D.announce for that. I'm hoping it'll be within the next few weeks, although that depends on real-world work... | |||
July 09, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Robert Jacques | "Robert Jacques" <sandford@jhu.edu> wrote in message news:op.vfjy3jlo26stm6@sandford... > > Check out Walter's slides and/or talk from the D conference. > (http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.announce&article_id=12555) > AST does stand for abstract syntax tree and they are much more like Lisp > macros as opposed to the C preprocessor. Hmm, looks like the D version is much more template-mixin/C-macro-like than what I made it sound like in a previous branch of the thread. Basically like template mixins, but where the arguments are snippets of code, and overloads are resolved by pattern-matching the syntax of their arguments. | |||
July 10, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to Robert Jacques | Thu, 08 Jul 2010 21:43:57 -0400, Robert Jacques wrote: > Check out Walter's slides and/or talk from the D conference. (http://www.digitalmars.com/webnews/newsgroups.php? art_group=digitalmars.D.announce&article_id=12555) > AST does stand for abstract syntax tree and they are much more like Lisp macros as opposed to the C preprocessor. Too bad nothing happened. He promised AST macros, but it will probably still take several years before we have a preliminary implementation and maybe 10-20 years before a written spec & almsot bug-free implementation. | |||
July 11, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to retard | retard wrote: > Thu, 08 Jul 2010 21:43:57 -0400, Robert Jacques wrote: > >> Check out Walter's slides and/or talk from the D conference. >> (http://www.digitalmars.com/webnews/newsgroups.php? > art_group=digitalmars.D.announce&article_id=12555) >> AST does stand for abstract syntax tree and they are much more like Lisp >> macros as opposed to the C preprocessor. > > Too bad nothing happened. He promised AST macros, but it will probably still take several years before we have a preliminary implementation and maybe 10-20 years before a written spec & almsot bug-free implementation. Part of what happened was that at the conference, I showed that the functionality of string mixins was a superset of what was planned for AST macros. Since that time, there has not been any kind of proposal, or even a concept. So it's a bit meaningless to talk about "AST macros" right now as if they were a planned-but-not-yet-implemented feature. | |||
July 11, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to Don Attachments:
| Don wrote:
>
> Part of what happened was that at the conference, I showed that the functionality of string mixins was a superset of what was planned for AST macros. Since that time, there has not been any kind of proposal, or even a concept. So it's a bit meaningless to talk about "AST macros" right now as if they were a planned-but-not-yet-implemented feature.
>
That's interesting. Do you have a link or any text I could read on that? String mixins sure are powerful, but I can't get ird of a feeling of 'cheating' when using them. Maybe with some kind of string interpolation they could be made more palatable to some?
Philippe
| |||
July 11, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
 Posted in reply to Philippe Sigaud | On Sun, 11 Jul 2010 14:26:51 +0200, Philippe Sigaud wrote:
> Don wrote:
>
>
>> Part of what happened was that at the conference, I showed that the functionality of string mixins was a superset of what was planned for AST macros. Since that time, there has not been any kind of proposal, or even a concept. So it's a bit meaningless to talk about "AST macros" right now as if they were a planned-but-not-yet-implemented feature.
>>
>>
> That's interesting. Do you have a link or any text I could read on that? String mixins sure are powerful, but I can't get ird of a feeling of 'cheating' when using them. Maybe with some kind of string interpolation they could be made more palatable to some?
I find that using token strings, q{ ... }, rather than ordinary "..." strings, sometimes makes string mixins feel less like a hack. Especially considering that my editor (vim) highlights token strings like ordinary code -- or, more likely, it doesn't recognise them as strings. ;)
-Lars
| |||
July 11, 2010 Re: What are AST Macros? | ||||
|---|---|---|---|---|
| ||||
Posted in reply to Philippe Sigaud | Philippe Sigaud wrote: > Don wrote: > > > Part of what happened was that at the conference, I showed that the > functionality of string mixins was a superset of what was planned > for AST macros. Since that time, there has not been any kind of > proposal, or even a concept. So it's a bit meaningless to talk about > "AST macros" right now as if they were a > planned-but-not-yet-implemented feature. > > > That's interesting. Do you have a link or any text I could read on that? Not really, it was just casual discussion on a whiteboard after the conference. I did post some example code on the newsgroup (announce, I think) which did lexing + parsing + basic semantic analysis in 300 lines of code. It gives an idea of what AST macros would need to be able to do. > String mixins sure are powerful, but I can't get ird of a feeling of 'cheating' when using them. Maybe with some kind of string interpolation they could be made more palatable to some? There's little doubt that it's the part of the language which needs the most attention in the long term. But the way forward is not at all obvious. So it's been deferred. | |||
Copyright © 1999-2021 by the D Language Foundation