View mode: basic / threaded / horizontal-split · Log in · Help
December 16, 2010
Re: New syntax for string mixins
> I've attached a part of how concurrency.d could look like translated 
> to my suggested syntax. It probably contains a lot of errors because 
> did a quick translation and I had some trouble understanding the mixins.
>
Yes, even I couldn't understand them even a week later. Maintenance is a 
real problem.

My initial reaction is that the proposed syntax helps a bit, but isn't 
really a game-changer. I will let you know if I change my mind on closer 
inspection.

-- 
Graham St Jack
December 17, 2010
Re: New syntax for string mixins
Hi,

I'm one of Nemerle developers.

> From an implementation point of view, the differences between Nemerle
> macros and string mixins are mostly syntactic.

You are deeply mistaken!
I'd advise to study the Nemerle macro-system.

You missed: pattern matching, quasi-quotation, running of fully functional Nemerle
code in compile time, access to compiler API in macros and IntelliSense support.

>The one thing about them that I find really impressive is the IDE
>integration, especially that they got syntax highlighting to work.
>I don't know they've done that.

"syntax highlighting" ? :)
We have full IntelliSense support: syntax highlighting, code completion, macro
expansion (in tooltips).
We simply use the Nemerle compiler to achive that.
December 17, 2010
Re: New syntax for string mixins
VladD2 wrote:
> Hi,
> 
> I'm one of Nemerle developers.

Cool!

>> From an implementation point of view, the differences between Nemerle
>> macros and string mixins are mostly syntactic.
> 
> You are deeply mistaken!
> I'd advise to study the Nemerle macro-system.
> 
> You missed: pattern matching, 

Yes, you've got me there. I've assumed that pattern matching, while a 
major feature, is not fundamental to the Nemerle macro-system, but I may 
be mistaken.

One obvious difference which *is* fundamental is that Nemerle macros 
allow new syntax.

Am I correct in thinking that Nemerle always requires complete ASTs?
That is, given a name "x", can you access variables "x1", "x2", "x3" ?

> quasi-quotation,

No, that's present in D. It's the primary reason I say that the 
differences are mostly syntactic, since I see this as THE fundamental 
feature. You need to be able to reenter the macro system after you have 
left it. Once you can do that, you can do pretty much anything.

BTW if you argued that D's current syntax is quite horrible, I couldn't 
disagree with you.

> running of fully functional Nemerle
> code in compile time, 

Yes, but that's a different issue. In D, running code at compile time is 
 regarded as an aspect of constant-folding, and is not restricted to 
macros.

> access to compiler API in macros

In practice, has to be a library, right? Otherwise the compiler 
internals would be exposed? (This is an issue we're struggling with).

Another one of the big differences is that D doesn't allow compile-time 
code to call external functions. Although it could certainly be done, it 
raises some big issues. Eg, we cannot assume that the target CPU is the 
same as the one we're running on. With a JIT compiler, you don't have 
that problem.

> and IntelliSense support.

I did mention that...

>> The one thing about them that I find really impressive is the IDE
>> integration, especially that they got syntax highlighting to work.
>> I don't know they've done that.
> 
> "syntax highlighting" ? :)
> We have full IntelliSense support: syntax highlighting, code completion, macro
> expansion (in tooltips).

I meant Intellisense, not just syntax highlighting. Though note that you 
can't do syntax highlighting *perfectly* unless the IDE understands the 
code.

> We simply use the Nemerle compiler to achive that.

Doesn't leave me any less impressed. <g>
December 17, 2010
Re: New syntax for string mixins
Nick Sabalausky wrote:

> "Jacob Carlborg" <doob@me.com> wrote in message
> news:iednio$2vj5$1@digitalmars.com...
>>
>> I can't quite visualize how the final code will look like and as you say
>> it's hard without a formal definition of AST macros. But I think somehow
>> it would be possible, I mean instead of combining strings one combine
>> expressions/syntax. But I don't know if it would be possible to combine
>> incomplete expressions.
>>
> 
> One parallel that may or may not be applicable, but might be worth
> considering, is dynamically building XHTML: Using a string-template system
> is generally found to work very well, but building it via a DOM
> (essentially an AST) is often considered a bit of a pain. I guess the
> simplest take-away would be that string-based approaches may be easier get
> working well. FWIW.

The newest vb.net has this feature, if I understand you correctly, via xml 
literals and linq expressions. It is quite convenient.

http://msdn.microsoft.com/en-us/library/bb384460.aspx
December 17, 2010
Re: New syntax for string mixins
Don Wrote:

> VladD2 wrote:
> > Hi,
> > 
> > I'm one of Nemerle developers.
> 
> Cool!
> 
> >> From an implementation point of view, the differences between Nemerle
> >> macros and string mixins are mostly syntactic.
> > 
> > You are deeply mistaken!
> > I'd advise to study the Nemerle macro-system.
> > 
> > You missed: pattern matching, 
> 
> Yes, you've got me there. I've assumed that pattern matching, while a 
> major feature, is not fundamental to the Nemerle macro-system, but I may 
> be mistaken.
> 
> One obvious difference which *is* fundamental is that Nemerle macros 
> allow new syntax.
> 
> Am I correct in thinking that Nemerle always requires complete ASTs?
> That is, given a name "x", can you access variables "x1", "x2", "x3" ?
> 
>  > quasi-quotation,
> 
> No, that's present in D. It's the primary reason I say that the 
> differences are mostly syntactic, since I see this as THE fundamental 
> feature. You need to be able to reenter the macro system after you have 
> left it. Once you can do that, you can do pretty much anything.
> 
> BTW if you argued that D's current syntax is quite horrible, I couldn't 
> disagree with you.
> 
> > running of fully functional Nemerle
> > code in compile time, 
> 
> Yes, but that's a different issue. In D, running code at compile time is 
>   regarded as an aspect of constant-folding, and is not restricted to 
> macros.
> 
> > access to compiler API in macros
> 
> In practice, has to be a library, right? Otherwise the compiler 
> internals would be exposed? (This is an issue we're struggling with).
> 
> Another one of the big differences is that D doesn't allow compile-time 
> code to call external functions. Although it could certainly be done, it 
> raises some big issues. Eg, we cannot assume that the target CPU is the 
> same as the one we're running on. With a JIT compiler, you don't have 
> that problem.

Don, can you please elaborate on this point? 

Here's my understanding:
The D compiler is run once to both 'execute' compile time code which you refer above as constant folding AND to generate the binary to execute at run-time. 

Nemerle separates this into two distinct steps:
1. you compile regular code inside macro definitions into a compiler plugin. 
2. when compiling the intended run-time code you need to load the compiled macros from step 1 above on the command line of the compiler.

Since you're talking above about cross-compilation and let's say we run the compiler on X and compile for Y, I see no problem to load precompiled macros for X in order to compile the code for Y. The only limit as far as I can see is that it won't be possible to load macros compiled by the such a cross compiler on X unless as you say JIT is employed. Perhaps it makes sense to disable macro definition in a cross-compilation scenario and only allow usage.

Is there something else that I'm missing here? 

> 
> > and IntelliSense support.
> 
> I did mention that...
> 
> >> The one thing about them that I find really impressive is the IDE
> >> integration, especially that they got syntax highlighting to work.
> >> I don't know they've done that.
> > 
> > "syntax highlighting" ? :)
> > We have full IntelliSense support: syntax highlighting, code completion, macro
> > expansion (in tooltips).
> 
> I meant Intellisense, not just syntax highlighting. Though note that you 
> can't do syntax highlighting *perfectly* unless the IDE understands the 
> code.
> 
> > We simply use the Nemerle compiler to achive that.
> 
> Doesn't leave me any less impressed. <g>
December 18, 2010
Re: New syntax for string mixins
foobar wrote:
> Don Wrote:

>> Another one of the big differences is that D doesn't allow compile-time 
>> code to call external functions. Although it could certainly be done, it 
>> raises some big issues. Eg, we cannot assume that the target CPU is the 
>> same as the one we're running on. With a JIT compiler, you don't have 
>> that problem.
> 
> Don, can you please elaborate on this point? 
> 
> Here's my understanding:
> The D compiler is run once to both 'execute' compile time code which you refer above as constant folding AND to generate the binary to execute at run-time. 

> 
> Nemerle separates this into two distinct steps:
> 1. you compile regular code inside macro definitions into a compiler plugin. 
> 2. when compiling the intended run-time code you need to load the compiled macros from step 1 above on the command line of the compiler.
> 
> Since you're talking above about cross-compilation and let's say we run the compiler on X and compile for Y, I see no problem to load precompiled macros for X in order to compile the code for Y. The only limit as far as I can see is that it won't be possible to load macros compiled by the such a cross compiler on X unless as you say JIT is employed. Perhaps it makes sense to disable macro definition in a cross-compilation scenario and only allow usage.

Suppose the pre-compiled code, when run, asks what CPU it's on. What's 
the answer? Is it X or Y?
December 18, 2010
Re: New syntax for string mixins
Don Wrote:
> Suppose the pre-compiled code, when run, asks what CPU it's on. What's 
> the answer? Is it X or Y?

Current: X
Target: Y

Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic.
In any case macro is a meta-programm wich generate or/and transform code.
December 18, 2010
Re: New syntax for string mixins
VladD2 wrote:
> Don Wrote:
>> Suppose the pre-compiled code, when run, asks what CPU it's on. What's 
>> the answer? Is it X or Y?
> 
> Current: X
> Target: Y
> 
> Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic.
> In any case macro is a meta-programm wich generate or/and transform code. 

Yes. But in D there's no distinction between code which is destined for 
a macro, versus any other function. You can call a function once at 
compile time, and the same function at compile time. My understanding of 
Nemerle (which is quite likely to be wrong!) is that at least some 
functions are callable only at compile-time.

I'm also scared of the implications of allowing arbitrary code execution 
during compilation. Make a typo in your program, and then compilation 
may wipe files from your hard disk, or corrupt an external database, 
etc... On some platforms you may be able to sandbox it, but since it's 
running as part of the compilation process, rather than with the 
permissions it will eventually have, it just seems like a security 
nightmare.

All these problems would be reduced somewhat if it were only permissible 
to call pure functions at compile time. Although even that has issues, 
since not all compiler platforms may allow calling of shared libraries. 
If we build that into the language, we'd be cutting ourselves off from 
those platforms.

I'm not certain that there are any unsurmountable problems, but these 
issues make me really cautious.
December 18, 2010
Re: New syntax for string mixins
Don Wrote:

> VladD2 wrote:
> > Don Wrote:
> >> Suppose the pre-compiled code, when run, asks what CPU it's on. What's 
> >> the answer? Is it X or Y?
> > 
> > Current: X
> > Target: Y
> > 
> > Macro - a plugin to the compiler. It works on the same platform as the compiler, but generates code through the API which abstracts the macro from the target platform. If you need generate platform specific code, you should worry about it in macro logic.
> > In any case macro is a meta-programm wich generate or/and transform code. 
> 
> Yes. But in D there's no distinction between code which is destined for 
> a macro, versus any other function. You can call a function once at 
> compile time, and the same function at compile time. My understanding of 
> Nemerle (which is quite likely to be wrong!) is that at least some 
> functions are callable only at compile-time.
> 

I don't see how there needs to be different code to accomplish the above use case. You have a function that tests the hardware it's being run on. When this function is called in the compiler context it would return X, when it's called from the target executable it returns Y. 

> I'm also scared of the implications of allowing arbitrary code execution 
> during compilation. Make a typo in your program, and then compilation 
> may wipe files from your hard disk, or corrupt an external database, 
> etc... On some platforms you may be able to sandbox it, but since it's 
> running as part of the compilation process, rather than with the 
> permissions it will eventually have, it just seems like a security 
> nightmare.
> 

This is a void argument since templates are Turing complete.
It's *already* possible to do all of the above in D at compile time, it's just a matter of how much code is required to accomplish this. 

> All these problems would be reduced somewhat if it were only permissible 
> to call pure functions at compile time. Although even that has issues, 
> since not all compiler platforms may allow calling of shared libraries. 
> If we build that into the language, we'd be cutting ourselves off from 
> those platforms.
> 
> I'm not certain that there are any unsurmountable problems, but these 
> issues make me really cautious.
December 18, 2010
Re: New syntax for string mixins
On 2010-12-17 00:34, Graham St Jack wrote:
>
>> I've attached a part of how concurrency.d could look like translated
>> to my suggested syntax. It probably contains a lot of errors because
>> did a quick translation and I had some trouble understanding the mixins.
>>
> Yes, even I couldn't understand them even a week later. Maintenance is a
> real problem.
>
> My initial reaction is that the proposed syntax helps a bit, but isn't
> really a game-changer. I will let you know if I change my mind on closer
> inspection.

No it's basically just syntactic sugar.

-- 
/Jacob Carlborg
1 2 3 4 5 6
Top | Discussion index | About this forum | D home