On Monday, 15 July 2013 at 17:49:04 UTC, H. S. Teoh wrote:
> On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote:
> [...]
>> The use cases I am more interested in are not possible with make.
>> Having the ability to pass the arguments from within the language
>> itself allows you to define your use cases inline instead of having
>> to separately define the use case outside. Something where you would
>> have many different points in the code that needed different input
>> from what ever executable you were calling. With out ctec you would
>> have to maintain a list of every time you needed something generated
>> from the outside and add that to your make which is error prone. I
>> think it is in these cases where the ctec would be the most useful
>> as it would remove a lot of the burden of keeping track of all the
>> pregenerated code.
>
> You mean, you want to do something like this?
>
> 	static inputValues = [
> 		"data1", "data2", "data3"
> 	];
>
> 	static foreach (val; inputValues) {
> 		import exec("prog", val);
> 	}
>
> I can see why that could be useful.
>
>
> T
 Or something more complex where a template generates its code based on the output of an executable, so in that case the template args would be passed into the executable call.

On Mon, Jul 15, 2013 at 07:54:56PM +0200, Tofu Ninja wrote:
> On Monday, 15 July 2013 at 17:49:04 UTC, H. S. Teoh wrote:
> >On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote: [...]
> >>The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately define the use case outside. Something where you would have many different points in the code that needed different input from what ever executable you were calling. With out ctec you would have to maintain a list of every time you needed something generated from the outside and add that to your make which is error prone.  I think it is in these cases where the ctec would be the most useful as it would remove a lot of the burden of keeping track of all the pregenerated code.
> >
> >You mean, you want to do something like this?
> >
> >	static inputValues = [
> >		"data1", "data2", "data3"
> >	];
> >
> >	static foreach (val; inputValues) {
> >		import exec("prog", val);
> >	}
> >
> >I can see why that could be useful.
> >
> >
> >T
> Or something more complex where a template generates its code based on the output of an executable, so in that case the template args would be passed into the executable call.

Hmm. Let's see:

	module dmd;
	static grammar = import exec("wget", "http://dlang.org/spec");
	static c_code = import exec("yacc", grammar);
	void main() {
		import exec("d-ify", c_code);
	}

Hey look, we just solved the problem of rewriting DMD in D, *and* the problem of the spec being out-of-sync with the compiler! :-P

On a more serious note, though, if I ever needed a template of the sort you're describing, I'd prefer to use CTFE to generate it (and fix/improve CTFE if for whatever reason it can't yet do what I want it to do).

The problem with invoking an external program to produce template code is that you have to somehow serialize the template arguments so that you can pass them to the program, and the program has to somehow deserialize them and reconstruct what the compiler has already constructed. Whereas if you did everything in CTFE, you have access to whatever the compiler has internally built to describe the types and values.  Serializing / deserializing these structures generally are a big hiding place for bugs; I'd imagine that CTFE, being part of the compiler, is far more reliable at interpreting these structures correctly.

Besides, I'd hate for my D code to have opaque blocks of import exec() where I have to examine another program to figure out exactly what's being compiled, instead of just reading the source code.

So I'm still kinda on the fence about this whole thing (ignoring, of
course, the major security concerns involving having import exec() in
the first place).

What I really want to see, is for CTFE to improve to the point where you can essentially write an arbitrary program that runs in CTFE to produce whatever code you wish to produce for the executable, thus making import exec() redundant.


T

-- 
MASM = Mana Ada Sistem, Man!

On 7/15/13 6:26 AM, Don wrote:
> On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
>> So I had an idea recently, wouldn't it be cool to have the ability to
>> call an executable at compile time and capture its output. Something
>> like the string imports but instead of opening and reading a text
>> file, it run an executable, waits for it to finish, and grabs its output.
>>
>> It would get really cool if you could pass this executable some args
>> and then mix in its out put into your own code. It could be used
>> similarly to how CTFE are used but with out the overhead of trying to
>> compile that function and what not and with out the limitations on
>> what it can do.
>>
>> I could imagine all sorts of things that would be possible with this
>> that is currently not.
>>
>> Not sure if this is something that could be implemented easily, but
>> seems like something that could be done and something that would be
>> really cool.
>
> I personally think it's a *horrible* idea. It's one of those things
> which looks good in small cases but doesn't scale.

I think the right direction here is to factor compiler parts as a library. Then user code may insert external artifacts (notably REPL) on its own.

Andrei

On Friday, 12 July 2013 at 21:54:02 UTC, Walter Bright wrote:
> On 7/12/2013 1:42 PM, Tofu Ninja wrote:
>> So I had an idea recently, wouldn't it be cool to have the ability to call an
>> executable at compile time and capture its output. Something like the string
>> imports but instead of opening and reading a text file, it run an executable,
>> waits for it to finish, and grabs its output.
>>
>> It would get really cool if you could pass this executable some args and then
>> mix in its out put into your own code. It could be used similarly to how CTFE
>> are used but with out the overhead of trying to compile that function and what
>> not and with out the limitations on what it can do.
>>
>> I could imagine all sorts of things that would be possible with this that is
>> currently not.
>>
>> Not sure if this is something that could be implemented easily, but seems like
>> something that could be done and something that would be really cool.
>
> This is actually done in the makefile that builds DMD. A program (optabgen) is compiled and then run. Optabgen's output is several .c files which are then compiled into DMD's binary. The programs impcnvgen and idgen do similar things.
>
> It is a very powerful technique.

I did similar stuff for a mixed D/C++ program. The build contained some intermediary utilities used to spit out infos about the C++ code (typically sizeof and other characteristics like that) that could then be used to generate part of the binding in D.

On Tuesday, 16 July 2013 at 04:10:35 UTC, Andrei Alexandrescu wrote:
> I think the right direction here is to factor compiler parts as a library. Then user code may insert external artifacts (notably REPL) on its own.
>
> Andrei

SDC is already based on that principle, but still far from release state.

On Mon, Jul 15, 2013 at 9:10 PM, Andrei Alexandrescu < SeeWebsiteForEmail@erdani.org> wrote:

> On 7/15/13 6:26 AM, Don wrote:
>
>> On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
>>
>>> So I had an idea recently, wouldn't it be cool to have the ability to
>>> call an executable at compile time and capture its output. Something
>>> like the string imports but instead of opening and reading a text
>>> file, it run an executable, waits for it to finish, and grabs its output.
>>>
>>> It would get really cool if you could pass this executable some args and then mix in its out put into your own code. It could be used similarly to how CTFE are used but with out the overhead of trying to compile that function and what not and with out the limitations on what it can do.
>>>
>>> I could imagine all sorts of things that would be possible with this that is currently not.
>>>
>>> Not sure if this is something that could be implemented easily, but seems like something that could be done and something that would be really cool.
>>>
>>
>> I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale.
>>
>
> I think the right direction here is to factor compiler parts as a library. Then user code may insert external artifacts (notably REPL) on its own.
>
>
CTFE exec would still be needed (see yet another use case in EMAIL:"Proof of concept: automatically import C header files"), but having a compiler-as-a-library would make implementation easier and more customizable, eg by redirecting certain symbols  such as 'CTFE exec' to a given delegate.

myfile.d:
import std.process;
void main(){enum HOME=exec("echo $HOME", Redirect.all);}

driver.d:
import std.compiler;
void exec_dg(){...}
void main(){"myfile.d".readText.compile(&exec_dg);}

On 2013-07-16 06:10, Andrei Alexandrescu wrote:

> I think the right direction here is to factor compiler parts as a
> library.

Yes, I agree. I've been wanting this for a long time.

-- 
/Jacob Carlborg

On Wed, Jul 17, 2013 at 10:34:32AM +0200, Jacob Carlborg wrote:
> On 2013-07-16 06:10, Andrei Alexandrescu wrote:
> 
> >I think the right direction here is to factor compiler parts as a library.
> 
> Yes, I agree. I've been wanting this for a long time.
[...]

We've been talking about this for a while. Have we made any progress in this direction since?


T

-- 
What do you get if you drop a piano down a mineshaft? A flat minor.

On 2013-07-17 16:55, H. S. Teoh wrote:

> We've been talking about this for a while. Have we made any progress in
> this direction since?

There's been a bunch of pull requests merged related to adopting the DMD source code to make it work with the tool, I think it was, Daniel Murphy has created. This tool will be used to automatically port DMD to D. It's designed specifically for the DMD code base.

Look for pull request tagged with [DDMD] in the title.

Then I think the idea is refactor DMD to make it usable as a library.

-- 
/Jacob Carlborg

On Wed, Jul 17, 2013 at 05:40:03PM +0200, Jacob Carlborg wrote:
> On 2013-07-17 16:55, H. S. Teoh wrote:
> 
> >We've been talking about this for a while. Have we made any progress in this direction since?
> 
> There's been a bunch of pull requests merged related to adopting the DMD source code to make it work with the tool, I think it was, Daniel Murphy has created. This tool will be used to automatically port DMD to D. It's designed specifically for the DMD code base.
> 
> Look for pull request tagged with [DDMD] in the title.
> 
> Then I think the idea is refactor DMD to make it usable as a library.
[...]

Wow. That sounds like it's still a long ways off. But at least we're making progress.


T

-- 
The volume of a pizza of thickness a and radius z can be described by the following formula: pi zz a. -- Wouter Verhelst