On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
> So I had an idea recently, wouldn't it be cool to have the ability to call an executable at compile time and capture its output. Something like the string imports but instead of opening and reading a text file, it run an executable, waits for it to finish, and grabs its output.
>
> It would get really cool if you could pass this executable some args and then mix in its out put into your own code. It could be used similarly to how CTFE are used but with out the overhead of trying to compile that function and what not and with out the limitations on what it can do.
>
> I could imagine all sorts of things that would be possible with this that is currently not.
>
> Not sure if this is something that could be implemented easily, but seems like something that could be done and something that would be really cool.

I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale.

Don't underestimate the fact that the compiler is a deterministic program at present. When compiled with the same flags, on the same set of source files, the results should always be the same. It's a classic filter program.

input -> compiler -> output

Losing that purity is a HUGE thing to give up.

On Monday, 15 July 2013 at 13:26:13 UTC, Don wrote:
> On Friday, 12 July 2013 at 20:42:50 UTC, Tofu Ninja wrote:
>> So I had an idea recently, wouldn't it be cool to have the ability to call an executable at compile time and capture its output. Something like the string imports but instead of opening and reading a text file, it run an executable, waits for it to finish, and grabs its output.
>>
>> It would get really cool if you could pass this executable some args and then mix in its out put into your own code. It could be used similarly to how CTFE are used but with out the overhead of trying to compile that function and what not and with out the limitations on what it can do.
>>
>> I could imagine all sorts of things that would be possible with this that is currently not.
>>
>> Not sure if this is something that could be implemented easily, but seems like something that could be done and something that would be really cool.
>
> I personally think it's a *horrible* idea. It's one of those things which looks good in small cases but doesn't scale.
>
> Don't underestimate the fact that the compiler is a deterministic program at present. When compiled with the same flags, on the same set of source files, the results should always be the same. It's a classic filter program.
>
> input -> compiler -> output
>
> Losing that purity is a HUGE thing to give up.

Ehh... to say its deterministic is kinda a stretch, for one, compiling on different systems could result in different output with the ability to version different sections of code. Also as has been mentioned before, string imports already make the same kind of break in purity.

If the programmer wants to use it then they also should be well aware of the consequences of such a decision, which can be considered on a case by case basis.

We should be as flexible as possible and leave the decision up to the programmer. We shouldn't restrict a feature based on a fictional idea of purity or vague worry of feature bloat(as others have mentioned) especially in a case like this where the implementation is fairly simple and the use cases are immense.

On Monday, 15 July 2013 at 15:05:39 UTC, Tofu Ninja wrote:
> On Monday, 15 July 2013 at 13:26:13 UTC, Don wrote:
>> Don't underestimate the fact that the compiler is a deterministic program at present. When compiled with the same flags, on the same set of source files, the results should always be the same. It's a classic filter program.
>>
>> input -> compiler -> output
>>
>> Losing that purity is a HUGE thing to give up.
>
> Ehh... to say its deterministic is kinda a stretch, for one, compiling on different systems could result in different output with the ability to version different sections of code. Also as has been mentioned before, string imports already make the same kind of break in purity.

If you want to be technical then yes, the purity takes the operating system as an input.

String imports don't break purity, they just add an input.

By allowing arbitrary code execution, you essentially require the whole state of the host machine as an input.

> If the programmer wants to use it then they also should be well aware of the consequences of such a decision, which can be considered on a case by case basis.
>
> We should be as flexible as possible and leave the decision up to the programmer. We shouldn't restrict a feature based on a fictional idea of purity or vague worry of feature bloat(as others have mentioned) especially in a case like this where the implementation is fairly simple and the use cases are immense.

No, this is a bad attitude to have. If you add a feature that is easy to misuse then it WILL be misused. Just looks at macros in C.

Any program that uses compile time executable calling will almost certainly become non-portable. This is not a good thing.

On 07/13/2013 09:05 AM, Ary Borenszweig wrote:

> What would actually be cool is to run any kind of code at compile time
> (within a sandbox, if you want). For example reading a mysql database
> and generating classes for the tables at compile time. No need to run a
> separate executable and remember to run it before compiling your program.

Humans make mistakes all the time, especially with repetitive tasks. That's why such tasks are handled by tools like 'make'.

Ali

On 7/15/13 2:03 PM, Ali Çehreli wrote:
> On 07/13/2013 09:05 AM, Ary Borenszweig wrote:
>
>  > What would actually be cool is to run any kind of code at compile time
>  > (within a sandbox, if you want). For example reading a mysql database
>  > and generating classes for the tables at compile time. No need to run a
>  > separate executable and remember to run it before compiling your
> program.
>
> Humans make mistakes all the time, especially with repetitive tasks.
> That's why such tasks are handled by tools like 'make'.
>
> Ali

Why have CTFE at all if you can produce the desired code with an external program, output it to some file and then use it with import("file")? Just use 'make'.

I believe a programming language should be as self-contained as possible for building an executable out of it. The more you can do within the language itself, minimizing external dependencies, the better. Because users need only to understand the language. They don't need to understand make, build systems and whatnot.

But that's just my opinion...

On 07/15/2013 08:05 AM, Tofu Ninja wrote:

> We should be as flexible as possible and leave the decision up to the
> programmer. We shouldn't restrict a feature based on a fictional idea of
> purity or vague worry of feature bloat(as others have mentioned)

My worry is not vague at all. What stops one from adding emailing ability to the compiler then? That would be immensely useful too becaues the compiler could email me. (!) Where is the line?

A compiler is a tool that compiles source code in a preset environment. Simple is better.

> especially in a case like this where the implementation is fairly simple
> and the use cases are immense.

The use cases are not immense compared to the current situation. There are lots of free tools that can set the environment for compilation and the compiler compiles.

Ali

On Monday, 15 July 2013 at 17:08:48 UTC, Ali Çehreli wrote:
> On 07/15/2013 08:05 AM, Tofu Ninja wrote:
>
> > We should be as flexible as possible and leave the decision
> up to the
> > programmer. We shouldn't restrict a feature based on a
> fictional idea of
> > purity or vague worry of feature bloat(as others have
> mentioned)
>
> My worry is not vague at all. What stops one from adding emailing ability to the compiler then? That would be immensely useful too becaues the compiler could email me. (!) Where is the line?
>
> A compiler is a tool that compiles source code in a preset environment. Simple is better.
>
> > especially in a case like this where the implementation is
> fairly simple
> > and the use cases are immense.
>
> The use cases are not immense compared to the current situation. There are lots of free tools that can set the environment for compilation and the compiler compiles.
>
> Ali

The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately define the use case outside. Something where you would have many different points in the code that needed different input from what ever executable you were calling. With out ctec you would have to maintain a list of every time you needed something generated from the outside and add that to your make which is error prone. I think it is in these cases where the ctec would be the most useful as it would remove a lot of the burden of keeping track of all the pregenerated code.

On Mon, Jul 15, 2013 at 02:06:47PM -0300, Ary Borenszweig wrote:
> On 7/15/13 2:03 PM, Ali Çehreli wrote:
> >On 07/13/2013 09:05 AM, Ary Borenszweig wrote:
> >
> > > What would actually be cool is to run any kind of code at compile time (within a sandbox, if you want). For example reading a mysql database and generating classes for the tables at compile time. No need to run a separate executable and remember to run it before compiling your program.
> >
> >Humans make mistakes all the time, especially with repetitive tasks. That's why such tasks are handled by tools like 'make'.
> >
> >Ali
> 
> Why have CTFE at all if you can produce the desired code with an external program, output it to some file and then use it with import("file")? Just use 'make'.

That's an invalid argument. CTFE allows compile-time introspection of types and declarations within the language. To move this outside the compiler would require essentially re-implementing the compiler so that you can parse your code to extract types, declarations, etc., in order to perform CTFE. This means it's better for it to be in the compiler.

OTOH, executing arbitrary programs has nothing to do with the compiling of D source code. That belongs to the OS level of operations, not inside the compiler.


> I believe a programming language should be as self-contained as possible for building an executable out of it. The more you can do within the language itself, minimizing external dependencies, the better. Because users need only to understand the language. They don't need to understand make, build systems and whatnot.
[...]

I don't understand this reluctance to learn how to use build systems. Why insist on building a house with only a hammer, and refuse to learn how to use a screwdriver and wrench? Sure, you *can* build a house with only a hammer if you try hard enough, but why not use tools that already exist?

(Of course, the problem is exacerbated by make being a really poor example of a build system, but also the one most people know about. Perhaps that explains part of the bad rep of build systems.)


T

-- 
What are you when you run out of Monet? Baroque.

On Mon, Jul 15, 2013 at 07:35:44PM +0200, Tofu Ninja wrote: [...]
> The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately define the use case outside. Something where you would have many different points in the code that needed different input from what ever executable you were calling. With out ctec you would have to maintain a list of every time you needed something generated from the outside and add that to your make which is error prone. I think it is in these cases where the ctec would be the most useful as it would remove a lot of the burden of keeping track of all the pregenerated code.

You mean, you want to do something like this?

	static inputValues = [
		"data1", "data2", "data3"
	];

	static foreach (val; inputValues) {
		import exec("prog", val);
	}

I can see why that could be useful.


T

-- 
By understanding a machine-oriented language, the programmer will tend to use a much more efficient method; it is much closer to reality. -- D. Knuth

On Mon, Jul 15, 2013 at 10:35 AM, Tofu Ninja <emmons0@purdue.edu> wrote:

> On Monday, 15 July 2013 at 17:08:48 UTC, Ali Çehreli wrote:
>
>> On 07/15/2013 08:05 AM, Tofu Ninja wrote:
>>
>> > We should be as flexible as possible and leave the decision
>> up to the
>> > programmer. We shouldn't restrict a feature based on a
>> fictional idea of
>> > purity or vague worry of feature bloat(as others have
>> mentioned)
>>
>> My worry is not vague at all. What stops one from adding emailing ability to the compiler then? That would be immensely useful too becaues the compiler could email me. (!) Where is the line?
>>
>> A compiler is a tool that compiles source code in a preset environment. Simple is better.
>>
>> > especially in a case like this where the implementation is
>> fairly simple
>> > and the use cases are immense.
>>
>> The use cases are not immense compared to the current situation. There
>> are lots of free tools that can set the environment for compilation and the
>> compiler compiles.
>>
>> Ali
>>
>
> The use cases I am more interested in are not possible with make. Having the ability to pass the arguments from within the language itself allows you to define your use cases inline instead of having to separately define the use case outside. Something where you would have many different points in the code that needed different input from what ever executable you were calling. With out ctec you would have to maintain a list of every time you needed something generated from the outside and add that to your make which is error prone. I think it is in these cases where the ctec would be the most useful as it would remove a lot of the burden of keeping track of all the pregenerated code.
>
>
Instead of debating forever we should implement it and make a pull request for it. People can test it individually and see whether it creates havoc or enables interesting scenarios. Then we can decide after some time.