Here's a suggestion that I actually sent a couple of of years ago that actually make sense now with the new mixin stuff.  What if you could plugin dlls into DMD that would be run at compile time.  The requirement of use would be that the input parameters would be constant.


ie

importplugin("perl");

mixin(Perl("perl code"));

//In the Perl dll
char[] Perl(char[] source)
{
 ...
}

Of course the other option would be to compile the Perl code on the fly.    Parhaps you could have "char[] str = static func()" or something (ie prefix calls to these functions with static).

of course templates could be used to represent almost anything but I find them hard to read when they start to try to do something weird.

The only downside I see is that people could write really slow dlls which would slow down compile time.  It probably would be a good idea for DMD to actually printout how much time is spent in each on of these processes (ie point its finger).

If this technique where to be enabled, we would be able to take all the string handling routines in the standard lib and use them now.

-Joel

janderson wrote:
> Here's a suggestion that I actually sent a couple of of years ago that actually make sense now with the new mixin stuff.  What if you could plugin dlls into DMD that would be run at compile time.  The requirement of use would be that the input parameters would be constant.

The main problem with this is the malware author who puts some source code up, and when you try to compile it, it would install a rootkit.

Hi, why not use D instead of perl ?
Since "sh style script syntax" is supported, I would like the following to be possible:

import std.stdio, std.string, std.metastrings;

void foo(A...)(A a)
{
    writefln(format(typeid(typeof(a[1]))) ~ " " ~ a[0] ~ " = \"" ~ a[1] ~ "\"");
}

void main (char[][] args)
{
    debug(templates)
        foo("var", "whatever");

    foo!("var", "whatever");
    /* This will call dmd -run on this file. But instead of starting at main(),
     * it starts by invoking foo() with the tuple ("var", "whatever").
     * Mixed into the context will be the console output of the
     * dmd -run call, in this case:
     * char[8] var = "whatever";
    */

    writefln(var);
}

Wouldn't this be very neat and powerful ?

Walter, about your concerns on malware autors:
Of course script interpretation within the compile-process could be used for malware. But also the malware could be put into the compiled code itself and get's started if the app is run. So I think it doesn't make any difference.

Best regards,
Daniel

Daniel919 wrote:
> Hi, why not use D instead of perl ?

I agree. I don't see any reason to use perl, unless you're using a pre-existing perl script.


> Walter, about your concerns on malware autors:
> Of course script interpretation within the compile-process could be used for malware.
> But also the malware could be put into the compiled code itself and get's started if
> the app is run. So I think it doesn't make any difference.

At a technical level, you're right. But people don't expect this to be possible when just the compiler is run, and won't be looking out for it. I also don't want D compilers to be maligned as vectors for malware, even if such maligning is undeserved. I don't want D compilers to be blocked by security conscious administrators.

Daniel919 wrote:
> Hi, why not use D instead of perl ?
> Since "sh style script syntax" is supported, I would like the following to be possible:
> 
> import std.stdio, std.string, std.metastrings;
> 
> void foo(A...)(A a)
> {
>     writefln(format(typeid(typeof(a[1]))) ~ " " ~ a[0] ~ " = \"" ~ a[1] ~ "\"");
> }
> 
> void main (char[][] args)
> {
>     debug(templates)
>         foo("var", "whatever");
> 
>     foo!("var", "whatever");
>     /* This will call dmd -run on this file. But instead of starting at main(),
>      * it starts by invoking foo() with the tuple ("var", "whatever").
>      * Mixed into the context will be the console output of the
>      * dmd -run call, in this case:
>      * char[8] var = "whatever";
>     */
> 
>     writefln(var);
> }
> 
> Wouldn't this be very neat and powerful ?
> 
> Walter, about your concerns on malware autors:
> Of course script interpretation within the compile-process could be used for malware. But also the malware could be put into the compiled code itself and get's started if the app is run. So I think it doesn't make any difference.
> 
> Best regards,
> Daniel

Also, people download and run makefiles to build their software all the time. Who knows what that makefile is doing.  There could be an rm -rf / buried in there somewhere in all that tab-sensitive gobbledy-gook.

--bb

On Wed, 07 Feb 2007 05:13:06 +0900, Bill Baxter wrote:

> Daniel919 wrote:
>> Hi, why not use D instead of perl ?
>> Since "sh style script syntax" is supported, I would like the following to be possible:
>> 
>> import std.stdio, std.string, std.metastrings;
>> 
>> void foo(A...)(A a)
>> {
>>     writefln(format(typeid(typeof(a[1]))) ~ " " ~ a[0] ~ " = \"" ~ a[1] ~ "\"");
>> }
>> 
>> void main (char[][] args)
>> {
>>     debug(templates)
>>         foo("var", "whatever");
>> 
>>     foo!("var", "whatever");
>>     /* This will call dmd -run on this file. But instead of starting at main(),
>>      * it starts by invoking foo() with the tuple ("var", "whatever").
>>      * Mixed into the context will be the console output of the
>>      * dmd -run call, in this case:
>>      * char[8] var = "whatever";
>>     */
>> 
>>     writefln(var);
>> }
>> 
>> Wouldn't this be very neat and powerful ?
>> 
>> Walter, about your concerns on malware autors:
>> Of course script interpretation within the compile-process could be used for malware. But also the malware could be put into the compiled code itself and get's started if the app is run. So I think it doesn't make any difference.
>> 
>> Best regards,
>> Daniel
> 
> Also, people download and run makefiles to build their software all the time. Who knows what that makefile is doing.  There could be an rm -rf / buried in there somewhere in all that tab-sensitive gobbledy-gook.
> 
> --bb


Very good point.

Any auxillary tool in the compilation process could could do the same.  I have difficult recognizing the validity of Walter's concern in this context.

-JJR

Walter Bright wrote:
> janderson wrote:
>> Here's a suggestion that I actually sent a couple of of years ago that actually make sense now with the new mixin stuff.  What if you could plugin dlls into DMD that would be run at compile time.  The requirement of use would be that the input parameters would be constant.
> 
> The main problem with this is the malware author who puts some source code up, and when you try to compile it, it would install a rootkit.

Good point.  What about the static version.  ie

//This would be compiled on the fly. It could be put into its own object file so it doesn't need to be re-built everytime.
mixin char[] stateMachine(char[] input)
{
 ...
}

void main()
{
    stateMachine("ect...");
}

or

plugin char[] stateMachine(char[] input)
{
 ...
}

void main()
{
     mixin(stateMachine("ect..."));
}

or

char[] stateMachine(char[] input)
{
 ...
}

void main()
{
     mixin(plugin stateMachine("ect..."));
}


Something like that.


I guess malware may still be possible, but you can do that with unit tests now.  Essentially these are not much more then unit tests.

-Joel

janderson wrote:
> Walter Bright wrote:
>> janderson wrote:
>>> Here's a suggestion that I actually sent a couple of of years ago that actually make sense now with the new mixin stuff.  What if you could plugin dlls into DMD that would be run at compile time.  The requirement of use would be that the input parameters would be constant.
>>
>> The main problem with this is the malware author who puts some source code up, and when you try to compile it, it would install a rootkit.
> 
> Good point.  What about the static version.  ie
> 
> //This would be compiled on the fly. It could be put into its own object file so it doesn't need to be re-built everytime.
> mixin char[] stateMachine(char[] input)
> {
>  ...
> }
> 
> void main()
> {
>     stateMachine("ect...");
> }
> 
> or
> 
> plugin char[] stateMachine(char[] input)
> {
>  ...
> }
> 
> void main()
> {
>      mixin(stateMachine("ect..."));
> }
> 
> or
> 
> char[] stateMachine(char[] input)
> {
>  ...
> }
> 
> void main()
> {
>      mixin(plugin stateMachine("ect..."));
> }
> 
> 
> Something like that.
> 
> 
> I guess malware may still be possible, but you can do that with unit tests now.  Essentially these are not much more then unit tests.
> 
> -Joel

Actually you could go one step better with this.  No calls to functions outside of D.  That way you'd have access to all of D's string/int/float/array operations but you wouldn't be able to make systems calls at all.   You could also disabled asm calls if you wanted to get really restrictive although I don't see that as necessary.

The only way you'd be able to hack the system is to do some sort of funky memory accessing.

-JOel

John Reimer wrote:
> On Wed, 07 Feb 2007 05:13:06 +0900, Bill Baxter wrote:
> 
>> Daniel919 wrote:
>>> Hi, why not use D instead of perl ?
>>> Since "sh style script syntax" is supported, I would like the following to be possible:
>>>
>>> import std.stdio, std.string, std.metastrings;
>>>
>>> void foo(A...)(A a)
>>> {
>>>     writefln(format(typeid(typeof(a[1]))) ~ " " ~ a[0] ~ " = \"" ~ a[1] ~ "\"");
>>> }
>>>
>>> void main (char[][] args)
>>> {
>>>     debug(templates)
>>>         foo("var", "whatever");
>>>
>>>     foo!("var", "whatever");
>>>     /* This will call dmd -run on this file. But instead of starting at main(),
>>>      * it starts by invoking foo() with the tuple ("var", "whatever").
>>>      * Mixed into the context will be the console output of the
>>>      * dmd -run call, in this case:
>>>      * char[8] var = "whatever";
>>>     */
>>>
>>>     writefln(var);
>>> }
>>>
>>> Wouldn't this be very neat and powerful ?
>>>
>>> Walter, about your concerns on malware autors:
>>> Of course script interpretation within the compile-process could be used for malware. But also the malware could be put into the compiled code itself and get's started if the app is run. So I think it doesn't make any difference.
>>>
>>> Best regards,
>>> Daniel
>> Also, people download and run makefiles to build their software all the time. Who knows what that makefile is doing.  There could be an rm -rf / buried in there somewhere in all that tab-sensitive gobbledy-gook.
>>
>> --bb
> 
> 
> Very good point.
> 
> Any auxillary tool in the compilation process could could do the same.  I
> have difficult recognizing the validity of Walter's concern in this context.
> 
> -JJR

I think the concern is that an awful lot of people who notice this capability will also /not/ notice the context|precedant.  And perhaps also a bit of good conscience... but that's no real obstacle in and of itself.

-- Chris Nicholson-Sauls

On Wed, 07 Feb 2007 02:06:44 +0200, John Reimer <terminal.node@gmail.com> wrote:

> On Wed, 07 Feb 2007 05:13:06 +0900, Bill Baxter wrote:
>
>> Also, people download and run makefiles to build their software all the time. Who knows what that makefile is doing.  There could be an rm -rf / buried in there somewhere in all that tab-sensitive gobbledy-gook.
>
> Very good point.
>
> Any auxillary tool in the compilation process could could do the same.  I have difficult recognizing the validity of Walter's concern in this context.

I believe there must be a strict distinction between tools which just work with data or code, and tools which run code. The sole definition of a compiler is transform human-readable code to machine code - not run any of those outside the purpose of generating other code or data.

Unit tests are a separate issue, because to run them the user must specify a switch on the compiler's command-line - in which case he is well-aware that code will be run. Thus, if any such feature does get implemented, the compiler should not execute any code and give off an error instead, and force the user to specify a switch which allows this, to prevent any naive mistakes and annoyances.

That, or write in big letters on the DMD download page... "This compiler may run some of the compiled code" in big red letters. But that'll just scare more people off...

-- 
Best regards,
  Vladimir                          mailto:thecybershadow@gmail.com