In article <e1g3sr$212p$1@digitaldaemon.com>, Lionello Lunesu says...
>
>Just some idea:
>
>Wouldn't it be nice if there were an "extern (System)" calling convention that would start external programs?
>
>For example:
>
>#extern (System) int wget(...);
>#
>#int main() {
>#  return wget("http://whatever");
>#}
>
>This would boost productivity especially for D scripts. Only too often I need a small, temporary program to process some data and I wish I had the compilable source of wget/grep/utility-x at hand to incorporate their functionality.
>
>Seems easy enough to implement using spawning/waiting. The program can be declared with "..." or some defined arugments, but all arguments to the function would have to be cast to char[] for passing to the new process. Would of course be even cooler if the linker could pull in the external program's main(), but I suppose that's a bit tricky :)
>
>L.



As for accessing stdin/stdout, let them return a Stream type spliced in with some pipes. You'd have to ditch the wait though.

extern (System) Stream sed(...);

int main()
{
auto io sed("s/hello/word");
io.writeLine(“hello to all you out there”);
wrietf(io.readLine());
// prints “word to all you out there”
return 0;
}

I’m not sure how to get the return value.

In article <e1g3sr$212p$1@digitaldaemon.com>, Lionello Lunesu says...
>
>Just some idea:
>
>Wouldn't it be nice if there were an "extern (System)" calling convention that would start external programs?
>
>For example:
>
>#extern (System) int wget(...);
>#
>#int main() {
>#  return wget("http://whatever");
>#}
>
>This would boost productivity especially for D scripts. Only too often I need a small, temporary program to process some data and I wish I had the compilable source of wget/grep/utility-x at hand to incorporate their functionality.

Nice idea.  The (System) convention is really just a macro to wrap a call to
std.system() right?

If it helps, I recently came up with a solution to handle cross-platform scripting using D.  It handles all the differences between Linux and Windows shells, including handling forward and backward slash conventions.

'Script' Library: http://www.dsource.org/projects/ddl/browser/trunk/utils/Script.d

Example Script: http://www.dsource.org/projects/ddl/browser/trunk/buildutils.d

The symbols from Script are imported using a template so the script can be run with a very minimal command line (a techinique that sadly only works under Windows), provided the utils/ directory is in scope.  Otherwise, the developer has the option of compiling it into an .exe.


>Seems easy enough to implement using spawning/waiting. The program can be declared with "..." or some defined arugments, but all arguments to the function would have to be cast to char[] for passing to the new process. Would of course be even cooler if the linker could pull in the external program's main(), but I suppose that's a bit tricky :)

This is actually quite tricky to do - pull this off and you're half-way to writing your own shell.  Using System() is probably more efficent in terms of overhead and RAM/CPU usage.  Plus, you don't have to worry about trying to emulate shell features like the current $path and command aliases.

- EricAnderton at yahoo

BCS wrote:
> In article <e1g3sr$212p$1@digitaldaemon.com>, Lionello Lunesu says...
>> Just some idea:
>>
>> Wouldn't it be nice if there were an "extern (System)" calling convention that would start external programs?
>>
>> For example:
>>
>> #extern (System) int wget(...);
>> #
>> #int main() {
>> #  return wget("http://whatever");
>> #}
>>
>> This would boost productivity especially for D scripts. Only too often I need a small, temporary program to process some data and I wish I had the compilable source of wget/grep/utility-x at hand to incorporate their functionality.
>>
>> Seems easy enough to implement using spawning/waiting. The program can be declared with "..." or some defined arugments, but all arguments to the function would have to be cast to char[] for passing to the new process. Would of course be even cooler if the linker could pull in the external program's main(), but I suppose that's a bit tricky :)
>>
>> L.
> 
> 
> 
> As for accessing stdin/stdout, let them return a Stream type spliced in with
> some pipes. You'd have to ditch the wait though.


Wooh, that's a whole different ball game. The "extern (System)" seemed nice, since the declared function will behave much like any other imported function. You can replace the external "wget" (or whatever) with your implementation at any time, without having to change the rest of the program.

The "calling convention" that you suggested is pretty tricky. If you were to do that for a normal function, you'd have to put it in a separate thread and synchronize communications with that thread, much like the stream in your example. IO with another process should definitely be made easier, but I suppose a good wrapper class can take care of that.

L.

pragma wrote:
> In article <e1g3sr$212p$1@digitaldaemon.com>, Lionello Lunesu says...
>> Just some idea:
>>
>> Wouldn't it be nice if there were an "extern (System)" calling convention that would start external programs?
>>
>> For example:
>>
>> #extern (System) int wget(...);
>> #
>> #int main() {
>> #  return wget("http://whatever");
>> #}
>>
>> This would boost productivity especially for D scripts. Only too often I need a small, temporary program to process some data and I wish I had the compilable source of wget/grep/utility-x at hand to incorporate their functionality.
> 
> Nice idea.  The (System) convention is really just a macro to wrap a call to
> std.system() right?

It's in std.process, apparently, but that's the idea. It's just a little less typing then

#import std.process;
#int wget( char[] args )
#{
#  return system( "wget " ~ args );
#}

and with an added feature that it can check the argument types and convert them to (quoted) strings.

> If it helps, I recently came up with a solution to handle cross-platform
> scripting using D.  It handles all the differences between Linux and Windows
> shells, including handling forward and backward slash conventions.

Nice! Now imagine your code like this

#extern (System) {
#  void build( char[] options, char[] path );
#  void copy( char[] src, char[] dest );
#  void del( char[] );
#  void move( char[] src, char[] dest );
#}

and calling them would invoke the same system calls.

>> Seems easy enough to implement using spawning/waiting. The program can be declared with "..." or some defined arugments, but all arguments to the function would have to be cast to char[] for passing to the new process. Would of course be even cooler if the linker could pull in the external program's main(), but I suppose that's a bit tricky :)
> 
> This is actually quite tricky to do - pull this off and you're half-way to
> writing your own shell.  Using System() is probably more efficent in terms of
> overhead and RAM/CPU usage.  Plus, you don't have to worry about trying to
> emulate shell features like the current $path and command aliases.

Yeah, I thought about those problems too. Furthermore, might not even be allowed in the GPL to link to wget this way :)

L.

Lionello Lunesu wrote:
> pragma wrote:
>> In article <e1g3sr$212p$1@digitaldaemon.com>, Lionello Lunesu says...
>>> Just some idea:
>>>
>>> Wouldn't it be nice if there were an "extern (System)" calling convention that would start external programs?
>>>
>>> For example:
>>>
>>> #extern (System) int wget(...);
>>> #
>>> #int main() {
>>> #  return wget("http://whatever");
>>> #}
>>>
>>> This would boost productivity especially for D scripts. Only too often I need a small, temporary program to process some data and I wish I had the compilable source of wget/grep/utility-x at hand to incorporate their functionality.
>>
>> Nice idea.  The (System) convention is really just a macro to wrap a call to
>> std.system() right?
> 
> It's in std.process, apparently, but that's the idea. It's just a little less typing then
> 
> #import std.process;
> #int wget( char[] args )
> #{
> #  return system( "wget " ~ args );
> #}
> 
> and with an added feature that it can check the argument types and convert them to (quoted) strings.
> 
>> If it helps, I recently came up with a solution to handle cross-platform
>> scripting using D.  It handles all the differences between Linux and Windows
>> shells, including handling forward and backward slash conventions.
> 
> Nice! Now imagine your code like this
> 
> #extern (System) {
> #  void build( char[] options, char[] path );
> #  void copy( char[] src, char[] dest );
> #  void del( char[] );
> #  void move( char[] src, char[] dest );
> #}
> 
> and calling them would invoke the same system calls.
> 
>>> Seems easy enough to implement using spawning/waiting. The program can be declared with "..." or some defined arugments, but all arguments to the function would have to be cast to char[] for passing to the new process. Would of course be even cooler if the linker could pull in the external program's main(), but I suppose that's a bit tricky :)
>>
>> This is actually quite tricky to do - pull this off and you're half-way to
>> writing your own shell.  Using System() is probably more efficent in terms of
>> overhead and RAM/CPU usage.  Plus, you don't have to worry about trying to
>> emulate shell features like the current $path and command aliases.
> 
> Yeah, I thought about those problems too. Furthermore, might not even be allowed in the GPL to link to wget this way :)
> 
> L.

Its not linking, all it is is syntactic sugar.

Tydr Schnubbis wrote:
> Regan Heath wrote:
>> On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  <Stuart_member@pathlink.com> wrote:
>>> There's a bug in the makeBlock functions in process.d. The first  parameter to
>>> calloc should be 1 not 0. With that change (and the private readLine  one) it
>>> works fine for me.  Don't have an answer to the OP's DNS problem though.
>> 
>> You're dead right. With those changes it works for me too.
> 
> I've fixed the calloc calls too, but it doesn't help.  If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence".  Maybe my windows installation is screwed, it's getting really old...
> 
> Btw, I use dmd 0.148, haven't tried this with any other version yet.
Could it be that CreateProcessA is used wrong somehow?  I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network?  Where would I start looking if I want to fix this?  The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'.  But it seems to be the same function.

Here's what the call looks like:
CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)

On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake@address.dude> wrote:
> Tydr Schnubbis wrote:
>> Regan Heath wrote:
>>> On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney  <Stuart_member@pathlink.com> wrote:
>>>> There's a bug in the makeBlock functions in process.d. The first  parameter to
>>>> calloc should be 1 not 0. With that change (and the private readLine  one) it
>>>> works fine for me.  Don't have an answer to the OP's DNS problem though.
>>>  You're dead right. With those changes it works for me too.
>>  I've fixed the calloc calls too, but it doesn't help.  If I try to ping google's IP, which is 64.233.167.99 according to ping, I get "Pinging Error: 4invalid UTF-8 sequence".  Maybe my windows installation is screwed, it's getting really old...
>>  Btw, I use dmd 0.148, haven't tried this with any other version yet.
> Could it be that CreateProcessA is used wrong somehow?  I don't know the win32 api, but does anyone know if using it wrong (security settings or sth) could block new process from accessing the network?  Where would I start looking if I want to fix this?  The msdn docs didn't help much. Couldn't even find mention of this function, only CreateProcess, without the trailing 'A'.  But it seems to be the same function.
>
> Here's what the call looks like:
> CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)

CreateProcessA is the ascii version.
CreateProcessW is the unicode (UTF-16) version.

std.string.toStringz converts the UTF-8 char[] into ascii
std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if you want to call CreateProcessW instead.

The common cause of the "4invalid UTF-8 sequence" error is trying to output non-ascii characters to the windows console. Can you post your current code here.

Regan

Regan Heath wrote:
> On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake@address.dude>  wrote:
>> Tydr Schnubbis wrote:
>>> Regan Heath wrote:
>>>> On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   <Stuart_member@pathlink.com> wrote:
>>>>> There's a bug in the makeBlock functions in process.d. The first   parameter to
>>>>> calloc should be 1 not 0. With that change (and the private readLine   one) it
>>>>> works fine for me.  Don't have an answer to the OP's DNS problem  though.
>>>>  You're dead right. With those changes it works for me too.
>>>  I've fixed the calloc calls too, but it doesn't help.  If I try to  ping google's IP, which is 64.233.167.99 according to ping, I get  "Pinging Error: 4invalid UTF-8 sequence".  Maybe my windows  installation is screwed, it's getting really old...
>>>  Btw, I use dmd 0.148, haven't tried this with any other version yet.
>> Could it be that CreateProcessA is used wrong somehow?  I don't know the  win32 api, but does anyone know if using it wrong (security settings or  sth) could block new process from accessing the network?  Where would I  start looking if I want to fix this?  The msdn docs didn't help much.  Couldn't even find mention of this function, only CreateProcess, without  the trailing 'A'.  But it seems to be the same function.
>>
>> Here's what the call looks like:
>> CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
> 
> CreateProcessA is the ascii version.
> CreateProcessW is the unicode (UTF-16) version.
> 
> std.string.toStringz converts the UTF-8 char[] into ascii
> std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if  you want to call CreateProcessW instead.
> 
> The common cause of the "4invalid UTF-8 sequence" error is trying to  output non-ascii characters to the windows console. Can you post your  current code here.

Sure.  As you can see it's full of weird characters...

import lib.process;
import std.stdio;

void main()
{
	Process proc;
	// ping my router
	proc = new Process("ping 192.168.0.1");
	writefln(proc.readLine());
	writefln(proc.readLine());
	writefln(proc.readLine());
	writefln(proc.readLine());
}


I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files.

Compile with: dmd test.d lib/process.d lib/pipestream.d

This prints:
Pinging Error: 4invalid UTF-8 sequence


If I ping google.com instead, I get this:
Ping request could not find host google.com. Please check the name and try again
.

Error: ReadFile: The pipe has been ended.

---------

Does both of these work for you?  I have no idea what would cause any of these problems.  I have winxp SP2 US.  CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158).  dmd version 0.154, got the same results with 0.148.

:/

On Wed, 19 Apr 2006 16:53:18 +0200, Tydr Schnubbis <fake@address.dude> wrote:
> Regan Heath wrote:
>> On Wed, 19 Apr 2006 03:57:44 +0200, Tydr Schnubbis <fake@address.dude>  wrote:
>>> Tydr Schnubbis wrote:
>>>> Regan Heath wrote:
>>>>> On Thu, 6 Apr 2006 14:46:09 +0000 (UTC), Stuart Delaney   <Stuart_member@pathlink.com> wrote:
>>>>>> There's a bug in the makeBlock functions in process.d. The first   parameter to
>>>>>> calloc should be 1 not 0. With that change (and the private readLine   one) it
>>>>>> works fine for me.  Don't have an answer to the OP's DNS problem  though.
>>>>>  You're dead right. With those changes it works for me too.
>>>>  I've fixed the calloc calls too, but it doesn't help.  If I try to  ping google's IP, which is 64.233.167.99 according to ping, I get  "Pinging Error: 4invalid UTF-8 sequence".  Maybe my windows  installation is screwed, it's getting really old...
>>>>  Btw, I use dmd 0.148, haven't tried this with any other version yet.
>>> Could it be that CreateProcessA is used wrong somehow?  I don't know the  win32 api, but does anyone know if using it wrong (security settings or  sth) could block new process from accessing the network?  Where would I  start looking if I want to fix this?  The msdn docs didn't help much.  Couldn't even find mention of this function, only CreateProcess, without  the trailing 'A'.  But it seems to be the same function.
>>>
>>> Here's what the call looks like:
>>> CreateProcessA(null,std.string.toStringz(command),null,null,true,DETACHED_PROCESS,env,null,&startup,info)
>>  CreateProcessA is the ascii version.
>> CreateProcessW is the unicode (UTF-16) version.
>>  std.string.toStringz converts the UTF-8 char[] into ascii
>> std.string.toUTF16 can be used to convert the UTF-8 char[] into UTF-16 if  you want to call CreateProcessW instead.
>>  The common cause of the "4invalid UTF-8 sequence" error is trying to  output non-ascii characters to the windows console. Can you post your  current code here.
>
> Sure.  As you can see it's full of weird characters...
>
> import lib.process;
> import std.stdio;
>
> void main()
> {
> 	Process proc;
> 	// ping my router
> 	proc = new Process("ping 192.168.0.1");
> 	writefln(proc.readLine());
> 	writefln(proc.readLine());
> 	writefln(proc.readLine());
> 	writefln(proc.readLine());
> }
>
>
> I have only made readLine public, and fixed the four calloc calls, no other changes have been made to your files.
>
> Compile with: dmd test.d lib/process.d lib/pipestream.d
>
> This prints:
> Pinging Error: 4invalid UTF-8 sequence
>
>
> If I ping google.com instead, I get this:
> Ping request could not find host google.com. Please check the name and try again
> .
>
> Error: ReadFile: The pipe has been ended.
>
> ---------
>
> Does both of these work for you?  I have no idea what would cause any of these problems.  I have winxp SP2 US.  CreateProcessA is in kernel32.dll, of which I have version 5.1.2600.2180 (xpsp_sp2_rtm.040803-2158).  dmd version 0.154, got the same results with 0.148.

It's quite odd, try this:

import lib.process;
import std.stdio;
import std.string;
import std.c.string;

extern(C) extern char **_environ;

void main()
{
	Process proc;
	proc = new Process();
	for(int i = 0; _environ[i]; i++) {
		proc.addEnv(toString(_environ[i]).dup);
	}
	//proc.execute("ping www.google.com");
	proc.execute("ping 192.168.0.1");
	while(true) writefln("%s",proc.readLine());
}

without the addEnv calls above I get the behaviour you're describing. With them it works.

Without them, and using printf I can see that ping responds with:

"Pinging °ÿ with 32 bytes of data:"

note the weird characters there. At first I thought maybe the command line I was passing to CreateProcessA was temporary and being collected by the GC, so I changed process.d to use:

cmd = strdup(std.string.toStringz(command));

where cmd is a member of Process - so will persist as long as it does. That made no difference. I have no idea why it's doing that, perhaps it reads it's args in a strange way?? I might write a debug program and run that passing different args etc to see if I can replicate the odd behaviour and figure out where it comes from.

Regan

Regan Heath wrote:
>>>  The common cause of the "4invalid UTF-8 sequence" error is trying to   output non-ascii characters to the windows console. Can you post your   current code here.
>>
>> Sure.  As you can see it's full of weird characters...
>>
>> import lib.process;
>> import std.stdio;
>>
>> void main()
>> {
>> 	Process proc;
>> 	// ping my router
>> 	proc = new Process("ping 192.168.0.1");
>> 	writefln(proc.readLine());
>> 	writefln(proc.readLine());
>> 	writefln(proc.readLine());
>> 	writefln(proc.readLine());
>> }
>>
>>
>> I have only made readLine public, and fixed the four calloc calls, no  other changes have been made to your files.
>>
>> Compile with: dmd test.d lib/process.d lib/pipestream.d
>>
>> This prints:
>> Pinging Error: 4invalid UTF-8 sequence
>>
>>
>> If I ping google.com instead, I get this:
>> Ping request could not find host google.com. Please check the name and  try again
>> .
>>
>> Error: ReadFile: The pipe has been ended.
>>
>> ---------
>>
>> Does both of these work for you?  I have no idea what would cause any of  these problems.  I have winxp SP2 US.  CreateProcessA is in  kernel32.dll, of which I have version 5.1.2600.2180  (xpsp_sp2_rtm.040803-2158).  dmd version 0.154, got the same results  with 0.148.
> 
> It's quite odd, try this:
> 
> import lib.process;
> import std.stdio;
> import std.string;
> import std.c.string;
> 
> extern(C) extern char **_environ;
> 
> void main()
> {
> 	Process proc;
> 	proc = new Process();
> 	for(int i = 0; _environ[i]; i++) {
> 		proc.addEnv(toString(_environ[i]).dup);
> 	}
> 	//proc.execute("ping www.google.com");
> 	proc.execute("ping 192.168.0.1");
> 	while(true) writefln("%s",proc.readLine());
> }
> 
> without the addEnv calls above I get the behaviour you're describing. With  them it works.
> 
Works for me too, thanks!


> Without them, and using printf I can see that ping responds with:
> 
> "Pinging °ÿ with 32 bytes of data:"
> 
> note the weird characters there. At first I thought maybe the command line  I was passing to CreateProcessA was temporary and being collected by the  GC, so I changed process.d to use:
> 
> cmd = strdup(std.string.toStringz(command));
> 
> where cmd is a member of Process - so will persist as long as it does.  That made no difference. I have no idea why it's doing that, perhaps it  reads it's args in a strange way?? I might write a debug program and run  that passing different args etc to see if I can replicate the odd  behaviour and figure out where it comes from.

Not sure if this helps:
http://www.digitalmars.com/techtips/windows_utf.html