On Fri, 12 Apr 2013 11:24:37 -0400, Andrej Mitrovic <andrej.mitrovich@gmail.com> wrote:

> On 4/12/13, Manu <turkeyman@gmail.com> wrote:
>> string[string] is used in the main API to receive environment variables;
>> perhaps kinda convenient, but it's impossible to supply environment
>> variables with loads of allocations.
>
> I think the functions which take the environment as an associative
> array can be changed without breaking any user code. E.g.:
>
> Pid spawnProcess(Env)(in char[][] args,
>                  File stdin = std.stdio.stdin,
>                  File stdout = std.stdio.stdout,
>                  File stderr = std.stdio.stderr,
>
>                  /* was const string[string] env = null */
>                  const Env env = Env.init,
>
>                  Config config = Config.none)
>     if (isSomeAssociativeArray!Env)  // introduce constraint
> {
> }
>
> Then the user can provide their custom stack-based type which implements opIn_r.

It's a good idea.  I believe it can be done later without breaking any code.

-Steve

yes

On Fri, 12 Apr 2013 04:14:15 -0400, Manu <turkeyman@gmail.com> wrote:

> On 12 April 2013 17:35, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
> What would be your suggestion?  string[string] is the built-in map type.
>>  How do you pass an environment map without having some allocations?
>
>
> I'd use string[].

You mean with format "a=b"?  I suppose that's possible, though horrible to work with before passing in.  Plus what happens if you have ["a=b", "a=c"] ?  AA's prevent these kinds of mistakes/contradictions.

I think someone suggested using a templated map type, that is probably a good idea.  Then you can provide whatever type you wish, with opIndex capability.

>>> toStringz is used liberally; alternatively, alloca() could allocate the
>>>  c-string's on the stack and zero terminate them there, passing a pointer to
>>>  the stack string to the OS functions.
>
>>  This would be a lot of effort for pretty much zero gain for the majority of cases.
>
> A trivial ... mixin template (?) could wrap it up, I don't see it particularly less convenient than calling toStringz().
> Perhaps there are other tools missing from phobos if this is somehow hard...

alloca is hard to use, specifically because it can't be wrapped into a function.  I don't know how easy it would be to do with a mixin, I haven't tried it.

Note that toStringz likely is going to avoid reallocation or copying if possible -- it's very likely that a string already has a 0 after it, and also very likely that if it doesn't, it has room to append a 0.

I still feel that if it can't be done easily, it over-complicates code for very little gain.

As others have said too, this is an implementation detail.  It can be done as a later pull request if you feel up to making it work!

-Steve

On Fri, 12 Apr 2013 06:14:35 -0400, Regan Heath <regan@netmail.co.nz> wrote:

> On Fri, 12 Apr 2013 10:17:54 +0100, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

>> Optimizing code often implies making it more complicated.
>>
>> Straight-forward code is self-documenting.
>>
>> More complicated code is harder to read and review.
>>
>> More complicated code more easily hides bugs - including security bugs, such as buffer overflows.
>>
>> Maintaining optimized code requires understanding not only the high-level logic, but also the low-level optimization details.
>>
>> The benefits of optimizing std.process are likely to be so small, as to be difficult to measure.
>
> All true.  However, complexity can and should be packaged in such a way as to localise, and this localised complex code should be tested to death and maintained by someone who understands it.  It should be bracketed by sufficient comments and warnings about how/why it does what it does.  The resulting packaged complexity, with it's associated cost can be re-used many times over for all the benefit it gives.

I would say I agree with both of you.  In this particular instance, maintainability trumps performance, since creating processes is fundamentally not cheap.  But if someone is willing to step up an give us a good alternative, there is no reason not to use it.  It should be readable and well documented.

I think in order to achieve good performance, we should be focused on custom allocators.  That really is the roadblock to all of this.

-Steve

On Fri, 12 Apr 2013 13:00:29 +0200
"Vladimir Panteleev" <vladimir@thecybershadow.net> wrote:
> 
> There's also the question of priorities. Would you rather than effort is spent on optimizing std.process (and dealing with all the fallout from any such optimizations), or working on something that is acutely missing and hurting D?
> 

While I'm not necessarily disagreeing with the other points you raise, Phobos's excess allocations *are* verifiability hurting D. A couple of the biggest potential user bases for D are videogames and embedded. I feel confident in saying that no other language has as much potential in these areas as D has. But these groups have *already*, vocally, been facing the problem of avoiding/rewriting potentially-large parts of Phobos, or sticking with C/C++.

Now, you could argue that many of these people are simply being overly fearful of a merely imagined problem, but even if that's true, the problem is still real in it's effects: Hindering D adoption and causing people to (perhaps needlessly) avoid/rewrite parts of Phobos.

Now, I'm not suggesting that we do *or* don't start a big effort to minimize allocations throughout Phobos, I'm simply objecting to the implication that unnecessary allocations in Phobos aren't hurting D in any way.

On Friday, 12 April 2013 at 16:42:18 UTC, Nick Sabalausky wrote:
> While I'm not necessarily disagreeing with the other points you raise,
> Phobos's excess allocations *are* verifiability hurting D. A couple of
> the biggest potential user bases for D are videogames and embedded. I
> feel confident in saying that no other language has as much potential
> in these areas as D has. But these groups have *already*, vocally, been
> facing the problem of avoiding/rewriting potentially-large parts of
> Phobos, or sticking with C/C++.
>
> Now, you could argue that many of these people are simply being overly
> fearful of a merely imagined problem, but even if that's true, the
> problem is still real in it's effects: Hindering D adoption and causing
> people to (perhaps needlessly) avoid/rewrite parts of Phobos.
>
> Now, I'm not suggesting that we do *or* don't start a big effort to
> minimize allocations throughout Phobos, I'm simply objecting to the
> implication that unnecessary allocations in Phobos aren't hurting D in
> any way.

No, that's not what I was trying to say. I agree with your post completely. My point was specifically about std.process in particular, since I was misunderstanding why Manu was using std.process as an example.

Yes.

-- 
Mike Wey

On Friday, 12 April 2013 at 15:43:27 UTC, Steven Schveighoffer wrote:
> On Fri, 12 Apr 2013 04:14:15 -0400, Manu <turkeyman@gmail.com>
>> I'd use string[].
>
> You mean with format "a=b"?  I suppose that's possible, though horrible to work with before passing in.  Plus what happens if you have ["a=b", "a=c"] ?  AA's prevent these kinds of mistakes/contradictions.
>

I prefer Manu's idea with the API accepting string[], it's closer to the native format.

Then you could simply provide a convenience conversion from a map...
ex env!["foo" : "bar"] which would convert it to ["foo=bar"].

This also has the added benefit of being self documenting(considering the lack of named parameters).

But most importantly So the user has a free choice of constructing the env parameter manually in the most efficient way or using the lazy convenience function.

On Fri, 12 Apr 2013 15:26:12 -0400, Tove <tove@fransson.se> wrote:

> On Friday, 12 April 2013 at 15:43:27 UTC, Steven Schveighoffer wrote:
>> On Fri, 12 Apr 2013 04:14:15 -0400, Manu <turkeyman@gmail.com>
>>> I'd use string[].
>>
>> You mean with format "a=b"?  I suppose that's possible, though horrible to work with before passing in.  Plus what happens if you have ["a=b", "a=c"] ?  AA's prevent these kinds of mistakes/contradictions.
>>
>
> I prefer Manu's idea with the API accepting string[], it's closer to the native format.

That's great for the library on POSIX systems, but for the user, it's not as straightforward.  One must construct "x=y" strings instead of setting x to y.  Unless you are using literals, this involves an allocation anyway.

Also, this is not the native format on Windows which passes the environment in one large string delimited by nulls.

>
> Then you could simply provide a convenience conversion from a map...
> ex env!["foo" : "bar"] which would convert it to ["foo=bar"].
>
> This also has the added benefit of being self documenting(considering the lack of named parameters).

So for the most convenient/common case, you want to add an allocation?

> But most importantly So the user has a free choice of constructing the env parameter manually in the most efficient way or using the lazy convenience function.

I think the best suggestion so far is to allow a templated version for which you can provide an opIndex method, and a possible toEnv method (versioned for both Posix and Windows of course).  Then you have the power to avoid allocation as much as possible.  At the moment with AA's being the interface, we have no possibility to avoid allocation, and I understand that point.  But I don't want to eliminate that case or make it specifically more inefficient.

-Steve

On Friday, 12 April 2013 at 20:24:05 UTC, Steven Schveighoffer wrote:
> On Fri, 12 Apr 2013 15:26:12 -0400, Tove <tove@fransson.se> wrote:
>
>
> So for the most convenient/common case, you want to add an allocation?
>

with the original proposal there is one anyway...

But with my suggested approach you could create many processes reusing the same env... only paying the conversion/allocation cost once(outside of the loop).