On Friday, 12 April 2013 at 08:26:48 UTC, Manu wrote:
> Again, I'm just suggesting possibilities, and trying to illustrate that
> it's a STANDARD library, you can never predict where users will want to use
> it.

So you are talking about an imaginary system where memory allocation is expensive but process creation is cheap.

It's impossible to design and optimize software for some intangible goals. None of the systems that D targets, or aims to target, meet that criteria.

> Can you perhaps quantify how any of those things would be reduced by
> addressing at least the details I highlight?
> I'm basically advocating making a habit of using the stack where possible.
> It's not exactly hard, or cryptic.

Yes. However, I suggest the following instead:

Please rewrite some part of std.process with performance in mind, and post it here for review. This way, we can analyze the benefits and drawbacks based on a concrete example, instead of vapor and hot air.

Am Fri, 12 Apr 2013 17:04:08 +1000
schrieb Manu <turkeyman@gmail.com>:


> 
> toStringz is used liberally; alternatively, alloca() could allocate the c-string's on the stack and zero terminate them there, passing a pointer to the stack string to the OS functions.
> 

Are you sure this is possible in this specific case?
The execve manpage says:

execve() does not return on success, and the text, data, bss, and _stack_ of the calling process are overwritten by that of the program loaded.

There is no documentation if the environment variables are copied internally in execve.

On Fri, 12 Apr 2013 09:38:53 +0100, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

> On Friday, 12 April 2013 at 08:26:48 UTC, Manu wrote:
>> Again, I'm just suggesting possibilities, and trying to illustrate that
>> it's a STANDARD library, you can never predict where users will want to use
>> it.
>
> So you are talking about an imaginary system where memory allocation is expensive but process creation is cheap.
>
> It's impossible to design and optimize software for some intangible goals. None of the systems that D targets, or aims to target, meet that criteria.

I don't understand why you're so "against" these comments.  There is no real "argument" here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less.

Yes, premature optimisation is generally frowned upon in "user" code, but library code should ideally be optimised.  Should this percieved lack of optimisation prevent it's inclusion now, no because optimisation wont change the API (it may extend it however).

>> Can you perhaps quantify how any of those things would be reduced by
>> addressing at least the details I highlight?
>> I'm basically advocating making a habit of using the stack where possible.
>> It's not exactly hard, or cryptic.
>
> Yes. However, I suggest the following instead:
>
> Please rewrite some part of std.process with performance in mind, and post it here for review. This way, we can analyze the benefits and drawbacks based on a concrete example, instead of vapor and hot air.

Fair point, after all someone has to do the work and it seems logical that the person with the most understand of the issue should so it.  Whether he has the time however is the Q/problem.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote:
> I don't understand why you're so "against" these comments.  There is no real "argument" here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less.

This is a fallacy.

Performance is not free.

Somebody must do the work.
D contributors do not have an infinity of time and motivation.

Optimizing code often implies making it more complicated.

Straight-forward code is self-documenting.

More complicated code is harder to read and review.

More complicated code more easily hides bugs - including security bugs, such as buffer overflows.

Maintaining optimized code requires understanding not only the high-level logic, but also the low-level optimization details.

The benefits of optimizing std.process are likely to be so small, as to be difficult to measure.

Would you still say that the above costs are worth the nearly-intangible gain?

On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote:
> ...

It is not something to discuss in this topic. Yes, phobos has allocations issues. Yes, it is worth discussing. Does std.process do something special in that sense? No, possible problems are more related to lack of general configurable allocator.

Please do not fill voting thread with noise if you have no _blocking_ objections.

On Friday, 12 April 2013 at 08:34:15 UTC, Johannes Pfau wrote:
> Am Fri, 12 Apr 2013 17:04:08 +1000
> schrieb Manu <turkeyman@gmail.com>:
>
>> 
>> I've said before, I sadly have to avoid phobos like the plague. Some
>> modules (like this one) that provide fundamental functionality - not
>> just helper functions - can't be avoided. Requirements for those
>> should be extra strict in my opinion.
>> 
>
> Most (GC) allocations could be fixed without breaking the API. I can
> see 2 places where the API forces _GC_ allocations:
> * string[string]
> * thrown Exceptions are allocated with the GC
>
> There is no simple solution for these. Maybe we'll have a hashtable in
> the standard library at some point which will allow to use custom
> allocators. Then std.process could use these. Exceptions require some
> thinking. You can of course allocate them using any allocator, but
> freeing them is difficult...
>
> If you want to get rid of all (not only GC) allocations, there's another
> issue: As D strings are not zero terminated we'll always have
> allocations passing those to C code. Maybe we should have a cstring
> type in phobos which would just be a string which is guaranteed to be
> zero terminated.
>
> http://dpaste.dzfl.pl/e76aa995 (needs some inout though)


This is the Object Pascal way, although you can reach to a point where you get language native strings and C strings mixed all over the place.

Not to mention that C strings should not be allowed in safe code.

--
Paulo

On Fri, Apr 12, 2013 at 06:46:51AM +0200, Jesse Phillips wrote:
> It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no.

Yes!


--T

On Friday, 12 April 2013 at 04:46:52 UTC, Jesse Phillips wrote:
> It is that time, If you would like to see the proposed std.process include into Phobos please vote yes.

Yes! :)

On Fri, 12 Apr 2013 10:17:54 +0100, Vladimir Panteleev <vladimir@thecybershadow.net> wrote:

> On Friday, 12 April 2013 at 08:57:19 UTC, Regan Heath wrote:
>> I don't understand why you're so "against" these comments.  There is no real "argument" here, you both want std.process to be as good as it can be and it will clearly be better if it performs faster and allocates less.
>
> This is a fallacy.

Nope, you're attacking a Strawman here.  At most my statement is incomplete because it doesn't address the costs (which I left out for brevity).

> Performance is not free.

I never claimed it was (thus the Strawman).  I simply claimed std.process would be "better" if it performed faster and allocated less - which is true.

> Somebody must do the work.
> D contributors do not have an infinity of time and motivation.

I know this as well as anyone, my own free time is in short supply.

> Optimizing code often implies making it more complicated.
>
> Straight-forward code is self-documenting.
>
> More complicated code is harder to read and review.
>
> More complicated code more easily hides bugs - including security bugs, such as buffer overflows.
>
> Maintaining optimized code requires understanding not only the high-level logic, but also the low-level optimization details.
>
> The benefits of optimizing std.process are likely to be so small, as to be difficult to measure.

All true.  However, complexity can and should be packaged in such a way as to localise, and this localised complex code should be tested to death and maintained by someone who understands it.  It should be bracketed by sufficient comments and warnings about how/why it does what it does.  The resulting packaged complexity, with it's associated cost can be re-used many times over for all the benefit it gives.

Stack allocating the environment variables need not be a localised improvement but could be a standard library function which can be reused, for example.

> Would you still say that the above costs are worth the nearly-intangible gain?

"nearly-intangible" is wrong. Library code is code which is used by (hopefully) millions of people, writing millions of applications, running for millions of hours, on millions of systems, creating thousands of processes, etc..  In short, a little effort now pays massive dividends over it's lifetime.

So, yes, IMO the costs shown above are worth the resulting gains.

D is a systems programming language, there is hope that it will penetrate a wide range of systems and environments - sure in many cases a little bit of memory use or performance loss is unimportant, but for many it will be the decisive factor which makes D unusable there.

D is constantly being compared to other languages on the basis of performance, so it's clearly an important aspect of D's success.

Library code needs to work first time and work well or people will roll their own wasting time, energy and in many cases getting some aspect of it just plain wrong.

Again, I'm not suggesting (nor was Manu I think) that this in any way blocks the inclusion of std.process.  But I do share his desire for performance to sometimes be a little higher up the priority list.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

On Fri, 12 Apr 2013 05:46:51 +0100, Jesse Phillips <Jessekphillips+d@gmail.com> wrote:

> It is that time, If you would like to see the proposed std.process include into Phobos please vote yes. If one condition must be met specify under what condition, otherwise vote no.

Yes.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/