Uh... I used toStringz, but didn't know that it's return type is const and that people have troubles with this :) AFAIK the only function, writing to its sz argument is strtok, C libraries usually follow popular convention that input strings are "in".

ore-sama wrote:
> Uh... I used toStringz, but didn't know that it's return type is
> const and that people have troubles with this :) AFAIK the only
> function, writing to its sz argument is strtok, C libraries usually
> follow popular convention that input strings are "in".

strcpy sprintf strcat gets, off the top of my head. They are so bad and dangerous, for some safer versions were introduced and are universally recommended: strncpy snprintf strncat.

Andrei

Steven Schveighoffer wrote:
> "Andrei Alexandrescu" wrote
>> Steven Schveighoffer wrote:
>>> But you probably won't add it.  That's ok, I don't use Phobos anyways. I'll be sure to add an appropriate function to Tango while porting it to D2.
>> You may want to rethink before putting dangerous functions in widely-used libraries. Returning a writable zero-terminated char* is as dangerous as it gets, and fostering bad coding style too.
> 
> Nonsense.  Tango currently has such a function with D 1.x, and I've never heard of any issues with it.

That doesn't mean much of anything.

> I think you have overblown the danger here.

And it's not me.

http://marc.info/?l=bugtraq&m=105673479925709&q=p4
http://seclists.org/vulnwatch/2007/q1/0069.html
http://www.mail-archive.com/popt-devel@rpm5.org/msg00072.html
http://sourceware.org/ml/gdb-patches/2005-03/msg00237.html
http://mailman.mit.edu/pipermail/krbdev/2008-October/007022.html

I got bored of pasting. Just google for strcpy sprintf strcat gets. Again: please rethink.


Andrei

Andrei Alexandrescu Wrote:

> ore-sama wrote:
> > Uh... I used toStringz, but didn't know that it's return type is const and that people have troubles with this :) AFAIK the only function, writing to its sz argument is strtok, C libraries usually follow popular convention that input strings are "in".
> 
> strcpy sprintf strcat gets, off the top of my head. They are so bad and dangerous, for some safer versions were introduced and are universally recommended: strncpy snprintf strncat.

that's because they get char buffer, not sz, so they can't determine end of buffer scanning for null (strcat does, but it's still buffer). toStringz has nothing to do with these functions.

ore-sama wrote:
> Andrei Alexandrescu Wrote:
> 
>> ore-sama wrote:
>>> Uh... I used toStringz, but didn't know that it's return type is const and that people have troubles with this :) AFAIK the only function, writing to its sz argument is strtok, C libraries
>>> usually follow popular convention that input strings are "in".
>> strcpy sprintf strcat gets, off the top of my head. They are so bad
>> and dangerous, for some safer versions were introduced and are
>> universally recommended: strncpy snprintf strncat.
> 
> that's because they get char buffer, not sz, so they can't determine
> end of buffer scanning for null (strcat does, but it's still buffer).
> toStringz has nothing to do with these functions.

Good point.

Andrei

"Andrei Alexandrescu" wrote
> Steven Schveighoffer wrote:
>> "Andrei Alexandrescu" wrote
>>> Steven Schveighoffer wrote:
>>>> But you probably won't add it.  That's ok, I don't use Phobos anyways. I'll be sure to add an appropriate function to Tango while porting it to D2.
>>> You may want to rethink before putting dangerous functions in widely-used libraries. Returning a writable zero-terminated char* is as dangerous as it gets, and fostering bad coding style too.
>>
>> Nonsense.  Tango currently has such a function with D 1.x, and I've never heard of any issues with it.
>
> That doesn't mean much of anything.

OK, it's only the most used D library.  Sorry for the unrelated reference. Phobos 1 had the same issue BTW.

> > I think you have overblown the danger here.
>
> And it's not me.
>
> http://marc.info/?l=bugtraq&m=105673479925709&q=p4 http://seclists.org/vulnwatch/2007/q1/0069.html http://www.mail-archive.com/popt-devel@rpm5.org/msg00072.html http://sourceware.org/ml/gdb-patches/2005-03/msg00237.html http://mailman.mit.edu/pipermail/krbdev/2008-October/007022.html
>
> I got bored of pasting. Just google for strcpy sprintf strcat gets. Again: please rethink.

I got bored of reading.  Like ore-sama said, your examples have nothing to do with zero-terminated strings.  Some of them even outlaw functions that take const char *.  So by your argument that proof is in these pages, if you are going to condemn functions that take mutable zero-terminated strings, you have to do the same for const zero terminated strings.  And most of them are talking about buffer overflow attacks, which would be unlikely (maybe impossible?  I don't know much about it) with heap-allocated strings, which is what alwaysCopyAndReturnMutableStringz returns.

None of this changes my view.  And I'm done arguing, take away from this what you will.

-Steve

Steven Schveighoffer wrote:
> "Andrei Alexandrescu" wrote
>> Steven Schveighoffer wrote:
>>> "Andrei Alexandrescu" wrote
>>>> Steven Schveighoffer wrote:
>>>>> But you probably won't add it.  That's ok, I don't use Phobos anyways. I'll be sure to add an appropriate function to Tango while porting it to D2.
>>>> You may want to rethink before putting dangerous functions in widely-used libraries. Returning a writable zero-terminated char* is as dangerous as it gets, and fostering bad coding style too.
>>> Nonsense.  Tango currently has such a function with D 1.x, and I've never heard of any issues with it.
>> That doesn't mean much of anything.
> 
> OK, it's only the most used D library.  Sorry for the unrelated reference. Phobos 1 had the same issue BTW.
> 
>>> I think you have overblown the danger here.
>> And it's not me.
>>
>> http://marc.info/?l=bugtraq&m=105673479925709&q=p4
>> http://seclists.org/vulnwatch/2007/q1/0069.html
>> http://www.mail-archive.com/popt-devel@rpm5.org/msg00072.html
>> http://sourceware.org/ml/gdb-patches/2005-03/msg00237.html
>> http://mailman.mit.edu/pipermail/krbdev/2008-October/007022.html
>>
>> I got bored of pasting. Just google for strcpy sprintf strcat gets. Again: please rethink.
> 
> I got bored of reading.  Like ore-sama said, your examples have nothing to do with zero-terminated strings.

They do because dealing in mutable char* fosters transporting bare pointers as entire mutable strings. (A slightly safer scenario used in C is to always transport the length along with the mutable char*, or to traffic in char** with an understanding it is heap allocated (as e.g. GNU's readline does).) So the connection is this: you almost never need, even in C routines, a bald pointer to mutable char*. When you do, it's almost always in order to pass to a function that is dangerous and unrecommended to start with. Safe functions take char* and length, which a char[] offers no problem. From all of the above the net result is that toStringzAlwaysCopy is useless.

> Some of them even outlaw functions that take const char *.

Yes, the potential format attack. That is an unrelated issue that has to do with tainted vs. untainted strings. (Some proposed languages and language extensions allow "tainted" as a qualifier... a good potential issue to discuss in the future.)

> So by your argument that proof is in these pages, if you are going to condemn functions that take mutable zero-terminated strings, you have to do the same for const zero terminated strings.

That would be a simplistic view of what I meant to say. Const zero-terminated strings may be dangerous when formatting/execution information that they contain leads to erratic behavior elsewhere, as in e.g. printf, scanf, or system. That is a distinct danger from that of not passing the length of a buffer around and running over its length.

> And most of them are talking about buffer overflow attacks, which would be unlikely (maybe impossible?  I don't know much about it) with heap-allocated strings, which is what alwaysCopyAndReturnMutableStringz returns.

Buffer overflow is a concrete and spectacular consequence of memory unsafety, but that doesn't make other consequences less bad.

> None of this changes my view.  And I'm done arguing, take away from this what you will.

I wasn't in this for arguing, saving face, or having the last word. I simply inferred from your post you did not know the information about the stir on unsafe functions in C and just wanted to share that with you. So I wasn't discussing a view as much as sharing a fact. That fact hamstrings the motivation for toStringzMutable, hence the relevance to the discussion. I confess I was even hoping for the casual "thanks" for going through the trouble of explaining and picking links etc :o).


Andrei

Andrei Alexandrescu Wrote:

> > that's because they get char buffer, not sz, so they can't determine end of buffer scanning for null (strcat does, but it's still buffer). toStringz has nothing to do with these functions.
> 
> Good point.

But what can really simplify interoperability is modification of cat and idup operators so that they allocate an extra char and write null to it, so toStringz can be replaced with idup ar avoided completely.

ore-sama wrote:
> Andrei Alexandrescu Wrote:
> 
>>> that's because they get char buffer, not sz, so they can't
>>> determine end of buffer scanning for null (strcat does, but it's
>>> still buffer). toStringz has nothing to do with these functions.
>> Good point.
> 
> But what can really simplify interoperability is modification of cat
> and idup operators so that they allocate an extra char and write null
> to it, so toStringz can be replaced with idup ar avoided completely.

At a point we thought of quite a few schemes along those lines. At the end of the day, however, it's not that often one does need zero-terminated strings in D so it seemed onerous to do the work always just in case someone needs a stringz.

Andrei