On Sat, May 23, 2020 at 09:05:47PM -0700, Walter Bright via Digitalmars-d wrote:
> On 5/23/2020 6:18 AM, Dibyendu Majumdar wrote:
> > > https://www.digitalmars.com/articles/C-biggest-mistake.html
> >
> > I do not think it was a mistake at all. Treating pointers and arrays uniformly is one of the great innovations in C.
>
> The amount of buffer overflows enabled by it is legendary.
On Sat, May 23, 2020 at 09:15:18PM -0700, Walter Bright via Digitalmars-d wrote:
> On 5/23/2020 4:50 PM, Andrei Alexandrescu wrote:
> > This "no fat pointers" decision cascaded into another disastruous choice - zero terminated strings. Worse than even autodecoding :o).
>
> Ironically, a lot of C's speed advantage is lost in endlessly scanning strings to find their length.
Hear, hear.
Having worked for the past 25 years in the industry primarily with C code, I have to say that the amount of C code with hidden potential buffer overflows is staggering. In spite of a big push in the past 10 or so years towards safer C coding practices, I still regularly come across code that have potential buffer overflows hidden deep within a large codebase, and code being freshly written that *still* depend on unfounded assumptions about array length -- because manually passing the length is just too cumbersome, and people either don't bother with it, or make mistakes while doing it (nothing like passing the wrong length to screw up your day -- esp. when it gets missed by QA and proceeds to explode in the customer's production environment -- or worse, it *doesn't* get noticed by the customer until a hacker decides to exploit it).
Not to mention the gigantic pain in writing said code in C in the first place -- you constantly have to be burning mental calories trying to keep track of which pointer goes with which length, making sure to manually check bounds, making sure to think thrice about your loop to prevent overruns, endless calls to strlen() without even thinking about the performance consequences. I'm serious, I work with a huge codebase of almost 2 million LOC, and strlen calls are everywhere, including inside macros that nobody really takes the time to understand but just sprinkle everywhere in their code. The complexity of managing all of this is just beyond any normal person's capacity given the tight deadlines we have to meet, so people just call strlen willy-nilly without even thinking about the performance consequences. And of course, the huge bulk of code that expects null-terminated strings means it's just not worth trying to encapsulate the length in any sort of way -- you make yourself incompatible with just about everything else, and end up spending endless effort converting to/from char* instead, which is not any better. So the problem perpetuates itself, and becomes an accumulated cost that nobody can feasibly improve without rewriting the entire darned thing from ground up. (And even then, all it takes is for *someone* to start using char* again, and suddenly you're back to square one. No sane C coder uses anything *but* char* for strings in "normal" C code. Esp. code that needs to talk to other C libraries not written by your team. It's inextricably ingrained in C culture.)
And strlen isn't even the end of it; because of manual memory management, everyone defensively strdup's every darned string that they intend to keep, because there's just no other sane way of ensuring the pointer won't get invalidated later (and no guarantee, thanks to default mutability, that someone won't change the contents of the string down the line and screw up the assumptions in your code). So in addition to strlen, your average typical C code is plagued with endless calls to strdup or equivalent, even for simple things like taking substrings.
Some people complain about D strings being immutable(char)[], but I tell you, encapsulated in that seemingly-trivial construct is a ton of experience-backed insight about string management in C-like programming paradigms that's worth careful study. People like bashing D because of the GC, but nobody talks about how *not* needing to call strlen() or strdup() endlessly is such a performance booster, not to mention it makes your code a LOT simpler and your APIs a lot cleaner to work with.
Considering just how much work has been poured into mitigating the horrible consequences of "no fat pointers" and null-terminated strings -- think about how much effort has been poured into dealing with buffer overflow bugs over the past 10 years: entire cottage industries have grown up around developing tools for detecting and fixing these sorts of things, and who knows how much money poured into cleaning up the consequences of the countless security exploits enabled by said buffer overflows -- the laughable "benefits" of saving a couple of bytes by using only lean pointers hardly measures up to anything less than a colossal design mistake in retrospect. If you can even call them "benefits": think of all the costly workarounds people have had to put up with: everyone inventing their own way of passing array length instead of having a standardized API, and inevitably doing it poorly / with costly slip-ups, and the memory cost of needing to invent, store, and manage data structures needed to manage all of this -- I surmise this in itself already counteracts any meager savings one may have gained by avoiding fat pointers. (Just think: in any persistent struct that carries pointers to arrays: you're already paying for the cost of a fat pointer because you need to store the length somehow anyway, except everyone invents their own way of doing this so you're essentially already paying for fat pointers but with none of the benefits of having a standardized fat pointer type that's been tested to do it correctly: you run the risk of human error at every turn.)
D arrays being fat pointers is a HUGE step at getting rid of the nonsensical churn C's array-pointers lead to. And immutable(char)[] is a huge saver in terms of performance. The two taken together is one of D's strengths.
T
--
"I suspect the best way to deal with procrastination is to put off the procrastination itself until later. I've been meaning to try this, but haven't gotten around to it yet. " -- swr
| 
Permalink
Reply
