"Matthew" <admin@stlsoft.dot.dot.dot.dot.org> wrote in message news:d00fgo$16uf$3@digitaldaemon.com...
>     1. I have found bugs in my C/C++ code involving
> unintended/undesirable integral narrowing. This is a fact. If you cannot
> accept this, then we have no basis for discussion. Do you accept this to
> be so?

Yes, and I've already done so to you (in the post about the risk of bugs
with it).

>     2. D provides, by definition, no facility for
> detecting/warning/remedying unintended/undesirable integral narrowing.
> This is a fact. Do you accept this to be so?

Yes.

> What I AM saying is that, if 1 is true, and 2 is true, then one can draw one of two conclusions:
>
>     A. D is flawed, by design, since errors due to integral narrowing
> cannot be automatically detected, other than by tools that do not
> represent the absolute letter of the language definition. Or,
>     B. D is different from C/C++ in a fundamental way such that the
> truth of 1. does not affect 2.

Let's just dismiss (B) as wrong. I have trouble with (A), though. All constructs in any language can result in bugs if used wrong, including explicit casts, and no compiler can detect them. The nirvana is to be able to figure out a language design that eliminates all possible error, but that's a research project I am not equipped to attempt.

The issue is a judgement call - what are the benefits vs risks of a particular construct? You assign the risk of an unintended narrowing as unacceptably high. I do not understand why you place it so high, but I am convinced you feel this to be so. Factoring in to that risk is the cost of the bug produced. I submit that such a bug will likely be reproducible and hence relatively easy to find. (This is as opposed to an uninitialized pointer bug, which can be really, really hard to reproduce and find. Part of my failure here is not understanding why you seem to regard the former problem as more dangerous than the latter, which is endemic in C++. I really loathe irreproducible bugs, and the design of D is clearly oriented towards stamping those out.) Clearly there is at least *some* benefit to the implicit narrowing conversions, as it has survived unscathed in two revisions of the C standard, one in C++, and I know of no proposal to C++ to make it an error.

Then there are the benefits and risks of the alternatives. The proposed solution is, issue a warning, and then eliminate the warning by inserting a cast. A cast is powerful but very blunt. It won't just do narrowing conversions, it will heroically attempt all sorts of things to convert the type. I submit that these kinds of casts can hide bugs. So far, I have not convinced you that this is a risk greater than 0. Running mango through this with warnings on will not prove that mango doesn't have such bugs in it, I don't know any way to automatically detect such things.

You've proposed a special cast type, narrow(t)e, to provide a finer point to casting. I know you like having many different kinds of casting available as seperate constructs. I find the multitude of them, even the C++ ones, confusing and find myself constantly going back to just ordinary casts.

The same points here generally apply to the other correctness issues we've discussed here - is the cure worse than the problem? It's not that there is *no* problem, it's that the cure is a worse problem. This is a point I have obviously failed to make in an understandable manner.

First, let me say, good post Matthew, I think this cuts down to the real issue here.
Which IMO is a basic disagreement to an initial proposition or suposition. (*see end)

I'll just sprinkle my opinions in below, no-one asked me to, but this _is_ a discussion group of sorts...

On Tue, 1 Mar 2005 12:11:48 +1100, Matthew <admin@stlsoft.dot.dot.dot.dot.org> wrote:
> While I may not necessarily be in full accord with either Kris's content
> or his presentation in this particular post, I do think he has something
> of a point.
>
> Let me put a question to you Walter:
>
>     1. I have found bugs in my C/C++ code involving
> unintended/undesirable integral narrowing. This is a fact. If you cannot
> accept this, then we have no basis for discussion. Do you accept this to
> be so?
>
>     2. D provides, by definition, no facility for
> detecting/warning/remedying unintended/undesirable integral narrowing.
> This is a fact. Do you accept this to be so?
>
> Now I am NOT saying that D's integral conversion is wrong. I'm NOT
> saying it has to be changed. I'm NOT saying DMD (and every other
> compiler) must have narrowing warnings.
>
> What I AM saying is that, if 1 is true, and 2 is true, then one can draw
> one of two conclusions:
>
>     A. D is flawed, by design, since errors due to integral narrowing
> cannot be automatically detected, other than by tools that do not
> represent the absolute letter of the language definition. Or,
>     B. D is different from C/C++ in a fundamental way such that the
> truth of 1. does not affect 2.

I think there are other conclusions, for example:

C. Integral narrowing might be a bug, it might not. It is outside the scope of the D compiler to identify/notify you of this possibility.

The reason for the above is that fact that D does not have warnings, so, unless it can be 100% sure something is a bug then it cannot do anything about it.

Perhaps _this_ is the flaw with D?

That said, a number of decisions have been made to call things which are 'likely' to be erroneous as simply erroneous, for example:

if (a) ;  <- error, must use {}

technically it cannot know for sure if that _is_ an error, but, it's 'likely' to be an error, and there is another easy alternative, so it calls it an error.

So, is it 'likely' to be an error? What percentage of implicit integer narrowing occurances result in an error?

The alternative is explicit casts, or refactoring the code in some way, Walter has given some examples of the results of that... they don't look pretty, at least to me.

If the answer is that it's not likely and/or the alternative is bad, then I don't think D can give an error, in which case, given it has no warnings, it is a "lint processing" task.

I view Walters responses as attempts to show that it's unlikely and/or the alternative is bad.

* I think the basic disagreement is a combination of "disagreement over how likely this is to cause a bug" and "the responsibility of the compiler to notify the programmer", which comes back to "D doesn't give warnings".

So, first you need to agree whether it is likely enough to warrant an error... if not, then isn't it a "lint processing" task?

I personally like the concept of segregating "compile" and "lint processing".

I view notifying you of any and every even remotely possible bug a "lint processing" task.

I think "lint processing" needs to be done once and only once on the same code, meaning, you run it when you compile on your dev box, but not when you compile the same code on your other x machines.

If anyone asked my opinion I would say Matthew was the best person for the task of writing the lint processor.

Regan

"Regan Heath" <regan@netwin.co.nz> wrote in message news:opsmxt9puo23k2f5@ally...
> I view Walters responses as attempts to show that it's unlikely and/or the alternative is bad.

Yes, that's correct. I'd like to add that not all bugs are equal - some are easy to reproduce and find, some are very hard to track down. I argue that the latter are so costly that they are worth considerable cost to try and prevent, whereas the former are less costly and so less risky.

D expends a lot more cost on trying to prevent the latter than the former. Things like array bounds checking, guaranteed initialization, garbage collection, etc., are examples of such steps.

I'm ignoring much of the attendant stuff as the reflective airborne particulate matter that it is:

    "nirvana", "research project", "opposed to an uninitialized pointer
bug, which can be really, really hard to reproduce and find" - smoke

    "I do not understand why you place it so high, but I am convinced
you feel this to be so" - irrelevant

    "I submit that such a bug will likely be reproducible and hence
relatively easy to find" - conjecture. (I've just been submitting
STLSoft 1.8.3 to many-compiler-compilations, and in a more organised and
comprehensive fashion than ever before, and I've found several long
dormant bugs. Either I'm a complete moron, or you're wrong. Could be
either ...)

    "The proposed solution is, issue a warning, and then eliminate the
warning by inserting a cast." - invalid assumption of the response to
the problem, based on ... ??what?? (That is to say, maybe I might change
the lhs type?)

    "A cast is powerful but very blunt" - says who? It is in C. Less so
in C++. Why must it be in D?


Thankfully, you've provided enough substance within to _finally_ allow me to understand your POV. Here's the best I can make of your argument, then. Please point out where I'm misrepresenting.

    1. D supports all integral=>integral conversions, including
narrowing ones. This is a potential source of bugs, which you
acknowledge, though you suggest that the alternative is worse.

    2. The reason for 1. is that you believe that providing a warning
may incline diligent developers to develop a bad habit of applying casts
with insufficient thought. Furthermore, the problem with this is not
that they will do so for narrowing, since the compiler already does that
for them, but rather than they will *also* do so in other circumstances.
I acknowledge that this is a very real danger.

    3. You do not think that separating the concerns of casts into
different casts operators is a good idea. Eureka! Now we're at the nub
of the matter, at last.

Your whole argument rests, then, on the fact that using a cast for narrowing might incline people to (mis)use casts elsewhere. As a consequence, narrowing is always allowed.

You agree that the current behaviour is manifestly a cause of bugs (naturally we would debate frequency of incidence, detectability, seriousness, etc....). I agree that requiring a simple cast may lead to cast-happy behaviour, which is also a source of bugs. All agree so far?

The problem, then, simply boils down to the notion of whether we split up cast functionality. You have not argued that a narrow_cast() would fail to address the requirement for narrowing conversions to be part of the programmer's cognisance. Nor have you argued that it would fail by inclining unrelated cast abuse (e.g. people'd start going mad with cross_cast(), down_cast(), etc. etc.). No, what you've said is:

    "I know you like having many different kinds of casting available as
    seperate constructs. I find the multitude of them, even the C++
ones,
    confusing and find myself constantly going back to just ordinary
casts."

This is bogus because:

    1. It's based on your own personal tastes and experiences. (As the
only commentator in recent times to have published a C++ book that, in
one circumstance only, advocates eschewing them for the wicked old C
cast, I think I am qualified to confidently say that C++ casts are well
received.)

    2. It's based on the (naive, IMO) dogma of simplicity, which
pervades D. Einstein said "as simple as possible, but no simpler". I
submit that, in this case at least, you've forgotten the second half of
that sentence. cast() is _too_ simple. It's needs complicationating. (I
confess that, prior to this topic, I hadn't really realised that D's
cast() has the same multifaceted nature as the C cast in C++. Now I
have, I must say I'm completely appalled, and will inevitably be banging
on about that too very soon.)

We've moved from two informed, intelligent, reasoned, reasonable, but opposing points of view, to an arbitrary decision based on whim and partial experience. While I would acknowledge that your experience may be greater than that of anyone else on this newsgroup, it does not represent the totality of software engineering, and I think this is a real and growing problem for D.

From a more general perspective, I wonder why you don't see how this will look to other people. You've now acknowledged that D has (at least one) flaw-by-design, and we've pretty much identified the rationale behind it. I submit that people coming from outside this community will look at that and dismiss D. Not because they care more about integral conversions above all else (though some might). But because they will naturally infer from this relatively simple and straightforward flaw that there will be other, deeper, and potentially nastier flaws. After all, how can one fault the reasoning: "I don't know all about X, but I have seen that X has done something very simple quite badly, and I therefore assume that X does non-so-simple things very badly indeed. I'll avoid X."

One last comment: I would agree with Kris in so far as that you compound the arbitrary nature of these decisions by refusing us ready access to the means by which we might prove/disprove them. If you're so confident of your position, allow us a pre-1.0 cast warning to play with, and instrument the inappropriate use of casts for non-narrowing things. Data's far more convincing that partial argument, however much it might be informed by experience.

Matthew

> "Regan Heath" <regan@netwin.co.nz> wrote in message news:opsmxt9puo23k2f5@ally...
>> I view Walters responses as attempts to show that it's unlikely
>> and/or the
>> alternative is bad.
>
> Yes, that's correct. I'd like to add that not all bugs are equal -
> some are
> easy to reproduce and find, some are very hard to track down. I argue
> that
> the latter are so costly that they are worth considerable cost to try
> and
> prevent, whereas the former are less costly and so less risky.
>
> D expends a lot more cost on trying to prevent the latter than the
> former.
> Things like array bounds checking, guaranteed initialization, garbage
> collection, etc., are examples of such steps.

Oh, come on!!

Is this truly _your_ experience, or supposition? Whichever, it is *NOT* my experience.

    I have not had an array bounds error since somewhere early last
year.
    I have not had an uninitialised variable bug since late last year.
    I had a memory leak (one) in January (I assume this is the equiv to
GC item)
    I have had two truncation bugs last month, one of which I only found
out about when running STLSoft through multiple compilers.

Now either you're representative of every other developer on the planet bar one, or I am, or we each represent but a small set of the spectrum. Of course it's the latter, which means that _you_ cannot cherry pick which issues you think matter and which do not. (Well, you can, it being your language, but you are not going to be right, and it ain't going to make D all it can be.)

Does no one else get this?!?!

"Martin M. Pedersen" <martin@moeller-pedersen.dk> wrote in message news:d00i1o$1974$1@digitaldaemon.com...
> "Walter" <newshound@digitalmars.com> skrev i en meddelelse news:cvvqp9$d89$1@digitaldaemon.com...
> > I challenge you to take the C++ program in
> > www.digitalmars.com/d/cppstrings.html and make it faster than the D
> > version.
> > Use any C++ technique, hack, trick, you need to.
>
> I could not resist having a go at it.
>
> The following are my results for the attached version:
>
>     dmd -o -release          359 ms
>     dmc -o -6                   375 ms
>     cl -Ox -G6 -ML         296 ms
>
> I used an input file with 64 times "alice30.txt" to have something measurable. "cl" is MSVC 7.1.

I noticed that you're using memory mapped files! Cheater! <g> Trying the D version with std.mmfile, I get about a 10% speedup. It adds one line of code and an import.

Anyhow, I think the effort you made just shows the point!

On Tue, 1 Mar 2005 14:39:18 +1100, Matthew <admin@stlsoft.dot.dot.dot.dot.org> wrote:
>> "Regan Heath" <regan@netwin.co.nz> wrote in message
>> news:opsmxt9puo23k2f5@ally...
>>> I view Walters responses as attempts to show that it's unlikely
>>> and/or the
>>> alternative is bad.
>>
>> Yes, that's correct. I'd like to add that not all bugs are equal -
>> some are
>> easy to reproduce and find, some are very hard to track down. I argue
>> that
>> the latter are so costly that they are worth considerable cost to try
>> and
>> prevent, whereas the former are less costly and so less risky.
>>
>> D expends a lot more cost on trying to prevent the latter than the
>> former.
>> Things like array bounds checking, guaranteed initialization, garbage
>> collection, etc., are examples of such steps.
>
> Oh, come on!!
>
> Is this truly _your_ experience, or supposition? Whichever, it is *NOT*
> my experience.
>
>     I have not had an array bounds error since somewhere early last
> year.
>     I have not had an uninitialised variable bug since late last year.
>     I had a memory leak (one) in January (I assume this is the equiv to
> GC item)
>     I have had two truncation bugs last month, one of which I only found
> out about when running STLSoft through multiple compilers.
>
> Now either you're representative of every other developer on the planet
> bar one, or I am, or we each represent but a small set of the spectrum.
> Of course it's the latter, which means that _you_ cannot cherry pick
> which issues you think matter and which do not. (Well, you can, it being
> your language, but you are not going to be right, and it ain't going to
> make D all it can be.)
>
> Does no one else get this?!?!

Sure, but how do you pick the right thing in this case? without canvasing every programmer in existance?

Someone has to make a decision, that person is Walter, D is his project. D will become all that Walter makes it, plus all that we contribute. My opinion is that he's (and we are) doing a fairly good job so far.

There is nothing stopping you taking the D front end and writing a lint-like program, I think having one, written by you, would be the best possible thing for D for several reasons:

1. you're going to do your damndest (sp?) to include as many possible things (like this) which you think should be done by the compiler.

2. you already use 10 different C++ compilers to get a series of unique, and useful warnings.

3. you have a good understanding of the many pitfalls and have thought about how to avoid them in the best possible manner.

The only problem being the amount of time it might take, so, perhaps you or someone else organises a team of people to work on this. I'd help, but I have a lack of spare time myself.

Regan

On Tue, 1 Mar 2005 14:39:18 +1100, Matthew wrote:


[snipped a lot of good stuff that I might come back to]

> Does no one else get this?!?!

I must admit that if I really look at what has been said so far, I feel a little offended by Walter's apparent attitude. It's sort of like Walter assumes I still wear nappies (diapers) and can't be trusted to use a toilet correctly. I guess that's the crux as I see it now; Walter doesn't trust me; its not personal, as he doesn't trust any programmer.

If I could be trusted to write responsible code, then DMD would be free to show me what could possibly be mistakes that I've made, and then let me act on them in a responsible manner, or at least take responsibility for my coding.

It could be that Walter has seen so much irresponsible code that he assumes everybody (including himself?) will always do the wrong thing if they could. And that position hurts a little, so maybe I'm too much of a boy scout.

-- 
Derek
Melbourne, Australia
1/03/2005 3:01:54 PM

"Derek Parnell" <derek@psych.ward> wrote in message news:1myhii2vp4h97$.5zhponxkjp52$.dlg@40tude.net...
> On Tue, 1 Mar 2005 14:39:18 +1100, Matthew wrote:
>
>
> [snipped a lot of good stuff that I might come back to]
>
>> Does no one else get this?!?!
>
> I must admit that if I really look at what has been said so far, I feel a
> little offended by Walter's apparent attitude. It's sort of like Walter
> assumes I still wear nappies (diapers) and can't be trusted to use a
> toilet
> correctly. I guess that's the crux as I see it now; Walter doesn't trust
> me; its not personal, as he doesn't trust any programmer.

I get the opposite impression - the situation he's arguing for is allow implicit casting without any warnings. That's putting all the responsibility on the coder (or other tools besides the compiler) to make sure it's right.

> If I could be trusted to write responsible code, then DMD would be free to
> show me what could possibly be mistakes that I've made, and then let me
> act
> on them in a responsible manner, or at least take responsibility for my
> coding.

I expect the grumbling masses will eventually make Walter put a warning or two in the "verbose mode".

> It could be that Walter has seen so much irresponsible code that he
> assumes
> everybody (including himself?) will always do the wrong thing if they
> could. And that position hurts a little, so maybe I'm too much of a boy
> scout.

He's not a backseat driver - he's letting us drive on our own without someone nagging them to signal when changing lanes, etc.

> -- 
> Derek
> Melbourne, Australia
> 1/03/2005 3:01:54 PM

"Regan Heath" <regan@netwin.co.nz> wrote in message news:opsmxxnciy23k2f5@ally...
> On Tue, 1 Mar 2005 14:39:18 +1100, Matthew <admin@stlsoft.dot.dot.dot.dot.org> wrote:
>>> "Regan Heath" <regan@netwin.co.nz> wrote in message news:opsmxt9puo23k2f5@ally...
>>>> I view Walters responses as attempts to show that it's unlikely
>>>> and/or the
>>>> alternative is bad.
>>>
>>> Yes, that's correct. I'd like to add that not all bugs are equal -
>>> some are
>>> easy to reproduce and find, some are very hard to track down. I
>>> argue
>>> that
>>> the latter are so costly that they are worth considerable cost to
>>> try
>>> and
>>> prevent, whereas the former are less costly and so less risky.
>>>
>>> D expends a lot more cost on trying to prevent the latter than the
>>> former.
>>> Things like array bounds checking, guaranteed initialization,
>>> garbage
>>> collection, etc., are examples of such steps.
>>
>> Oh, come on!!
>>
>> Is this truly _your_ experience, or supposition? Whichever, it is
>> *NOT*
>> my experience.
>>
>>     I have not had an array bounds error since somewhere early last
>> year.
>>     I have not had an uninitialised variable bug since late last
>> year.
>>     I had a memory leak (one) in January (I assume this is the equiv
>> to
>> GC item)
>>     I have had two truncation bugs last month, one of which I only
>> found
>> out about when running STLSoft through multiple compilers.
>>
>> Now either you're representative of every other developer on the
>> planet
>> bar one, or I am, or we each represent but a small set of the
>> spectrum.
>> Of course it's the latter, which means that _you_ cannot cherry pick
>> which issues you think matter and which do not. (Well, you can, it
>> being
>> your language, but you are not going to be right, and it ain't going
>> to
>> make D all it can be.)
>>
>> Does no one else get this?!?!
>
> Sure, but how do you pick the right thing in this case? without canvasing  every programmer in existance?

That's my point. The only person is the programmer writing the code that's about to be compiled. Any other judgement is flawed.

> Someone has to make a decision, that person is Walter, D is his project. D  will become all that Walter makes it, plus all that we contribute. My  opinion is that he's (and we are) doing a fairly good job so far.

Good, but not great. Get this: I believe that D has the potential to be truly fantastic. At its going I suspect it'll be hamstrung by its flaws, and become YAGI

> There is nothing stopping you taking the D front end and writing a lint-like program, I think having one, written by you, would be the best  possible thing for D for several reasons:

But that's the whole bleeding point. (Not swearing at you, btw.) I don't _want_ to write something that's going to go _against_ the standard. If I wanted that I'd just write a front end that gave me a boolean type, prevented any non-boolean subexprssions, etc. etc. But what's the point in that. That's not a commercial solution, it's a hobby.

> 1. you're going to do your damndest (sp?) to include as many possible things (like this) which you think should be done by the compiler.
>
> 2. you already use 10 different C++ compilers to get a series of unique,  and useful warnings.

I do, because I'm a pragmatist, and that's the pragmatic choice for C++. Were I to be presented with having to do the same thing for D, the pragmatic choice would be to stick to C++. (I'm serious.)

> 3. you have a good understanding of the many pitfalls and have thought about how to avoid them in the best possible manner.
>
> The only problem being the amount of time it might take, so, perhaps you  or someone else organises a team of people to work on this. I'd help, but  I have a lack of spare time myself.

I do plan to write such a thing in the future, just as I did for Java some years ago, to help with things that are outside the language, e.g. bracing styles, import lists, checking for doc comments, etc.. I have no intention to do it for such fundamental things _within_ the language. If D needs a lint for such things, I think D will not be worth using. (I base this on the supposition that many/most engineers do not use a lint, and belief that they should not have to use a lint to be able to write robust programs.)