Sean Kelly <sean@f4.ca> wrote:

[...]
> Then this should be changed.  The properties of inout parameters should be well-defined.

True. But this deficiency of the spec is not my main target.

Lets wait for Burton to solve the task he cried for.

-manfred

On Mon, 8 Aug 2005 17:01:52 +0000 (UTC), Manfred Nowak wrote:

> Derek Parnell <derek@psych.ward> wrote:
> [...]
>> I'm guessing that you are hinting that D has some really big security holes in it
> [...]
> 
> You are right. And because I have learned in this group, that I have to talk until I am blue in the face when I present such a problem,  I now try the other way round: let all of you talk until you are blue in the face and do find the problem, which releases me of that burdon, or do not find the problem, which makes you aware of the fact that there might be a problem as your post shows.

You may not be aware, but these are not the only two possible alternatives. Even if you do (secretly?) regard the rest of us unwashed and incompetent neophytes, there are still some 'rules' of social etiquette that allows patricians to discourse with plebeians.

For example, you may have chosen to phrase your concern thusly ...

  It appears that DMD allows this sort of situation to exist without
  reporting on it ...

    void f( inout int i, inout int j){
      i= 1;
      j= 2;
    }
    . . .
    int a;
    f(a,a);

  Do you think that this should be allowed to continue? Or is there
  some functionality that can be given to either D or DMD to help
  coders become aware of this situation if they choose to?

Or one of the myriad of alternative, but less abrasive and more informative, methods than the method that you did employ.


> If on the other hand nobody starts talking, then it is of no interest to the language and I have not wasted my time. So, because you got this pack running I answer your question as a representative of this group.

LOL ... I don't think so ... ;-)

> The problem with this function is indeed, that the outcome of giving the same variable to it, is undefined by design.

Well, the outcome is not 'undefined' in that the outcome is quite predicable, but it is still likely to be not what was expected.

> Therefore the task of any quality assurer is to reject this implementation: i.e. the outcome of step 2 is FAIL, with the reason above used for step 3.

Well, actually I see this slightly differently. It depends a lot on what 'quality' is declared to be. Our simple definition here is that something that conforms to all the specifications will pass the quality check.

Thus, if the *User* specification for this function stated that overlapping memory was allowable then it would pass. If the specification stated the opposite then it would fail. If it wasn't stated, then the specification is technically at fault and whatever the user received would pass. Of course, the unwritten expectation, or commonly assumed position, is that overlapping arguments would be a mistake and ought not to be provided to the function. In the case of unspecified behaviour, what usually happens is that runtime performance is given priority. However, you did mention the critical nature of the impact that this function could have, so I would be leaning more towards runtime checking of arguments, in that case.

> 
> So that piece of work comes back to the developing team.
> 
> Now the team has at least two choices from which this two are closest:
> 
> 1) define the outcome
> 2) forbid this case
> 
> But according to the specs none of these can succesfully be implemented in D.

Which specs? The D Language specs or the user requirement specs?

In my opinion, the team could implement either of these two choices.

I think the most useful position for us would be to have D continue to allow this possibility, and provide a standard function that we can use to detect overlapping arguments. Then the coder can choose how to react to the situation.

-- 
Derek
Melbourne, Australia
9/08/2005 12:13:56 PM

On Mon, 8 Aug 2005 18:31:10 +0000 (UTC), Sean Kelly wrote:

> In article <dd8360$2sm9$1@digitaldaemon.com>, Manfred Nowak says...
>> 
>>So that piece of work comes back to the developing team.
>>
>>
>>Now the team has at least two choices from which this two are closest:
>>
>>1) define the outcome
>>2) forbid this case
>>
>>But according to the specs none of these can succesfully be implemented in D.
> 
> Why not?  DBC should take care of both these problems:
> 
> # void f( inout int a, inout int b ) {
> # in {
> #     assert( &a != &b );
> # }
> # out {
> #     assert( &a != &b || ( a == b && b == 2 ) );
> # }
> # body {
> #     a = 1; b = 2;
> # }
> 
> I grant that the in and out clauses are a bit contradictory, but I'm assuming that either one or the other would be implemented.  As for contracts not being included in release builds--some applications ship with contract checking in place.  If this is a system where lives hang in the balance, I would expect the contract code to remain (coupled with an improved assertion handler).

There are at least two problems with using Contract code as an input argument checking device. One is that one cannot always guarantee that it will be compiled in; by accident it might be compiled with "-release". The other problem is that some contract code might only be intended for non-production editions of the application and there is no way to separate the two uses of Contract code.

The simpler, and more *reliable*, method is to perform argument checking outside of contract code and provide messages that will always get to the user. In other words, never assume that your application will always be compiled without "-release", because someday it will be. (Murphy's law?)

I still think that in{} and out{} blocks are useful for in-house testing, just not for production editions of the application. If you suspect that 'testing' might be needed at some customer's site, provide two editions of the application - one without contracts and one with contracts (and unit tests).

-- 
Derek
Melbourne, Australia
9/08/2005 12:42:52 PM

Hi,

>> There are languages that specialize in writing easily parallelisable code (via syntax). D, coming roughly from normal C/C++, is not one of them. An extended version of C, called Cilk, does just that:
>> 
>> http://supertech.lcs.mit.edu/cilk/
>> 
>Thanks for the link - that looks really cool.

Sure thing.

>It is becoming more and more clear to me that C's biggest weakness (for certain purposes, anyhow) is that, because it lets you do so much, without being explicit about what you really *intend*, the compiler really can't infer too much
>  about what you're specifically asking for; much of what your code asks
>for is not actually important. In the example, if you know that i and j will never refer to the same thing, (and there's no thread issues) then the order doesn't matter. But you have to write one before the other, and the C (or D) compiler will have to do them in that order.

That's why I'm a fan of more "dense" expressions. The less fluff the better. Of course, that's not to say sacrifice readability gratuitously.

>So a lot of optimizations which you'd like to happen become unsafe and don't get done. Contrast to fortran: there are very few ways to manipulate a fortran array. This means coding tends to be long and tedious, but the compiler can infer what you are doing more easily.

Have you checked restricted C pointers? They accomplish some of this, but are hella ugly.

>>>Is there any guarantee that the operations are carried out in the order
>>>they have been typed? (if so, what is the point of the volatile block
>>>statement?)
>>>http://www.digitalmars.com/d/statement.html#volatile
>> 
>> 
>> The guarantee (real or simulated) of order of execution is the very foundation of logical programming. If you couldn't rely on your statements being executed in the order you intended, then there is utter chaos.
> >
>Well put; but if you don't mind my saying, I think you glossed over the key issue a bit. 'real or simulated' is the key thing. Statements aren't always executed in the order you wrote them, but the results must be indistinguishable.

You are right, I did gloss over it. My IA32 is a little rusty ;). And the "simulated" part can get _real_ gory under the hood.

>> Btw, that's why the NON-guarantee of order of execution in parameter calls C) is such a hotly debated issue. It is one exception to the rule. It's a mistake, IMO, that should be fixed.
>>
>It's one of those things, I haven't been bothered by it, any more than that the order of calls in f(x) + g(x) is undefined. The cost (in terms of code efficiency) of breaking up your expressions to eliminate problems with side-effects has basically gone to zero. I.e.  consider
>
>       func(2*i,i++,f2(i));	/* not predictable, some wish it were */

Myself included.

>vs.
>       int f2res = f2(i+1)
>       func(2*i, i, f2res );	/*assuming this is what you wanted :-)*/
>       i++;
>
>The second one is predictable, won't warp your brain, and should not be any less efficient with a modern compiler (as opposed to a 20-year C compiler, where it could be somewhat slower).

Hehehe... I'm a fan of the first one, to be honest. The second one has some of that fluff I was talking about. ;)

>An expression is really just a flat description of a tree, I've never seen any reason why the different parts of the tree should be evaluated in any order (except as specified for || etc, of course), especially when the compiler can make better code by doing it in some particular order. Again, the more implicit control you give to the programmer, the more you are tying the compiler's hands in all the many cases where that control isn't actually needed.

You're probably right. Though I'm not sure what else could be done syntax-wise to help.

>And, unfortunately, if K&R had been forced to define an order, I suspect they would have gone with right-to-left (stack-push order) which is probably why they didn't. On some machines (especially older ones) there could be a serious penalty for left-to-right evaluation.

Ah, yes. This delicious relic still haunts me at night.

>>>In other words, is it possible for i to be 1 or 2 depending on what the optimising compiler decides to do and/or the enviroment the code is running in.
>> 
>> 
>> Optimization should not alter the behaviour (intrinsic order) of such code, otherwise it is erroneous. I do not recall seeing anywhere "variable references may not refer to the same thing" but please do correct me if I'm wrong.
>> 
>Right.
>As I see it, the solution is this: if you write a function where two or
>more reference parameters could refer to the same thing, and at least
>one of them is non-const, you should make an explicit note to potential
>users of the function as to whether that's allowed, and what the results
>are. Sometimes it's quite reasonable to pass the same thing in twice,
>and, by careful ordering of the writes (& reads) you can arrange to get
>the expected result. If you intend to *call* such a function with
>overlapped operands, and the author has *not* made such a note, then
>you're on your own... Even if the behaviour happens to be what you want,
>there's no contract that it won't change.

I think we're back to restricted pointers here. IIRC, the compiler trusts the programmer not to send the same thing twice. I wish there were automated ways to check for this kind of thing. Walter said it's not possible "in the general case," so it always kinda comes down to trusting the programmer.

Cheers,
--AJG.

On Tue, 9 Aug 2005 03:00:49 +0000 (UTC), AJG wrote:


[snip]
> I think we're back to restricted pointers here. IIRC, the compiler trusts the programmer not to send the same thing twice. I wish there were automated ways to check for this kind of thing. Walter said it's not possible "in the general case," so it always kinda comes down to trusting the programmer.

Well... there are ways of automating it, but just not at compile time ;-)

-- 
Derek
Melbourne, Australia
9/08/2005 1:15:00 PM

>> You are right. And because I have learned in this group, that I have
>> to talk until I am blue in the face when I present such a problem,  I
>> now try the other way round: let all of you talk until you are blue
>> in the face and do find the problem, which releases me of that
>> burdon, or do not find the problem, which makes you aware of the fact
>> that there might be a problem as your post shows.
>
> You may not be aware, but these are not the only two possible alternatives.
> Even if you do (secretly?) regard the rest of us unwashed and incompetent
> neophytes, there are still some 'rules' of social etiquette that allows
> patricians to discourse with plebeians.
>
> [snip]
>
> Or one of the myriad of alternative, but less abrasive and more
> informative, methods than the method that you did employ.

I feel this way, too. The way you did it sounded like "I see something you don't see ... hehe". I did not even bother to answer because of that, and i wonder why anyone else did... Too much spare time, obviously. :)

Anyways, i do not understand the whole point. If somebody passes overlapping out/inout arguments to a function, it is a niffy little bug of the programmer, not something i would write into a spec or even try to prevent with compiler errors. Neither would i write a assert into the in contract to test for that. As far as i can remember, i never made such an error in my programming practice. And yes, i do make some errors, and i program for 10+ years now. So this seems simply irrelevant to me. Remember that this is also possible with this function:

####
void doSomething(int* i, int* j)
{
  *i = 1;
  *j = 3;
}
####

But i did not read anything like "you must not pass two identical addresses to a function that is going to write to them" in the C specs, even if this could be a big security hole in the C/C++ and a lot of other languages... ;o)

Anyways, i could even think of senseful applications:

####
struct SomeStruct
{
  // members, lots of members.
}

void someFunc(inout SomeStruct st1, int i, inout SomeStruct st2)
{
  st1.someMember = someValue;
  st2.someOtherMember = someOtherValue;
}
####

Eh?

Ciao
uwe

Bad grammar in this posting... Sorry :(

Ciao
uwe

Derek Parnell <derek@psych.ward> wrote:

[ wise arguments]

Applause.

-manfred

Derek Parnell <derek@psych.ward> wrote:

[...]
> You may not be aware, but these are not the only two possible alternatives. Even if you do (secretly?) regard the rest of us unwashed and incompetent neophytes, there are still some 'rules' of social etiquette that allows patricians to discourse with plebeians.
> 
> For example, you may have chosen to phrase your concern thusly ...
[...]

You are right. In order to become a loved person to everyone you can do a lot of things. But if this little example makes you feel unwashed and incompetent then please never try to become a quality assurer.

My management once decided to buy a company. One of the first steps was to forbid to give out unauthorized patches, which was routine in that company. So my first contact to them was a developer coming with a patch in a 4000 lines program, written in basic, and declared that there is no documentation for this peace of software and that the originator of this whole thing was forced to leave the company.


[...]
>> The problem with this function is indeed, that the outcome of giving the same variable to it, is undefined by design.
> 
> Well, the outcome is not 'undefined' in that the outcome is quite predicable, but it is still likely to be not what was expected.

Please be aware of the differences of "by design" and "by implementation". The outcome by implementation is defined by the order of assignments and the other restrictions that are already mentioned in this thread.

[...]
> If it wasn't stated, then the specification is technically at fault and whatever the user received would pass.

And in this case the QA wouldn't have seen the code at all, because
that would be a mere waste of time?

[...]
> Which specs? The D Language specs or the user requirement specs?

D language.

[...]
> I think the most useful position for us would be to have D continue to allow this possibility, and provide a standard function that we can use to detect overlapping arguments. Then the coder can choose how to react to the situation.

Agreed. But now to something completely different: the details :-)

-manfred

"Uwe Salomon" <post@uwesalomon.de> wrote:

[...]
> The way you did it sounded like "I see something you  don't see ... hehe".

Have you ever received a compiler error or taken over a piece of source code with an error, because your colleague was trapped in a street accident? How do you feel, when your machine says "GPF"? Do you think: "Oh you little bastard! If you say again 'I see something you  don't see ... hehe' I will power you off and will work elsewhere."


> void doSomething(int* i, int* j)

Dont you see the differece? Here your interface requires and gives you full control over the actual parameters. But still the outcome is undefined by design.

-manfred