Menu
Search

Forums

Forum Index

April 01, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to H. S. Teoh		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to H. S. Teoh


Reply

On 4/1/22 5:20 PM, H. S. Teoh wrote:

>

On Fri, Apr 01, 2022 at 01:56:19PM -0700, Ali Çehreli via Digitalmars-d wrote:

>

On 4/1/22 13:47, Steven Schveighoffer wrote:

> >

I think this rule is related to C's accepting wildly different
platforms, some of which may have different kinds of memory. Two
pointers to different kinds of memory may not be subtracted.

Well, can the pointers be subtracted? Yes.

My understanding is that depending on the CPU, certain operations
would make the CPU barf. I am not sure but the old protected memory,
extended memory, etc. systems might not be able to subtract between
the systems at all. (Not sure; I am making this up.)
[...]

In the bad ole days of segmented protected memory (around the days of
the 386 or 486, IIRC), you could have memory with different segment
prefixes, referenced using a convoluted scheme of far ptrs and near
ptrs. Near ptrs are relative to a particular segment; subtracting near
ptrs associated with diverse segment pointers yields nonsensical values.
You can subtract far ptrs, sorta-kinda, but the results are likely to be
either garbage, or else refer to an address that can't be addressed with
existing segment pointers. So basically, it's Trouble with a capital T.

On modern machines, though, this is no longer relevant. I think people
figured out real quick that segmented addressing is just way more
trouble than it's worth, so we came running back, tail between legs, to
the flat memory model and embraced it like there's no tomorrow. :-D

Right, but my larger point was, the subtraction itself is not harmful.

There's two ways to look at this. First, if you subtract two pointers that aren't to the same block, then the data is garbage. The other way is that it is undefined behavior. I think using that subtracted difference to e.g. index a pointer is what would be UB. But the subtraction itself is ok.

-Steve

April 02, 2022
Gravatar of Salih DincerPosted by Salih Dincer
in reply to Ali Çehreli		Reply
Salih Dincer
Gravatar of Salih Dincer
Posted in reply to Ali Çehreli


Reply

On Friday, 1 April 2022 at 15:52:39 UTC, Ali Çehreli wrote:

>

As the following quote from a Microsoft document claims, and as I've already known, pointer subtraction is legal only if the pointers are into the same array: "ANSI 3.3.6, 4.1.1 The type of integer required to hold the difference between two pointers to elements of the same array, ptrdiff_t." ( https://docs.microsoft.com/en-us/cpp/c-language/pointer-subtraction?view=msvc-170 )

I suspect "array" means "a block of memory" there because arrays are ordinarily malloc'ed pieces of memory in C.
[...]

 
import std.stdio;

enum testLimit = 1024 * 1024 * 62;
void main()
{
  char[] first ="a12345".dup;      // length =>6
  first.length = testLimit;
  first[$-1] = 'z';

  auto hLen = first.length/2;      // =>3
  auto sliceRight = first[hLen..$];// "345"

  char* oneHalf = &first[hLen++];  // =>[3]
  char[] half12 = first[0..hLen];  // =>[0..4]
  char[] half22 = oneHalf[0..hLen];// =>[0..4]

  // legal? Ok.

  --hLen;                          // =>3
  for(int i; i < hLen; i++)
    assert(&sliceRight[i] - &half12[i] == hLen);

  // legal? Ok.

  half12[0..3].write("...");       // "a12..."
  sliceRight[$-3..$].writeln;      // "��z"

  char* a = &half12[0];            // "a"
  char* z = &sliceRight[$-1];      // "z"

  writefln("[%c]%s\n[%c]%s", *a, &(*a), *z, &(*z));
  assert( (&(*z) - &(*a)) ==
          (testLimit - 1) );

  // legal? Ok.

  auto test = half22.ptr - half12.ptr;
  assert(test == hLen);/*
  test.writeln;//*/

  // legal? Ok.

  auto last = first;
  assert(first.ptr - last.ptr ==
 0);

  // legal? Ok.

  last ~= '.';
  assert(&last[$-1] - &first[$-1] == 1);

}

Everything is legal...

SDB@79

April 02, 2022
Gravatar of ElronndPosted by Elronnd
in reply to Steven Schveighoffer		Reply
Elronnd
Gravatar of Elronnd
Posted in reply to Steven Schveighoffer


Reply
On Friday, 1 April 2022 at 17:39:24 UTC, Steven Schveighoffer wrote:
> In practice, I don't see how it affects the behavior *of the compiler*. When you subtract two pointers, I don't see how the compiler/optimizer can make some other decision based on the subtraction not being between two pointers to the same block of memory.

Unfortunately, they can and do.  For instance, consider this snippet of c code:

#include <stddef.h>
#include <stdlib.h>
int f() {
	int *x = malloc(1), *y = malloc(1);
	ptrdiff_t d = y - x;
	return y == x + d;
}

GCC compiles this to the same code as:

int f() { return 1; }

This is intertwined with issues of provenance.
April 02, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Elronnd		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Elronnd


Reply

On 4/2/22 5:38 AM, Elronnd wrote:

>

On Friday, 1 April 2022 at 17:39:24 UTC, Steven Schveighoffer wrote:

>

In practice, I don't see how it affects the behavior of the compiler. When you subtract two pointers, I don't see how the compiler/optimizer can make some other decision based on the subtraction not being between two pointers to the same block of memory.

Unfortunately, they can and do.  For instance, consider this snippet of c code:

#include <stddef.h>
#include <stdlib.h>
int f() {
    int *x = malloc(1), *y = malloc(1);
    ptrdiff_t d = y - x;
    return y == x + d;
}

GCC compiles this to the same code as:

int f() { return 1; }

This is intertwined with issues of provenance.

Wait, how does that differ from how it would handle pointers to the same block? If we use something other than pointers, it becomes obvious why this happens:

int x = ...;
int y = ...;
int d = x - y;
return y == x + d;

This is trivially always going to be true.

-Steve

April 02, 2022
Gravatar of Paul BackusPosted by Paul Backus
in reply to Steven Schveighoffer		Reply
Paul Backus
Gravatar of Paul Backus
Posted in reply to Steven Schveighoffer


Reply

On Saturday, 2 April 2022 at 02:20:34 UTC, Steven Schveighoffer wrote:

>

Right, but my larger point was, the subtraction itself is not harmful.

There's two ways to look at this. First, if you subtract two pointers that aren't to the same block, then the data is garbage. The other way is that it is undefined behavior. I think using that subtracted difference to e.g. index a pointer is what would be UB. But the subtraction itself is ok.

It's UB just to perform the subtraction:

>

C11 § 6.5.6 ¶ 9 When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object

 >

C11 § 4 ¶ 2 If a ''shall'' or ''shall not'' requirement that appears outside of a constraint or runtime- constraint is violated, the behavior is undefined.
April 02, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Paul Backus		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Paul Backus


Reply

On 4/2/22 1:08 PM, Paul Backus wrote:

>

On Saturday, 2 April 2022 at 02:20:34 UTC, Steven Schveighoffer wrote:

>

Right, but my larger point was, the subtraction itself is not harmful.

There's two ways to look at this. First, if you subtract two pointers that aren't to the same block, then the data is garbage. The other way is that it is undefined behavior. I think using that subtracted difference to e.g. index a pointer is what would be UB. But the subtraction itself is ok.

It's UB just to perform the subtraction:

>

C11 § 6.5.6 ¶ 9 When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object

 >

C11 § 4 ¶ 2 If a ''shall'' or ''shall not'' requirement that appears outside of a constraint or runtime- constraint is violated, the behavior is undefined.

Yep. That's a steep penalty.

It looks like C is trying to avoid having to specify how memory works without actually specifying it (likely on purpose so it doesn't tie down hardware developers to one memory model).

An interesting read I found on C and pointer comparisons here:

https://stefansf.de/post/pointers-are-more-abstract-than-you-might-expect/

-Steve

April 02, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Ali Çehreli		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Ali Çehreli


Reply
You're right that the spec should add some commentary saying that pointer arithmetic should be confined to being within the same memory object.
April 03, 2022
Gravatar of Tobias PankrathPosted by Tobias Pankrath
in reply to Walter Bright		Reply
Tobias Pankrath
Gravatar of Tobias Pankrath
Posted in reply to Walter Bright


Reply
On Saturday, 2 April 2022 at 18:54:58 UTC, Walter Bright wrote:
> You're right that the spec should add some commentary saying that pointer arithmetic should be confined to being within the same memory object.

What's the definition of memory object here? Does the C Standard treat malloc/calloc and co as special functions and memory objects are what is returned by them?
April 03, 2022
Gravatar of Patrick SchluterPosted by Patrick Schluter
in reply to Ali Çehreli		Reply
Patrick Schluter
Gravatar of Patrick Schluter
Posted in reply to Ali Çehreli


Reply
On Friday, 1 April 2022 at 20:22:46 UTC, Ali Çehreli wrote:
> On 4/1/22 10:39, Steven Schveighoffer wrote:
>
> > I don't see how the compiler/optimizer
> > can make some other decision based on the subtraction not
> being between
> > two pointers to the same block of memory.
>
> I think this rule is related to C's accepting wildly different platforms, some of which may have different kinds of memory. Two pointers to different kinds of memory may not be subtracted.
>
> Ali

In C you're not even allowed to cast a function pointer to a data pointer. Posix requires that they are (or else no dynamic linking).

These C restictions are put in place because of the all the platforms it is supposed to work (subtracting pointers on MS-DOS or on PDP-10 is not an easy proposition).
For simplicity sake D was defined to be implemented only on at least 32 bit machines with memory protection and linear address range.
April 03, 2022
Gravatar of Paul BackusPosted by Paul Backus
in reply to Tobias Pankrath		Reply
Paul Backus
Gravatar of Paul Backus
Posted in reply to Tobias Pankrath


Reply
On Sunday, 3 April 2022 at 12:22:22 UTC, Tobias Pankrath wrote:
> On Saturday, 2 April 2022 at 18:54:58 UTC, Walter Bright wrote:
>> You're right that the spec should add some commentary saying that pointer arithmetic should be confined to being within the same memory object.
>
> What's the definition of memory object here? Does the C Standard treat malloc/calloc and co as special functions and memory objects are what is returned by them?

In C: http://port70.net/~nsz/c/c11/n1570.html#3.15

In D: https://dlang.org/spec/intro.html#object-model
« First   ‹ Prev
1 2
Next ›   Last »