http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #10 from Michel Fortin <michel.fortin@michelf.com> 2012-03-20 07:45:08 EDT ---
(In reply to comment #6)
> One possibility is to allow arbitrary sizes but have the compiler insert checks for all field accesses through pointer or reference when the field offset is beyond the OS's protected area.

That'd work. It'd introduce uneven performance characteristics, but that could be the price to pay for safety. (To not pay this price we'd need statically-enforced non-nullable pointers.)

On second thought, it needs a little tweak. Take this example:

struct A {
  /* a lot of fields */
  int f; // offset is 3 Kb - 4 bytes
}
struct B {
  A a1; // field offset is 0
  A a2; // field offset is 3 Kb
}

int foo(ref A a) {
  return a.f; // offset is 3K-4, considered safe?
}
int bar(B * b) {
  return foo(b.a2); // offset is 3K, considered safe?
}

Here, if you call bar(null), it'll call foo() with address null + 3K. foo()
will in turn add 3K-4 to the pointer, making it 6K-4 bytes, beyond the 4K
protected range.

What you need to take into account when deciding to insert the null check is not whether the offset is beyond the protected range, but whether the memory area of the field is entirely encompassed in the protected range. So you need to use field.offsetof+field.sizeof < 4K (or whatever the protected range is for a specific machine).

Note that you also need to do this same check when taking the address of a field:

B * baz() {
  return null;
}
void coz() {
  B * b = baz();
  A * a = &b.a2; // b.a2 spans on 3K..6K range, unsafe
  int f = a.f; // address of field is 6K-4
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #11 from deadalnix <deadalnix@gmail.com> 2012-03-20 04:50:47 PDT ---
(In reply to comment #10)
> (In reply to comment #6)
> > One possibility is to allow arbitrary sizes but have the compiler insert checks for all field accesses through pointer or reference when the field offset is beyond the OS's protected area.
> 
> That'd work. It'd introduce uneven performance characteristics, but that could be the price to pay for safety. (To not pay this price we'd need statically-enforced non-nullable pointers.)
> 
> On second thought, it needs a little tweak. Take this example:
> 
> struct A {
>   /* a lot of fields */
>   int f; // offset is 3 Kb - 4 bytes
> }
> struct B {
>   A a1; // field offset is 0
>   A a2; // field offset is 3 Kb
> }
> 

Classes are references types. So both a1.f and a2.f will have the same offset.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176


hsteoh@quickfur.ath.cx changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hsteoh@quickfur.ath.cx


--- Comment #13 from hsteoh@quickfur.ath.cx 2013-01-09 17:21:29 PST ---
Very large structs can also corrupt memory not just via null references, but via stack overflow. (Just pass a large struct by value down a few levels of recursion, and the stack will overflow.) And it's easier than one might think:

struct S {
  dchar[256] buffer;
}

Looks small, right? Well, sizeof(dchar)=4, which means sizeof(S)=1024. On Linux, the default stacksize is about 4KB. That means passing S by value down just 3-4 levels of recursion is enough to overflow the stack. And currently, I don't think DMD does any handling for stack overflow; it just crashes (and may corrupt memory as well, I didn't look into it).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #14 from deadalnix <deadalnix@gmail.com> 2013-01-09 17:41:47 PST ---
(In reply to comment #13)
> Very large structs can also corrupt memory not just via null references, but via stack overflow. (Just pass a large struct by value down a few levels of recursion, and the stack will overflow.) And it's easier than one might think:
> 
> struct S {
>   dchar[256] buffer;
> }
> 
> Looks small, right? Well, sizeof(dchar)=4, which means sizeof(S)=1024. On Linux, the default stacksize is about 4KB. That means passing S by value down just 3-4 levels of recursion is enough to overflow the stack. And currently, I don't think DMD does any handling for stack overflow; it just crashes (and may corrupt memory as well, I didn't look into it).

A page is reserved after the stack to detect it I think. It require to limit the size of what can be put on stack in a single operation. Or to add the checks that matter.

But a better option IMO is to reserve an insane amount of memory space for fiber's stack (and then some page to be protected for stack overflow detection) and run everything within a fiber. This is not very 32 bits compliant, but can work really well on 64bits systems.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #15 from bearophile_hugs@eml.cc 2013-01-09 18:30:05 PST ---
(In reply to comment #13)
> Very large structs can also corrupt memory not just via null references, but via stack overflow. (Just pass a large struct by value down a few levels of recursion, and the stack will overflow.) And it's easier than one might think:
> 
> struct S {
>   dchar[256] buffer;
> }
> 
> Looks small, right? Well, sizeof(dchar)=4, which means sizeof(S)=1024. On Linux, the default stacksize is about 4KB. That means passing S by value down just 3-4 levels of recursion is enough to overflow the stack.

A 1 kb struct should be considered very little for the stack. Generally it's a good idea to use the stack in D, because it avoids many slow heap allocations and decreases pressure on the GC.

On a modern PC with 1 GB RAM a D programmer should be free to use 1-10 MB of RAM for the stack. On Windows there is a linker command to increase the stack size.


> And currently, I
> don't think DMD does any handling for stack overflow; it just crashes (and may
> corrupt memory as well, I didn't look into it).

DMD used to give a readable error message on Windows, but now programs just crash.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #16 from bearophile_hugs@eml.cc 2013-01-09 18:37:19 PST ---
(In reply to comment #14)

> Or to add the checks that matter.

Adding checks in non-release mode is a good idea. See also Issue 9242

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugzilla@digitalmars.com


--- Comment #17 from Walter Bright <bugzilla@digitalmars.com> 2013-01-09 21:24:23 PST ---
Windows inserts a "guard page" that is hardware protected beyond the end of the allocated stack. A stack overflow runs into that guard page, which throws a seg fault.

It does not corrupt memory.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176


Jacob Carlborg <doob@me.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |doob@me.com


--- Comment #18 from Jacob Carlborg <doob@me.com> 2013-01-09 23:22:14 PST ---
(In reply to comment #17)
> Windows inserts a "guard page" that is hardware protected beyond the end of the allocated stack. A stack overflow runs into that guard page, which throws a seg fault.
> 
> It does not corrupt memory.

What about the rest of the platforms?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176



--- Comment #19 from Walter Bright <bugzilla@digitalmars.com> 2013-01-09 23:52:41 PST ---
They all do it. It's a well-known technique (several decades) used when you've got protected mode memory.

Of course, if you malloc your own stack, you don't have such protection.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=5176


Walter Bright <bugzilla@digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Platform|Other                       |All
            Version|unspecified                 |D2
         OS/Version|Mac OS X                    |All
           Severity|normal                      |enhancement


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------