December 31, 2021

On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:

>

While D offers buffer overflow detection, it does not protect against buffer overflows resulting from an array size calculation overflow:

T* p = cast(T*)malloc(len * T.sizeof);

What if len*T.sizeof overflows? malloc() will succeed, but the result will be too small for the data.

For projects using Phobos, an easy way to avoid this is to use Mallocator and makeArray from the std.experimental.allocator package.

T[] array = Mallocator.instance.makeArray!T(len);

makeArray will perform an overflow check internally and return null if the check fails.

December 31, 2021
On 12/31/21 5:05 AM, max haughton wrote:
> On Friday, 31 December 2021 at 08:46:59 UTC, claptrap wrote:
>> On Friday, 31 December 2021 at 00:13:56 UTC, Walter Bright wrote:
>>> [...]
>>
>> are the asserts not taken out in release mode?
> 
> The compiler ships with assertions turned on.

And to remind, dmd's -release picks 'safeonly' for the following command line switch, which can separately be selected to override the default:

  -boundscheck=[on|safeonly|off]
                    bounds checks on, in @safe only, or off

---
@safe: // Assertions are checked by-default for @safe code

int main(string[] args) {
  int[] arr;
  return arr[args.length];
}
---

Ali
December 31, 2021

On Friday, 31 December 2021 at 13:52:26 UTC, Paul Backus wrote:

>

For projects using Phobos, an easy way to avoid this is to use [Mallocator][1] and [makeArray][2] from the std.experimental.allocator package.

T[] array = Mallocator.instance.makeArray!T(len);

makeArray will perform an overflow check internally and return null if the check fails.

This. D code should not keep calling C malloc when we can do better. It's unfortunate that the import and the call above are quite awkward to remember and type. It's a shame core.memory.pureMalloc repeats this vulnerable design. Perhaps add an overload for ease of use?

import core.memory;
T[] array = pureMalloc!T(len);
December 31, 2021
On 12/30/2021 4:37 PM, sarn wrote:
> Good thing to do, but Walter's talking about integer overflow with the `len * T.sizeof` calculation itself.
> 
> calloc() doesn't have this problem.

The calculation of `len` can also have overflow problems. `calloc` is not sufficient. The provenance of `len` needs to be carefully checked.
December 31, 2021
Allocators should be first class citizen anyways, malloc/free should not be used directly, not only they are unsafe, they are most of the time not what people need

And i hope what will come out of `std.experimental.allocator` won't be using classes..something light, value, extensible and made with betterC in mind

It should be the root of everything that comes out of D, not something lightly thought

A base to move forward and be future proof
December 31, 2021
On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:
> Allocators should be first class citizen anyways, malloc/free should not be used directly, not only they are unsafe, they are most of the time not what people need
>
> And i hope what will come out of `std.experimental.allocator` won't be using classes..something light, value, extensible and made with betterC in mind
>
> It should be the root of everything that comes out of D, not something lightly thought
>
> A base to move forward and be future proof

Follow up, it's roten: https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception

There are words, and facts, unfortunately..

According to DConf, Atila is working to get that mess out of experimental, hopefully he won't repeat same mistakes as Andrei, and step up to remove all of that crap that holds the language back

I can offer a hand if he wants to, i personally would hold them for a D3, if that effect happen, making a bigger impact is better than a few in an ocean full of mistakes
December 31, 2021
On Fri, Dec 31, 2021 at 12:12:45PM -0800, Walter Bright via Digitalmars-d wrote:
> On 12/30/2021 4:37 PM, sarn wrote:
> > Good thing to do, but Walter's talking about integer overflow with the `len * T.sizeof` calculation itself.
> > 
> > calloc() doesn't have this problem.
> 
> The calculation of `len` can also have overflow problems. `calloc` is not sufficient. The provenance of `len` needs to be carefully checked.

At my day job we use Coverity to identify potential issues with our C codebase.  One of the issues it reports is using external inputs as the length of a memory allocation, the typical case being reading an int or long from a file/socket and then passing that to malloc, et al (potentially with a sizeof multipler).  So imagine a carefully-crafted malicious input designed to overflow a 64-bit integer just a little -- the malloc call would end up allocating just a tiny amount of memory while the rest of the code thinks that the buffer has more memory than can be addressed.

Of course, that tempts the following check (obviously wrong, but I've seen this scarily often in "professional" code):

	size_t n = ... /* read from file/socket */;
	size_t nbytes = n * sizeof(Element); // oops
	if (nbytes > INT64_MAX)	// never true
		error();	// dead code
	void *p = malloc(n * sizeof(Element)); // uh-oh
	for (i=0; i < n; i++) {
		... /* use p: kaboom */
	}


T

-- 
INTEL = Only half of "intelligence".
December 31, 2021
On Friday, 31 December 2021 at 21:58:14 UTC, russhy wrote:
> On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:
>> Allocators should be first class citizen anyways, malloc/free should not be used directly, not only they are unsafe, they are most of the time not what people need
>>
>> And i hope what will come out of `std.experimental.allocator` won't be using classes..something light, value, extensible and made with betterC in mind
>>
>> It should be the root of everything that comes out of D, not something lightly thought
>>
>> A base to move forward and be future proof
>
> Follow up, it's roten: https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception
>
> There are words, and facts, unfortunately..

As far as I can tell, the "uses" of exceptions found by that search consist of

1. Code to handle exceptions thrown by copy constructors and postblits.
2. Optional features that must be explicitly enabled by the programmer at compile-time.
3. A try/catch block in GCAllocator.
3. Unit-test code.

So, nothing here should block compatibility with BetterC, although it might require some `version` blocks to disable the runtime-dependent stuff.
January 01, 2022
What I can confirm as working right now is make, makeArray and dispose.

What I don't think works right now and that needs to work is RCISharedAllocator.

In saying all this, I'm planning on writing my own allocators anyway, so it may be all moot for me.
December 31, 2021
On Friday, 31 December 2021 at 22:13:15 UTC, Paul Backus wrote:
> On Friday, 31 December 2021 at 21:58:14 UTC, russhy wrote:
>> On Friday, 31 December 2021 at 21:37:56 UTC, russhy wrote:
>>> Allocators should be first class citizen anyways, malloc/free should not be used directly, not only they are unsafe, they are most of the time not what people need
>>>
>>> And i hope what will come out of `std.experimental.allocator` won't be using classes..something light, value, extensible and made with betterC in mind
>>>
>>> It should be the root of everything that comes out of D, not something lightly thought
>>>
>>> A base to move forward and be future proof
>>
>> Follow up, it's roten: https://github.com/dlang/phobos/search?q=path%3Astd%2Fexperimental%2Fallocator+exception
>>
>> There are words, and facts, unfortunately..
>
> As far as I can tell, the "uses" of exceptions found by that search consist of
>
> 1. Code to handle exceptions thrown by copy constructors and postblits.
> 2. Optional features that must be explicitly enabled by the programmer at compile-time.
> 3. A try/catch block in GCAllocator.
> 3. Unit-test code.
>
> So, nothing here should block compatibility with BetterC, although it might require some `version` blocks to disable the runtime-dependent stuff.

Ok great, sounds manageable, i'm starting the work already, we'll see where that'll go