On Sunday, 16 January 2022 at 23:34:41 UTC, Ali Çehreli wrote:
>
> Definitely a -profile=gc bug. Here are the existing ones:
>
>   https://issues.dlang.org/buglist.cgi?quicksearch=profile%20gc
>
> Ali

yeah, a bug makes more sense ... otherwise I really would have had a slice to data that doesn't exist anywhere.

some of those reported bugs are pretty old.

.. oh well... just another thing in D, that I can't rely on :-(

On Sun, Jan 16, 2022 at 08:21:29AM -0800, Ali Çehreli via Digitalmars-d-learn wrote:
> On 1/16/22 07:32, Salih Dincer wrote:
> > On Sunday, 16 January 2022 at 11:43:40 UTC, Ali Çehreli wrote:
> >>
> >> void main() {
> >>   enum count = 7;
> >>
> >>   // Allocate some memory
> >>   void* rawData = malloc(int.sizeof * count);
> 
> In practice, malloc'ed memory is cleared e.g. by memset(). Or, there is
> calloc() which returns memory filled with zeros.

Correction: malloc() is not guaranteed to clear the allocated memory. Possibly for "performance reasons".  Usually, though, I'd recommend using calloc() instead to initialize the allocated memory and prevent surprising results. (E.g., you allocate a buffer expecting it would be zeroed, but it isn't so the code that assumes it does produces garbled results. Or worse, if you're allocating memory for, e.g., a packet buffer to be transmitted across the network, failing to initialize it may leak the past contents of that memory to the internet. Cf. HeartBleed.)


[...]
> > If count is not equal to 8 I get weird results! The reason of
> > course, is the free():
> > // [93947717336544, 1, 2, 3, 4, 5, 6]
> 
> I didn't know free wrote into the freed buffer but since it's undefined behavior, we shouldn't even be able to know whether it did or not. :/
[...]

This is likely caused by the implementation of malloc/free. Some implementations store pointers and other tracking information inside the free blocks themselves instead of using a separate data structure to track free memory (e.g., the start of the block may contain a pointer to the next available block).  After calling free(), that memory is now a free block, so the implementation may start storing such information within the block.

Some memory allocator implementations may also store a canary value at the beginning of freed blocks to detect double free's (i.e., a value unlikely to occur in user data that, if detected by free(), may indicate a bug in the code that's trying to free the same block twice).


T

-- 
Debian GNU/Linux: Cray on your desktop.

On 1/16/22 17:51, H. S. Teoh wrote:

>> In practice, malloc'ed memory is cleared e.g. by memset(). Or, there is
>> calloc() which returns memory filled with zeros.
>
> Correction: malloc() is not guaranteed to clear the allocated memory.

Agreed. What I meant is, the programmer ordinarily clears it with memset().

>> I didn't know free wrote into the freed buffer but since it's
>> undefined behavior, we shouldn't even be able to know whether it did
>> or not. :/
> [...]
>
> This is likely caused by the implementation of malloc/free. Some
> implementations store pointers and other tracking information inside the
> free blocks themselves

Yes but they do not (and cannot) put it inside what they returned. They store such information in the area right before the returned pointer.

> Some memory allocator implementations may also store a canary value at
> the beginning of freed blocks to detect double free's

I thought of that as well but I doubt they would go against the mentioned performance ideals.

My current theory is that our writeln() calls after the free needed memory for the output strings and it got the last freed memory. (Assuming a common implementation of a free list where the next allocated memory is the one most recently freed.) Well... speculation... :)

> unlikely to occur in user data that, if detected by free()

Yes, those are human-identifiable as well but I don't think it's related in this case because they look random values. (Speculation: A string's .ptr value. :) )

Ali

On Monday, 17 January 2022 at 00:54:19 UTC, forkit wrote:
> ..


module test;

import std;

@safe void main()
{
    // mArr1 is a dynamic array allocated on the gc heap.
    int[][] mArr1 = [[1, 2], [3, 4], [5, 6], [7, 8]];
    static assert(is(typeof(mArr1.array())));
    alias R1 = typeof(mArr1);
    static assert(isRandomAccessRange!R1 &&
                  isForwardRange!R1 &&
                  isInputRange!R1 &&
                  isBidirectionalRange!R1 &&
                  hasAssignableElements!R1 &&
                  !isInfinite!R1 &&
                  hasSlicing!R1);

    // mArr2 is a dynamic array allocated on the gc heap
    // although compiling with '-profile=gc' incorrectly suggests otherwise.
    int[][] mArr2 = iota(1, 9).chunks(2).map!array.array;
    static assert(is(typeof(mArr2.array())));
    alias R2 = typeof(mArr2);
    static assert(isRandomAccessRange!R2 &&
                  isForwardRange!R2 &&
                  isInputRange!R2 &&
                  isBidirectionalRange!R2 &&
                  hasAssignableElements!R2 &&
                  !isInfinite!R2 &&
                  hasSlicing!R2);

    static assert(is(typeof(mArr1) == typeof(mArr2)));

}

/+

if add @nogc to above code:
(8): Error: array literal in `@nogc` function `D main` may cause a GC allocation
(22): Error: `@nogc` function `D main` cannot call non-@nogc function `std.array.array!(MapResult!(array, Chunks!(Result))).array`

+/

On Monday, 17 January 2022 at 03:11:50 UTC, Steven Schveighoffer wrote:
>
> The profile=gc appears to only show GC allocations that the *compiler* initiates (i.e. via `new`, array operations (like appending) or closure allocations). It does not detect that the functions that actually allocate memory themselves (such as `core.memory.GC.malloc`) are GC allocations. This actually does not care whether a function might or might not allocate, but records when it actually does allocate.
>
> -Steve

yes, that seems to be the case:

// -----
module test;

import std;

@safe void main()
{
    int[][] mArr2;
    mArr2 ~= iota(1, 9).chunks(2).map!array.array; // profilegc.log created ok.
}
// ----