Jump to page: 1 24  
Page
Thread overview
malloc error when trying to assign the returned pointer to a struct field
Sep 08, 2023
rempas
Sep 08, 2023
evilrat
Sep 08, 2023
rempas
Sep 08, 2023
evilrat
Sep 08, 2023
rempas
Sep 08, 2023
rempas
Sep 08, 2023
rempas
Sep 10, 2023
ryuukk_
Sep 08, 2023
Kagamin
Sep 08, 2023
rempas
Sep 08, 2023
Hipreme
Sep 08, 2023
rempas
Sep 08, 2023
bachmeier
Sep 08, 2023
rempas
Sep 08, 2023
Basile B.
Sep 08, 2023
rempas
Sep 08, 2023
H. S. Teoh
Sep 08, 2023
Basile B.
Sep 09, 2023
rempas
Sep 09, 2023
rempas
Sep 08, 2023
Basile B.
Sep 09, 2023
rempas
Sep 09, 2023
rempas
Sep 09, 2023
Brad Roberts
Sep 09, 2023
rempas
Sep 09, 2023
rempas
Sep 09, 2023
H. S. Teoh
Sep 09, 2023
rempas
Sep 09, 2023
rempas
Sep 09, 2023
bachmeier
Sep 09, 2023
rempas
September 08, 2023

I do have the following struct:

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }
}

That's some minimal code that I do have just to showcase it. So, some times, this work will works, some others, it will give me the following error:

Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed: (old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)

The problem seems to happen when the pointer that is returned from malloc is assigned to the _ptr field. If I just assign it to a variable and don't assign anything to _ptr, it will work!

Is there any possible that there is a compiler bug? I do use ldc2 and betterC!

September 08, 2023

On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:

>

I do have the following struct:

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }
}

That's some minimal code that I do have just to showcase it. So, some times, this work will works, some others, it will give me the following error:

Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed: (old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)

The problem seems to happen when the pointer that is returned from malloc is assigned to the _ptr field. If I just assign it to a variable and don't assign anything to _ptr, it will work!

Is there any possible that there is a compiler bug? I do use ldc2 and betterC!

Hard to tell from that code but it is quite unlikely there is a compiler bug in such simple use case.

I assume you already tried debugging your program, right?

So how about to diagnose a bit more, what if you enforce check before malloc that size>0, and second - from that example it is unclear how you are using that struct, so maybe add else statement static assert to see if it is misused somewhere else in your codebase?

Also this example doesn't have len field, depending on how you use with regard to cap this could be a source of problems too.

September 08, 2023

On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:

>

I do have the following struct:

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }
}

That's some minimal code that I do have just to showcase it. So, some times, this work will works, some others, it will give me the following error:

Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed: (old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)

The problem seems to happen when the pointer that is returned from malloc is assigned to the _ptr field. If I just assign it to a variable and don't assign anything to _ptr, it will work!

Is there any possible that there is a compiler bug? I do use ldc2 and betterC!

Hello, not completely unrelated to your problem, I have also done something like that, and when you're in D, don't use simply a pointer and length like that, use the slice operator.

See references:

https://tour.dlang.org/tour/en/basics/slices
https://dlang.org/spec/operatoroverloading.html#array-ops

For example, you can make your pointer a lot safer by doing:

size_t length = 5;
int* pointer = cast(int*)malloc(int.sizeof * length); //Don't
int[] mallocArray = (cast(int*)malloc(int.sizeof * length))[0..length]; //Do

On the second way, you'll get bound checks, thus, making it safer. Also, no need to keep track of your length separately anymore.

This is good practice in D language and you'll find yourself using this instead in the future.

And yes, this works in betterC, it is a simple runtime check, completely @nogc @safe nothrow and every other kind of thing you would want.

September 08, 2023

On Friday, 8 September 2023 at 09:07:12 UTC, evilrat wrote:

>

Hard to tell from that code but it is quite unlikely there is a compiler bug in such simple use case.

I assume you already tried debugging your program, right?

Yep! I have spent days and it's these kinds of bugs that burn me off and make me want to give up.

>

So how about to diagnose a bit more, what if you enforce check before malloc that size>0,

In the real code, there is a check! I just removed it from this example to make the code more clear.

>

and second - from that example it is unclear how you are using that struct, so maybe add else statement static assert to see if it is misused somewhere else in your codebase?

That's interesting, I wasn't able to find something else! The bug happens when I run the testing suit and well... the tests before pass so I cannot find anything that goes wrong except for the fact that I do not free the memory that is allocated (on purpose).

But still, why would the bug be there when I assign the _ptr on the field and it won't happen if I don't assign it and just use a regular local variable instead (and don't assign anything to the _ptr field)?

>

Also this example doesn't have len field, depending on how you use with regard to cap this could be a source of problems too.

Oh right, I forgot about that! The _len field normally exists and it's it's also an unsigned 64-bit (u64) number! Yeah, the code is a little bit changed from the original to make it more simple for you guys...

September 08, 2023

On Friday, 8 September 2023 at 09:25:59 UTC, Hipreme wrote:

>

Hello, not completely unrelated to your problem, I have also done something like that, and when you're in D, don't use simply a pointer and length like that, use the slice operator.

See references:

https://tour.dlang.org/tour/en/basics/slices
https://dlang.org/spec/operatoroverloading.html#array-ops

For example, you can make your pointer a lot safer by doing:

size_t length = 5;
int* pointer = cast(int*)malloc(int.sizeof * length); //Don't
int[] mallocArray = (cast(int*)malloc(int.sizeof * length))[0..length]; //Do

On the second way, you'll get bound checks, thus, making it safer. Also, no need to keep track of your length separately anymore.

This is good practice in D language and you'll find yourself using this instead in the future.

And yes, this works in betterC, it is a simple runtime check, completely @nogc @safe nothrow and every other kind of thing you would want.

Thank you for the reply! Using slices will give me the same result (my code has another field that I forgot to mention btw and it's _len (which holds the length that is used from the capacity). However, this will just make the API worse to use so I choose to not use slices in this case as using separate fields is more convenient.

September 08, 2023

On Friday, 8 September 2023 at 11:50:52 UTC, rempas wrote:

>

That's interesting, I wasn't able to find something else! The bug happens when I run the testing suit and well... the tests before pass so I cannot find anything that goes wrong except for the fact that I do not free the memory that is allocated (on purpose).

You run with -unittest compiler flag? Well, that does nothing for me with betterc (without it is ok).

I did stupid and unsafe things like malloc(0) and writing out of bounds but still no crash, it works fine.

I guess it depends on your libc, tested this on ubuntu 23.04 with gcc12 install and ldc 1.32.2

import core.stdc.stdlib;
import core.stdc.stdio;

alias u64 = ulong;
alias i64 = long;

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store
  u64 _len = 0;

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }

  ref T opIndex(size_t idx) { return _ptr[idx]; }
}

extern(C)
void main()
//unittest
{
    enum el = 3;
    auto vec = Vec!char(10);
    assert(vec._ptr);
    vec[el] = 'h';
    assert(vec[el] == 'h');
    printf("ptr = %p\n", vec._ptr);
    printf("vec ptr = %p\n", &vec[el]);
    printf("vec local = %p\n", &vec);
    printf("vec[%d] = %c\n", el, vec[el]);
    foreach (i; 0..vec._cap) {
      printf("-");
    }
    printf("\n");
    foreach (i; 0..vec._cap) {
      printf("%d", vec[i]);
    }
    printf("\n");
    printf("run ok\n");
}

ldc2 -betterC -run membug.d

output

ptr = 0x55cb701de2a0
vec ptr = 0x55cb701de2a3
vec local = 0x7fffa1542258
vec[3] = h
----------
000104000000
run ok
September 08, 2023

On Friday, 8 September 2023 at 07:59:37 UTC, rempas wrote:

>

I do have the following struct:

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }
}

That's some minimal code that I do have just to showcase it. So, some times, this work will works, some others, it will give me the following error:

Fatal glibc error: malloc.c:2594 (sysmalloc): assertion failed: (old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)

The problem seems to happen when the pointer that is returned from malloc is assigned to the _ptr field. If I just assign it to a variable and don't assign anything to _ptr, it will work!

Is there any possible that there is a compiler bug? I do use ldc2 and betterC!

I've had an error message like that before. This was the answer: https://stackoverflow.com/questions/46803671/sysmalloc-assertion

Without additional code it's hard to say if that's your problem.

September 08, 2023

On Friday, 8 September 2023 at 13:05:47 UTC, evilrat wrote:

>

You run with -unittest compiler flag? Well, that does nothing for me with betterc (without it is ok).

I did stupid and unsafe things like malloc(0) and writing out of bounds but still no crash, it works fine.

I guess it depends on your libc, tested this on ubuntu 23.04 with gcc12 install and ldc 1.32.2

import core.stdc.stdlib;
import core.stdc.stdio;

alias u64 = ulong;
alias i64 = long;

struct Vec(T) {
private:
  T* _ptr = null; // The pointer to the data
  u64 _cap = 0;   // Total amount of elements (not bytes) we can store
  u64 _len = 0;

public:
  /* Create a vector by just allocating memory for it. The null terminator is not set for
     strings as, the vector is considered empty and we should  first push something to it
     in order to use it! */
  this(i64 size) {
    this._len = 0;
    this._cap = size;

    static if (is(T == char)) { size += 1; } // Additional space for the null terminator
    this._ptr = cast(T*)malloc(size);
  }

  ref T opIndex(size_t idx) { return _ptr[idx]; }
}

extern(C)
void main()
//unittest
{
    enum el = 3;
    auto vec = Vec!char(10);
    assert(vec._ptr);
    vec[el] = 'h';
    assert(vec[el] == 'h');
    printf("ptr = %p\n", vec._ptr);
    printf("vec ptr = %p\n", &vec[el]);
    printf("vec local = %p\n", &vec);
    printf("vec[%d] = %c\n", el, vec[el]);
    foreach (i; 0..vec._cap) {
      printf("-");
    }
    printf("\n");
    foreach (i; 0..vec._cap) {
      printf("%d", vec[i]);
    }
    printf("\n");
    printf("run ok\n");
}

ldc2 -betterC -run membug.d

output

ptr = 0x55cb701de2a0
vec ptr = 0x55cb701de2a3
vec local = 0x7fffa1542258
vec[3] = h
----------
000104000000
run ok

As D's "uninttest" feature is disabled on BetterC, I have wrote my own testing suit (with is very simple). I just said that to point out that I'm testing this data structure (along side other things) and I cannot find anything wrong.

I have made a search on the web and I found out one thread that pointed out that it may be a Glibc error. However, because like I said the problem only happens when I assign the returned value to the _ptr field, I just wanted to post here in case someone has a similar experience and if it's a compiler bug in which case, we should report it.

I will use a VM to test my code in another environment and I'll be back to report!

September 09, 2023
In case you didn't know, all you need to get unittests working in -betterC is:

```d
			foreach (module_; allModules) {
				foreach (unitTest; __traits(getUnitTests, module_)) {
					unitTest();
				}
			}
```

You'd need to provide allModules somehow, like dub does.

https://github.com/dlang/dub/blob/2ea883833adf085095b07a7dba8250fb3db79a71/source/dub/project.d#L1937
September 08, 2023

On Friday, 8 September 2023 at 13:34:42 UTC, Richard (Rikki) Andrew Cattermole wrote:

>

In case you didn't know, all you need to get unittests working in -betterC is:

			foreach (module_; allModules) {
				foreach (unitTest; __traits(getUnitTests, module_)) {
					unitTest();
				}
			}

You'd need to provide allModules somehow, like dub does.

https://github.com/dlang/dub/blob/2ea883833adf085095b07a7dba8250fb3db79a71/source/dub/project.d#L1937

Oh wow!!! How did I missed that? It's either recent (past months) or I'm blind, lol!

As I don't remember unittests however, I would like to as. Do they have automatic symbol order resolution? Which is, testing symbols that other symbol depend on first? Or is it random?

« First   ‹ Prev
1 2 3 4