Menu
Search

Forums

Forum Index

September 29, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Andrey Zherikov		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Andrey Zherikov


Reply

On 9/29/22 9:57 AM, Andrey Zherikov wrote:

>

On Thursday, 29 September 2022 at 03:19:08 UTC, Steven Schveighoffer wrote:

>

On 9/28/22 9:01 PM, Andrey Zherikov wrote:

>

On Thursday, 29 September 2022 at 00:48:03 UTC, Andrey Zherikov wrote:

>

I actually see that DMD is very unstable.

This seems like it might be an out of memory problem. When the compiler runs out of memory, any of a number of things might happen, including segfaults.

Why doesn't it fail with "out of memory exception"? I remember it did in previous versions (I use 2.100.2 now).

Classic problem for memory allocation is nobody checks if it succeeds, or if they do, they don't handle it properly. This likely includes various places in the compiler.

-Steve

October 02, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
On 9/28/2022 8:19 PM, Steven Schveighoffer wrote:
> This seems like it *might be* an out of memory problem. When the compiler runs out of memory, any of a number of things might happen, including segfaults.

All memory allocations in DMD are checked for returning null, and if so, the program exits with a fatal error.

If you find any that do not, please post to bugzilla and let me know.
October 02, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
On 9/29/2022 7:50 AM, Steven Schveighoffer wrote:
> Classic problem for memory allocation is nobody checks if it succeeds, or if they do, they don't handle it properly. This likely includes various places in the compiler.

If you can find one, I owe you a beer for the next BeerConf.
October 03, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Walter Bright		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Walter Bright


Reply

On 10/2/22 5:12 PM, Walter Bright wrote:

>

On 9/29/2022 7:50 AM, Steven Schveighoffer wrote:

>

Classic problem for memory allocation is nobody checks if it succeeds, or if they do, they don't handle it properly. This likely includes various places in the compiler.

If you can find one, I owe you a beer for the next BeerConf.

haha, now I need to go look for one!

In all seriousness though, I've definitely had 4 types of results when DMD runs out of memory:

  1. OOM Killer kills the process (this is the most frequent on Linux)
  2. Segfault
  3. Seemingly random assert error or other exception.
  4. Out of memory notification (this is what should always happen, except for OOM killer)

Not saying any of this is proof the compiler (or used library) incorrectly ignores a failed memory allocation, but it would fit the pattern.

-Steve

October 03, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
On 10/3/2022 7:45 AM, Steven Schveighoffer wrote:
> In all seriousness though, I've definitely had 4 types of results when DMD runs out of memory:
> 
> 1. OOM Killer kills the process (this is the most frequent on Linux)
> 2. Segfault
> 3. Seemingly random assert error or other exception.
> 4. Out of memory notification (this is what should always happen, except for OOM killer)

2 and 3 should be investigated and fixed.
October 03, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
grep reveals:

https://github.com/dlang/dmd/pull/14517
October 03, 2022
Gravatar of Steven SchveighofferPosted by Steven Schveighoffer
in reply to Walter Bright		Reply
Steven Schveighoffer
Gravatar of Steven Schveighoffer
Posted in reply to Walter Bright


Reply

On 10/3/22 10:13 PM, Walter Bright wrote:

>

grep reveals:

https://github.com/dlang/dmd/pull/14517

Does prodding you to check count as me finding it? ;)

🍺

-Steve

October 03, 2022
Gravatar of Walter BrightPosted by Walter Bright
in reply to Steven Schveighoffer		Reply
Walter Bright
Gravatar of Walter Bright
Posted in reply to Steven Schveighoffer


Reply
On 10/3/2022 7:17 PM, Steven Schveighoffer wrote:
> Does prodding you to check count as me finding it? ;)
> 
> 🍺
> 
> -Steve

For that you get a virtual beer!

🍺
October 04, 2022
Gravatar of DennisPosted by Dennis
in reply to Walter Bright		Reply
Dennis
Gravatar of Dennis
Posted in reply to Walter Bright


Reply

On Tuesday, 4 October 2022 at 02:13:16 UTC, Walter Bright wrote:

>

grep reveals:

https://github.com/dlang/dmd/pull/14517

I have a few more :)

The backend has 4 calls to util_malloc, 28 calls to mem_malloc, 2 calls to mem_malloc2, 8 calls to mem_fmalloc (why so many malloc wrappers?), and 16 calls to mem_calloc. I don't see any checks on those.

Here's one with plain malloc:

dmd/backend/dcgcv.d:264

d = cast(debtyp_t *) malloc(debtyp_t.sizeof - (d.data).sizeof + length);
d.length = cast(ushort)length;

This one checks, but after memcpy'ing to null!

dmd/common/file.d:148

name = cast(char*) memcpy(malloc(totalNameLength), filename, totalNameLength);
name || assert(0, "FileMapping: Out of memory.");
October 04, 2022
Gravatar of ryuukk_Posted by ryuukk_
in reply to Dennis		Reply
ryuukk_
Gravatar of ryuukk_
Posted in reply to Dennis


Reply

On Tuesday, 4 October 2022 at 10:10:35 UTC, Dennis wrote:

>

On Tuesday, 4 October 2022 at 02:13:16 UTC, Walter Bright wrote:

>

grep reveals:

https://github.com/dlang/dmd/pull/14517

I have a few more :)

The backend has 4 calls to util_malloc, 28 calls to mem_malloc, 2 calls to mem_malloc2, 8 calls to mem_fmalloc (why so many malloc wrappers?), and 16 calls to mem_calloc. I don't see any checks on those.

Here's one with plain malloc:

dmd/backend/dcgcv.d:264

d = cast(debtyp_t *) malloc(debtyp_t.sizeof - (d.data).sizeof + length);
d.length = cast(ushort)length;

This one checks, but after memcpy'ing to null!

dmd/common/file.d:148

name = cast(char*) memcpy(malloc(totalNameLength), filename, totalNameLength);
name || assert(0, "FileMapping: Out of memory.");

Another case for a proper allocator API in the std!

« First   ‹ Prev
1 2 3
Next ›   Last »