This is a tool + article I wrote in February, but never got around to finishing / publishing until today.

https://blog.thecybershadow.net/2018/02/07/dmdprof/

Hopefully someone will find it useful.

On Tue, Nov 06, 2018 at 06:00:22PM +0000, Vladimir Panteleev via Digitalmars-d-announce wrote:
> This is a tool + article I wrote in February, but never got around to finishing / publishing until today.
> 
> https://blog.thecybershadow.net/2018/02/07/dmdprof/
> 
> Hopefully someone will find it useful.

I don't have the time to look into this right now, but at a cursory glance, WOW.  This is awesome!  It looks like it would be really useful one day when I try to tackle the dmd-on-lowmem-system problem again. This will greatly help identify which Phobos modules cause big slowdowns on a low-memory host system.


T

-- 
Recently, our IT department hired a bug-fix engineer. He used to work for Volkswagen.

On Tuesday, 6 November 2018 at 19:01:58 UTC, H. S. Teoh wrote:
> I don't have the time to look into this right now, but at a cursory glance, WOW.  This is awesome!  It looks like it would be really useful one day when I try to tackle the dmd-on-lowmem-system problem again. This will greatly help identify which Phobos modules cause big slowdowns on a low-memory host system.

Great. Also, assuming that DMD still never deallocates, it should be pretty easy to turn this into a memory usage profiler by measuring the delta between the resident set size in samples.

On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev wrote:
> This is a tool + article I wrote in February, but never got around to finishing / publishing until today.
>
> https://blog.thecybershadow.net/2018/02/07/dmdprof/
>
> Hopefully someone will find it useful.

Awesome, great work!

I really really hate waiting for the compiler.

On Tue, Nov 06, 2018 at 07:44:41PM +0000, Atila Neves via Digitalmars-d-announce wrote:
> On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev wrote:
> > This is a tool + article I wrote in February, but never got around to finishing / publishing until today.
> > 
> > https://blog.thecybershadow.net/2018/02/07/dmdprof/
> > 
> > Hopefully someone will find it useful.
> 
> Awesome, great work!
> 
> I really really hate waiting for the compiler.

OTOH, I really really hate that the compiler, in the name of faster compilation, eats up all available RAM and gets OOM-killed on a low memory system, so no amount of waiting will get me an executable.


T

-- 
Famous last words: I wonder what will happen if I do *this*...

On Tuesday, 6 November 2018 at 21:52:30 UTC, H. S. Teoh wrote:
> On Tue, Nov 06, 2018 at 07:44:41PM +0000, Atila Neves via Digitalmars-d-announce wrote:
>> On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev wrote:
>> > This is a tool + article I wrote in February, but never got around to finishing / publishing until today.
>> > 
>> > https://blog.thecybershadow.net/2018/02/07/dmdprof/
>> > 
>> > Hopefully someone will find it useful.
>> 
>> Awesome, great work!
>> 
>> I really really hate waiting for the compiler.
>
> OTOH, I really really hate that the compiler, in the name of faster compilation, eats up all available RAM and gets OOM-killed on a low memory system, so no amount of waiting will get me an executable.
>
Now that the compiler is completely in D, wouldn't it be a good idea to activate the GC in the compiler. I know that it requires some care for bootstrapping the compiler when there are dependencies to the D runtime, but the compiler would be an excellent example of the advantage of the GC (i.e. dumb fast allocations as long as there's memory, collection when no memory left which is miles away better than to get OOM-killed).

On 08/11/2018 1:46 AM, Patrick Schluter wrote:
> Now that the compiler is completely in D, wouldn't it be a good idea to activate the GC in the compiler. I know that it requires some care for bootstrapping the compiler when there are dependencies to the D runtime, but the compiler would be an excellent example of the advantage of the GC (i.e. dumb fast allocations as long as there's memory, collection when no memory left which is miles away better than to get OOM-killed).

No, that would be a bad idea currently.

https://issues.dlang.org/show_bug.cgi?id=18811

On Thu, 08 Nov 2018 01:49:49 +1300, rikki cattermole wrote:
> On 08/11/2018 1:46 AM, Patrick Schluter wrote:
>> Now that the compiler is completely in D, wouldn't it be a good idea to
>> activate the GC in the compiler. I know that it requires some care for
>> bootstrapping the compiler when there are dependencies to the D
>> runtime,
>> but the compiler would be an excellent example of the advantage of the
>> GC (i.e. dumb fast allocations as long as there's memory, collection
>> when no memory left which is miles away better than to get OOM-killed).
> 
> No, that would be a bad idea currently.
> 
> https://issues.dlang.org/show_bug.cgi?id=18811

That issue describes a memory leak. A memory leak of half a megabyte per full compiler invocation on a small file should still result in a lot of code being compilable on low-end machines that currently isn't.

On 08/11/2018 4:17 AM, Neia Neutuladh wrote:
> On Thu, 08 Nov 2018 01:49:49 +1300, rikki cattermole wrote:
>> On 08/11/2018 1:46 AM, Patrick Schluter wrote:
>>> Now that the compiler is completely in D, wouldn't it be a good idea to
>>> activate the GC in the compiler. I know that it requires some care for
>>> bootstrapping the compiler when there are dependencies to the D
>>> runtime,
>>> but the compiler would be an excellent example of the advantage of the
>>> GC (i.e. dumb fast allocations as long as there's memory, collection
>>> when no memory left which is miles away better than to get OOM-killed).
>>
>> No, that would be a bad idea currently.
>>
>> https://issues.dlang.org/show_bug.cgi?id=18811
> 
> That issue describes a memory leak. A memory leak of half a megabyte per
> full compiler invocation on a small file should still result in a lot of
> code being compilable on low-end machines that currently isn't.

Its a symptom of a larger set of problems. The frontend is not quite ready to have the GC turned on full time.

Based upon my testing, that little memory leak prevents pretty much *all* memory allocated by the GC to not be collected. I don't know why, but for some reason it pins it. Mind you, my testing could have been flawed, and needs more eyes on it *shrug*.

On Thu, 08 Nov 2018 14:35:29 +1300, rikki cattermole wrote:
> Its a symptom of a larger set of problems. The frontend is not quite ready to have the GC turned on full time.
> 
> Based upon my testing, that little memory leak prevents pretty much *all* memory allocated by the GC to not be collected. I don't know why, but for some reason it pins it. Mind you, my testing could have been flawed, and needs more eyes on it *shrug*.

In the short term, that means turning on the GC won't do much, but it at least probably won't hurt.

In the long term, that bug might be covering up crashes and memory corruption, so turning it on by default is not a great idea.