Thread overview | ||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
October 07, 2014 On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
I made a proposal to quantatively measure and tabulate all GC allocations in Phobos before coming up with solutions to "@nogc Phobos". After approving node from Andrei I've come up with a piece of automation to extract this data and post it on wiki. So here is the exhustive list of everything calling into GC in Phobos (-vgc compiler flag): http://wiki.dlang.org/Stuff_in_Phobos_That_Generates_Garbage Including source links, a wild guess at function's name and the compiler's warning message for potential GC call. As far as data goes this is about as good as we can get, the next phase is labeling this stuff with potential solution(s). Again doing all by hand is tedious and hardly useful. Instead we need to observe patterns and label it automatically until the non-trivial subset remains. So everybody, please take time and identify simple patterns and post back your ideas on solution(s). So far I see the most frequent cases: - `new SomeException` - switch to RC exceptions - AA access - ??? (use user-defined AA type as parameter?) - array concat - ??? - closure - ??? --- Dmitry Olshansky |
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | 1.) It may be helpful to reduce the noise in that every match after a new is ignored (and probaly multiple 'operator ~' alarms within the same statement). 2.) There seems to be a problem with repeated alarms: When viewing the page source, this link shows up numerous times. See https://github.com/D-Programming-Language//phobos/blob/d4d98124ab6cbef7097025a7cfd1161d1963c87e/std/conv.d#L688 /Gerhard |
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to grm | On Tuesday, 7 October 2014 at 16:23:19 UTC, grm wrote:
> 2.) There seems to be a problem with repeated alarms:
> When viewing the page source, this link shows up numerous times. See
> https://github.com/D-Programming-Language//phobos/blob/d4d98124ab6cbef7097025a7cfd1161d1963c87e/std/conv.d#L688
That's because of multiple template instantiations of the same function. These should probably be filtered for this use case.
|
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | On 10/7/2014 8:57 AM, Dmitry Olshansky wrote:
> I made a proposal to quantatively measure and tabulate all GC allocations in
> Phobos before coming up with solutions to "@nogc Phobos".
>
> After approving node from Andrei I've come up with a piece of automation to
> extract this data and post it on wiki.
Thanks, Dmitri, this is great work. I suggest at a minimum that all of those get notes added to their documentation that they gc allocate.
|
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Walter Bright | On Tuesday, 7 October 2014 at 19:24:34 UTC, Walter Bright wrote:
> Thanks, Dmitri, this is great work. I suggest at a minimum that all of those get notes added to their documentation that they gc allocate.
Seems like that's something that should just be automated where possible instead of trying to update the documentation of hundreds of functions.
|
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | On 2014-10-07 17:57, Dmitry Olshansky wrote: > I made a proposal to quantatively measure and tabulate all GC > allocations in Phobos before coming up with solutions to "@nogc Phobos". > > After approving node from Andrei I've come up with a piece of automation > to extract this data and post it on wiki. > > So here is the exhustive list of everything calling into GC in Phobos > (-vgc compiler flag): > > http://wiki.dlang.org/Stuff_in_Phobos_That_Generates_Garbage > > Including source links, a wild guess at function's name and the > compiler's warning message for potential GC call. > > As far as data goes this is about as good as we can get, the next phase > is labeling this stuff with potential solution(s). Again doing all by > hand is tedious and hardly useful. > > Instead we need to observe patterns and label it automatically until the > non-trivial subset remains. So everybody, please take time and identify > simple patterns and post back your ideas on solution(s). > > So far I see the most frequent cases: > - `new SomeException` - switch to RC exceptions > - AA access - ??? (use user-defined AA type as parameter?) > - array concat - ??? > - closure - ??? I did some processing of the data and this is the results I got: 772 | 'new' causes GC allocation 515 | operator ~= may cause GC allocation 380 | operator ~ may cause GC allocation 113 | array literal may cause GC allocation 90 | setting 'length' may cause GC allocation 77 | indexing an associative array may cause GC allocation 34 | using closure causes GC allocation 16 | 'delete' requires GC 5 | associative array literal may cause GC allocation Total 9 I didn't look at any source code to see what "new" is actually allocating, for example. -- /Jacob Carlborg |
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Jacob Carlborg | On Tuesday, 7 October 2014 at 20:13:32 UTC, Jacob Carlborg wrote:
> I didn't look at any source code to see what "new" is actually allocating, for example.
I did some random sampling, and it's 90% exceptions, with the occasional array allocation.
I noticed that a lot of the ~ and ~= complaints are in code that only ever runs at compile time (generating strings for mixin). I wonder if there's any way we can silence these false positives.
|
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Brad Anderson | On 10/7/14, 12:31 PM, Brad Anderson wrote:
> On Tuesday, 7 October 2014 at 19:24:34 UTC, Walter Bright wrote:
>> Thanks, Dmitri, this is great work. I suggest at a minimum that all of
>> those get notes added to their documentation that they gc allocate.
>
> Seems like that's something that should just be automated where possible
> instead of trying to update the documentation of hundreds of functions.
Could ddoc do that? -- Andrei
|
October 07, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | On 10/7/14, 8:57 AM, Dmitry Olshansky wrote:
> I made a proposal to quantatively measure and tabulate all GC
> allocations in Phobos before coming up with solutions to "@nogc Phobos".
>
> After approving node from Andrei I've come up with a piece of automation
> to extract this data and post it on wiki.
>
> So here is the exhustive list of everything calling into GC in Phobos
> (-vgc compiler flag):
>
> http://wiki.dlang.org/Stuff_in_Phobos_That_Generates_Garbage
Awesome! I've started adding explanations to the first few entries, let's use that crowdsourcing thing to fill this! -- Andrei
|
October 08, 2014 Re: On Phobos GC hunt | ||||
---|---|---|---|---|
| ||||
Posted in reply to Dmitry Olshansky | On Tuesday, 7 October 2014 at 15:57:59 UTC, Dmitry Olshansky wrote:
> So here is the exhustive list of everything calling into GC in Phobos (-vgc compiler flag):
>
> http://wiki.dlang.org/Stuff_in_Phobos_That_Generates_Garbage
A correction on TimSortImpl, it does actually generate garbage by calling uninitializedArray to allocate the buffer. The check for __ctfe is unnecessary (it may have been needed sometime ago or was added naively).
Timsort is an O(n/2) algorithm and requires a buffer, but there's no reason for it to be GC-allocated. It could simply be malloc'd and free'd before the function returns.
|
Copyright © 1999-2021 by the D Language Foundation