January 10, 2007
Sean Kelly wrote:

>> This seems to be a very big issue.  The GC seems to be pretty much useless right now if you're going to have a lot of floating point data in your app.
> 
> For what it's worth, I ran the test above with the modified GC in Tango, for 10000 iterations of the "while(1)" loop.  The default behavior roughly matched Phobos, with an 89 second run time and over 340MB of memory consumed and growing steadily.  Then I told the GC to not scan the arrays using the following calls:
> 
>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
> 
> A test with these changes in place dropped the run time to 7 seconds with 43MB of memory consumed and not growing.
> 
> I grant that this isn't quite as nice as if D just figured out whether to scan the block using TypeInfo, but at least it grants the programmer a way to customize GC behavior somewhat to tune application performance.  The only stipulation with the current implementation is that block attributes will not be preserved if an array is resized.  This isn't terribly difficult to fix, but I haven't done so yet.

I'd certainly be willing to do that sort of thing in my code to make the problem go away in the short term.  In my case all of those setAttr calls would be hidden away in library code, so it wouldn't make life any harder for everyday use.

But it's not much use as a workaround until Tango is actually downloadable somewhere.  :-)

Once Tango is available, though, that would certainly be better than the alternative of rewriting my library to use malloc/free.  If that's going to be necessary then I think I'll just as soon go back to C++ where at least I can use smart pointers.  The integrated GC is the #1 reason I'm using D in the first place, and if I can't realistically use that then I might as well be using C++.

--bb
January 10, 2007
== Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
> Sean Kelly wrote:
> >> This seems to be a very big issue.  The GC seems to be pretty much useless right now if you're going to have a lot of floating point data in your app.
> >
> > For what it's worth, I ran the test above with the modified GC in Tango, for 10000 iterations of the "while(1)" loop.  The default behavior roughly matched Phobos, with an 89 second run time and over 340MB of memory consumed and growing steadily.  Then I told the GC to not scan the arrays using the following calls:
> >
> >     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
> >     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
> >
> > A test with these changes in place dropped the run time to 7 seconds with 43MB of memory consumed and not growing.
> >
> > I grant that this isn't quite as nice as if D just figured out whether
> > to scan the block using TypeInfo, but at least it grants the programmer
> > a way to customize GC behavior somewhat to tune application performance.
> >  The only stipulation with the current implementation is that block
> > attributes will not be preserved if an array is resized.  This isn't
> > terribly difficult to fix, but I haven't done so yet.
> I'd certainly be willing to do that sort of thing in my code to make the
> problem go away in the short term.  In my case all of those setAttr
> calls would be hidden away in library code, so it wouldn't make life any
> harder for everyday use.
> But it's not much use as a workaround until Tango is actually
> downloadable somewhere.  :-)
> Once Tango is available, though, that would certainly be better than the
> alternative of rewriting my library to use malloc/free.  If that's going
> to be necessary then I think I'll just as soon go back to C++ where at
> least I can use smart pointers.  The integrated GC is the #1 reason I'm
> using D in the first place, and if I can't realistically use that then I
> might as well be using C++.
> --bb

I feel the same way. Without GC, D is just C++ with a few more features.
January 10, 2007
%u wrote:
> == Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
>> Sean Kelly wrote:
>>>> This seems to be a very big issue.  The GC seems to be pretty much
>>>> useless right now if you're going to have a lot of floating point data
>>>> in your app.
>>> For what it's worth, I ran the test above with the modified GC in Tango,
>>> for 10000 iterations of the "while(1)" loop.  The default behavior
>>> roughly matched Phobos, with an 89 second run time and over 340MB of
>>> memory consumed and growing steadily.  Then I told the GC to not scan
>>> the arrays using the following calls:
>>>
>>>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
>>>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
>>>
>>> A test with these changes in place dropped the run time to 7 seconds
>>> with 43MB of memory consumed and not growing.
>>>
>>> I grant that this isn't quite as nice as if D just figured out whether
>>> to scan the block using TypeInfo, but at least it grants the programmer
>>> a way to customize GC behavior somewhat to tune application performance.
>>>  The only stipulation with the current implementation is that block
>>> attributes will not be preserved if an array is resized.  This isn't
>>> terribly difficult to fix, but I haven't done so yet.
>> I'd certainly be willing to do that sort of thing in my code to make the
>> problem go away in the short term.  In my case all of those setAttr
>> calls would be hidden away in library code, so it wouldn't make life any
>> harder for everyday use.
>> But it's not much use as a workaround until Tango is actually
>> downloadable somewhere.  :-)
>> Once Tango is available, though, that would certainly be better than the
>> alternative of rewriting my library to use malloc/free.  If that's going
>> to be necessary then I think I'll just as soon go back to C++ where at
>> least I can use smart pointers.  The integrated GC is the #1 reason I'm
>> using D in the first place, and if I can't realistically use that then I
>> might as well be using C++.
>> --bb
> 
> I feel the same way. Without GC, D is just C++ with a few more features.

I do not think that this is even remotely valid. Think of everything you arent doing in D. You arent hacking macros with the preprocessor, you arent declaring methods outside of classes, globally overloading operators... To say that D is C++ + GC + some other features is just plain ignorant.
January 10, 2007
== Quote from Kyle Furlong (kylefurlong@gmail.com)'s article
> %u wrote:
> > == Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
> >> Sean Kelly wrote:
> >>>> This seems to be a very big issue.  The GC seems to be pretty much useless right now if you're going to have a lot of floating point data in your app.
> >>> For what it's worth, I ran the test above with the modified GC in Tango, for 10000 iterations of the "while(1)" loop.  The default behavior roughly matched Phobos, with an 89 second run time and over 340MB of memory consumed and growing steadily.  Then I told the GC to not scan the arrays using the following calls:
> >>>
> >>>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
> >>>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
> >>>
> >>> A test with these changes in place dropped the run time to 7 seconds with 43MB of memory consumed and not growing.
> >>>
> >>> I grant that this isn't quite as nice as if D just figured out whether
> >>> to scan the block using TypeInfo, but at least it grants the programmer
> >>> a way to customize GC behavior somewhat to tune application performance.
> >>>  The only stipulation with the current implementation is that block
> >>> attributes will not be preserved if an array is resized.  This isn't
> >>> terribly difficult to fix, but I haven't done so yet.
> >> I'd certainly be willing to do that sort of thing in my code to make the
> >> problem go away in the short term.  In my case all of those setAttr
> >> calls would be hidden away in library code, so it wouldn't make life any
> >> harder for everyday use.
> >> But it's not much use as a workaround until Tango is actually
> >> downloadable somewhere.  :-)
> >> Once Tango is available, though, that would certainly be better than the
> >> alternative of rewriting my library to use malloc/free.  If that's going
> >> to be necessary then I think I'll just as soon go back to C++ where at
> >> least I can use smart pointers.  The integrated GC is the #1 reason I'm
> >> using D in the first place, and if I can't realistically use that then I
> >> might as well be using C++.
> >> --bb
> >
> > I feel the same way. Without GC, D is just C++ with a few more features.
> I do not think that this is even remotely valid. Think of everything you arent doing in D. You arent hacking macros with the preprocessor, you arent declaring methods outside of classes, globally overloading operators... To say that D is C++ + GC + some other features is just plain ignorant.

It's not ignorant from my viewpoint. You're assuming I care about the features you described.
January 10, 2007
Oskar Linde wrote:
> Oskar Linde wrote:
>> After having fought a while with D programs with runaway memory leaks, I've unfortunately had to come to the conclusion that the D GC is not ready for production use. The problem is what I'd call "spurious pointers". That is random data (strings, numbers, image data, audio or whatever) appearing to the GC to be full of pointers to all over the memory space.
> 
> I feel kind of bad for making it sound like this is a problem related specifically to the D garbage collector. It is rather a general and apparently well known problem of all conservative garbage collectors. The D garbage collector is still excellent for a large domain of problems.
> 
> Lots of people seem to be having similar problems though, so a better understanding of under what conditions a conservative garbage collector will and will not work seems to be called for.

This link may be relevant:

http://www.hpl.hp.com/techreports/2001/HPL-2001-251.html

"Bounding Space Usage of Conservative Garbage Collectors" - Hans J. Boehm


Sean
January 10, 2007
"%u" <duser@hotmail.com> wrote in message news:eo324t$2ui5$1@digitaldaemon.com...
>
> It's not ignorant from my viewpoint. You're assuming I care about the
> features you
> described.

A bit OT, but would you please get a newsreader so you can stop being called "%u"?


January 10, 2007
%u wrote:
> == Quote from Kyle Furlong (kylefurlong@gmail.com)'s article
>> %u wrote:
>>> == Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
>>>> Sean Kelly wrote:
>>>>>> This seems to be a very big issue.  The GC seems to be pretty much
>>>>>> useless right now if you're going to have a lot of floating point data
>>>>>> in your app.
>>>>> For what it's worth, I ran the test above with the modified GC in Tango,
>>>>> for 10000 iterations of the "while(1)" loop.  The default behavior
>>>>> roughly matched Phobos, with an 89 second run time and over 340MB of
>>>>> memory consumed and growing steadily.  Then I told the GC to not scan
>>>>> the arrays using the following calls:
>>>>>
>>>>>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
>>>>>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
>>>>>
>>>>> A test with these changes in place dropped the run time to 7 seconds
>>>>> with 43MB of memory consumed and not growing.
>>>>>
>>>>> I grant that this isn't quite as nice as if D just figured out whether
>>>>> to scan the block using TypeInfo, but at least it grants the programmer
>>>>> a way to customize GC behavior somewhat to tune application performance.
>>>>>  The only stipulation with the current implementation is that block
>>>>> attributes will not be preserved if an array is resized.  This isn't
>>>>> terribly difficult to fix, but I haven't done so yet.
>>>> I'd certainly be willing to do that sort of thing in my code to make the
>>>> problem go away in the short term.  In my case all of those setAttr
>>>> calls would be hidden away in library code, so it wouldn't make life any
>>>> harder for everyday use.
>>>> But it's not much use as a workaround until Tango is actually
>>>> downloadable somewhere.  :-)
>>>> Once Tango is available, though, that would certainly be better than the
>>>> alternative of rewriting my library to use malloc/free.  If that's going
>>>> to be necessary then I think I'll just as soon go back to C++ where at
>>>> least I can use smart pointers.  The integrated GC is the #1 reason I'm
>>>> using D in the first place, and if I can't realistically use that then I
>>>> might as well be using C++.
>>>> --bb
>>> I feel the same way. Without GC, D is just C++ with a few more features.
>> I do not think that this is even remotely valid. Think of everything you
>> arent doing in D. You arent hacking macros with the preprocessor, you
>> arent declaring methods outside of classes, globally overloading
>> operators... To say that D is C++ + GC + some other features is just
>> plain ignorant.
> 
> It's not ignorant from my viewpoint. You're assuming I care about the features you
> described.

What I'm trying to say is, that to say a language is just a sum of its features is incorrect. There are behaviors that arise from the interaction of features that can be good or bad. In D, I've found that these interactions are on the whole good. In C++, I've found that these interactions are on the whole bad.

I'm not trying to say your experiences with both are invalid, maybe you have been using them in a way in which the opposite is true. But for me, C++ as a whole doesn't feel right, and D as a whole does.
January 11, 2007
Kyle Furlong wrote:
> %u wrote:
>> == Quote from Kyle Furlong (kylefurlong@gmail.com)'s article
>>> %u wrote:
>>>> == Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
>>>>> Sean Kelly wrote:
>>>>>>> This seems to be a very big issue.  The GC seems to be pretty much
>>>>>>> useless right now if you're going to have a lot of floating point data
>>>>>>> in your app.
>>>>>> For what it's worth, I ran the test above with the modified GC in Tango,
>>>>>> for 10000 iterations of the "while(1)" loop.  The default behavior
>>>>>> roughly matched Phobos, with an 89 second run time and over 340MB of
>>>>>> memory consumed and growing steadily.  Then I told the GC to not scan
>>>>>> the arrays using the following calls:
>>>>>>
>>>>>>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
>>>>>>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
>>>>>>
>>>>>> A test with these changes in place dropped the run time to 7 seconds
>>>>>> with 43MB of memory consumed and not growing.
>>>>>>
>>>>>> I grant that this isn't quite as nice as if D just figured out whether
>>>>>> to scan the block using TypeInfo, but at least it grants the programmer
>>>>>> a way to customize GC behavior somewhat to tune application performance.
>>>>>>  The only stipulation with the current implementation is that block
>>>>>> attributes will not be preserved if an array is resized.  This isn't
>>>>>> terribly difficult to fix, but I haven't done so yet.
>>>>> I'd certainly be willing to do that sort of thing in my code to make the
>>>>> problem go away in the short term.  In my case all of those setAttr
>>>>> calls would be hidden away in library code, so it wouldn't make life any
>>>>> harder for everyday use.
>>>>> But it's not much use as a workaround until Tango is actually
>>>>> downloadable somewhere.  :-)
>>>>> Once Tango is available, though, that would certainly be better than the
>>>>> alternative of rewriting my library to use malloc/free.  If that's going
>>>>> to be necessary then I think I'll just as soon go back to C++ where at
>>>>> least I can use smart pointers.  The integrated GC is the #1 reason I'm
>>>>> using D in the first place, and if I can't realistically use that then I
>>>>> might as well be using C++.
>>>>> --bb
>>>> I feel the same way. Without GC, D is just C++ with a few more features.
>>> I do not think that this is even remotely valid. Think of everything you
>>> arent doing in D. You arent hacking macros with the preprocessor, you
>>> arent declaring methods outside of classes, globally overloading
>>> operators... To say that D is C++ + GC + some other features is just
>>> plain ignorant.
>>
>> It's not ignorant from my viewpoint. You're assuming I care about the features you
>> described.
> 
> What I'm trying to say is, that to say a language is just a sum of its features is incorrect. There are behaviors that arise from the interaction of features that can be good or bad. In D, I've found that these interactions are on the whole good. In C++, I've found that these interactions are on the whole bad.
> 
> I'm not trying to say your experiences with both are invalid, maybe you have been using them in a way in which the opposite is true. But for me, C++ as a whole doesn't feel right, and D as a whole does.

Yes, that's certainly a valid point.  But I think maybe you're taking Mr. Percent a little too literally.

All I was trying to say originally was that, to me, D without garbage collection -- despite all the other nice features -- is not compelling enough to abandon C++.

Also I think Mr. Percent and I would both agree that whether or not the language "feels right" is less important than being able to get our work done in a timely manner.  I'm interested in D primarily because I believe it will help me get things done faster.  Right now, I think with GC and all the other nice D features, for the kinds of things I want to do, D and C++ are just about neck-and-neck.  What C++ lacks vs D, it makes up for with great debuggers and tools, and gobs of free libraries.  Take away the GC from D, and what's left is simply not enough to overcome the advantages of C++'s debuggers, tools, and libraries *for the kinds of things I want to do*.  Your mileage may vary.

--bb
January 11, 2007
Bill Baxter wrote:
> Kyle Furlong wrote:
>> %u wrote:
>>> == Quote from Kyle Furlong (kylefurlong@gmail.com)'s article
>>>> %u wrote:
>>>>> == Quote from Bill Baxter (dnewsgroup@billbaxter.com)'s article
>>>>>> Sean Kelly wrote:
>>>>>>>> This seems to be a very big issue.  The GC seems to be pretty much
>>>>>>>> useless right now if you're going to have a lot of floating point data
>>>>>>>> in your app.
>>>>>>> For what it's worth, I ran the test above with the modified GC in Tango,
>>>>>>> for 10000 iterations of the "while(1)" loop.  The default behavior
>>>>>>> roughly matched Phobos, with an 89 second run time and over 340MB of
>>>>>>> memory consumed and growing steadily.  Then I told the GC to not scan
>>>>>>> the arrays using the following calls:
>>>>>>>
>>>>>>>     gc.setAttr( data.ptr, GC.BlkAttr.NO_SCAN );
>>>>>>>     gc.setAttr( incoming.ptr, GC.BlkAttr.NO_SCAN );
>>>>>>>
>>>>>>> A test with these changes in place dropped the run time to 7 seconds
>>>>>>> with 43MB of memory consumed and not growing.
>>>>>>>
>>>>>>> I grant that this isn't quite as nice as if D just figured out whether
>>>>>>> to scan the block using TypeInfo, but at least it grants the programmer
>>>>>>> a way to customize GC behavior somewhat to tune application performance.
>>>>>>>  The only stipulation with the current implementation is that block
>>>>>>> attributes will not be preserved if an array is resized.  This isn't
>>>>>>> terribly difficult to fix, but I haven't done so yet.
>>>>>> I'd certainly be willing to do that sort of thing in my code to make the
>>>>>> problem go away in the short term.  In my case all of those setAttr
>>>>>> calls would be hidden away in library code, so it wouldn't make life any
>>>>>> harder for everyday use.
>>>>>> But it's not much use as a workaround until Tango is actually
>>>>>> downloadable somewhere.  :-)
>>>>>> Once Tango is available, though, that would certainly be better than the
>>>>>> alternative of rewriting my library to use malloc/free.  If that's going
>>>>>> to be necessary then I think I'll just as soon go back to C++ where at
>>>>>> least I can use smart pointers.  The integrated GC is the #1 reason I'm
>>>>>> using D in the first place, and if I can't realistically use that then I
>>>>>> might as well be using C++.
>>>>>> --bb
>>>>> I feel the same way. Without GC, D is just C++ with a few more features.
>>>> I do not think that this is even remotely valid. Think of everything you
>>>> arent doing in D. You arent hacking macros with the preprocessor, you
>>>> arent declaring methods outside of classes, globally overloading
>>>> operators... To say that D is C++ + GC + some other features is just
>>>> plain ignorant.
>>>
>>> It's not ignorant from my viewpoint. You're assuming I care about the features you
>>> described.
>>
>> What I'm trying to say is, that to say a language is just a sum of its features is incorrect. There are behaviors that arise from the interaction of features that can be good or bad. In D, I've found that these interactions are on the whole good. In C++, I've found that these interactions are on the whole bad.
>>
>> I'm not trying to say your experiences with both are invalid, maybe you have been using them in a way in which the opposite is true. But for me, C++ as a whole doesn't feel right, and D as a whole does.
> 
> Yes, that's certainly a valid point.  But I think maybe you're taking Mr. Percent a little too literally.
> 
> All I was trying to say originally was that, to me, D without garbage collection -- despite all the other nice features -- is not compelling enough to abandon C++.
> 
> Also I think Mr. Percent and I would both agree that whether or not the language "feels right" is less important than being able to get our work done in a timely manner.  I'm interested in D primarily because I believe it will help me get things done faster.  Right now, I think with GC and all the other nice D features, for the kinds of things I want to do, D and C++ are just about neck-and-neck.  What C++ lacks vs D, it makes up for with great debuggers and tools, and gobs of free libraries.  Take away the GC from D, and what's left is simply not enough to overcome the advantages of C++'s debuggers, tools, and libraries *for the kinds of things I want to do*.  Your mileage may vary.
> 
> --bb

I agree. This is only good news for D, however, because libraries and tools come with time. D is still young.
January 11, 2007
The only comment I'd like to add is this:

At the same time as we're thinking about how to use a precise collector instead of a conservative collector, we should also think about how to implement a generational copying collector, since it might be necessary to add new constructs to the core language (pointer pinning, for example) to facilitate the new collector semantics.

--benji