November 05, 2009
Steven Schveighoffer wrote:
> On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> 
>>>  Most of the usages are like this:
>>>  ubyte[1024] buffer;
>>> functionThatNeedsBufferSpace(buffer);
>>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking an address of the local data.
>>>  So it's not explicit address taking, but it's the same thing under the hood.  There always exists the potential for the stack reference to escape.
>>
>> I see, thank you. I am confident that a trusted reap could be implemented in the standard library. (google reap)
> 
> I did.  Couldn't find anything.

Damn acronyms, sorry. Better results: reap memory allocation

ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf


Andrei
November 05, 2009
grauzone wrote:
> Ary Borenszweig wrote:
>> grauzone wrote:
>>> Frank Benoit wrote:
>>>> safe should be the default. The unsafe part should take the extra
>>>> typing, not the other way. Make the user prefer the safe way.
>>>
>>> No. D is not C#.
>>
>> D is an unsafe language.
>> C# is a safe language.
>>
>> Like that? :)
> 
> If you mean memory safety, then yes and will probably forever be for all practical uses (unless D gets implemented on a Java or .net like VM).

Oh how cool. So it turns out that SafeD can be 100% implemented on a safe VM. It's great to give a well-defined target to potential implementers.

Andrei
November 05, 2009
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> Steven Schveighoffer wrote:
> > On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> >
> >>>  Most of the usages are like this:
> >>>  ubyte[1024] buffer;
> >>> functionThatNeedsBufferSpace(buffer);
> >>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking
> >>> an address of the local data.
> >>>  So it's not explicit address taking, but it's the same thing under
> >>> the hood.  There always exists the potential for the stack reference
> >>> to escape.
> >>
> >> I see, thank you. I am confident that a trusted reap could be implemented in the standard library. (google reap)
> >
> > I did.  Couldn't find anything.
> Damn acronyms, sorry. Better results: reap memory allocation ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf Andrei

Ok, I understand the basic principle of a reap, but if it's going to convert to a heap when you try to delete something, why not just improve the standard GC heap, i.e. by making per-thread heaps?  If you're not going to delete stuff, why not just use a regular old region or stack (not necessarily the call stack, but a stack of some kind)?
November 05, 2009
dsimcha wrote:
> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
>> Steven Schveighoffer wrote:
>>> On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu
>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>
>>>>>  Most of the usages are like this:
>>>>>  ubyte[1024] buffer;
>>>>> functionThatNeedsBufferSpace(buffer);
>>>>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking
>>>>> an address of the local data.
>>>>>  So it's not explicit address taking, but it's the same thing under
>>>>> the hood.  There always exists the potential for the stack reference
>>>>> to escape.
>>>> I see, thank you. I am confident that a trusted reap could be
>>>> implemented in the standard library. (google reap)
>>> I did.  Couldn't find anything.
>> Damn acronyms, sorry. Better results: reap memory allocation
>> ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf
>> Andrei
> 
> Ok, I understand the basic principle of a reap, but if it's going to convert to a
> heap when you try to delete something, why not just improve the standard GC heap,
> i.e. by making per-thread heaps?  If you're not going to delete stuff, why not
> just use a regular old region or stack (not necessarily the call stack, but a
> stack of some kind)?

Perhaps a region could also be defined as a @trusted facility! So much good stuff to do, so little time...

Andrei
November 05, 2009
Andrei Alexandrescu wrote:
> grauzone wrote:
>> Ary Borenszweig wrote:
>>> grauzone wrote:
>>>> Frank Benoit wrote:
>>>>> safe should be the default. The unsafe part should take the extra
>>>>> typing, not the other way. Make the user prefer the safe way.
>>>>
>>>> No. D is not C#.
>>>
>>> D is an unsafe language.
>>> C# is a safe language.
>>>
>>> Like that? :)
>>
>> If you mean memory safety, then yes and will probably forever be for all practical uses (unless D gets implemented on a Java or .net like VM).
> 
> Oh how cool. So it turns out that SafeD can be 100% implemented on a safe VM. It's great to give a well-defined target to potential implementers.

I'm not sure. On Java you probably had trouble emulating ref params/returns, and on .net, you can use pointers anyway. By the way, I remember someone saying T[new] would be good for implementing arrays/slices on .net, but it got dumped (I find this funny and absurd in this context).

> Andrei
November 05, 2009
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> >> Steven Schveighoffer wrote:
> >>> On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> >>>
> >>>>>  Most of the usages are like this:
> >>>>>  ubyte[1024] buffer;
> >>>>> functionThatNeedsBufferSpace(buffer);
> >>>>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking
> >>>>> an address of the local data.
> >>>>>  So it's not explicit address taking, but it's the same thing under
> >>>>> the hood.  There always exists the potential for the stack reference
> >>>>> to escape.
> >>>> I see, thank you. I am confident that a trusted reap could be implemented in the standard library. (google reap)
> >>> I did.  Couldn't find anything.
> >> Damn acronyms, sorry. Better results: reap memory allocation ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf Andrei
> >
> > Ok, I understand the basic principle of a reap, but if it's going to convert to a heap when you try to delete something, why not just improve the standard GC heap, i.e. by making per-thread heaps?  If you're not going to delete stuff, why not just use a regular old region or stack (not necessarily the call stack, but a stack of some kind)?
> Perhaps a region could also be defined as a @trusted facility! So much
> good stuff to do, so little time...
> Andrei

I have my doubts only b/c I've worked on similar things before and the thing that keeps biting me in the @$$ is how to handle the GC.  If you allocate some huge block of memory to parcel out and have it all be scanned by the GC, you're asking for slow scan times and lots of false pointers, thus largely defeating the purpose of the region scheme.  (My precise heap scanning patch doesn't address this, as it assumes that untyped memory blocks will be allocated using the conservative bit mask and scanned conservatively, or not scanned at all.)

If you don't have it scanned by the GC, then you can't store the only reference to a GC-allocated object in the region.  I chose the latter for TempAlloc, and it's still ridiculously useful in the small niche of dstats, where I need to allocate tons of temporary arrays of numbers or copies of arrays that are already in memory but need to be sorted, etc.  However, it's a shoe with a gun built in.  If it wasn't, I'd recommend it for Phobos.  If I made it scanned by the GC performance and space overhead would likely be so bad that it would be useless.
November 05, 2009
dsimcha wrote:
> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
>> dsimcha wrote:
>>> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
>>>> Steven Schveighoffer wrote:
>>>>> On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu
>>>>> <SeeWebsiteForEmail@erdani.org> wrote:
>>>>>
>>>>>>>  Most of the usages are like this:
>>>>>>>  ubyte[1024] buffer;
>>>>>>> functionThatNeedsBufferSpace(buffer);
>>>>>>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking
>>>>>>> an address of the local data.
>>>>>>>  So it's not explicit address taking, but it's the same thing under
>>>>>>> the hood.  There always exists the potential for the stack reference
>>>>>>> to escape.
>>>>>> I see, thank you. I am confident that a trusted reap could be
>>>>>> implemented in the standard library. (google reap)
>>>>> I did.  Couldn't find anything.
>>>> Damn acronyms, sorry. Better results: reap memory allocation
>>>> ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf
>>>> Andrei
>>> Ok, I understand the basic principle of a reap, but if it's going to convert to a
>>> heap when you try to delete something, why not just improve the standard GC heap,
>>> i.e. by making per-thread heaps?  If you're not going to delete stuff, why not
>>> just use a regular old region or stack (not necessarily the call stack, but a
>>> stack of some kind)?
>> Perhaps a region could also be defined as a @trusted facility! So much
>> good stuff to do, so little time...
>> Andrei
> 
> I have my doubts only b/c I've worked on similar things before and the thing that
> keeps biting me in the @$$ is how to handle the GC.  If you allocate some huge
> block of memory to parcel out and have it all be scanned by the GC, you're asking
> for slow scan times and lots of false pointers, thus largely defeating the purpose
> of the region scheme.

Well I'm thinking that often when you use a region, the memory consumption is not really large. If it's really large, then you may be better off just using the GC because it means you do a lot of stuff. But I'm sure I'm ignoring a few important applications.


Andrei
November 05, 2009
dsimcha wrote:
> Ok, I understand the basic principle of a reap, but if it's going to convert to a
> heap when you try to delete something, why not just improve the standard GC heap,
> i.e. by making per-thread heaps?

The problem with per-thread heaps is immutable data can be passed between threads.
November 05, 2009
Andrei Alexandrescu wrote:
> Nick Sabalausky wrote:
>> Sounds great! The lower-grained safeness makes a lot of sense, and I'm thrilled at the idea of safe D finally encompassing more than just memory safety - I'd been hoping to see that happen ever since I first heard that "safeD" only ment memory-safe. 
> 
> I can think of division by zero as an example. What others are out there?

Casting away const/immutable/shared.
November 05, 2009
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> >> dsimcha wrote:
> >>> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> >>>> Steven Schveighoffer wrote:
> >>>>> On Thu, 05 Nov 2009 16:30:42 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
> >>>>>
> >>>>>>>  Most of the usages are like this:
> >>>>>>>  ubyte[1024] buffer;
> >>>>>>> functionThatNeedsBufferSpace(buffer);
> >>>>>>>  where functionThatNeedsBufferSpace takes a ubyte[], thereby taking
> >>>>>>> an address of the local data.
> >>>>>>>  So it's not explicit address taking, but it's the same thing under
> >>>>>>> the hood.  There always exists the potential for the stack reference
> >>>>>>> to escape.
> >>>>>> I see, thank you. I am confident that a trusted reap could be implemented in the standard library. (google reap)
> >>>>> I did.  Couldn't find anything.
> >>>> Damn acronyms, sorry. Better results: reap memory allocation ftp://ftp.cs.utexas.edu/pub/emery/papers/reconsidering-custom.pdf Andrei
> >>> Ok, I understand the basic principle of a reap, but if it's going to convert
to a
> >>> heap when you try to delete something, why not just improve the standard GC
heap,
> >>> i.e. by making per-thread heaps?  If you're not going to delete stuff, why not just use a regular old region or stack (not necessarily the call stack, but a stack of some kind)?
> >> Perhaps a region could also be defined as a @trusted facility! So much
> >> good stuff to do, so little time...
> >> Andrei
> >
> > I have my doubts only b/c I've worked on similar things before and the thing that keeps biting me in the @$$ is how to handle the GC.  If you allocate some huge block of memory to parcel out and have it all be scanned by the GC, you're asking for slow scan times and lots of false pointers, thus largely defeating the purpose of the region scheme.
> Well I'm thinking that often when you use a region, the memory
> consumption is not really large. If it's really large, then you may be
> better off just using the GC because it means you do a lot of stuff. But
> I'm sure I'm ignoring a few important applications.
> Andrei

By not really large, how big are we talking?  Less than a few 10s of KB?  If so, I think just having the whole thing scanned would be feasible.