View mode: basic / threaded / horizontal-split · Log in · Help
November 05, 2009
Regions and Heaps and Reaps, Oh my (Was: Safety, undefined behavior, @safe, @trusted )
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> dsimcha wrote:
> > == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> >> Well I'm thinking that often when you use a region, the memory
> >> consumption is not really large. If it's really large, then you may be
> >> better off just using the GC because it means you do a lot of stuff. But
> >> I'm sure I'm ignoring a few important applications.
> >> Andrei
> >
> > By not really large, how big are we talking?  Less than a few 10s of KB?  If so, I
> > think just having the whole thing scanned would be feasible.
> Honest, I don't know. Only real-life usage might tell. At any rate, a
> few 10s of KBs would definitely work for many of my own applications.
> Andrei

Ok, now we're getting somewhere.  I guess if enough ppl find a few 10s of k
useful, we could just make GC scanning and block size configurable.  Maybe through
some static if's we could make it only @trusted if it's scanned by the GC.  I'm
hoping to make a region template that can go into Phobos, and, in some
instantiation, can satisfy just about everyone, and then make TempAlloc simply an
instantiation of this template for my own personal use.  Here are some questions
about how this should work:

1.  What should happen if you try to allocate more space than you have in the
region?  Should it silently fall back to heap allocation?  Should it throw an
exception?  Should it return null?  Should it silently allocate another region block?

2.  Should the region also allow freeing memory in last in, first out order,
behaving somewhat as a stack?  The advantage to doing so would be to increase
flexibility.  The downside is that you would have to do a little bookkeeping
internally, and the overhead of this might be too high for the use case of lots of
tiny allocations.  Maybe this should also be a policy.  Also, if it's a region + a
stack, can we call it a "rack"?

3.  Should it be designed as a normal object (more flexible but less convenient)
or a thread-local singleton that lazily initializes itself and is just there, kind
of like malloc (more convenient but less flexible)?

4.  Any other generic comments on how this should be designed?
November 05, 2009
Re: Safety, undefined behavior, @safe, @trusted
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> Walter Bright wrote:
> > Jason House wrote:
> >> I posted in the other thread how casting to immutable/shared can be
> >> just as bad. A leaked reference prior to casting to immutable/shared
> >> is in effect the same as casting away shared. No matter how you mix
> >> thread local and shared, or mutable and immutable, you still have the
> >> same undefined behavior
> >
> > Not undefined, it's just that the compiler can't prove it's defined
> > behavior. Hence, such code would go into a trusted function.
> Are we in agreement that @safe functions have bounds checking on
> regardless of -release?
> Andrei

I'd vote for this.  I've wanted, for a while, a way to have finer-grained control
over bounds checking anyhow.  In non-performance-critical pieces of code it seems
like a no-brainer to leave it on all the time, just to be safe.  In
performance-critical code, it's a no-brainer that it has to be turned off after
debugging.

Right now I almost never use bounds checking except when I already know I have a
bug and am trying to find it because it's just too slow.  I'd love to have it as a
safety net in the 90+% of my code that isn't performance-critical.
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Steven Schveighoffer wrote:
> On Thu, 05 Nov 2009 17:49:33 -0500, Walter Bright 
> <newshound1@digitalmars.com> wrote:
> 
>> Jason House wrote:
>>> I posted in the other thread how casting to immutable/shared can be
>>> just as bad. A leaked reference prior to casting to immutable/shared
>>> is in effect the same as casting away shared. No matter how you mix
>>> thread local and shared, or mutable and immutable, you still have the
>>> same undefined behavior
>>
>> Not undefined, it's just that the compiler can't prove it's defined 
>> behavior. Hence, such code would go into a trusted function.
> 
> But how does such a trusted function guarantee that the invariant/shared 
> reference has no other aliases?

It doesn't. Trusted code is verified by the programmer, not the compiler.


> The point is, there is no way to write 
> such a function in good faith because you can't guarantee it's actually 
> safe, it's still up to the user of the function.  My understanding is 
> that a @trusted function should be provably safe even if the compiler 
> can't prove it.
> 
> -Steve
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Steven Schveighoffer wrote:
> On Thu, 05 Nov 2009 17:49:33 -0500, Walter Bright 
> <newshound1@digitalmars.com> wrote:
> 
>> Jason House wrote:
>>> I posted in the other thread how casting to immutable/shared can be
>>> just as bad. A leaked reference prior to casting to immutable/shared
>>> is in effect the same as casting away shared. No matter how you mix
>>> thread local and shared, or mutable and immutable, you still have the
>>> same undefined behavior
>>
>> Not undefined, it's just that the compiler can't prove it's defined 
>> behavior. Hence, such code would go into a trusted function.
> 
> But how does such a trusted function guarantee that the invariant/shared 
> reference has no other aliases?

It doesn't. Trusted code is verified by the programmer, not the compiler.


> The point is, there is no way to write 
> such a function in good faith because you can't guarantee it's actually 
> safe, it's still up to the user of the function.  My understanding is 
> that a @trusted function should be provably safe even if the compiler 
> can't prove it.
> 
> -Steve
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Andrei Alexandrescu wrote:
> Walter Bright wrote:
>> Jason House wrote:
>>> I posted in the other thread how casting to immutable/shared can be
>>> just as bad. A leaked reference prior to casting to immutable/shared
>>> is in effect the same as casting away shared. No matter how you mix
>>> thread local and shared, or mutable and immutable, you still have the
>>> same undefined behavior
>>
>> Not undefined, it's just that the compiler can't prove it's defined 
>> behavior. Hence, such code would go into a trusted function.
> 
> Are we in agreement that @safe functions have bounds checking on 
> regardless of -release?

You're right from a theoretical perspective, but not from a practical 
one. People ought to be able to flip on 'safe' without large performance 
penalties.

If it came with inescapable large performance penalties, then it'll get 
a bad rap and people will be reluctant to use it, defeating its purpose.
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Michel Fortin wrote:
> On 2009-11-05 17:11:38 -0500, Walter Bright <newshound1@digitalmars.com> 
> said:
> 
>> dsimcha wrote:
>>> Ok, I understand the basic principle of a reap, but if it's going to 
>>> convert to a
>>> heap when you try to delete something, why not just improve the 
>>> standard GC heap,
>>> i.e. by making per-thread heaps?
>>
>> The problem with per-thread heaps is immutable data can be passed 
>> between threads.
> 
> Well, if that's a problem you could fix it by making immutable not 
> shared unless you also put the shared attribute:
> 
>     immutable Object o;        // thread-local
>     shared immutable Object o; // visible from all threads


Aaggghhhh !!! <g>

> 
> I think having per-thread heaps is a worthy goal.
>
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
On Thu, 05 Nov 2009 19:11:34 -0500, Walter Bright  
<newshound1@digitalmars.com> wrote:

> Steven Schveighoffer wrote:
>> On Thu, 05 Nov 2009 17:49:33 -0500, Walter Bright  
>> <newshound1@digitalmars.com> wrote:
>>
>>> Jason House wrote:
>>>> I posted in the other thread how casting to immutable/shared can be
>>>> just as bad. A leaked reference prior to casting to immutable/shared
>>>> is in effect the same as casting away shared. No matter how you mix
>>>> thread local and shared, or mutable and immutable, you still have the
>>>> same undefined behavior
>>>
>>> Not undefined, it's just that the compiler can't prove it's defined  
>>> behavior. Hence, such code would go into a trusted function.
>>  But how does such a trusted function guarantee that the  
>> invariant/shared reference has no other aliases?
>
> It doesn't. Trusted code is verified by the programmer, not the compiler.

OK, you totally ignored my point though.  How do you write such a function?

That is, I have a mutable reference x, I want to make it immutable.  How  
do you write a function to do that?

i.e.:

@safe void foo()
{
   x = new X();
   x.modifyState(5);
   immutable(X) ix = ???; // how to write this part
}

-Steve
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Andrei Alexandrescu wrote:
> Oh how cool. So it turns out that SafeD can be 100% implemented on a
> safe VM.

This is also true of regular unsafe C.


-- 
Rainer Deyke - rainerd@eldwood.com
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
On 2009-11-05 19:14:47 -0500, Walter Bright <newshound1@digitalmars.com> said:

> Andrei Alexandrescu wrote:
>> Are we in agreement that @safe functions have bounds checking on 
>> regardless of -release?
> 
> You're right from a theoretical perspective, but not from a practical 
> one. People ought to be able to flip on 'safe' without large 
> performance penalties.
> 
> If it came with inescapable large performance penalties, then it'll get 
> a bad rap and people will be reluctant to use it, defeating its purpose.

But if you remove bound checking, it isn't safe anymore, is it?

Sometime safety is more important than performance. If I needed 
performance in a safe program, I'd profile and find the bottlenecks, 
review carefully those parts of the code slowing down the program, then 
when I trust them perfectly I'd add the @trusted attribute. @trusted 
should remove bound checks (in release mode). @safe should keep them to 
keep other less trustworthy pieces of of the program truly safe.

That said, I'd be in favor of a compiler switch to enable/disable 
runtime checks in release mode... perhaps "-safe" could return as way 
to generate truly safe binaries even in release mode. This would also 
make it pretty easy to evaluate how much impact those runtime checks 
have on final executable (by turning on and off the compiler switch).

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
November 06, 2009
Re: Safety, undefined behavior, @safe, @trusted
Steven Schveighoffer wrote:
> That is, I have a mutable reference x, I want to make it immutable.  How 
> do you write a function to do that?
> 
> i.e.:
> 
> @safe void foo()
> {
>    x = new X();
>    x.modifyState(5);
>    immutable(X) ix = ???; // how to write this part
> }

If you, the writer of foo(), know that there are no other mutable 
references to x you can cast it to immutable - but you'll have to mark 
the function as @trusted.
2 3 4 5 6 7 8 9 10
Top | Discussion index | About this forum | D home