View mode: basic / threaded / horizontal-split · Log in · Help
February 15, 2009
ref?
Walter and I have been discussing what the regime of statically-sized 
arrays should be. In short, the behavior that's most consistent with 
everything else is to treat them as values.

This also brings the problems of e.g. containers - should they have 
consistent value semantics (like in STL) or consistent reference 
semantics (like in Java)? Or both, subject to a policy parameter?

At any rate, it looks like the days of cheap by-value copying are over. 
Even today, large structs become onerous to pass by value. If copy 
constructors start being used, we can't simply assume that copying 
things around is not a problem.

Today we use:

void fun(T)(T value) { ... }

or

void fun(T)(ref T value) { ... }

to make a choice in parameter passing. But neither is perfect. The 
former copies too much, and the latter does not accept rvalues. C++ has 
found a solution to this problem in the guise of references to const 
objects. Those can accept lvalues and rvalues alike. It looked like a 
good solution at the time, but it's caused huge issues in other areas of 
the language, which ultimately led to a complicated feature called 
rvalue references. So I'd like to veer a different way.

I was thinking of something like:

void fun(T)(ref? T value) { ... }

This really generates two overloads:

void fun(T)(ref T value) { ... }
void fun(T)(T value) { return (cast(void (ref T)) fun)(value); }

meaning that there are two fun instantiations, one actually doing the 
work, and the other just forwarding rvalues to it.

There's one little extra thing that the feature does. A function may 
want to return its incoming argument, in which case it might say:

ref? T fun(T)(ref? T value) { ... return value; }

in which case the generated code is:

ref T fun(T)(ref T value) { ... }
T fun(T)(T value) { return (cast(void (ref T)) fun)(value); }

I wanted to discuss this to gather more ideas and insights on the topic.


Andrei
February 15, 2009
Re: ref?
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> Walter and I have been discussing what the regime of statically-sized
> arrays should be. In short, the behavior that's most consistent with
> everything else is to treat them as values.
> This also brings the problems of e.g. containers - should they have
> consistent value semantics (like in STL)

Oh God no!  I always thought the value semantics of STL containers were largely a
kludge to work around the fact that C++ doesn't have garbage collection.  They
make it easier to use RAII for memory management, since every object has a clear
owner.  This leads to tons and tons of copying if code is written the obvious way,
and lots of kludges to prevent it.

> or consistent reference
> semantics (like in Java)?

Yes.  Largely to be consistent with builtin dynamic arrays and AAs, and for
efficiency, to avoid hidden copying.  I consider the builtin static arrays to be
just a niche performance hack anyhow, and hardly ever use them because they're too
inflexible, so consistency with them isn't as important.  If you want to duplicate
something before passing it to something else, attaching a simple .dup to the end
as in foo(myArray.dup) works pretty darn well.  All the standard containers should
simply have a .dup method, which works similarly to the builtin dynamic array .dup
method.
February 15, 2009
Re: ref?
dsimcha wrote:
> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
>> Walter and I have been discussing what the regime of statically-sized
>> arrays should be. In short, the behavior that's most consistent with
>> everything else is to treat them as values.
>> This also brings the problems of e.g. containers - should they have
>> consistent value semantics (like in STL)
> 
> Oh God no!  I always thought the value semantics of STL containers were largely a
> kludge to work around the fact that C++ doesn't have garbage collection.  They
> make it easier to use RAII for memory management, since every object has a clear
> owner.  This leads to tons and tons of copying if code is written the obvious way,
> and lots of kludges to prevent it.

Well I never seemed to have a problem with it, quite the contrary in 
fact. Values are easier to reason about because they don't have aliasing 
and don't foster long-distance interdependencies.

>> or consistent reference
>> semantics (like in Java)?
> 
> Yes.  Largely to be consistent with builtin dynamic arrays and AAs, and for
> efficiency, to avoid hidden copying.

Today's dynamic arrays and AAs are really ranges. This becomes most (and 
most embarrassingly) visible when expanding a dynamic array with ~= 
comes into discussion.

> I consider the builtin static arrays to be
> just a niche performance hack anyhow, and hardly ever use them because they're too
> inflexible, so consistency with them isn't as important.  If you want to duplicate
> something before passing it to something else, attaching a simple .dup to the end
> as in foo(myArray.dup) works pretty darn well.  All the standard containers should
> simply have a .dup method, which works similarly to the builtin dynamic array .dup
> method.

Attaching a .dup doesn't always work that pretty darn well because 
sometimes you really want .deepdup without realizing it. The .dup 
thingie copies only one layer of the onion. To bring this more into 
perspective, threading with STM does necessitate guarantees of full, 
alias-free copies of entire graphs of objects.

So I guess I disagree with most of your post :o).


Andrei
February 15, 2009
Re: ref?
dsimcha Wrote:

> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> > Walter and I have been discussing what the regime of statically-sized
> > arrays should be. In short, the behavior that's most consistent with
> > everything else is to treat them as values.
> > This also brings the problems of e.g. containers - should they have
> > consistent value semantics (like in STL)
> 
> Oh God no!  I always thought the value semantics of STL containers were largely a
> kludge to work around the fact that C++ doesn't have garbage collection.  They
> make it easier to use RAII for memory management, since every object has a clear
> owner.  This leads to tons and tons of copying if code is written the obvious way,
> and lots of kludges to prevent it.

On the other hand, for relatively small objects, the last thing I want is to have everything be a reference.  This is one of the major flaws of Java, I think, from a memory efficiency standpoint.  I'd hate to have to have references to 16 byte objects if I don't need the sharing behavior.

Having classes with reference semantics in D largely solves this issue, doesn't it?  Use a class when you have things that make sense to share, and use structs for when it makes more sense to have distinct copies.

Jerry
February 15, 2009
Re: ref?
Jerry Quinn wrote:
> dsimcha Wrote:
> 
>> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s
>> article
>>> Walter and I have been discussing what the regime of
>>> statically-sized arrays should be. In short, the behavior that's
>>> most consistent with everything else is to treat them as values. 
>>> This also brings the problems of e.g. containers - should they
>>> have consistent value semantics (like in STL)
>> Oh God no!  I always thought the value semantics of STL containers
>> were largely a kludge to work around the fact that C++ doesn't have
>> garbage collection.  They make it easier to use RAII for memory
>> management, since every object has a clear owner.  This leads to
>> tons and tons of copying if code is written the obvious way, and
>> lots of kludges to prevent it.
> 
> On the other hand, for relatively small objects, the last thing I
> want is to have everything be a reference.  This is one of the major
> flaws of Java, I think, from a memory efficiency standpoint.  I'd
> hate to have to have references to 16 byte objects if I don't need
> the sharing behavior.
> 
> Having classes with reference semantics in D largely solves this
> issue, doesn't it?  Use a class when you have things that make sense
> to share, and use structs for when it makes more sense to have
> distinct copies.

That implies all structs should have copy semantics. I'm not sure 
whether this is good or bad, I'm just saying :o).

Andrei
February 15, 2009
Re: ref?
Andrei Alexandrescu wrote:
> Well I never seemed to have a problem with it, quite the contrary in
> fact. Values are easier to reason about because they don't have aliasing
> and don't foster long-distance interdependencies.

This.

When the container is used in a struct, then it must not only be copied,
but deep-copied when the outer struct is copied.  The same applies when
the container is used in a class that has a cloning operation defined.

When the container is used in a class that cannot be cloned or is
directly placed on the stack, then it doesn't really matter if the
container has reference or value semantics, since the variable that
holds the container isn't being copied anyway.

For those very rare cases where you /want/ the container to be shared,
you can always use a pointer to the container.


-- 
Rainer Deyke - rainerd@eldwood.com
February 15, 2009
Re: ref?
Andrei Alexandrescu wrote:

> Walter and I have been discussing what the regime of statically-sized
> arrays should be. In short, the behavior that's most consistent with
> everything else is to treat them as values.

I'm curious, what use cases did you discuss?  I don't know of any functions where a statically sized array is passed in, so the whole ref vs. value has no meaning for the cases I'm aware of.



> This also brings the problems of e.g. containers - should they have
> consistent value semantics (like in STL) or consistent reference
> semantics (like in Java)? Or both, subject to a policy parameter?

I don't know about others, but I always pass around STL containers as either a reference or a const reference.  C++ is designed to allow everything be passed by value, but with little guarantee of what will happen.  It's up to the programmer to know what they're doing.  At least reference semantics are consistent...



> At any rate, it looks like the days of cheap by-value copying are over.
> Even today, large structs become onerous to pass by value. If copy
> constructors start being used, we can't simply assume that copying
> things around is not a problem.
> 
> Today we use:
> 
> void fun(T)(T value) { ... }
> 
> or
> 
> void fun(T)(ref T value) { ... }

Don't forget void fun(T)(in T value) { ... }


> to make a choice in parameter passing. But neither is perfect. The
> former copies too much, and the latter does not accept rvalues. C++ has
> found a solution to this problem in the guise of references to const
> objects. Those can accept lvalues and rvalues alike. It looked like a
> good solution at the time, but it's caused huge issues in other areas of
> the language, which ultimately led to a complicated feature called
> rvalue references. So I'd like to veer a different way.

I'd love to see some concrete examples of where that went wrong for C++.  If not, it's tough to judge what benefit alternatives provide.


> I was thinking of something like:
> 
> void fun(T)(ref? T value) { ... }
> 
> This really generates two overloads:
> 
> void fun(T)(ref T value) { ... }
> void fun(T)(T value) { return (cast(void (ref T)) fun)(value); }


Based on those two overloads, it appears the first can modify value, but the effect on the caller will still depend on if T is either a value or reference type.  This style of strange behavior seems really dangerous to me.
February 15, 2009
Re: ref?
Rainer Deyke wrote:
> Andrei Alexandrescu wrote:
>> Well I never seemed to have a problem with it, quite the contrary in
>> fact. Values are easier to reason about because they don't have aliasing
>> and don't foster long-distance interdependencies.
> 
> This.
> 
> When the container is used in a struct, then it must not only be copied,
> but deep-copied when the outer struct is copied.  The same applies when
> the container is used in a class that has a cloning operation defined.
> 
> When the container is used in a class that cannot be cloned or is
> directly placed on the stack, then it doesn't really matter if the
> container has reference or value semantics, since the variable that
> holds the container isn't being copied anyway.
> 
> For those very rare cases where you /want/ the container to be shared,
> you can always use a pointer to the container.

Spot on. My ambitions are actually a tad higher. I want to implement 
containers as by-value structs defining value semantics and the needed 
primitives. Then, using introspection, I want to define a template Class 
that takes a struct and turns it into a class. For example:

struct SomeContainer(T)
{
    ref T opIndex(size_t n) { ... }
}

auto byval = SomeContainer!(int)();
auto javalike = new Class!(SomeContainer!(int));

The type Class!(SomeContainer!(int)) will store a SomeContainer!(int) 
object and will define (by using introspection) a method opIndex of 
which implementation will do what the struct's method does. The net 
effect is as if the programmer sat down with SomeContainer and changed 
"struct" to "class". Of course without the code duplication and the 
maintenance nightmare :o).

Class should work for primitive types, e.g. Class!(int) should do what 
Integer does in Java and so on.


Andrei
February 15, 2009
Re: ref?
Andrei Alexandrescu wrote:
> Spot on. My ambitions are actually a tad higher. I want to implement 
> containers as by-value structs defining value semantics and the needed 
> primitives. Then, using introspection, I want to define a template Class 
> that takes a struct and turns it into a class. For example:
> 
> struct SomeContainer(T)
> {
>     ref T opIndex(size_t n) { ... }
> }
> 
> auto byval = SomeContainer!(int)();
> auto javalike = new Class!(SomeContainer!(int));
> 
> The type Class!(SomeContainer!(int)) will store a SomeContainer!(int) 
> object and will define (by using introspection) a method opIndex of 
> which implementation will do what the struct's method does. The net 
> effect is as if the programmer sat down with SomeContainer and changed 
> "struct" to "class". Of course without the code duplication and the 
> maintenance nightmare :o).
> 
> Class should work for primitive types, e.g. Class!(int) should do what 
> Integer does in Java and so on.

An interesting idea. Would Class!(T) allow polymorphism, or would all 
methods be implicitly final?
Would it also delegate template methods?

And what would happen to any T methods accepting or returning T? Would 
they keep returning T or would they be replaced by Class!(T)? Or perhaps 
both, through overloading?
Same question for types derived from T (e.g. Nullable!(T), or 
SomeContainer!(T))?

For instance in the case of a container it would be nice to allow 
Class!(LinkedList!(T)).splice!(Class!(LinkedList!(T)) other) if there's 
a LinkedList!(T).splice(LinkedList(T) other).

If such unwrapping were done, how would such methods react if the 
Class!(LinkedList!(T)) passed into slice() were null?
February 15, 2009
Re: ref?
Frits van Bommel wrote:
> Andrei Alexandrescu wrote:
>> Spot on. My ambitions are actually a tad higher. I want to
>> implement containers as by-value structs defining value semantics
>> and the needed primitives. Then, using introspection, I want to
>> define a template Class that takes a struct and turns it into a
>> class. For example:
>> 
>> struct SomeContainer(T) { ref T opIndex(size_t n) { ... } }
>> 
>> auto byval = SomeContainer!(int)(); auto javalike = new
>> Class!(SomeContainer!(int));
>> 
>> The type Class!(SomeContainer!(int)) will store a
>> SomeContainer!(int) object and will define (by using introspection)
>> a method opIndex of which implementation will do what the struct's
>> method does. The net effect is as if the programmer sat down with
>> SomeContainer and changed "struct" to "class". Of course without
>> the code duplication and the maintenance nightmare :o).
>> 
>> Class should work for primitive types, e.g. Class!(int) should do
>> what Integer does in Java and so on.
> 
> An interesting idea. Would Class!(T) allow polymorphism, or would all
>  methods be implicitly final?

Polymorphic. Then of course there's the Finalize!(C) template that takes
a class C and makes all of its methods final. Compile-time introspection
is a treasure trove.

> Would it also delegate template methods?

Good question. I don't know how template methods could be handled.

> And what would happen to any T methods accepting or returning T?
> Would they keep returning T or would they be replaced by Class!(T)?
> Or perhaps both, through overloading? Same question for types derived
> from T (e.g. Nullable!(T), or SomeContainer!(T))?

I don't know. We need more experience to figure that out.

> For instance in the case of a container it would be nice to allow 
> Class!(LinkedList!(T)).splice!(Class!(LinkedList!(T)) other) if
> there's a LinkedList!(T).splice(LinkedList(T) other).
> 
> If such unwrapping were done, how would such methods react if the 
> Class!(LinkedList!(T)) passed into slice() were null?

Segfault :o).


Andrei
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home