February 15, 2009
Walter and I have been discussing what the regime of statically-sized arrays should be. In short, the behavior that's most consistent with everything else is to treat them as values.

This also brings the problems of e.g. containers - should they have consistent value semantics (like in STL) or consistent reference semantics (like in Java)? Or both, subject to a policy parameter?

At any rate, it looks like the days of cheap by-value copying are over. Even today, large structs become onerous to pass by value. If copy constructors start being used, we can't simply assume that copying things around is not a problem.

Today we use:

void fun(T)(T value) { ... }

or

void fun(T)(ref T value) { ... }

to make a choice in parameter passing. But neither is perfect. The former copies too much, and the latter does not accept rvalues. C++ has found a solution to this problem in the guise of references to const objects. Those can accept lvalues and rvalues alike. It looked like a good solution at the time, but it's caused huge issues in other areas of the language, which ultimately led to a complicated feature called rvalue references. So I'd like to veer a different way.

I was thinking of something like:

void fun(T)(ref? T value) { ... }

This really generates two overloads:

void fun(T)(ref T value) { ... }
void fun(T)(T value) { return (cast(void (ref T)) fun)(value); }

meaning that there are two fun instantiations, one actually doing the work, and the other just forwarding rvalues to it.

There's one little extra thing that the feature does. A function may want to return its incoming argument, in which case it might say:

ref? T fun(T)(ref? T value) { ... return value; }

in which case the generated code is:

ref T fun(T)(ref T value) { ... }
T fun(T)(T value) { return (cast(void (ref T)) fun)(value); }

I wanted to discuss this to gather more ideas and insights on the topic.


Andrei
February 15, 2009
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> Walter and I have been discussing what the regime of statically-sized
> arrays should be. In short, the behavior that's most consistent with
> everything else is to treat them as values.
> This also brings the problems of e.g. containers - should they have
> consistent value semantics (like in STL)

Oh God no!  I always thought the value semantics of STL containers were largely a kludge to work around the fact that C++ doesn't have garbage collection.  They make it easier to use RAII for memory management, since every object has a clear owner.  This leads to tons and tons of copying if code is written the obvious way, and lots of kludges to prevent it.

> or consistent reference
> semantics (like in Java)?

Yes.  Largely to be consistent with builtin dynamic arrays and AAs, and for efficiency, to avoid hidden copying.  I consider the builtin static arrays to be just a niche performance hack anyhow, and hardly ever use them because they're too inflexible, so consistency with them isn't as important.  If you want to duplicate something before passing it to something else, attaching a simple .dup to the end as in foo(myArray.dup) works pretty darn well.  All the standard containers should simply have a .dup method, which works similarly to the builtin dynamic array .dup method.
February 15, 2009
dsimcha wrote:
> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
>> Walter and I have been discussing what the regime of statically-sized
>> arrays should be. In short, the behavior that's most consistent with
>> everything else is to treat them as values.
>> This also brings the problems of e.g. containers - should they have
>> consistent value semantics (like in STL)
> 
> Oh God no!  I always thought the value semantics of STL containers were largely a
> kludge to work around the fact that C++ doesn't have garbage collection.  They
> make it easier to use RAII for memory management, since every object has a clear
> owner.  This leads to tons and tons of copying if code is written the obvious way,
> and lots of kludges to prevent it.

Well I never seemed to have a problem with it, quite the contrary in fact. Values are easier to reason about because they don't have aliasing and don't foster long-distance interdependencies.

>> or consistent reference
>> semantics (like in Java)?
> 
> Yes.  Largely to be consistent with builtin dynamic arrays and AAs, and for
> efficiency, to avoid hidden copying.

Today's dynamic arrays and AAs are really ranges. This becomes most (and most embarrassingly) visible when expanding a dynamic array with ~= comes into discussion.

> I consider the builtin static arrays to be
> just a niche performance hack anyhow, and hardly ever use them because they're too
> inflexible, so consistency with them isn't as important.  If you want to duplicate
> something before passing it to something else, attaching a simple .dup to the end
> as in foo(myArray.dup) works pretty darn well.  All the standard containers should
> simply have a .dup method, which works similarly to the builtin dynamic array .dup
> method.

Attaching a .dup doesn't always work that pretty darn well because sometimes you really want .deepdup without realizing it. The .dup thingie copies only one layer of the onion. To bring this more into perspective, threading with STM does necessitate guarantees of full, alias-free copies of entire graphs of objects.

So I guess I disagree with most of your post :o).


Andrei
February 15, 2009
dsimcha Wrote:

> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s article
> > Walter and I have been discussing what the regime of statically-sized
> > arrays should be. In short, the behavior that's most consistent with
> > everything else is to treat them as values.
> > This also brings the problems of e.g. containers - should they have
> > consistent value semantics (like in STL)
> 
> Oh God no!  I always thought the value semantics of STL containers were largely a kludge to work around the fact that C++ doesn't have garbage collection.  They make it easier to use RAII for memory management, since every object has a clear owner.  This leads to tons and tons of copying if code is written the obvious way, and lots of kludges to prevent it.

On the other hand, for relatively small objects, the last thing I want is to have everything be a reference.  This is one of the major flaws of Java, I think, from a memory efficiency standpoint.  I'd hate to have to have references to 16 byte objects if I don't need the sharing behavior.

Having classes with reference semantics in D largely solves this issue, doesn't it?  Use a class when you have things that make sense to share, and use structs for when it makes more sense to have distinct copies.

Jerry

February 15, 2009
Jerry Quinn wrote:
> dsimcha Wrote:
> 
>> == Quote from Andrei Alexandrescu (SeeWebsiteForEmail@erdani.org)'s
>> article
>>> Walter and I have been discussing what the regime of
>>> statically-sized arrays should be. In short, the behavior that's
>>> most consistent with everything else is to treat them as values. This also brings the problems of e.g. containers - should they
>>> have consistent value semantics (like in STL)
>> Oh God no!  I always thought the value semantics of STL containers
>> were largely a kludge to work around the fact that C++ doesn't have
>> garbage collection.  They make it easier to use RAII for memory
>> management, since every object has a clear owner.  This leads to
>> tons and tons of copying if code is written the obvious way, and
>> lots of kludges to prevent it.
> 
> On the other hand, for relatively small objects, the last thing I
> want is to have everything be a reference.  This is one of the major
> flaws of Java, I think, from a memory efficiency standpoint.  I'd
> hate to have to have references to 16 byte objects if I don't need
> the sharing behavior.
> 
> Having classes with reference semantics in D largely solves this
> issue, doesn't it?  Use a class when you have things that make sense
> to share, and use structs for when it makes more sense to have
> distinct copies.

That implies all structs should have copy semantics. I'm not sure whether this is good or bad, I'm just saying :o).

Andrei
February 15, 2009
Andrei Alexandrescu wrote:
> Well I never seemed to have a problem with it, quite the contrary in fact. Values are easier to reason about because they don't have aliasing and don't foster long-distance interdependencies.

This.

When the container is used in a struct, then it must not only be copied, but deep-copied when the outer struct is copied.  The same applies when the container is used in a class that has a cloning operation defined.

When the container is used in a class that cannot be cloned or is directly placed on the stack, then it doesn't really matter if the container has reference or value semantics, since the variable that holds the container isn't being copied anyway.

For those very rare cases where you /want/ the container to be shared, you can always use a pointer to the container.


-- 
Rainer Deyke - rainerd@eldwood.com
February 15, 2009
Andrei Alexandrescu wrote:

> Walter and I have been discussing what the regime of statically-sized arrays should be. In short, the behavior that's most consistent with everything else is to treat them as values.

I'm curious, what use cases did you discuss?  I don't know of any functions where a statically sized array is passed in, so the whole ref vs. value has no meaning for the cases I'm aware of.



> This also brings the problems of e.g. containers - should they have
> consistent value semantics (like in STL) or consistent reference
> semantics (like in Java)? Or both, subject to a policy parameter?

I don't know about others, but I always pass around STL containers as either a reference or a const reference.  C++ is designed to allow everything be passed by value, but with little guarantee of what will happen.  It's up to the programmer to know what they're doing.  At least reference semantics are consistent...



> At any rate, it looks like the days of cheap by-value copying are over. Even today, large structs become onerous to pass by value. If copy constructors start being used, we can't simply assume that copying things around is not a problem.
> 
> Today we use:
> 
> void fun(T)(T value) { ... }
> 
> or
> 
> void fun(T)(ref T value) { ... }

Don't forget void fun(T)(in T value) { ... }


> to make a choice in parameter passing. But neither is perfect. The former copies too much, and the latter does not accept rvalues. C++ has found a solution to this problem in the guise of references to const objects. Those can accept lvalues and rvalues alike. It looked like a good solution at the time, but it's caused huge issues in other areas of the language, which ultimately led to a complicated feature called rvalue references. So I'd like to veer a different way.

I'd love to see some concrete examples of where that went wrong for C++.  If not, it's tough to judge what benefit alternatives provide.


> I was thinking of something like:
> 
> void fun(T)(ref? T value) { ... }
> 
> This really generates two overloads:
> 
> void fun(T)(ref T value) { ... }
> void fun(T)(T value) { return (cast(void (ref T)) fun)(value); }


Based on those two overloads, it appears the first can modify value, but the effect on the caller will still depend on if T is either a value or reference type.  This style of strange behavior seems really dangerous to me.
February 15, 2009
Rainer Deyke wrote:
> Andrei Alexandrescu wrote:
>> Well I never seemed to have a problem with it, quite the contrary in
>> fact. Values are easier to reason about because they don't have aliasing
>> and don't foster long-distance interdependencies.
> 
> This.
> 
> When the container is used in a struct, then it must not only be copied,
> but deep-copied when the outer struct is copied.  The same applies when
> the container is used in a class that has a cloning operation defined.
> 
> When the container is used in a class that cannot be cloned or is
> directly placed on the stack, then it doesn't really matter if the
> container has reference or value semantics, since the variable that
> holds the container isn't being copied anyway.
> 
> For those very rare cases where you /want/ the container to be shared,
> you can always use a pointer to the container.

Spot on. My ambitions are actually a tad higher. I want to implement containers as by-value structs defining value semantics and the needed primitives. Then, using introspection, I want to define a template Class that takes a struct and turns it into a class. For example:

struct SomeContainer(T)
{
    ref T opIndex(size_t n) { ... }
}

auto byval = SomeContainer!(int)();
auto javalike = new Class!(SomeContainer!(int));

The type Class!(SomeContainer!(int)) will store a SomeContainer!(int) object and will define (by using introspection) a method opIndex of which implementation will do what the struct's method does. The net effect is as if the programmer sat down with SomeContainer and changed "struct" to "class". Of course without the code duplication and the maintenance nightmare :o).

Class should work for primitive types, e.g. Class!(int) should do what Integer does in Java and so on.


Andrei
February 15, 2009
Andrei Alexandrescu wrote:
> Spot on. My ambitions are actually a tad higher. I want to implement containers as by-value structs defining value semantics and the needed primitives. Then, using introspection, I want to define a template Class that takes a struct and turns it into a class. For example:
> 
> struct SomeContainer(T)
> {
>     ref T opIndex(size_t n) { ... }
> }
> 
> auto byval = SomeContainer!(int)();
> auto javalike = new Class!(SomeContainer!(int));
> 
> The type Class!(SomeContainer!(int)) will store a SomeContainer!(int) object and will define (by using introspection) a method opIndex of which implementation will do what the struct's method does. The net effect is as if the programmer sat down with SomeContainer and changed "struct" to "class". Of course without the code duplication and the maintenance nightmare :o).
> 
> Class should work for primitive types, e.g. Class!(int) should do what Integer does in Java and so on.

An interesting idea. Would Class!(T) allow polymorphism, or would all methods be implicitly final?
Would it also delegate template methods?

And what would happen to any T methods accepting or returning T? Would they keep returning T or would they be replaced by Class!(T)? Or perhaps both, through overloading?
Same question for types derived from T (e.g. Nullable!(T), or SomeContainer!(T))?

For instance in the case of a container it would be nice to allow Class!(LinkedList!(T)).splice!(Class!(LinkedList!(T)) other) if there's a LinkedList!(T).splice(LinkedList(T) other).

If such unwrapping were done, how would such methods react if the Class!(LinkedList!(T)) passed into slice() were null?
February 15, 2009
Frits van Bommel wrote:
> Andrei Alexandrescu wrote:
>> Spot on. My ambitions are actually a tad higher. I want to
>> implement containers as by-value structs defining value semantics
>> and the needed primitives. Then, using introspection, I want to
>> define a template Class that takes a struct and turns it into a
>> class. For example:
>> 
>> struct SomeContainer(T) { ref T opIndex(size_t n) { ... } }
>> 
>> auto byval = SomeContainer!(int)(); auto javalike = new
>> Class!(SomeContainer!(int));
>> 
>> The type Class!(SomeContainer!(int)) will store a
>> SomeContainer!(int) object and will define (by using introspection)
>> a method opIndex of which implementation will do what the struct's
>> method does. The net effect is as if the programmer sat down with
>> SomeContainer and changed "struct" to "class". Of course without
>> the code duplication and the maintenance nightmare :o).
>> 
>> Class should work for primitive types, e.g. Class!(int) should do
>> what Integer does in Java and so on.
> 
> An interesting idea. Would Class!(T) allow polymorphism, or would all
>  methods be implicitly final?

Polymorphic. Then of course there's the Finalize!(C) template that takes
a class C and makes all of its methods final. Compile-time introspection
is a treasure trove.

> Would it also delegate template methods?

Good question. I don't know how template methods could be handled.

> And what would happen to any T methods accepting or returning T?
> Would they keep returning T or would they be replaced by Class!(T)?
> Or perhaps both, through overloading? Same question for types derived
> from T (e.g. Nullable!(T), or SomeContainer!(T))?

I don't know. We need more experience to figure that out.

> For instance in the case of a container it would be nice to allow Class!(LinkedList!(T)).splice!(Class!(LinkedList!(T)) other) if
> there's a LinkedList!(T).splice(LinkedList(T) other).
> 
> If such unwrapping were done, how would such methods react if the Class!(LinkedList!(T)) passed into slice() were null?

Segfault :o).


Andrei
« First   ‹ Prev
1 2
Top | Discussion index | About this forum | D home