Wed, 15 Oct 2008 14:04:44 -0500,
Andrei Alexandrescu wrote:
> Sergey Gromov wrote:
> > Wed, 15 Oct 2008 09:49:08 -0500,
> > Andrei Alexandrescu wrote:
> >> Sergey Gromov wrote:
> >>> Wed, 15 Oct 2008 09:17:57 -0500,
> >>> Andrei Alexandrescu wrote:
> >>>> I see how it breaks those conventions, but I've never heard of them. The convention I heard of is that if you want dynamic polymorphism you use classes, otherwise you don't.
> >>> How about if you want pass-by-value you use structs, otherwise you don't? My argument against structs-collections is that I want to toss collections around and build other collections out of them without messing with pointers.
> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
> > 
> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
> 
> No. "Don't submit to the dogma that struct == value semantics, because that was never true".

I don't understand.  Structs are value types, everything else are implementation details.  And yes, I assume that struct copying yields a separate, independent object unless documented otherwise.  I don't usually need such documentation in my code because I either remember what I'm doing or the reference components of structs are obvious.  But with a library struct with undocumented, private implementation it becomes an issue.

> > So how do I return it?  Should I return it by value, hoping that the implied reference semantics will stay that way in future releases?  How does your Array!() behave?
> 
> That's not finished, but e.g. Appender can be copied around and always refers to the same underlying array.

This means that without reading docs I won't be able to tell whether my code is going to work or not.  Because the only truly copyable struct- based solution I can imagine is a struct with a single pointer field pointing at a heap-allocated implementation.  And I doubt that your Appender is implemented this way.

I'm afraid I'll have to actually look into library sources to make sure Appender does what I expect.

On Thu, Oct 16, 2008 at 5:46 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
> Wed, 15 Oct 2008 14:04:44 -0500,
> Andrei Alexandrescu wrote:
>> Sergey Gromov wrote:
>> > Wed, 15 Oct 2008 09:49:08 -0500,
>> > Andrei Alexandrescu wrote:
>> >> Sergey Gromov wrote:
>> >>> Wed, 15 Oct 2008 09:17:57 -0500,
>> >>> Andrei Alexandrescu wrote:
>> >>>> I see how it breaks those conventions, but I've never heard of them. The convention I heard of is that if you want dynamic polymorphism you use classes, otherwise you don't.
>> >>> How about if you want pass-by-value you use structs, otherwise you don't? My argument against structs-collections is that I want to toss collections around and build other collections out of them without messing with pointers.
>> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
>> >
>> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
>>
>> No. "Don't submit to the dogma that struct == value semantics, because that was never true".
>
> I don't understand.  Structs are value types, everything else are implementation details.

I think the point is that value *type* doesn't necessarily imply value
*semantics*.
Like with D's built-in arrays.

--bb

Thu, 16 Oct 2008 06:44:30 +0900,
Bill Baxter wrote:
> On Thu, Oct 16, 2008 at 5:46 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
> > Wed, 15 Oct 2008 14:04:44 -0500,
> > Andrei Alexandrescu wrote:
> >> Sergey Gromov wrote:
> >> > Wed, 15 Oct 2008 09:49:08 -0500,
> >> > Andrei Alexandrescu wrote:
> >> >> Sergey Gromov wrote:
> >> >>> Wed, 15 Oct 2008 09:17:57 -0500,
> >> >>> Andrei Alexandrescu wrote:
> >> >>>> I see how it breaks those conventions, but I've never heard of them. The convention I heard of is that if you want dynamic polymorphism you use classes, otherwise you don't.
> >> >>> How about if you want pass-by-value you use structs, otherwise you don't? My argument against structs-collections is that I want to toss collections around and build other collections out of them without messing with pointers.
> >> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
> >> >
> >> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
> >>
> >> No. "Don't submit to the dogma that struct == value semantics, because that was never true".
> >
> > I don't understand.  Structs are value types, everything else are implementation details.
> 
> I think the point is that value *type* doesn't necessarily imply value
> *semantics*.
> Like with D's built-in arrays.

Arrays are very widely used and very well documented to the point they're quite obvious.  There are no hidden implementation details. Nevertheless, there are well-known problems which directly follow from the arrays' sort-of-reference semantics.

On Thu, Oct 16, 2008 at 7:47 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
> Thu, 16 Oct 2008 06:44:30 +0900,
> Bill Baxter wrote:
>> On Thu, Oct 16, 2008 at 5:46 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
>> > Wed, 15 Oct 2008 14:04:44 -0500,
>> > Andrei Alexandrescu wrote:
>> >> Sergey Gromov wrote:
>> >> > Wed, 15 Oct 2008 09:49:08 -0500,
>> >> > Andrei Alexandrescu wrote:
>> >> >> Sergey Gromov wrote:
>> >> >>> Wed, 15 Oct 2008 09:17:57 -0500,
>> >> >>> Andrei Alexandrescu wrote:
>> >> >>>> I see how it breaks those conventions, but I've never heard of them. The convention I heard of is that if you want dynamic polymorphism you use classes, otherwise you don't.
>> >> >>> How about if you want pass-by-value you use structs, otherwise you don't? My argument against structs-collections is that I want to toss collections around and build other collections out of them without messing with pointers.
>> >> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
>> >> >
>> >> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
>> >>
>> >> No. "Don't submit to the dogma that struct == value semantics, because that was never true".
>> >
>> > I don't understand.  Structs are value types, everything else are implementation details.
>>
>> I think the point is that value *type* doesn't necessarily imply value
>> *semantics*.
>> Like with D's built-in arrays.
>
> Arrays are very widely used and very well documented to the point they're quite obvious.  There are no hidden implementation details.

And I think the issues are pretty much the same with all struct-based container.  So if you understand the issues with D's arrays, then all you have to know is "same thing for other struct containers".

> Nevertheless, there are well-known problems which directly follow from the arrays' sort-of-reference semantics.

Yeh, these do make me wonder about whether it was a good idea to make D's arrays behave the way they do.  But since they do, and since you can't really avoid having to learn how they work and what their pitfalls are, I don't think it's so bad to make other containers behave the same way.  Less to learn that way.

But I'd definitely be willing to entertain new ground-up designs in which arrays behaved more like pure references, or were not copyable, or similar.

--bb

"Sergey Gromov" wrote
> Wed, 15 Oct 2008 14:04:44 -0500,
> Andrei Alexandrescu wrote:
>> Sergey Gromov wrote:
>> > Wed, 15 Oct 2008 09:49:08 -0500,
>> > Andrei Alexandrescu wrote:
>> >> Sergey Gromov wrote:
>> >>> Wed, 15 Oct 2008 09:17:57 -0500,
>> >>> Andrei Alexandrescu wrote:
>> >>>> I see how it breaks those conventions, but I've never heard of them.
>> >>>> The
>> >>>> convention I heard of is that if you want dynamic polymorphism you
>> >>>> use
>> >>>> classes, otherwise you don't.
>> >>> How about if you want pass-by-value you use structs, otherwise you
>> >>> don't?
>> >>> My argument against structs-collections is that I want to toss
>> >>> collections around and build other collections out of them without
>> >>> messing with pointers.
>> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
>> >
>> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
>>
>> No. "Don't submit to the dogma that struct == value semantics, because that was never true".
>
> I don't understand.  Structs are value types, everything else are implementation details.  And yes, I assume that struct copying yields a separate, independent object unless documented otherwise.  I don't usually need such documentation in my code because I either remember what I'm doing or the reference components of structs are obvious.  But with a library struct with undocumented, private implementation it becomes an issue.

First, there is no compiler required semantics.  It is up to the designer. If the designer says 'this is return by reference', then the designer is telling you what it is.

Second, just because you have your expectations doesn't mean they should be right ;)  The only things that are guaranteed are done so by the compiler. Everything else is up to the designer.

>> > So how do I return it?  Should I
>> > return it by value, hoping that the implied reference semantics will
>> > stay
>> > that way in future releases?  How does your Array!() behave?
>>
>> That's not finished, but e.g. Appender can be copied around and always refers to the same underlying array.
>
> This means that without reading docs I won't be able to tell whether my code is going to work or not.  Because the only truly copyable struct- based solution I can imagine is a struct with a single pointer field pointing at a heap-allocated implementation.  And I doubt that your Appender is implemented this way.
>
> I'm afraid I'll have to actually look into library sources to make sure Appender does what I expect.

All you should need to do is look at the docs.  If the docs suck, you have a gripe.  If the implementation isn't what you would have done, but there is nothing wrong with it, tough.  Go write your own.

-Steve

Thu, 16 Oct 2008 08:42:23 +0900,
Bill Baxter wrote:
> On Thu, Oct 16, 2008 at 7:47 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
> > Thu, 16 Oct 2008 06:44:30 +0900,
> > Bill Baxter wrote:
> >> On Thu, Oct 16, 2008 at 5:46 AM, Sergey Gromov <snake.scaly@gmail.com> wrote:
> >> > Wed, 15 Oct 2008 14:04:44 -0500,
> >> > Andrei Alexandrescu wrote:
> >> >> Sergey Gromov wrote:
> >> >> > Wed, 15 Oct 2008 09:49:08 -0500,
> >> >> > Andrei Alexandrescu wrote:
> >> >> >> Sergey Gromov wrote:
> >> >> >>> Wed, 15 Oct 2008 09:17:57 -0500,
> >> >> >>> Andrei Alexandrescu wrote:
> >> >> >>>> I see how it breaks those conventions, but I've never heard of them. The convention I heard of is that if you want dynamic polymorphism you use classes, otherwise you don't.
> >> >> >>> How about if you want pass-by-value you use structs, otherwise you don't? My argument against structs-collections is that I want to toss collections around and build other collections out of them without messing with pointers.
> >> >> >> A struct can choose to implement value or reference semantics as it pleases. The only thing it can't readily implement is dynamic polymorphism.
> >> >> >
> >> >> > I can imagine the documentation: "Ignore the fact it's a struct, it's actually a reference internally."
> >> >>
> >> >> No. "Don't submit to the dogma that struct == value semantics, because that was never true".
> >> >
> >> > I don't understand.  Structs are value types, everything else are implementation details.
> >>
> >> I think the point is that value *type* doesn't necessarily imply value
> >> *semantics*.
> >> Like with D's built-in arrays.
> >
> > Arrays are very widely used and very well documented to the point they're quite obvious.  There are no hidden implementation details.
> 
> And I think the issues are pretty much the same with all struct-based container.  So if you understand the issues with D's arrays, then all you have to know is "same thing for other struct containers".

No.  Built-in array is transparent.  Appender is not.  Because if Appender is transparent, too, and there is a change in GC which makes its current implementation slow, you won't be able to fix that.  Therefore Appender must be opaque and must specify its interface.  And copy semantics becomes a part of that interface.  It may transfer ownership on assignment for instance.  Or declare behavior undefined if there are multiple copies of it.  You see, I can't even say "multiple references" because they are not, and "multiple copies" sounds ambiguous.

The bottom line is, your knowledge of built-in arrays is useless here. You must study docs for every container you use.

> > Nevertheless, there are well-known problems which directly follow from the arrays' sort-of-reference semantics.
> 
> Yeh, these do make me wonder about whether it was a good idea to make D's arrays behave the way they do.  But since they do, and since you can't really avoid having to learn how they work and what their pitfalls are, I don't think it's so bad to make other containers behave the same way.  Less to learn that way.

They cannot behave the same way unless they're very similar to the built- in arrays, which makes them pretty useless.  Even built-in AA behaves differently.  BTW I was quite surprised to find out that it was implemented exactly as a struct with a single pointer inside.  Well, there could have been reasons to implement a built-in type like this.  I cannot see such reasons for Appender.

> But I'd definitely be willing to entertain new ground-up designs in which arrays behaved more like pure references, or were not copyable, or similar.
> 
> --bb
> 

Andrei Alexandrescu wrote:
> I don't want to seem like picking on you, but I would like to answer
> this message too.

I don't feel like you're picking on me! Without disagreements, there wouldn't be much to discuss :)

Though I do think you should lay off with the "beginner" business. The people who disagree with you disagree for legitimate reasons, not because they're inexperienced. Perhaps Josh Bloch is a beginner?

> In my humble opinion, collections APIs are the poster-child of the
> misguided joy of the beginnings of object-orientation, right next to the
> mammal isa animal example. Organizing collections in hierarchies is of
> occasional benefit, but the real benefit comes from decoupling
> containers from algorithms via iterators, the way the STL did. The
> savings there were that an O(m * n) problem has been solved with writing
> O(m + n) code. Container hierarchies do not achieve such a reduction
> unless efforts are being made that have little to do with hierarchical
> organization. At most they factor out some code in the upper classes,
> but hierarchies help when types have many commonalities and few
> difference, and that is very rarely the case with containers.

Nonsense.

Iterators are not the only way to "decouple containers from algorithms". An iterator in STL is just a compile-time interface for accessing the elements in a container.

You can argue that using type hierarchies or interface inheritance is an undue runtime burden and that iterators solve that problem effectively. But, from a data-modeling perspective, there is absolutely no difference between these two:

   sort(List<T> list);

   sort(iterator<T> start, iterator<T> end);

>> Structs can't inherit from abstract classes. A good deal of functionality in a collection class can be implemented once in an abstract base class, making it simpler for subclass authors to implement the API without duplicating a bunch of boilerplate.
> 
> This also reveals a beginner's view of inheritance. You don't inherit to
> reuse. You inherit to be reused. If all you want is to reuse, use
> composition.

When the rubber hits the road, I'll take my "beginners view of inheritance any time".

One of my favorite tiny classes in Java is this little nugget:

   import java.util.LinkedHashMap;
   import java.util.Map;

   public class CacheMap<K, V> extends LinkedHashMap<K, V> {
      int maxCapacity;
      public CacheMap(int maxCapacity) {
         this.maxCapacity = maxCapacity;
      }
      protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
         return size() > maxCapacity;
      }
   }

Now I have a fully-functional Map class that automatically removes entries (in FIFO order) when the collection reaches some maximum size. In every other way, it performs exactly like any other LinkedHashMap (which always iterates in insertion-order).

If I used composition, and if I wanted to comply with with Map interface, I'd have to manually re-implement every single method, wrapping the underlying functionality and delegating the calls myself. If the type system can do it for me, why not?

And since I've implemented Map<K, V>, I can pass my container to any method that cares about "mappiness", regardless of whether it cares about "cachiness".

>> Conceptually -- and there's no hard and fast rule about this -- structs usually either represent small logically-atomic values (DateTime), or some fixed-size collection of logically-atomic values (Vector3). Using a struct as an arbitrarily-sized container seems, on the face of it, to break those conventions.
> 
> I see how it breaks those conventions, but I've never heard of them. The
> convention I heard of is that if you want dynamic polymorphism you use
> classes, otherwise you don't.

The D documentation implies otherwise:

http://www.digitalmars.com/d/2.0/struct.html

Though I suppose you're free to use whatever conventions you like.

--benji

Sergey Gromov:
> Or declare behavior undefined if there are multiple copies of it.  You see, I can't even say "multiple references" because they are not, and "multiple copies" sounds ambiguous.

Just a note: my ArrayBuilder is a struct (with 3 fields inside, each sizeof 1 CPU word). I have already used it a lot in my libs, probably about 50 times, but I've never had to copy one of them. I have used a struct because a class isn't necessary, and because the struct methods are faster at my benchmarks.

Bye,
bearophile

Benji Smith wrote:
> Andrei Alexandrescu wrote:
>> In my humble opinion, collections APIs are the poster-child of the
>> misguided joy of the beginnings of object-orientation, right next to the
>> mammal isa animal example. Organizing collections in hierarchies is of
>> occasional benefit, but the real benefit comes from decoupling
>> containers from algorithms via iterators, the way the STL did. The
>> savings there were that an O(m * n) problem has been solved with writing
>> O(m + n) code. Container hierarchies do not achieve such a reduction
>> unless efforts are being made that have little to do with hierarchical
>> organization. At most they factor out some code in the upper classes,
>> but hierarchies help when types have many commonalities and few
>> difference, and that is very rarely the case with containers.
> 
> Nonsense.

I know you're going to hate this, but I'm unable to follow through with this discussion. There is much background and many aspects to discuss, and I simply can't afford the time. Since not too long ago it has dawned on the programming community that OOP is not all it's cracked to be, and that there are quite a few amendments and qualifications that need to be made to what once seemed to be very attractive principles. But such developments are quite recent, and they may sound downright heretic and  indeed nonsensical to many. That's why there's need for a lot of explaining.

There's a book I do recommend though to bring some good info on the subject: Agile Software Development by Robert Martin. The book is good, and for this particular subject I recommend Chapter 10.


Andrei

Andrei Alexandrescu wrote:
> Benji Smith wrote:
>> Andrei Alexandrescu wrote:
>>> In my humble opinion, collections APIs are the poster-child of the
>>> misguided joy of the beginnings of object-orientation, right next to the
>>> mammal isa animal example. Organizing collections in hierarchies is of
>>> occasional benefit, but the real benefit comes from decoupling
>>> containers from algorithms via iterators, the way the STL did. The
>>> savings there were that an O(m * n) problem has been solved with writing
>>> O(m + n) code. Container hierarchies do not achieve such a reduction
>>> unless efforts are being made that have little to do with hierarchical
>>> organization. At most they factor out some code in the upper classes,
>>> but hierarchies help when types have many commonalities and few
>>> difference, and that is very rarely the case with containers.
>>
>> Nonsense.
> 
> I know you're going to hate this, but I'm unable to follow through with this discussion. There is much background and many aspects to discuss, and I simply can't afford the time.

Naw, I don't hate it. I'm up to my eyeballs too, and I had no delusions that I'd convince you anyhow.

I just wanted to put those ideas on the table, since I think there are a lot of other people like me out there. Those people might like to use D too, and their development styles are valid too, so I'm standing up for them.

Anyhoo... catcha later!

--benji