In article <ceb6ur$1hn3$1@digitaldaemon.com>, Ben Hinkle says...
>
>Could TypeInfo's help here? Structs don't currently have TypeInfo so it won't solve the struct issue but Walter has said he knows TypeInfos for structs needs improvement so eventually it should get fixed. I've been using TypeInfo for compare, swap, equals and getHash and found them very useful.

I suggested the same thing, but TypeInfo fails in eone important manner: TypeInfo is evaluated at run-time.  I'm only speculating, but I have a feeling that we may also need some compile-time means for differentiating types.  C++ doesn't have this particular problem because structs and classes are the same thing.


Sean

In article <ceamcj$1bco$1@digitaldaemon.com>, Arcane Jill says...
>
>In article <ceaeid$18ir$1@digitaldaemon.com>, Matthew says...
>>
>>I'm almost at the end of my tether here. There are so many problems with the language/compiler, that I'm wondering seriously whether it's going to be possible to have a decent generic template library.
>
>This is a real bummer. I was rather hoping that D would end up being /better/ than C++. If that's not to be so, it's a little disheartening.

I think it still will, but the language may have to become a bit more complicated to do so (which is a bit contrary to its original design parameters).  It's probably a good time to start taking a critical look at the language and see if we can find holes in its capabilities.  It would be nice if the issue were just one of mindset and not of feature support.


Sean

Matthew,

   Compiler bugs aside, you forté make a few silly arguments.  Such as,
that you can't support iterators because you can't overload the *
operator.  What kind of BS is that?

Overloading operators for no apparent good reason is STLs forté.  Obj-C and Cocoa have lots of templates and it doesn't have any operator overloading what-so-ever.

STLs and C++ use operator overloading incessantly  because somehow their designers thinks it looks cool.   IT confuses whether or not an iterator is a pointer or an object.  Why not just have a real function call: Iterator.getMyFuckingInstanceNow()

No instead we must confuse things by going (*Iterator)

Iterators in STL suck, plain and simple.  Most of C++'s standard template library is overly complicated and obtuse for the job.  Not to mention buggy between implementations.


As for implicit instantation,  Why is this required to make an iterator?

Maybe I'm missing the boat, but what's wrong with something like this:

class List(T) {
   alias ListIterator(T) iterator;

   ...
}

List!(int).iterator iter;
// Hell you could probably even take your existing instance and do:
// Foo.iterator iter;  In fact you should be able to but it seems you
can't  This would allow you to also do aliases for stuff like:

iter.BaseType t; and do your generic manipulation functions. or List.BaseType; etc.  And have aliases in your class.

I say we bitch at walter.

for( iter = myList.begin();  iter != myList.end(); iter++ )
{
   printf(iter.getMyFuckingInstanceNow().toString());
}

This is nearly identical to what C++ does, and i really don't see why it wouldn't work.  You could probably leave the instance-getter function name a little shorter though.

Maybe you could expand as to why this sucks? (Besides that it's similar to STL which sucks)


And as for your template example:


#    template dump(T) {
#    void dump(IContainer!(T) c)
#    {
#        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
#        {
#
#        }
#    }
#    }
#
#    List!(int, IContainer!(int))           l = new ...;
#
#    dump(l);

For one, making those aliases all over is just as ugly and hard to remember as just specifying it.  Second:

You're passing in l of type List!(int, IContainer!(int)), and then proceed to do this with it:

IContainer!(List!(int, IContainer!(int))) c.

Is that really what you were intending to do?

It seems like templates should support something along these lines:

#   template dump( T : IContainer!(T) ) {
#      void dump( IContainer!(t) ) {
#      }
#   }

Although I seriously doubt that works as it is.

In article <ceb98u$1iff$1@digitaldaemon.com>, Sean Kelly says...
>
>In article <ceb6ur$1hn3$1@digitaldaemon.com>, Ben Hinkle says...
>>
>>Could TypeInfo's help here? Structs don't currently have TypeInfo so it won't solve the struct issue but Walter has said he knows TypeInfos for structs needs improvement so eventually it should get fixed. I've been using TypeInfo for compare, swap, equals and getHash and found them very useful.
>
>I suggested the same thing, but TypeInfo fails in eone important manner: TypeInfo is evaluated at run-time.  I'm only speculating, but I have a feeling that we may also need some compile-time means for differentiating types.  C++ doesn't have this particular problem because structs and classes are the same thing.
>

I don't profess to have done anywhere near as much template programming as Matthew, but I've run into some quirks in D that force me to write some really painful runarounds.  That aside, it _is_ possible to use templates based on TypeInfo to get the compile-time typing you're after.

template Foo(TI: TypeInfo,T){ /* T is a primitive type */ }
template Foo(TI: TypeInfoClass,T){ /* T is a class */ }
template Bar(T: void){ /* T is void */ }
template Bar(T: Object){ /* T is a class */ }
template Bar(T){ /* T is a primitive */ }

// examples
Foo!(typeid(int),int);
Foo!(typeid(MyClass),MyClass);
Bar(void);
Bar(MyClass);
Bar(int);

- Pragma

"Matthew" <admin.hat@stlsoft.dot.org> wrote in message news:ceaeid$18ir$1@digitaldaemon.com...
> I'm almost at the end of my tether here. There are so many problems with
the language/compiler, that I'm wondering
> seriously whether it's going to be possible to have a decent generic
template library.

I think that's to be expected with a new generic language design. After all, the creator of STL (Stepanov?) went through multiple design iterations with Stroustrup. None of us are smart enough to get it all right the first, second, or even third time. You do have a tendency to push the envelope past its limits (I see that in your C++ code!), and that's good for finding what those limits and weaknesses are.

Generic programming in D is also going to be in a different style than in C++, and use different techniques. We have to carefully think about whether the problems are bugs in the compiler, bugs in the language design, or bugs in our thinking - perhaps we haven't discovered the right D way to do generic programming. One thing I am pretty sure of, the C++ way of doing STL isn't the right way for D. I personally find STL to be impenetrably obtuse. I refuse to believe that is the best way of doing generics. I think it's great that you are taking a different approach than STL does, and are open to inventive new ways of thinking about it.


> Alas, this crashes the compiler. Sigh ...

That's my problem, send me the test case!


> 4. Iterator based approach. There are two reasons why STL-iterators are
not possible in D: (i) there is no implicit
> instantiation, and (ii) there is no operator *(). Neither of these
preclude the creation of iterators - and I've done a
> little of that - including adaptors for arrays/pointers, but it means
they're not really very usable.

Not having implicit instantiation can make the code a bit more verbose, but
it shouldn't be that bad. As for operator*(), why should that be a problem?
Just define a member function called, I'm sure there's a better name,
operatorStar(), and instead of *foo, type foo.operatorStar().

> 1. Beyond a significant, but indefinable level of complexity, the
linker/compiler loose the ability to find all required
> symbols, while chopping out any *unrelated* parts makes them locatable
again. Boil it down? I wish!!
> Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood
up, since one can usually find workarounds,
> and at this stage a little bugginess is not the crime of the century.

If you can't reduce it, you can't reduce it. Send a reproducible example.


> 2. Templates in D are instantiated en masse. What "en masse" actually
means is beyond me, since I am yet to work out the
> true rules regarding instantation.

The true rule is the whole template is compiled when it is instantiated. (This is unlike C++, which does "lazy instantiation", which means it only compiles the bits of a template that are actually used.)

> But what I can say is that having a template such as the following is
totally
> f**cked:
>
>     template TransformedRange(R, F, T) { class TransformedRange
>      : public NotionalRange!(T)
>     {
>       . . .
>         this(R r, F f)
>         {
>             m_r = r.dup;
>             m_f = f;
>         }
>         this(R r)
>         {
>             m_r = r.dup;
>             m_f = new filter_type();
>         }
>
> Without worrying too much about the details of what a TransformedRange
does, the problem is pretty obvious. If F does
> not have a default constructor, one cannot instantiate TransformedRange
even in the case where the single parameter ctor
> *is never called*!

I don't get it from the example given.

> 3. There's no implicit instantiation. This has *massive* consequences,
including:
>
> - We can't have iterators (as noted above). This is fine for reading from
containers, but consider how we might
> "generically" insert into container ranges in the same (or any analogous)
way as is the case with STL.

Can you give a canonical example?


> - In order to support the parameterisable interface (e.g.
IContainer!(int)) described above, there needs to be a common
> way to manipulate built-in types, objects and structs. For some things,
one can use traits, for others, overloaded
> functions. Alas, there seems to be no way to discriminate structs via
overloaded functions. Hence, currently the DTL
> containers do not support polymorphic interfaces when storing structs.
Naturally, this has much wider consequences

I hadn't realized that structs and classes cannot be discriminated in
template specialization, you're right. How does the following look:
    template foo(T : struct)
and:
    template foo(T : class)
?

> 4. D's import stuff just blows my mind! As a related issue to the boxing
utility class described above, I'm running into
> conflicts between the toString() functions in std.string and my box and
boxutil modules. I've tried all kinds of use of
> private imports, to no avail. I concede that this might be my fault, and I
might have just failed to grok D's import
> rules, but as it currently looks to me, it looks bad and stupid.

I suspect the problem you're running into is that there's a toString in std.string, and a toString in foo. Then, in bar.d:

    import std.string;
    import foo;

    ...
    x = toString(y);
    ...

This will bring up a conflict, because which toString do you want, the one in std.string or the one in foo? The compiler doesn't know. The rule is that name lookup happens first, *then* overload matching. If you want to overload them together:

    import std.string;
    import foo;
    alias std.string.toString  toString;
    alias foo.toString  toString;

    ...
    x = toString(y);
    ...

now there's no longer a conflict, because toString is found in the current module (the aliases), and the aliases are chased down for overload resolution. It is this way because it gives you complete control over overloading.


> So, what do we do about this? Walter's stock response is to boil it down,
but it's either impossible to do so with
> non-trivial projects such as DTL, or I'm simply not smart enough to do it.
Sure, one might argue that this is indicative
> of a too-complex design, but I think that's crap: the concept is simple,
and the code is simple; it just doesn't work.
> (Very similar techniques that I've experimented on in C++ work fine.)

If you can't do it, you can't do it. Send what you can do.

> Rather than having things boiled down, I think the compiler should be
amended to provide *copious" debugging
> information, so we can email a dump of that to Walter, and which will be
useful to him.

Trust me, sending me dumps would be quite useless.

Walter wrote:
> I hadn't realized that structs and classes cannot be discriminated in
> template specialization, you're right. How does the following look:
>     template foo(T : struct)
> and:
>     template foo(T : class)
> ?

Looks promising to me... how likely would the following be, as well?
	template foo(T : signed)
	template foo(T : unsigned)
	template foo(T : enum)

I figure the more ways to narrow down the parameters, the better.  Maybe templates could be further revised to have complex filters, such as:
	template foo(T : struct || (signed && enum))

This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..).  It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..).  The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only.

But I'm sure a certain amount of this is dreaming on my part.  :)

-Chris S.
-Invironz

I like your idea.
Theres only a tiny drawback. For the compiler it's harder to find the right template. That should be something big W can handle *g*

Stephan

C. Sauls wrote:
> 
> 
> Looks promising to me... how likely would the following be, as well?
>     template foo(T : signed)
>     template foo(T : unsigned)
>     template foo(T : enum)
> 
> I figure the more ways to narrow down the parameters, the better.  Maybe templates could be further revised to have complex filters, such as:
>     template foo(T : struct || (signed && enum))
> 
> This would take as parameter wither a struct, or an enum that derived from a signed type (byte,short,int,long,cent..).  It would reject anything else, including an /un/signed enum (enum : ubyte,ushort,uint,ulong.ucent..).  The use of conditional operators is arbitrary, I guess anything could be used... maybe even new 'or' and 'and' keywords that are template-only.
> 
> But I'm sure a certain amount of this is dreaming on my part.  :)
> 
> -Chris S.
> -Invironz

"C. Sauls" <ibisbasenji@yahoo.com> wrote in message news:cebjlk$1n2q$1@digitaldaemon.com...
> Walter wrote:
> > I hadn't realized that structs and classes cannot be discriminated in
> > template specialization, you're right. How does the following look:
> >     template foo(T : struct)
> > and:
> >     template foo(T : class)
> > ?
>
> Looks promising to me... how likely would the following be, as well?
> template foo(T : signed)
> template foo(T : unsigned)
> template foo(T : enum)
>
> I figure the more ways to narrow down the parameters, the better.  Maybe
> templates could be further revised to have complex filters, such as:
> template foo(T : struct || (signed && enum))

It's a good idea. I've been thinking something along those lines for 2.0.

I personally think generics are over-rated, if i wanted typeless variables i'd use a scripting language ( or VB ).


>    template Vector(T, B = EmptyBase)
>    {
>        class Vector
>            : public BaseSelector!(B).selected_type

I realize you do alot of C++ and D programming, so you want to keep the syntax roughly the same, but this looks dangerously close to C++.  Why noy use the D way :)

class Vector(T,B = EmptyBase) : BaseSelector!(B).selected_type

I only mention this because I wonder how much ( unconsiously ) your applying
C++'s soloutions to D.

I for one am investing a large amount of
>time to the D cause, with all the consequences wrt to more materially lucrative activities. Since I find that boiling down the code generally takes me at least as long to write the code in the first place, I just don't see that it's justified that I should do the boiling as well. Give me a -dump switch, and I can send it all off to Walter.

I hear you on that one.

>[btw: both of the above forms work now and, fwiw, I hope to release some of the containers supporting this in the next c/o days.]

Youve been saying that for months ;).  But im not downing you I know your busy and all good things take time, but _please_ do release it , so that we can look at it too, theres lots of talent on the newsgroup.

>4. D's import stuff just blows my mind!

Maybe we can convince walter to give us a written and posted on the website description, as this seems to be a problem everyone has had at one time or another.  I know hes answered it a few times, but it needs to be easily accessible.

>If all this sounds like I'm down on D, and I've spent several weeks - actually it's several months, but who's counting? - working against all these issues and more, then you'd be about spot on. I can't remember being this technically frustrated in the last 10 years, and I'm generally known as a practical pragmatist! :-(

Well i hope you don't give up, hate to lose you at this stage, but I feel your frustration :S.

Charlie


In article <ceaeid$18ir$1@digitaldaemon.com>, Matthew says...
>
>I'm almost at the end of my tether here. There are so many problems with the language/compiler, that I'm wondering seriously whether it's going to be possible to have a decent generic template library.
>
>Before I preface this, I should reiterate the "modes" of DTL, to inform on any thoughts anyone might have on this:
>
>1. foreach - client code uses foreach. This all works well for all containers. The problem with this is only that it's an unsophisticated view of looking at a container: you get everything, and have to do all filtering and transformations yourself in client code. This can result in verbose and tedious coding. Good libraries should provide more power.
>
>2. Transformations/filtering - based on "Ranges" (a concept John Torjo and I concocted for C++, but which is eminently suitable for D.). Basically, each container provides a set of methods (eventually via a mixin) that returns a "range". For example, the "select()" method takes either a function/delegate/functor predicate, and returns a freachable range that "contains" only the elements matching the predicate, e.g.
>
>    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
>    bool IsOdd(int i)
>    {
>        return 0 != (i & 0x2);
>    }
>    foreach(int i; list.select(IsOdd))
>    {
>        printf("%d ", i);
>    }
>
>This will print out "1 3 5 7 9"
>
>The power of this is that the returned "range" itself provides the same transformation methods, which means that the transformations are composable, as in:
>
>    List!(int)        list = . . .; // Assume it's filled with numbers 0 - 9
>    bool IsOdd(int i)
>    {
>        return 0 != (i & 0x2);
>    }
>    int Times2(int i)
>    {
>        return 2 * i;
>    }
>    foreach(int i; list.select(IsOdd).collect(Times2))
>    {
>        printf("%d ", i);
>    }
>
>This will print out "2 6 10 14 18"
>
>I'm sure you can see the power in this approach, as well as the potential efficiency savings, where the transformation "criteria" may be tunnelled into the original container's foreach (I've not done that yet).
>
>3. Interface-based containers
>
>Each container in DTL is declared as follows:
>
>    template Vector(T, B = EmptyBase)
>    {
>        class Vector
>            : public BaseSelector!(B).selected_type
>
>
>This means that by default the container will be do as the STL does, and will just be all compile-time typing, e.g.
>
>    alias    Vector!(int)     int_vector_t;
>
>    int_vector_t    v = new ...;
>
>    v[10] = 3;
>    size_t n = v.length;
>
>    int    r = v[0] + v[1];
>
>Such a type is great, as long as we want to manipulate it in/with code that knows its exact type.
>
>Alternatively, we can also support the old-style Java approach, whereby one can specify a base interface, e.g.
>
>    void dump(IObjectContainer c)
>    {
>        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
>        {
>            printf("%.*s ", e.nextElement().toString());
>        }
>        printf("\n");
>    }
>
>    alias    Vector!(int, IObjectContainer)    int_vector_IObjectContainer_t;
>    alias    List!(int, IObjectContainer)        int_list_IObjectContainer_t;
>
>    int_vector_IObjectContainer_t   v = new ...;
>    int_list_IObjectContainer_t        l = new ...;
>
>    dump(v);
>    dump(l);
>
>Now we can pass instances of any container implementing IObjectContainer this to *any* function that knows how to manipulate that interface.
>
>[btw: both of the above forms work now and, fwiw, I hope to release some of the containers supporting this in the next c/o days.]
>
>The downside to the IObjectContainer type approach is that (i) fundamental types must be "boxed" (I've already worked up
>std.box), and (ii) it's not generic, since one must downcast object types to their "real" types from Object. Pretty
>pukey stuff, except in the minority of circumstances where the IObject-stuff is what you're after (e.g. a message
>board).
>
>Further to these two approaches, which I've been working on today, is inheritance via parameterised-interfaces, e.g.
>
>    template dump(T) { void dump(IContainer!(T) c)
>    {
>        for(IEnumerator e = c.enumerate(); e.hasMoreElements; )
>        {
>            T    t    =    e.nextElement();
>
>            // ... write generically, presumably via writef() (which I've not used yet <g>)
>        }
>        printf("\n");
>    }
>
>    alias    IContainer!(int)                         int_IContainer_t;
>    alias    Vector!(int, int_IContainer_t)    int_vector_IContainer_int_t;
>    alias    List!(int, int_IContainer_t)        int_list_IContainer_int_t;
>
>    int_vector_IContainer_int_t      v = new ...;
>    int_list_IContainer_int_t            l = new ...;
>
>    dump(v);
>    dump(l);
>
>Note: this approach does _not_ require boxing or downcasting.
>
>Alas, this crashes the compiler. Sigh ...
>
>4. Iterator based approach. There are two reasons why STL-iterators are not possible in D: (i) there is no implicit instantiation, and (ii) there is no operator *(). Neither of these preclude the creation of iterators - and I've done a little of that - including adaptors for arrays/pointers, but it means they're not really very usable. The only way to use them "generically" in algorithms would be polymorphically if the iterator classes derived from interfaces, and that would preclude efficiency mechanisms (such as Random Access iterator advancement in STL), without some nasty hacks.
>
>It's my belief that iterators are not for D, and that we thus have the modes 1-3 described above.
>
>And now on to my litany of woes ...
>
>Some tasters:
>
>1. Beyond a significant, but indefinable level of complexity, the linker/compiler loose the ability to find all required symbols, while chopping out any *unrelated* parts makes them locatable again. Boil it down? I wish!!
>
>Nonetheless, that's a bug in the compiler/linker, and doesn't get my blood up, since one can usually find workarounds, and at this stage a little bugginess is not the crime of the century.
>
>2. Templates in D are instantiated en masse. What "en masse" actually means is beyond me, since I am yet to work out the true rules regarding instantation. But what I can say is that having a template such as the following is totally f**cked:
>
>    template TransformedRange(R, F, T) { class TransformedRange
>     : public NotionalRange!(T)
>    {
>      . . .
>        this(R r, F f)
>        {
>            m_r = r.dup;
>            m_f = f;
>        }
>        this(R r)
>        {
>            m_r = r.dup;
>            m_f = new filter_type();
>        }
>
>Without worrying too much about the details of what a TransformedRange does, the problem is pretty obvious. If F does not have a default constructor, one cannot instantiate TransformedRange even in the case where the single parameter ctor *is never called*!
>
>I'm yet to work out a workaround to this one - all I do at the moment is not use certain operations with certain container instantiations in the test programs. Woo hoo!
>
>3. There's no implicit instantiation. This has *massive* consequences, including:
>
>- We can't have iterators (as noted above). This is fine for reading from containers, but consider how we might "generically" insert into container ranges in the same (or any analogous) way as is the case with STL.
>
>- In order to support the parameterisable interface (e.g. IContainer!(int)) described above, there needs to be a common way to manipulate built-in types, objects and structs. For some things, one can use traits, for others, overloaded functions. Alas, there seems to be no way to discriminate structs via overloaded functions. Hence, currently the DTL containers do not support polymorphic interfaces when storing structs. Naturally, this has much wider consequences
>
>4. D's import stuff just blows my mind! As a related issue to the boxing utility class described above, I'm running into conflicts between the toString() functions in std.string and my box and boxutil modules. I've tried all kinds of use of private imports, to no avail. I concede that this might be my fault, and I might have just failed to grok D's import rules, but as it currently looks to me, it looks bad and stupid.
>
>
>So, what do we do about this? Walter's stock response is to boil it down, but it's either impossible to do so with non-trivial projects such as DTL, or I'm simply not smart enough to do it. Sure, one might argue that this is indicative of a too-complex design, but I think that's crap: the concept is simple, and the code is simple; it just doesn't work. (Very similar techniques that I've experimented on in C++ work fine.)
>
>Rather than having things boiled down, I think the compiler should be amended to provide *copious" debugging information, so we can email a dump of that to Walter, and which will be useful to him. I don't know what that information should be, but I know that it's simply not practical for me, or anyone else, to "boil down" these precipitating bugs when they only manifest in highly complex code. I do know that it's totally bogus for us to be made to feel guilty for not having the time or the talent to do this boiling down. I for one am investing a large amount of time to the D cause, with all the consequences wrt to more materially lucrative activities. Since I find that boiling down the code generally takes me at least as long to write the code in the first place, I just don't see that it's justified that I should do the boiling as well. Give me a -dump switch, and I can send it all off to Walter.
>
>Anyway, that's just the compiler, and my main problem is with the language. I'm coming to the conclusion that D either will never be suitable for generic programming, or such suitability is years away. Given that, my aims for DTL are starting to seem naive at best, unattainable at worst.
>
>What's to be done? Well, one might say let's just have vanilla containers, and skip all the transformation stuff. That's fine, but then where're the generics? We can't have algorithms, remember, because we've not got implicit instantiation! The only remaining way to be generic is to follow the Java-route, and go with polymorphic container interfaces, but (i) they can't contain structures, and (ii) we're in Java-la-la land where everything has to be downcast. Yeuch! Even if we can get the compiler to accept parameterisable container interfaces, it's still a runtme indirection, with the concomitant efficiency costs.
>
>So please, someone enlighten me (since I am quite prepared to believe I've missed something simple and obvious here): how can we do generic programming in D?
>
>Matthew
>
>If all this sounds like I'm down on D, and I've spent several weeks - actually it's several months, but who's counting? - working against all these issues and more, then you'd be about spot on. I can't remember being this technically frustrated in the last 10 years, and I'm generally known as a practical pragmatist! :-(
>
>

"Ben Hinkle" <bhinkle@mathworks.com> wrote in message news:ceb6ur$1hn3$1@digitaldaemon.com...
> > - In order to support the parameterisable interface (e.g.
> IContainer!(int)) described above, there needs to be a common
> > way to manipulate built-in types, objects and structs. For some things,
> one can use traits, for others, overloaded
> > functions. Alas, there seems to be no way to discriminate structs via
> overloaded functions. Hence, currently the DTL
> > containers do not support polymorphic interfaces when storing structs.
> Naturally, this has much wider consequences
>
> Could TypeInfo's help here? Structs don't currently have TypeInfo so it won't solve the struct issue but Walter has said he knows TypeInfos for structs needs improvement so eventually it should get fixed. I've been using TypeInfo for compare, swap, equals and getHash and found them very useful.

Very probably. I confess my experience with/understanding of structs is not what you'd call comprehensive. :)