On Friday, 28 February 2014 at 18:42:57 UTC, Steve Teale wrote:
> All the D aficionados seem to wet their pants over
> meta-programming, but I struggle to find a place to use it.
>
> IIRC, I used it in a couple of places when I was trying to write
> library stuff for MySQL, but in my current project, I use it only
> once. That's when I want to stuff something onto my undo stack.
>
> For that I have two template functions - push(T)(MybaseClass* p,
> T t, int ID), and pushC, which is just the same except that it
> checks the top of the stack to see if the ID there is the same as what it is wanting to push.
>
> This has served me very reliably, but I struggle to find other
> places in the whole application where I would benefit from
> templates.
>
> Is this typical - libraries use templates, applications don't, or am I just being unimaginative?

This is certainly my experience with C++ and is why I wrote the
chapter on templates in the "Tango with D" book.  Personally
though, I use templates constantly.  For functions, the most
common case is to eliminate code duplication where I might
normally overload for different parameter types.

On 3/1/2014 1:00 PM, Steve Teale wrote:
> I have already dealt
> with the yada-yada cases by old-fashioned OOP.
>

As I see it, a big part of the benefit of templates is that they can help you avoid a lot of the downsides of OOP:

- OO Boilerplate.
- Multiple dispatch is ridiculously messy, having to resort to contortions like the visitor pattern.
- Upcasting looses compile-time type info.
- Decreased opportunity for compiler optimizations, because a single function handles multiple data types at runtime. So the optimizer can't generate optimal code for all the data types used.
- Extra indirections at runtime.
- Downcasting has a runtime cost.
- Poor cache behavior.
- Tendency for increased heap activity, which is not cheap at all.
- Lumping all data/functionality for a single "object" instance into the same physical chunk of memory causes problems for parallelization. And that's increasingly problematic on modern processors which work best when operating as "streaming-data processors". (See "Entity/Component Systems"[1] - There are good reasons videogame development has mostly switched from OOP to entity/component systems.)

Basically, while OOP certainly has benefits, it also has notable flaws that inherently slow down both the programmer and the computer. JVM goes to an enormous amount of trouble to mitigate that natural tendency, but it still has limits. And most languages (like D) can't expect to approach JVM's work on de-slow-ifying OO.

The runtime performance issues of OO aren't *always* a problem, but they can easily kill inner-loop/performance-sensitive sections of code. So with OO, you have to give up the generic/polymorphic benefits for any critical sections. Metaprogramming lets you still be generic even in the critical sections. This is demonstrated in sections #1-4 of an article I wrote a little while back[2]. It's a very contrived example, but even I was still surprised just *how* badly the OO version performed.

There's also this template primer[3] which might help, but I'm guessing it may be below your ability level.

[1] Entity/Component Systems: http://t-machine.org/index.php/2007/09/03/entity-systems-are-the-future-of-mmog-development-part-1/

[2] http://www.semitwist.com/articles/EfficientAndFlexible/MultiplePages/Page1/

[3] https://semitwist.com/articles/article/view/template-primer-in-d

Nick Sabalausky:

> - Lumping all data/functionality for a single "object" instance into the same physical chunk of memory causes problems for parallelization. And that's increasingly problematic on modern processors which work best when operating as "streaming-data processors". (See "Entity/Component Systems"[1] - There are good reasons videogame development has mostly switched from OOP to entity/component systems.)

Sometimes OOP is handy, for GUIs and other situations, but if today a more horizontal placement of fields is more efficient when high performance is needed, then perhaps a creative programmer has to find the courage to step out of his/her/hir language constraints to invent a better language. Is it possible to invent a language more fit for this different kind of data layout?

Bye,
bearophile

On 3/4/2014 7:42 PM, bearophile wrote:
> Nick Sabalausky:
>
>> - Lumping all data/functionality for a single "object" instance into
>> the same physical chunk of memory causes problems for parallelization.
>> And that's increasingly problematic on modern processors which work
>> best when operating as "streaming-data processors". (See
>> "Entity/Component Systems"[1] - There are good reasons videogame
>> development has mostly switched from OOP to entity/component systems.)
>
> Sometimes OOP is handy, for GUIs and other situations, but if today a
> more horizontal placement of fields is more efficient when high
> performance is needed, then perhaps a creative programmer has to find
> the courage to step out of his/her/hir language constraints to invent a
> better language. Is it possible to invent a language more fit for this
> different kind of data layout?
>

I don't think it's necessary to go as far as creating a new language. An entity-system in a low-level-capable language gives you enough power to control layouts however appropriate, and using them is pretty easy.

It seems intimidating at first, but it's really just like a relational DB: You have a unique ident for each "object" (ie, what an OO system would call an instantiated object, but in this case called an "entity"). And then all the data associated with the "object/entity" is stored in various tables/structs/whatever (in this case, called "components") with the entity's unique identifier being the primary key for each table/component. Functionality operates on a particular table/component, or a set of specific tables/components, either on a row-at-a-time basis or as the whole table as an aggregate.

Note that this imposes very little, if any, requirements on in-memory layouts, so things can be laid out however desired - possibly even with per-platform topologies thanks to the magic of metaprogramming. In any case, metaprogramming can help abstract away many of the internal memory-layout details.

Playing around with the Unity3D editor, and it's tutorial videos, can help grok the usage and modeling power of entities (like any good modern game engine, it uses an entity-based design). Although, FWIW, I'm not convinced Unity3D's design is able to *fully* take advantage of all the potential performance benefits of an entity system (mostly because of its CLR-based scripting and black-box closed-source engine).

But, I admit, I have wondered if a language could aid the creation/usage of entity systems with some special language features.

Nick Sabalausky:

> But, I admit, I have wondered if a language could aid the creation/usage of entity systems with some special language features.

I have seen that a good way to learn lazyness and purity is to try to write some Haskell code. Then you can use the same ideas in other languages, like D. Similarly I've studied regular expressions in dynamic languages, and now I am able to use them in Java, C#, D, etc.

So I've seen that it's good to learn a feature/style in a language that uses it a lot, or even in a language designed around such feature. Because languages shape your mind, and specialized languages train your mind to use few specific features. Later in real-world situations often you can't use such specialized/esoteric/rare language, and you have to use a common language as Java. And sometimes if people use a feature a lot in other languages, eventually it gets ported even to the common languages (like lambdas in Java).

So even if you can't or you don't want to use a new language to use entity systems, training your mind a bit thinking in a new language designed for it could help use it in a common language, or could even suggest you few features to add to a more general purpose language as D.

Wouter van Oortmerssen shows very well that designing many small specialized languages helps you sharpen your mind and later you apply those ideas to more general situations:
http://strlen.com/language-design-overview

Bye,
bearophile

On 3/3/2014 5:35 PM, Chris wrote:
>
> Maybe I'm a bit too philosophical about this. But consider the following
> (made up) case:
>
> struct MyTemp(T) {
>      // ...
>      T add(T a, T b) {
>          if (a is string && b is string) {
>                return a~b;  // or return a~"+"~b; or whatever
>          } else if (a is integer && a is integer) {
>                return a+b;
>          }
>      }
> }
>
> I don't know if this can be considered a "pure" template. It is a
> template of sorts, but the type specialization in the function add makes
> it a watered-down template,

Any useful template function already works that way. The mere + operator is *not* one single operation, but a whole category of opcodes which are chosen based on the operand's type. Thus, + can be thought of as a built-in template that [roughly] does this:

T opPlus(T a, T b) {
    if (T is integer) {
        [asm opcode for 32-bit addition] a, b
    } else if (T is ubyte) {
        [asm opcode for 8-bit addition] a, b
    } else if (T is float) {
        [asm opcode for floating point addition] a, b
    }
}

Specialization is what makes function templates useful. Some of the specialization is explicit (static if) and some is implicit (+ operator). But without specialization (explicit or implicit), all instantiations would be identical. If all instantiations are identical, why even have a template in the first place?

On 3/4/2014 1:27 PM, H. S. Teoh wrote:
> On Tue, Mar 04, 2014 at 06:19:38PM +0000, Chris wrote:
>>
>> Btw, the quote you have in this post:
>>
>> Never ascribe to malice that which is adequately explained by
>> incompetence. -- Napoleon Bonaparte
>>
>> I'm surprised that Napoleon would say something like this. Malice is
>> often a characteristic of the incompetent. The only way to get the
>> better of their betters. :-)
>
> I'm not sure if that attribution is accurate. Nick has pointed out to me
> that he knows the same quote attributed to someone else, so this may be
> a case of internet misattribution (I picked up that quote from somewhere
> online, way back when -- no idea if the source was reliable, y'know,
> being the internet and everything).
>

Hanlon's Razor, a tongue-in-cheek "corollary" (of sorts) to Occam's Razor. I'm a huge believer in Hanlon's Razor, BTW, as well as Occam's Razor which I see as an axiom (for a loose definition of "axiom") that's fundamental in making all of science actually work and separating science from folklore.

For all I know, Napoleon may have uttered Hanlon's Razor at some point. I'm sure he's said a lot of things :) If so, he may have been the first.

On 3/4/2014 9:00 PM, bearophile wrote:
> Nick Sabalausky:
>
>> But, I admit, I have wondered if a language could aid the
>> creation/usage of entity systems with some special language features.
>
> I have seen that a good way to learn lazyness and purity is to try to
> write some Haskell code. Then you can use the same ideas in other
> languages, like D. Similarly I've studied regular expressions in dynamic
> languages, and now I am able to use them in Java, C#, D, etc.
>
> So I've seen that it's good to learn a feature/style in a language that
> uses it a lot, or even in a language designed around such feature.
> Because languages shape your mind, and specialized languages train your
> mind to use few specific features. Later in real-world situations often
> you can't use such specialized/esoteric/rare language, and you have to
> use a common language as Java. And sometimes if people use a feature a
> lot in other languages, eventually it gets ported even to the common
> languages (like lambdas in Java).
>
> So even if you can't or you don't want to use a new language to use
> entity systems, training your mind a bit thinking in a new language
> designed for it could help use it in a common language, or could even
> suggest you few features to add to a more general purpose language as D.
>
> Wouter van Oortmerssen shows very well that designing many small
> specialized languages helps you sharpen your mind and later you apply
> those ideas to more general situations:
> http://strlen.com/language-design-overview
>

Yea, all good points. I'm reminded of one of my favorite quotes from Joel Spolsky:

"I have never met anyone who can do Scheme, Haskell, and C pointers who can't pick up Java in two days, and create better Java code than people with five years of experience in Java, but try explaining that to the average HR drone."
  - http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html

On Wednesday, 5 March 2014 at 02:28:00 UTC, Nick Sabalausky wrote:
> On 3/3/2014 5:35 PM, Chris wrote:
>>
>> Maybe I'm a bit too philosophical about this. But consider the following
>> (made up) case:
>>
>> struct MyTemp(T) {
>>     // ...
>>     T add(T a, T b) {
>>         if (a is string && b is string) {
>>               return a~b;  // or return a~"+"~b; or whatever
>>         } else if (a is integer && a is integer) {
>>               return a+b;
>>         }
>>     }
>> }
>>
>> I don't know if this can be considered a "pure" template. It is a
>> template of sorts, but the type specialization in the function add makes
>> it a watered-down template,
>
> Any useful template function already works that way. The mere + operator is *not* one single operation, but a whole category of opcodes which are chosen based on the operand's type. Thus, + can be thought of as a built-in template that [roughly] does this:
>
> T opPlus(T a, T b) {
>     if (T is integer) {
>         [asm opcode for 32-bit addition] a, b
>     } else if (T is ubyte) {
>         [asm opcode for 8-bit addition] a, b
>     } else if (T is float) {
>         [asm opcode for floating point addition] a, b
>     }
> }
>
> Specialization is what makes function templates useful. Some of the specialization is explicit (static if) and some is implicit (+ operator). But without specialization (explicit or implicit), all instantiations would be identical. If all instantiations are identical, why even have a template in the first place?

Yes, this is perfectly clear to me. But the opcodes only matter to the compiler not to the programmer, i.e. the logical concept that one number (integer, float ...) is added to another remains the same. That's what I was saying.

The use of templates to avoid code duplication has been mentioned (cf. Sean Kelly's post), however real duplication is rare in specialized programs, because each function or method has it's own specialized job, else we wouldn't have them. Even if we suppose that we might want to cater for string, dchar, wchar without having to write the function three times, there might be slight differences in handling the three types so we have to fork within the template. What you are basically doing then is to pack three different functions (or the parts in which they differ) in one, at programming time (not compile time). This means you introduce three _different_ types of logic / handling mechanisms. Philosophically this is not a pure template, imo. The question is where does it end? Can you turn the whole program into one big template that handles all the different types and types of logic? (Exaggerating :-) The example I showed above is maybe not so far fetched add(int, int) and add(string, string).

The points you made above about OOP are very good and I'm increasingly unhappy with OOP's limitations. However, turning to templates I find myself stuck sometimes too, and I don't feel good about "compromising" a template by using too many if's for different types or introducing new logic that should actually go into a separate function / method. I'm really interested in different approaches like templates and entities, especially if they help optimizing code for processors and memory. However, I sometimes think that in Software development we always hit the same wall, whether we go from assembly to C, to OOP, to templates. The wall is that every real world program is highly unique and specialized and abstraction goes only so far.

One good use for templates, I figure, would be writing an interpreter for a scripting or DSL. As in:

greeting = "Hello, " + "world!";
result = 1 + 1;

greeting = add(string, string)
result = add(int, int)

On Sunday, 2 March 2014 at 18:59:23 UTC, Steve Teale wrote:
> On Sunday, 2 March 2014 at 15:23:03 UTC, H. S. Teoh wrote:
>> This is a pretty good primer to templates:
>>
>> 	https://semitwist.com/articles/article/view/template-primer-in-d
>>
>>
>>
> The trouble is with most of these tutorials that they offer examples that are things you would probably never want to do. I can already add an int to an int, or a double to a double, or an int to a double.
>
> Perhaps the examples should pick on something like vector operations, but then who would be doing those with int, or some class? It would be doubles or pairs of, as in struct Coord.




import std.stdio;
import std.algorithm;

void add(T,size_t N)(ref T[N] result, const T[N] a, const T[N] b)
{
  result[0] = a[0]+b[0];
  static if (N > 1)
    result[1]=a[1]+b[1];
}

void main()
{
	int [2] a = [1,2];
	int [2] b= [3,4];
	int [2] result;

	result.add(a,b);
	writeln(result);

	result[] = a[] + b[];
	writeln(result);
}


>
> I believe readers would study documentation and examples much more carefully if they were things they might realistically want to do. And that won't be type conversion - std.conv already does a pretty good job on that. So what?
>
> We could really do with a place where template savvy open source contributors could publish interesting examples of template use.
>
> Otherwise, Joe Soap, like me, can spend a great deal of time and effort in:
>
> a) Determining when the use of a template might be advantageous,
> b) Hacking at test programs to determine what the documentation means, and what works, and what doesn't.
> c) After that, deciding whether it would be just as effective to use two or three separate methods.
>
> Steve
>
>
> Steve

Are there any disadvantages of using a fixed size array for fixed size
coordinates and vectors, over creating an actual typedef or struct Vec3?