As was recently pointed out on the Phobos list ( http://forum.dlang.org/post/CAEnAdhZTcuOL50TeRwCCG=E99nCA_Gh=y=TtHeUGKNuEaC072Q@mail.gmail.com ), we technically have std.typelist (which I'd never heard of before that post), which Bartosz Milewski put together a few years ago. However, it's undocumented and not actually included in the build at all. The question is what to do with it.

https://github.com/D-Programming-Language/phobos/blob/master/std/typelist.d

It probably needs a bit of cleanup (e.g. making sure it follows the proper naming coventions everywhere), but honestly, I think that it looks pretty cool, and it probably wouldn't take much work to make it properly releasable.

Do we want to have someone look it over, take it over, and submit it for review? Or do we want to simply give it a look over and fully add it after a few tweaks? Or do we want to just toss it altogether? What do we want to do with it? We really shouldn't be leaving it sitting there like it is. As it stands, it's just forgotten cruft, which is a shame, since it does look pretty cool.

- Jonathan M Davis

On 22-06-2012 20:06, Jonathan M Davis wrote:
> As was recently pointed out on the Phobos list (
> http://forum.dlang.org/post/CAEnAdhZTcuOL50TeRwCCG=E99nCA_Gh=y=TtHeUGKNuEaC072Q@mail.gmail.com
> ), we technically have std.typelist (which I'd never heard of before that
> post), which Bartosz Milewski put together a few years ago. However, it's
> undocumented and not actually included in the build at all. The question is
> what to do with it.
>
> https://github.com/D-Programming-Language/phobos/blob/master/std/typelist.d
>
> It probably needs a bit of cleanup (e.g. making sure it follows the proper
> naming coventions everywhere), but honestly, I think that it looks pretty
> cool, and it probably wouldn't take much work to make it properly releasable.
>
> Do we want to have someone look it over, take it over, and submit it for
> review? Or do we want to simply give it a look over and fully add it after a
> few tweaks? Or do we want to just toss it altogether? What do we want to do
> with it? We really shouldn't be leaving it sitting there like it is. As it
> stands, it's just forgotten cruft, which is a shame, since it does look pretty
> cool.
>
> - Jonathan M Davis
>

Yeah, I really think we need to ship this stuff.

I might have a poke at it within the next few days. I suspect some of the unit tests might fail due to various changes in the compile-time reflection parts of the compiler over the years, so I'll probably have to fix those while I'm at it.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On Fri, 22 Jun 2012 20:06:10 +0200, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> As was recently pointed out on the Phobos list (
> http://forum.dlang.org/post/CAEnAdhZTcuOL50TeRwCCG=E99nCA_Gh=y=TtHeUGKNuEaC072Q@mail.gmail.com
> ), we technically have std.typelist (which I'd never heard of before that
> post), which Bartosz Milewski put together a few years ago. However, it's
> undocumented and not actually included in the build at all.


I added a ticket for this back in March: http://d.puremagic.com/issues/show_bug.cgi?id=7797

Wow, I never noticed that this file exists, even though I'm routinely doing metaprogramming-heavy stuff…

The obligatory references to other »meta« code:
[1] https://github.com/sinfu/phobos-sandbox/blob/master/std/internal/meta/meta.d
[2] https://github.com/PhilippeSigaud/dranges/blob/master/templates.d
[3] https://gist.github.com/1191885 (this was a one-off selection of my own code for a NG discussion, feel free to ask for more/an updated version)

On Friday, 22 June 2012 at 18:10:38 UTC, Alex Rønne Petersen wrote:
> On 22-06-2012 20:06, Jonathan M Davis wrote:
>> The question is what to do with it. […]
> Yeah, I really think we need to ship this stuff.

I'd rather not ship it without more consideration and a proper review process. Yes, functionality supporting more advanced manipulation of compile time lists/tuples is definitely needed (cf. the countless std.meta discussions), but:

 1) The term TypeList does not make much sense at all, as compile time tuples/lists aren't at all restricted to types. In retrospect, giving TypeTuples the name they bear was a mistake, we shouldn't repeat it. The genreral template would become meta.List (no, I never got around to finishing my proposal), and a meta.TypeList constructor could then ensure that only types are accepted (similar for meta.ExprList, or whatever good names there might be).

 2)  I'm not sure if introducing a concept which is essentially the same as TypeTuple, but doesn't entirely replace it for »high-level« use, is the right way to go. Yes, functional-style head/tail lists are a natural fit for many meta-algorithms, since the semantics of templates mostly require a very functional style anyway (you'll frequently see T[0] and T[1 .. $] in type tuples-heavy code). And yes, the »auto-expanding« property of TypeTuples can sometimes lead to unexpected results, and for some »higher-order« templates, you need to confine them into another template (imagine TypeList with only toTuple, see [1], [3]).

But on the other hand, type tuples are deeply anchored in the language (variadics, ...), and better match the general array/range theme of D, thus probably reducing »mental overhead« for people not used to functional programming.

This is not to say that I don't find the std.typelist concept interesting. We just should be very clear on how to go forward with »meta« algorithms in Phobos before going forward with this. Developing std.typelist and std.typetuple side by side, and then at some later point also introducing std.meta doesn't make much sense, in my eyes.

 3) I think going for an explicit »apply« method for template predicate is not worth the hassle in the general case (in user code, which usually just passes predicates to higher order templates). Instead, I prefer using an explicit Apply/Instantiate template where needed to overcome grammar limitations, which mostly is in the implementation of the primitives.

 4) I know this is picking nits at this stage, but I think And/Or should implement short-cut evaluation, like in [3].

One more thing I noticed is that the module doc comment solely lists Bartosz as the author, but the file has another copyright comment mentioning Burton Radons. This situation should definitely be clarified.

David

On 22-06-2012 21:13, David Nadlinger wrote:
> Wow, I never noticed that this file exists, even though I'm routinely
> doing metaprogramming-heavy stuff…
>
> The obligatory references to other »meta« code:
> [1]
> https://github.com/sinfu/phobos-sandbox/blob/master/std/internal/meta/meta.d
>
> [2] https://github.com/PhilippeSigaud/dranges/blob/master/templates.d
> [3] https://gist.github.com/1191885 (this was a one-off selection of my
> own code for a NG discussion, feel free to ask for more/an updated version)
>
> On Friday, 22 June 2012 at 18:10:38 UTC, Alex Rønne Petersen wrote:
>> On 22-06-2012 20:06, Jonathan M Davis wrote:
>>> The question is what to do with it. […]
>> Yeah, I really think we need to ship this stuff.
>
> I'd rather not ship it without more consideration and a proper review
> process. Yes, functionality supporting more advanced manipulation of
> compile time lists/tuples is definitely needed (cf. the countless
> std.meta discussions), but:
>
>   1) The term TypeList does not make much sense at all, as compile time
> tuples/lists aren't at all restricted to types. In retrospect, giving
> TypeTuples the name they bear was a mistake, we shouldn't repeat it. The
> genreral template would become meta.List (no, I never got around to
> finishing my proposal), and a meta.TypeList constructor could then
> ensure that only types are accepted (similar for meta.ExprList, or
> whatever good names there might be).
>
>   2)  I'm not sure if introducing a concept which is essentially the
> same as TypeTuple, but doesn't entirely replace it for »high-level« use,
> is the right way to go. Yes, functional-style head/tail lists are a
> natural fit for many meta-algorithms, since the semantics of templates
> mostly require a very functional style anyway (you'll frequently see
> T[0] and T[1 .. $] in type tuples-heavy code). And yes, the
> »auto-expanding« property of TypeTuples can sometimes lead to unexpected
> results, and for some »higher-order« templates, you need to confine them
> into another template (imagine TypeList with only toTuple, see [1], [3]).
>
> But on the other hand, type tuples are deeply anchored in the language
> (variadics, ...), and better match the general array/range theme of D,
> thus probably reducing »mental overhead« for people not used to
> functional programming.
>
> This is not to say that I don't find the std.typelist concept
> interesting. We just should be very clear on how to go forward with
> »meta« algorithms in Phobos before going forward with this. Developing
> std.typelist and std.typetuple side by side, and then at some later
> point also introducing std.meta doesn't make much sense, in my eyes.
>
>   3) I think going for an explicit »apply« method for template predicate
> is not worth the hassle in the general case (in user code, which usually
> just passes predicates to higher order templates). Instead, I prefer
> using an explicit Apply/Instantiate template where needed to overcome
> grammar limitations, which mostly is in the implementation of the
> primitives.
>
>   4) I know this is picking nits at this stage, but I think And/Or
> should implement short-cut evaluation, like in [3].
>
> One more thing I noticed is that the module doc comment solely lists
> Bartosz as the author, but the file has another copyright comment
> mentioning Burton Radons. This situation should definitely be clarified.
>
> David

Well, author is not necessarily the same as copyright holder. Author usually means "maintainer" or "previous maintainer", while copyright could simply indicate that some code was borrowed.

-- 
Alex Rønne Petersen
alex@lycus.org
http://lycus.org

On 06/22/12 21:13, David Nadlinger wrote:
> This is not to say that I don't find the std.typelist concept interesting. We just should be very clear on how to go forward with »meta« algorithms in Phobos before going forward with this. Developing std.typelist and std.typetuple side by side, and then at some later point also introducing std.meta doesn't make much sense, in my eyes.

+1, I'd add std.typecons to this list ( std.typecons.Tuple in particular ). The boundaries of each of those modules are kind of blurry and overlapping.

On Friday, 22 June 2012 at 19:52:37 UTC, Guillaume Chatelet wrote:
> +1, I'd add std.typecons to this list ( std.typecons.Tuple in particular ).
> The boundaries of each of those modules are kind of blurry and overlapping.

std.typecons.Tuple lives in the run-time (and CTFE, for that matter) domain, while the stuff discussed here happens in the compile-time type system) world There are some situations in which this line is blurred (e.g. in the case of »type tuples of runtime values«, like you get for variadic template functions), but there is a deep conceptual difference between the two modules.

David

On 06/22/12 22:16, David Nadlinger wrote:
> 
> std.typecons.Tuple lives in the run-time (and CTFE, for that matter) domain, while the stuff discussed here happens in the compile-time type system) world There are some situations in which this line is blurred (e.g. in the case of »type tuples of runtime values«, like you get for variadic template functions), but there is a deep conceptual difference between the two modules.

Yes. I re-read std.typecons documentation, it's quite clear about that. I think I mixed them up because I tend to use both frequently. Thank you for clarifying.

On Friday, June 22, 2012 21:13:07 David Nadlinger wrote:
> Wow, I never noticed that this file exists, even though I'm routinely doing metaprogramming-heavy stuff…
> 
> The obligatory references to other »meta« code:
> [1]
> https://github.com/sinfu/phobos-sandbox/blob/master/std/internal/meta/meta.d
> [2]
> https://github.com/PhilippeSigaud/dranges/blob/master/templates.d
> [3] https://gist.github.com/1191885 (this was a one-off selection
> of my own code for a NG discussion, feel free to ask for more/an
> updated version)
> 
> On Friday, 22 June 2012 at 18:10:38 UTC, Alex Rønne Petersen
> 
> wrote:
> > On 22-06-2012 20:06, Jonathan M Davis wrote:
> >> The question is what to do with it. […]
> > 
> > Yeah, I really think we need to ship this stuff.
> 
> I'd rather not ship it without more consideration and a proper review process. Yes, functionality supporting more advanced manipulation of compile time lists/tuples is definitely needed (cf. the countless std.meta discussions), but:
> 
> 1) The term TypeList does not make much sense at all, as compile time tuples/lists aren't at all restricted to types. In retrospect, giving TypeTuples the name they bear was a mistake, we shouldn't repeat it. The genreral template would become meta.List (no, I never got around to finishing my proposal), and a meta.TypeList constructor could then ensure that only types are accepted (similar for meta.ExprList, or whatever good names there might be).
> 
> 2) I'm not sure if introducing a concept which is essentially the same as TypeTuple, but doesn't entirely replace it for »high-level« use, is the right way to go. Yes, functional-style head/tail lists are a natural fit for many meta-algorithms, since the semantics of templates mostly require a very functional style anyway (you'll frequently see T[0] and T[1 .. $] in type tuples-heavy code). And yes, the »auto-expanding« property of TypeTuples can sometimes lead to unexpected results, and for some »higher-order« templates, you need to confine them into another template (imagine TypeList with only toTuple, see [1], [3]).
> 
> But on the other hand, type tuples are deeply anchored in the language (variadics, ...), and better match the general array/range theme of D, thus probably reducing »mental overhead« for people not used to functional programming.
> 
> This is not to say that I don't find the std.typelist concept interesting. We just should be very clear on how to go forward with »meta« algorithms in Phobos before going forward with this. Developing std.typelist and std.typetuple side by side, and then at some later point also introducing std.meta doesn't make much sense, in my eyes.
> 
> 3) I think going for an explicit »apply« method for template predicate is not worth the hassle in the general case (in user code, which usually just passes predicates to higher order templates). Instead, I prefer using an explicit Apply/Instantiate template where needed to overcome grammar limitations, which mostly is in the implementation of the primitives.
> 
> 4) I know this is picking nits at this stage, but I think And/Or should implement short-cut evaluation, like in [3].
> 
> One more thing I noticed is that the module doc comment solely lists Bartosz as the author, but the file has another copyright comment mentioning Burton Radons. This situation should definitely be clarified.

I tend to agree with all of this. I think that std.typelist looks very cool, but we need to make sure that it fits in well with what we want to do. Simply adding it to the makefiles and documentation without much in the way of discussion is probably a bad idea. I suppose that the real question is where we want to go with this and who's going to spearhead sorting out the fancier metadata stuff - particularly since std.typelist is a lot more of a shift than simply adding a trait or two to std.traits.

- Jonathan M Davis

On Friday, June 22, 2012 21:13:07 David Nadlinger wrote:
>   2)  I'm not sure if introducing a concept which is essentially
> the same as TypeTuple, but doesn't entirely replace it for
> »high-level« use, is the right way to go. Yes, functional-style
> head/tail lists are a natural fit for many meta-algorithms, since
> the semantics of templates mostly require a very functional style
> anyway (you'll frequently see T[0] and T[1 .. $] in type
> tuples-heavy code). And yes, the »auto-expanding« property of
> TypeTuples can sometimes lead to unexpected results, and for some
> »higher-order« templates, you need to confine them into another
> template (imagine TypeList with only toTuple, see [1], [3]).
> 
> But on the other hand, type tuples are deeply anchored in the language (variadics, ...), and better match the general array/range theme of D, thus probably reducing »mental overhead« for people not used to functional programming.

But you pretty much _have_ to program stuff like eponymous templates and template constraints functionally. The only real exception that I can think of is

is(typeof({ my code here...}))

All the rest of them are functional. So, having a module which gives you the tools to handle them functionally will probably _reduce_ the mental overhead rather than reduce it. The real mental overhead is understanding how to program functionally. And we'd need to redesign how they work if we wanted to make it so that they aren't so functional in nature (which wouldn't be easy, because metaprogramming is inherently functional). So, I expect that std.typelist has the right basic approach, and I think that we should work towards adopting something along those lines. The question is how it all fits in with what we've been doing. But I'd very much like to see us adopting something similar to std.typelist as the normal way to handle a lot of this stuff rather than constantly doing stuff like T[0] and T[1 .. $].

- Jonathan M Davis


P.S. Bartosz actually wrote an interesting article a few years ago about how how template metaprogramming in C++ is like programming in Haskell:

http://bartoszmilewski.com/2009/10/21/what-does-haskell-have-to-do-with-c/