Am 17.01.19 um 23:20 schrieb Stefan Koch:
> For 2 years I have pondered this problem, and I did come up with a
> solution.
> It's actually not that hard to have CTFE interact with type-tuples.
> You can pass them as function parameters, or return them if you wish.
> Of course a type-tuple returning ctfe function, is compile-time only.
> This solved one more problem that ctfe has:
> helper functions required for ctfe can only be omitted from the binary,
> if you use the trick of putting them into a module which is the import
> path but never explicitly given on the command line.
> newCTFE has the facility to be extended for this, and implementing
> type-functions is at least on my personal roadmap.
> 
> At Dconf 2018 Andrei and Walter said, a DIP which is substantiated
> enough might make it.
> However due to lack of time, (and productivity-reducing internal
> changes) it will take some time until I can get started on this.
> 
> Also I plan for newCTFE to be in shape before I add type-manipulation abilities.
> 
> Cheers,
> 
> Stefan
> 
> P.S. There is one caveat: because of how type-functions work they
> cannot, you cannot create a non-anonymous symbol inside a type-function,
> because there is no way to infer a mangle.
> You can however create an anonymous symbol and alias it inside a
> template body, which gives it a mangle and it can behave like a regular
> symbol.
> 
> 
This is one of the most exciting things i have read in recent times!

On Thursday, 17 January 2019 at 22:44:08 UTC, H. S. Teoh wrote:
> On Thu, Jan 17, 2019 at 10:20:24PM +0000, Stefan Koch via Digitalmars-d-announce wrote:
>> P.S. There is one caveat: because of how type-functions work they cannot, you cannot create a non-anonymous symbol inside a type-function, because there is no way to infer a mangle.
>>
>> You can however create an anonymous symbol and alias it inside a template body, which gives it a mangle and it can behave like a regular symbol.
>
> Interesting.  Is it possible to assign a "fake" mangle to type functions that never actually gets emitted into the object code, but just enough to make various internal compiler stuff that needs to know the mangle work properly?

No this is not possible, a symbol which is only used at compile-time is actually really rare. Actually If the symbol is constrained to a compile-time only context (e.g. inside is() or taking the .sizeof) this problem does not arise, and it could be allowed.

Imagine you want to return a struct type  which has all the fields of a given base struct but adds a member.
>> module ct_helper;
> alias f(alias baseT, alias newMemberType, string newMember_name)
> {
>    struct Extended
>    {
>        baseT base;
>        mixin("newMemberType " ~ newMemberName);
>    }
>    return typeof(Extended.init);
>}
------
>> module user
struct S1 { int x; }
alias S2 = f!(S1, float, "y") // looks like a template-instantiation but it's not!, I am just reusing it, to not confuse the parser to much.

which mangle should this get?
S2 ?  - doesn't work there is no mangle for an alias.
 ct_helper.f.Extended? doesn't work if we call the type-function again with diffrent arguments
make it an anonymous type ? - If we do that than this means that the type-function is no longer pure as two anonymous types can never Equal each other

include the arguments to the type-function and it's parameters in the mangle? - that's possible but type-functions are not meant to leave any sign of their existence in order to not introduce ABI issues.

In short for now I'd rather side-step the problem by not allowing freshly minted types to escape into a runtime context without going through a template wrapper (which also handles caching and has proper behavior in is-expressions).

On 1/17/2019 8:06 AM, bpr wrote:
> Was that a pre C++11 version of C++, or a more modern one?

pre C++11

On 1/17/2019 11:31 AM, H. S. Teoh wrote:
> [...]

Thanks for the thoughtful and well-written piece.

But there is a counterpoint: symmetry in mathematics is one thing, but symmetry in human intuition is not. Anytime one is dealing in human interfaces, one runs into this. I certainly did with the way imports worked in D. The lookups worked exactly the same for any sort of symbol lookup. I thought it was great.

But I was unable to explain it to others. Nobody could understand it when I said imported symbol lookup worked exactly like any lookup in a name space. They universally said it was "unintuitive", filed bug reports, etc. Eventually, I had to give it up. Now import lookup follows special different rules, people are happy, and I learned (again) that symmetry doesn't always produce the best outcomes.

User interfaces (and programming languages certainly are user interfaces) are hard and (ironically) are anything but intuitive to design.

On 2019-01-17 23:44, H. S. Teoh wrote:

> YES!  This is the way it should be.  Type-tuples become first class
> citizens, and you can pass them around to functions and return them from
> functions
No no no, not only type-tuples, you want types to be first class citizens. This makes it possible to store a type in a variable, pass it to and return from functions. Instead of a type-tuple, you want a regular array of types. Then it would be possible to use the algorithms in std.algorithm to manipulate the arrays. I really hate that today one needs to resort to things like staticMap and staticIndexOf.

Of course, if we both get tuples and types as first class citizens it would be possible to store types in these tuples as well. But a tuple is usually immutable and I'm not sure if it would be possible to use std.algorithm on that.

It would be awesome to be able to do things like this:

type foo = int;

type bar(type t)
{
    return t;
}

auto u = [byte, short, int, long].map!(t => t.unsigned).array;
assert(u == [ubyte, ushort, uint, ulong];

-- 
/Jacob Carlborg

On 2019-01-17 23:44, H. S. Teoh wrote:

> Interesting.  Is it possible to assign a "fake" mangle to type functions
> that never actually gets emitted into the object code, but just enough
> to make various internal compiler stuff that needs to know the mangle
> work properly?

Not sure that would be possible. I tries to a support for pragma(mangle) on alias declarations. That opened a can of worms. It turns out that the compiler is using the mangling of a type to compare types internally.

-- 
/Jacob Carlborg

On Thu, Jan 17, 2019 at 05:32:52PM -0800, Walter Bright via Digitalmars-d-announce wrote:
> On 1/17/2019 11:31 AM, H. S. Teoh wrote:
> > [...]
> 
> Thanks for the thoughtful and well-written piece.
> 
> But there is a counterpoint: symmetry in mathematics is one thing, but symmetry in human intuition is not. Anytime one is dealing in human interfaces, one runs into this.  I certainly did with the way imports worked in D. The lookups worked exactly the same for any sort of symbol lookup. I thought it was great.
> 
> But I was unable to explain it to others. Nobody could understand it when I said imported symbol lookup worked exactly like any lookup in a name space.  They universally said it was "unintuitive", filed bug reports, etc.  Eventually, I had to give it up. Now import lookup follows special different rules, people are happy, and I learned (again) that symmetry doesn't always produce the best outcomes.

Alas, it's true, it's true, 100% symmetry is, in the general case, impossible to achieve.  If we wanted 100% mathematical symmetry, one could argue lambda calculus is the best programming language ever, because it's Turing complete, the syntax is completely uniform with no quirky exceptions, and the semantics are very clearly defined with no ambiguity anywhere.  Unfortunately, these very characteristics are also what makes lambda calculus impossible to work with for anything but the most trivial of programs. It's completely unmaintainable, extremely hard to read, and has non-trivial semantics that vary wildly from the smallest changes to the code.

For a human-friendly programming language, any symmetry must necessarily be based on human expectations.  Unfortunately, as you learned, human intuition varies from person to person, and indeed, is often inconsistent even with the same person, so trying to maximise symmetry in a way that doesn't become "counterintuitive" is a pretty tall order.

As somebody (perhaps you) said once, in Boeing's experience with designing intuitive UIs, they discovered that what people consider "intuitive" is partly colored by their experience, and their experience is in turn shaped by the UIs they interact with.  So it's a feedback loop, which means what's "intuitive" is not some static set of rules (even allowing for arbitrarily complex rules), but it's a *moving target*, the hardest thing to design for.  What's considered "intuitive" today may be considered "totally counter-intuitive" 10 years from now.

In the case of imports, I'd argue that the problem is with how people understand the word "import".  From a compiler's POV, the simplest, most straightforward (and most symmetric!) definition is "pull in the symbols into the local scope".  Unfortunately, that's not the understanding most programmers have.  Perhaps in an older, bygone era people might have been more open to that sort of definition, but in this day and age of encapsulation and modularity, "pull in symbols into the local scope" does not adequately capture people's expectations: it violates encapsulation, in the following sense: symbols from the imported module shadow local symbols, which goes against the expectation that the local module is an encapsulated thing, inviolate from outside interference. It breaks the expectation of encapsulation.  It breaks the symmetry that everywhere else, outside code cannot interfere with local symbols.

Consequently, the expectation is that imported symbols are somehow "second class" relative to local symbols -- imported symbols don't shadow local symbols (unless you explicitly ask for it), and thus encapsulation is preserved (in some sense).  So we have here a conflict between different axes of symmetry: the symmetry of every module being an inviolate, self-contained unit (encapsulation), and the symmetry of having the same rules for symbol lookup no matter where the symbol came from.  It's a toss-up which axis of symmetry one should strive for, and which one should be compromised.

I'd say the general principle ought to be that the higher-level symmetry (encapsulation of modules) should override the lower-level symmetry (the mechanics of symbol lookup).  But this is easy to say because hindsight is 20/20; it's not so simple at the time of decision-making because it's not obvious which symmetries are in effect and what their relative importance should be.  And there's always the bugbear that symmetry from the implementor's (compiler writer's) POV does not necessarily translate to symmetry from the user's (language user's) POV.

Still, I'd say that in a general sense, symmetry ought to be a relatively high priority as far as designing language features or adding/changing features are concerned.  Adding a new feature with little regard for how it interacts with existing features, what new corner cases it might introduce, etc., is generally a bad idea. Striving for maximal symmetry should at least give you a ballpark idea for where things should be headed, even if working out the details is generally not so straightforward.  It may be that you will discover mutually-exclusive symmetries that, no matter which way you take, will lead to trouble later on.  Conflicts of this sort usually cannot be fully resolved without essentially gutting the entire system to the core and rebuilding from scratch, which usually is not a practical thing to do.  So we're stuck with imperfection.  Nevertheless, IMO we should still strive for perfection, even if it's ultimately unattainable. Being slightly closer to perfection is better than being far off.


> User interfaces (and programming languages certainly are user
> interfaces) are hard and (ironically) are anything but intuitive to
> design.

Yes, it's very unintuitive to design intuitive interfaces. :-D  It's the paradox of user-interface design.


T

-- 
Life is too short to run proprietary software. -- Bdale Garbee

On Fri, Jan 18, 2019 at 11:23:11AM +0100, Jacob Carlborg via Digitalmars-d-announce wrote:
> On 2019-01-17 23:44, H. S. Teoh wrote:
> 
> > YES!  This is the way it should be.  Type-tuples become first class citizens, and you can pass them around to functions and return them from functions
> No no no, not only type-tuples, you want types to be first class citizens.  This makes it possible to store a type in a variable, pass it to and return from functions. Instead of a type-tuple, you want a regular array of types.  Then it would be possible to use the algorithms in std.algorithm to manipulate the arrays. I really hate that today one needs to resort to things like staticMap and staticIndexOf.

Yes, that would be the next level of symmetry. :-D  Types as first class citizens would eliminate another level of distinctions that leads to the necessity of staticMap, et al.  But it will also require changing the language in a much more fundamental, invasive way.

So I'd say, let's take it one step at a time.  Start with first-class type-tuples, then once that's ironed out and working well, take it to the next level and have first-class types.  Trying to leap from here to there in one shot is probably a little too ambitious, with too high a chance of failure.


[...]
> It would be awesome to be able to do things like this:
> 
> type foo = int;
> 
> type bar(type t)
> {
>     return t;
> }
> 
> auto u = [byte, short, int, long].map!(t => t.unsigned).array;
> assert(u == [ubyte, ushort, uint, ulong];
[...]

Yes this would be awesome.  But in order to avoid unmanageable complexity of implementation, all of this would have to be compile-time only constructs.


T

-- 
Your inconsistency is the only consistent thing about you! -- KD

On Friday, 18 January 2019 at 10:23:11 UTC, Jacob Carlborg wrote:
> On 2019-01-17 23:44, H. S. Teoh wrote:
>
>> YES!  This is the way it should be.  Type-tuples become first class
>> citizens, and you can pass them around to functions and return them from
>> functions
> No no no, not only type-tuples, you want types to be first class citizens. This makes it possible to store a type in a variable, pass it to and return from functions. Instead of a type-tuple, you want a regular array of types. Then it would be possible to use the algorithms in std.algorithm to manipulate the arrays. I really hate that today one needs to resort to things like staticMap and staticIndexOf.
>
> Of course, if we both get tuples and types as first class citizens it would be possible to store types in these tuples as well. But a tuple is usually immutable and I'm not sure if it would be possible to use std.algorithm on that.
>
> It would be awesome to be able to do things like this:
>
> type foo = int;
>
> type bar(type t)
> {
>     return t;
> }
>
> auto u = [byte, short, int, long].map!(t => t.unsigned).array;
> assert(u == [ubyte, ushort, uint, ulong];

Yes, you will be able to do exactly what you describe above.
type-tuples are strictly a superset of types; which also include true compile-time constants. (e.g. things you can use to instantiate a template with.)

Within type functions you are able to create `alias[]` which is in some ways equivalent to type-tuple (and will be converted to one upon being returned outside of compile-functions),which you can append to if you own it and type functions can also take other type-functions as parameters.
Therefore it's perfectly possible to implement staticMap in terms of type functions.
I already did the semantic sanity checks, and it shows promise.

The only difference that type-functions have from what you describe is that it does not need to occupy a keyword 'type'.

Cheers,
Stefan

On Thursday, 17 January 2019 at 20:47:38 UTC, Steven Schveighoffer wrote:
>
> well, there was no static foreach for that article (which I admit I didn't read, but I know what you mean).
>
> But it's DEFINITELY not as easy as it could be:
>
> import std.conv;
>
> alias AliasSeq(P...) = P;
>
> template staticMap(alias Transform, Params...)
> {
>     alias seq0 = Transform!(Params[0]);
>     static foreach(i; 1 .. Params.length)
>     {
>        mixin("alias seq" ~ i.to!string ~ " = AliasSeq!(seq" ~ (i-1).to!string ~ ", Transform!(Params[" ~ i.to!string ~ "]));");
>     }
>     mixin("alias staticMap = seq" ~ (Params.length-1).to!string ~ ";");
> }
>
> alias Constify(T) = const(T);
> void main()
> {
>     alias someTypes = AliasSeq!(int, char, bool);
>     pragma(msg, staticMap!(Constify, someTypes)); // (const(int), const(char), const(bool))
> }
>
> Note, that this would be a LOT easier with string interpolation...
>
> mixin("alias seq${i} = AliasSeq!(seq${i-1}, Transform!(Params[${i}]));".text);
>
> -Steve

Why not do away with AliasSeq and use strings all the way?

string Constify(string type)
{
    // can add input checks here
    return "const(" ~ type ~ ")";
}

void main()
{
    import std.algorithm : map;
    enum someTypes = ["int", "char", "bool"];
    enum constTypes = map!Constify(someTypes);
    mixin(constTypes[0] ~ "myConstInt = 42;"); // int myConstInt = 42;
}

Represent types as strings, CTFE them as you see fit, and output a string that can then be mixin'ed to use the actual type. :)