Versioning Phobos would free us from maintaining backward compatibility with a variety of decisions that did not withstand the test of time:

- autodecoding
- liberal use of the GC
- antediluvian run-time support library API (Object, Typeinfo, ...)
- liberal throwing of exceptions
- many others

Goals of library versioning:

- avoid copy-and-paste code duplication across versions
- support mixed use of versions for incremental migration
- allow library writers to structure code in a versioning-friendly manner

Non-goals:

- migrate code that was not written for versioning in a black-box manner (i.e. without touching it)
- support arbitrarily radical API changes with no effort

To open the discussion, here's a simple example: say we want to port the mismatch function in std.algorithm.comparison to a future std2 version that removes autodecoding. (I'll use the two-parameters version for simplicity.)

Tuple!(Range1, Range2)
mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
if (isInputRange!(Range1) && isInputRange!(Range2))
{
    for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
    {
        if (!binaryFun!(pred)(r1.front, r2.front)) break;
    }
    return tuple(r1, r2);
}

This same source code would work with or without autodecoding. It all depends on how the primitives front and popFront are implemented. If pasted within an std2 version without autodecoding, it would just work.

Of course, we don't want to paste it twice, so the challenge is to have the source code in a single place and use it both in std, with auto-decoding, and std2, without autodecoding.

One naive solution would be to simply instruct mismatch to look up front, popFront etc. in the same namespace in which it's called, much like a C-style macro. This is called "dynamic lookup" and its use is largely discredited at least in statically-typed languages.

So there's a need to parameterize mismatch with the namespace in which lookup is to be done. For example:


// In std/algorithm/comparison.d:

template mismatch(alias the_std)
{
    import the_std.typecons : Tuple, tuple;
    import the_std.range.primitives : isInputRange, empty, front, popFront;
    Tuple!(Range1, Range2)
    mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
    if (isInputRange!(Range1) && isInputRange!(Range2))
    {
        for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
        {
            if (!binaryFun!(pred)(r1.front, r2.front)) break;
        }
        return tuple(r1, r2);
    }
}

Then:

// In std/algorithm/comparison.d:
alias mismatch = mismatch!std;

// In std2/algorithm/comparison.d:
alias mismatch = mismatch!std2;

Pros:

- no duplication of implementation
- freedom to pull an old name in a new version, or simply redefine it

Cons:

- one line of scaffolding per name introduced; can't do bulk without a bit of support compile-time reflectioncode
- does not support easy migration of directory structure - what if we want to reorganized the modules such that Tuple is no longer in std2.typecons?
- does not work within the current language

About the last point: currently the language does allow passing a package name as an alias, but trying to import the_std.typecons ignores any alias definition and opens the hardcoded path "the_std/typcons.d". This would need to be changed. If breaking changes are to be allowed we need to introduce a new syntax such as:

    import alias(the_std).typecons : Tuple, tuple;

There are many other aspects to discuss, but I'll keep this short so the discussion doesn't meander too much.

Alternatively:

mixin template StandardAlgorithmComparison() {
    Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
    if (isInputRange!(Range1) && isInputRange!(Range2))
    {
        for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
        {
            if (!binaryFun!(pred)(r1.front, r2.front)) break;
        }
        return tuple(r1, r2);
    }
}

In std2.algorithm.comparison:

mixin StandardAlgorithmComparison;

In std.algorithm.comparison:

mixin StandardAlgorithmComparison;

On 10/25/21 6:19 PM, Andrei Alexandrescu wrote:

> template mismatch(alias the_std)
> {
>      import the_std.typecons : Tuple, tuple;
[...]
> }
[...]
> About the last point: currently the language does allow passing a
> package name as an alias, but trying to import the_std.typecons ignores
> any alias definition and opens the hardcoded path "the_std/typcons.d".

Sorry to stray from the main point but I needed that missing feature in the past: "This is a string template parameter but I don't want to use a string at call site." (Similar to how non-string uses of opDispatch are presented as strings.)

For example, Flag!"foo" does not look good. Instead, I want to say Flag!foo. I just thought about the following syntaxes:

a) template mismatch(auto mixin the_std) { /* ... */ }

b) template mismatch(string mixin the_std) { /* ... */ }

c) Something like that

So, at the call site:

  mismatch!std2;

So, the_std parameter would be the string literal "std2", and it would be automatically mixed in like a string mixin.

I know that we cannot mixin at that granularity but maybe this is a useful idea.

Ali

On 10/25/21 9:59 PM, Ali Çehreli wrote:
> On 10/25/21 6:19 PM, Andrei Alexandrescu wrote:
> 
>  > template mismatch(alias the_std)
>  > {
>  >      import the_std.typecons : Tuple, tuple;
> [...]
>  > }
> [...]
>  > About the last point: currently the language does allow passing a
>  > package name as an alias, but trying to import the_std.typecons ignores
>  > any alias definition and opens the hardcoded path "the_std/typcons.d".
> 
> Sorry to stray from the main point but I needed that missing feature in the past: "This is a string template parameter but I don't want to use a string at call site." (Similar to how non-string uses of opDispatch are presented as strings.)
> 
> For example, Flag!"foo" does not look good. Instead, I want to say Flag!foo.

Walter and I discussed this a long time ago, several times, under the proposal "new alias":

template Flag(new alias name) { ... }

If a template takes a "new alias" it means it takes an alias that does not refer to anything that currently exists. Inside the template the alias can be used to introduce a name or as a string.

On 10/25/21 9:41 PM, rikki cattermole wrote:
> 
> Alternatively:
> 
> mixin template StandardAlgorithmComparison() {
>      Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
>      if (isInputRange!(Range1) && isInputRange!(Range2))
>      {
>          for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
>          {
>              if (!binaryFun!(pred)(r1.front, r2.front)) break;
>          }
>          return tuple(r1, r2);
>      }
> }
> 
> In std2.algorithm.comparison:
> 
> mixin StandardAlgorithmComparison;
> 
> In std.algorithm.comparison:
> 
> mixin StandardAlgorithmComparison;

Good stuff. A great plus is that it works within the existing languages. Other pros and cons?

On 26/10/2021 3:04 PM, Andrei Alexandrescu wrote:
> Good stuff. A great plus is that it works within the existing languages. Other pros and cons?

My concerns with this approach would be auto completion and documentation.

If the documentation engine doesn't see past the mixin template and include the members transparently, then that it is a bug.

If the auto completion can't see into the mixin template that is a bug too. But one that may not be fixable without changing to dmd-fe and that is something the core developers should be owning: IDE integrations.

On 10/25/21 10:04 PM, Andrei Alexandrescu wrote:
> On 10/25/21 9:41 PM, rikki cattermole wrote:
>>
>> Alternatively:
>>
>> mixin template StandardAlgorithmComparison() {
>>      Tuple!(Range1, Range2) mismatch(alias pred = "a == b", Range1, Range2)(Range1 r1, Range2 r2)
>>      if (isInputRange!(Range1) && isInputRange!(Range2))
>>      {
>>          for (; !r1.empty && !r2.empty; r1.popFront(), r2.popFront())
>>          {
>>              if (!binaryFun!(pred)(r1.front, r2.front)) break;
>>          }
>>          return tuple(r1, r2);
>>      }
>> }
>>
>> In std2.algorithm.comparison:
>>
>> mixin StandardAlgorithmComparison;
>>
>> In std.algorithm.comparison:
>>
>> mixin StandardAlgorithmComparison;
> 
> Good stuff. A great plus is that it works within the existing languages. Other pros and cons?

Found a con:

- Cannot use local imports at all; the mixin must do all of its lookup in the top namespace.

On 26/10/2021 3:10 PM, Andrei Alexandrescu wrote:
> Found a con:
> 
> - Cannot use local imports at all; the mixin must do all of its lookup in the top namespace.

This works:

void main() {
    FooBar foobar;
    foobar.zar();
}

struct FooBar {
    import std;

    mixin Zar;

    void mar() {
        writeln("ugh");
    }
}

mixin template Zar() {
    void zar() {
        writeln("hi");
        mar();
    }
}

What is not working for you?

On 10/25/21 10:21 PM, rikki cattermole wrote:
> What is not working for you?

Most Phobos functions import std.something locally, as they should. Those imports would need to be all hoisted to top level for the mixin to work. Wouldn't want to lose the advantages of local imports.

On 26/10/2021 3:29 PM, Andrei Alexandrescu wrote:
> On 10/25/21 10:21 PM, rikki cattermole wrote:
>> What is not working for you?
> 
> Most Phobos functions import std.something locally, as they should. Those imports would need to be all hoisted to top level for the mixin to work. Wouldn't want to lose the advantages of local imports.

Ah yes, importing inside a function body.

Hmm, does anyone know what the cost is of a named import if unused is? That could resolve this issue.