On Wednesday, 27 October 2021 at 01:36:24 UTC, Adam D Ruppe wrote:
> On Wednesday, 27 October 2021 at 01:23:44 UTC, Basile B. wrote:
>> The A and W API are funadementally there because it's a C API.
>
> That's irrelevant. They do actually overload it with macros.
>
> The point is the old functions - the A ones - are kept around but simply forward to the new functions - the W ones - after doing some compatibility transformations.
>
> That is a viable path for Phobos. The existing std.x functions can forward to std.v2.x as appropriate.

Yes and no, if it's done subtle and you don't know which functions change it could possibly lead to silent bugs.

On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu wrote:
> I regret I forgot to mention in the initial post that we're not looking at v2. We're looking at v2, v3, v4, ... released e.g. at an annual or trienal pace. It's the having a fish vs. knowing how to fish problem.

That does change things. Obviously the copy/paste approach isn't going to work then.

Whenever people have to maintain several versions of a product what they often do is to implement the old version in terms of the new. From the outside it seems version 1 is still available but internally it is just version 2 + the quirks of version 1. That might not be possible everywhere, especially if the newer versions stray too much.

It is not much different from what you propose, except that it doesn't penalize all the code to have extra version boilerplate around it (mixins, alias module). That might not be possible in certain cases because of a too intrusive change, where you can't write version 1 with version 2, and you either have to copy it (if it is small) or wrap it in version/behavior-selection code.

You can end up in a situation where most of version 1 is implemented with version 2, except for a few annoying bits here and there. Which would also be acceptable as long as it isn't too much. The unanswered questions are "what is too much?", "how to measure/detect", "how to mitigate?".

Ultimately it depends a lot on how well you can extract generic code usable for both versions. If the ratio generic code to version-specific quirk code is favorable then it can definitely work.

It goes without saying, do this all in one repo please.

On Wednesday, 27 October 2021 at 07:44:19 UTC, Sebastiaan Koppe wrote:
> Whenever people have to maintain several versions of a product what they often do is to implement the old version in terms of the new. From the outside it seems version 1 is still available but internally it is just version 2 + the quirks of version 1. That might not be possible everywhere, especially if the newer versions stray too much.

This is can be problematic when you have advanced introspection. This could potentially ruin future language advances (more powerful introspection).

I don't think the community should assume that what works for other languages will serve D well.

Basically, maintaining multiple versions of a standard lib for a language that is evolving is overreaching. You get another layer of issues in addition to the ones you already have.

On Wednesday, 27 October 2021 at 06:10:46 UTC, bauss wrote:
> Yes and no, if it's done subtle and you don't know which functions change it could possibly lead to silent bugs.

Keep the existing unittests with the compatibility shim.

If they pass, it is good enough.

On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei Alexandrescu wrote:
> Agreed. I conflated the moment of "forking" with the moment of "releasing".
>
> The core issue remains the same. Consider e.g. we fork there and we set out to eliminate autodecoding in the new version. Once we release that, we have two trees in two repos on our hands. Back to versioning hell.

It seems to me like you only really end up in versioning hell if, after releasing v2 in Phobos (and committing to no further breaking changes), you go on to develop v3 (with new breaking changes) in the same repo. Otherwise, you can just mirror changes from the Phobos version to the dub version.

I think the root of the problem is, releasing N versions of the Phobos API means you are committing  to maintaining N versions of the Phobos API, more-or-less indefinitely. And with annual or even triennial releases, this maintenance burden will quickly grow beyond what the D community can handle.

As Steven Schveighoffer points out [1], having all N versions in the same tree does not really save you here--it gets you out of versioning hell, but you end up in #ifdef hell instead.

So, what if we didn't release N versions? What if we let the new library evolve through v2, v3, etc. on code.dlang.org, and only released it into Phobos when we were well and truly done with it, and willing to commit to supporting its API long-term?

[1] https://forum.dlang.org/post/sl9161$2ntu$1@digitalmars.com

On 10/27/21 9:47 AM, Paul Backus wrote:
> On Tuesday, 26 October 2021 at 21:59:33 UTC, Andrei Alexandrescu wrote:
>> Agreed. I conflated the moment of "forking" with the moment of "releasing".
>>
>> The core issue remains the same. Consider e.g. we fork there and we set out to eliminate autodecoding in the new version. Once we release that, we have two trees in two repos on our hands. Back to versioning hell.
> 
> It seems to me like you only really end up in versioning hell if, after releasing v2 in Phobos (and committing to no further breaking changes), you go on to develop v3 (with new breaking changes) in the same repo. Otherwise, you can just mirror changes from the Phobos version to the dub version.

On the contrary, I see repeated mirroring of changes hell, and mindful departure by a mix of reuse and rewriting much easier.

At least there is evidence in the way everybody maintains versions now that mirroring them changes is NOT easy. It remains to be seen whether better language support for versioning will improve the matter; I am hopeful it does.

> I think the root of the problem is, releasing N versions of the Phobos API means you are committing  to maintaining N versions of the Phobos API, more-or-less indefinitely. And with annual or even triennial releases, this maintenance burden will quickly grow beyond what the D community can handle.

We can and we should establish a retirement schedule.

Minimizing effort is exactly why I'm trying to improve on the status quo. Do we agree that extant methods of maintaining multiple versions is difficult?

> As Steven Schveighoffer points out [1], having all N versions in the same tree does not really save you here--it gets you out of versioning hell, but you end up in #ifdef hell instead.

There is no ifdef hell. The new version of a library either redoes an artifact or reuses it. Really the new version is a delta over the old.

> So, what if we didn't release N versions? What if we let the new library evolve through v2, v3, etc. on code.dlang.org, and only released it into Phobos when we were well and truly done with it, and willing to commit to supporting its API long-term?
> 
> [1] https://forum.dlang.org/post/sl9161$2ntu$1@digitalmars.com

I think breaking std would be a disaster.

On 10/27/21 3:44 AM, Sebastiaan Koppe wrote:
> On Tuesday, 26 October 2021 at 14:43:19 UTC, Andrei Alexandrescu wrote:
>> I regret I forgot to mention in the initial post that we're not looking at v2. We're looking at v2, v3, v4, ... released e.g. at an annual or trienal pace. It's the having a fish vs. knowing how to fish problem.
> 
> That does change things. Obviously the copy/paste approach isn't going to work then.
> 
> Whenever people have to maintain several versions of a product what they often do is to implement the old version in terms of the new. From the outside it seems version 1 is still available but internally it is just version 2 + the quirks of version 1. That might not be possible everywhere, especially if the newer versions stray too much.
> 
> It is not much different from what you propose, except that it doesn't penalize all the code to have extra version boilerplate around it (mixins, alias module). That might not be possible in certain cases because of a too intrusive change, where you can't write version 1 with version 2, and you either have to copy it (if it is small) or wrap it in version/behavior-selection code.
> 
> You can end up in a situation where most of version 1 is implemented with version 2, except for a few annoying bits here and there. Which would also be acceptable as long as it isn't too much. The unanswered questions are "what is too much?", "how to measure/detect", "how to mitigate?".
> 
> Ultimately it depends a lot on how well you can extract generic code usable for both versions. If the ratio generic code to version-specific quirk code is favorable then it can definitely work.
> 
> It goes without saying, do this all in one repo please.

Thanks. Lots of good stuff here.

I'll set out to put together a WIP PR in phobos showcasing a couple of artifacts migration. It will look ugly because I'll need to supplant language support with string mixins and other tricks.

Then you and others could improve it and/or provide alternatives.

On 10/26/21 10:08 PM, zjh wrote:
> On Wednesday, 27 October 2021 at 01:58:07 UTC, zjh wrote:
>> On Wednesday, 27 October 2021 at 01:17:54 UTC, zjh wrote:
> 
> while the old `std`, you only need to add `using std = std2021;`, Then change it to our forwarding code(compile time forwarding).
> Through compile time forwarding, the user's code speed should not change.
> 

We tried a number of approaches along these lines, we couldn't make anything work. My learning bit has been that you must modify code to make it versionable.

On 10/26/21 9:36 PM, Adam D Ruppe wrote:
> On Wednesday, 27 October 2021 at 01:23:44 UTC, Basile B. wrote:
>> The A and W API are funadementally there because it's a C API.
> 
> That's irrelevant. They do actually overload it with macros.
> 
> The point is the old functions - the A ones - are kept around but simply forward to the new functions - the W ones - after doing some compatibility transformations.
> 
> That is a viable path for Phobos. The existing std.x functions can forward to std.v2.x as appropriate.

It's interesting that both you and Sebastiaan mention an "inverted delta" of sorts whereby current std is redone to use std2.

The converse approach would be to do minimal changes on std and write the modified behavior in std2.

What are the comparative pros and cons of the two approaches?