October 27, 2021
On Wednesday, 27 October 2021 at 13:57:50 UTC, Andrei Alexandrescu wrote:
> It's interesting that both you and Sebastiaan mention an "inverted delta" of sorts whereby current std is redone to use std2.

Well, generally new functions end up being more fundamental than old, so this is just naturally simpler.

In the case of the Windows functions, the Unicode functions can take any string the ANSI functions can, which is not true vice versa. So naturally, you'd be forced to write the less functional one in terms of the more functional one.

In the case of various Phobos modules:

* Removing autodecoding in v2 makes it more general. You can always write an autodecode function in terms of a plain function - just do a wrapper that adds the decode adapter.

Doing it the other way happens to be possible here too, using a byCodeUnit wrapper, but generally it is easier to add something in a wrapper than remove something that was already added.

* Some people hate std.socket because it uses classes. Well, if you build a std2.socket on top of it, those people will still be angry. But std2.socket could be a struct api and then the original std.socket class is built on top of it.

(in that case it would probably be a waste of time; just keeping the original std.socket unmodified - it is already a fairly thin layer on the underlying C functions anyway - and writing std2.socket right on top of the underlying C functions too is easy enough.)

* A generalization of std.socket: you can always build GC functions on top of @nogc foundations (e.g. `string toLower(string s) { return s.asLowerCase.array; } `). Not true other way around.

* If you want to trim down an interface, again, you can define std.v2 to be minimal then layer on stuff for compatibility on top. Users who stick to std.v2, importing just it and not the old one, they will not pay the cost of carrying these functions around.

But if you defined std.v2 on top of std.v1, it is impossible to remove functions. If a user imports std.v2, they automatically drag all the std.v1 baggage along too, whether they like it or not.

* Just as a matter of merge conflicts, making the bulk of your edits on the forward moving version is a more natural way to work.

The con here is maintaining compatibility with the old one is a bit harder since you're making edits and the old one might rely on it. But.... worse case you just copy/paste when the divergence is too much and lock in the old version letting the new one evolve.

October 27, 2021

On 10/27/21 10:22 AM, Adam D Ruppe wrote:

>
  • Removing autodecoding in v2 makes it more general. You can always write an autodecode function in terms of a plain function - just do a wrapper that adds the decode adapter.

I wish this were true. But there is a fundamental problem with autodecode -- Phobos claims, for example, that a string has no indexing, yet you can index it. hasLength!string is false, yet someString.length works. There are a large amount of places where you can use strings, that you can't use some wrapped autodecoding range. Want to implicitly cast a string to a const(char)[]? No problem! Try doing this with a custom type.

There is literally no way to reproduce the abomination known as an autodecoding for backwards compatibility. I tried.

And given that a major focus of v2 is going to be ridding the library of autodecoding, it means a huge segment of code will need to be redone. I was actually kind of surprised how much I had to change when I was trying to do it. It gets to the point where you are thinking "why are we trying to save this code? Let's just rewrite it".

-Steve

October 27, 2021

On Wednesday, 27 October 2021 at 13:56:14 UTC, Andrei Alexandrescu wrote:

>

On 10/26/21 10:08 PM, zjh wrote:

Yes, we must change the language. But as long as it is confined within a small range.
I think it can satisfy the user, because the user only needs a small amount of modification to obtain the incremental function, which is worth it for the user.
For example, ahead of the module, in sc.ini, or in dub.sdl/dub.json, We can add a version declaration. I think we can minimize the modification to resolve the branch version.
In fact, as long as the version statement is added to std and forwarded to like std2, the user code that depends on std may not be destroyed.
At the same time, don't be afraid of change. The "big change" when the number of users is small is better than the "big change" when the number of users is large.
However, we should strive for the understanding of the users.

October 27, 2021
On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:
> There is literally no way to reproduce the abomination known as an autodecoding for backwards compatibility. I tried.

in that case realistically you can't make a v2 act sane to strings when built on v1 either, meaning the only choice is to rewrite (aka copy/paste aka fork) as needed.

I'm ok with that.
October 27, 2021
On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:
> On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:
> > There is literally no way to reproduce the abomination known as an autodecoding for backwards compatibility. I tried.
> 
> in that case realistically you can't make a v2 act sane to strings when built on v1 either, meaning the only choice is to rewrite (aka copy/paste aka fork) as needed.
> 
> I'm ok with that.

Couldn't we just do something like this?

	// Phobos v1
	auto someStringFunc(R)(R range)
	{
		static if (willAutoDecode!R) {
			// old v1 code here
		} else {
			// thin wrapper over v2 code
		}
	}


T

-- 
It is widely believed that reinventing the wheel is a waste of time; but I disagree: without wheel reinventers, we would be still be stuck with wooden horse-cart wheels.
October 27, 2021

On 10/27/21 12:33 PM, H. S. Teoh wrote:

>

On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:

>

On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:

>

There is literally no way to reproduce the abomination known as an
autodecoding for backwards compatibility. I tried.

in that case realistically you can't make a v2 act sane to strings
when built on v1 either, meaning the only choice is to rewrite (aka
copy/paste aka fork) as needed.

I'm ok with that.

Couldn't we just do something like this?

// Phobos v1
auto someStringFunc(R)(R range)
{
static if (willAutoDecode!R) {
// old v1 code here
} else {
// thin wrapper over v2 code
}
}

I'm not seeing the point. Why wrap v2 code when you are going to have to maintain the v1 code anyway?

Plus, now it's a template that accepts ranges. Like it or not, someStringFunc(string) is not the same as someStringFunc(R)(R).

-Steve

October 27, 2021
On Wed, Oct 27, 2021 at 01:04:25PM -0400, Steven Schveighoffer via Digitalmars-d wrote:
> On 10/27/21 12:33 PM, H. S. Teoh wrote:
> > On Wed, Oct 27, 2021 at 03:06:10PM +0000, Adam D Ruppe via Digitalmars-d wrote:
> > > On Wednesday, 27 October 2021 at 14:51:06 UTC, Steven Schveighoffer wrote:
> > > > There is literally no way to reproduce the abomination known as an autodecoding for backwards compatibility. I tried.
> > > 
> > > in that case realistically you can't make a v2 act sane to strings when built on v1 either, meaning the only choice is to rewrite (aka copy/paste aka fork) as needed.
> > > 
> > > I'm ok with that.
> > 
> > Couldn't we just do something like this?
> > 
> > 	// Phobos v1
> > 	auto someStringFunc(R)(R range)
> > 	{
> > 		static if (willAutoDecode!R) {
> > 			// old v1 code here
> > 		} else {
> > 			// thin wrapper over v2 code
> > 		}
> > 	}
> 
> I'm not seeing the point. Why wrap v2 code when you are going to have to maintain the v1 code anyway?

I thought the you guys were arguing for making Phobos v1 a thin wrapper that forwards to v2 code.  If you wanted to retain a copy of v1 code, then this problem wouldn't come up at all?


> Plus, now it's a template that accepts ranges. Like it or not,
> `someStringFunc(string)` is not the same as `someStringFunc(R)(R)`.
[...]

Point.  So what, does that mean we're stuck with keeping v1 code around forever?

Honestly, I think a more practical approach is to give up the perfectionist ideal of old code compiling forever (the D community simply does not have the resources to maintain such a thing, unlike, say, MS Windows), and just do a best-efforts v1 emulation in v2 code, that will mostly work but may have a few rare cases where it won't compile, etc.. Then users just change the few bits that broke, and can continue enjoying v2 benefits.

//

Alternatively, here's a different approach: make v1 code a dub package, then break upstream Phobos at will to make v2.

Well OK, maybe have a transitional period where we maintain a copy of v1 code around, but with big fat warnings that within N months/years/etc. you will need to use the v1 dub package to continue compiling the code, or migrate to v2 now.  Once the transition period is over, we shed v1 code from upstream, but keep the dub package around for people who need it. There will be no more new features, only non-breaking bugfixes.


T

-- 
It only takes one twig to burn down a forest.
October 27, 2021

On 10/27/21 1:16 PM, H. S. Teoh wrote:

>

On Wed, Oct 27, 2021 at 01:04:25PM -0400, Steven Schveighoffer via Digitalmars-d wrote:

>

I'm not seeing the point. Why wrap v2 code when you are going to have
to maintain the v1 code anyway?

I thought the you guys were arguing for making Phobos v1 a thin wrapper
that forwards to v2 code. If you wanted to retain a copy of v1 code,
then this problem wouldn't come up at all?

The point of making a v2 that is separate from v1 is that you can include a library that depends on v1 in the same application as one that depends on v2, and all work together.

Autodecoding is one of the worst problems, but there are so many more. I'll note there is a strong focus in this thread on how to make this maintainable from a library side. But not a lot of focus on how usable it would be to actually depend on both libraries at once.

Note that if Autodecoding was ascribed to a custom type and not just a char array, this probably isn't even close to as big an issue.

> >

Plus, now it's a template that accepts ranges. Like it or not,
someStringFunc(string) is not the same as someStringFunc(R)(R).
[...]

Point. So what, does that mean we're stuck with keeping v1 code around
forever?

I'm fine with just migrating everything to v2 and if you still want v1, you go get the old compiler, and good luck to you.

This is what I started with here, but it was decided this was undesirable, so I spent a bunch of time trying to figure out how to make tools that help with maintaining a v2/v1 split in the same code base, without success. Not to say that a solution isn't possible, I just couldn't find one that seemed reasonable. Every time I thought I had something that would work, I'd run into a critical problem that destroyed it.

Some things that we toyed with, but never implemented, were ways to parameterize imports (i.e. same source code, but treated as 2 different imports with the package symbols automatically migrated). This I think was the most promising, but it is hard to play with if you don't know how to augment the compiler.

>

Alternatively, here's a different approach: make v1 code a dub package,
then break upstream Phobos at will to make v2.

I think having dub packages for each major phobos version would be ideal. Then you select the package you depend on (and e.g. phobos v2 could depend on v1 if you want to share code). You still need tools to automatically change the imports and things like that (these have to be in different packages).

That is, if we want to maintain all-time backwards compatibility.

-Steve

October 27, 2021

On Tuesday, 26 October 2021 at 01:19:29 UTC, Andrei Alexandrescu wrote:

>

Versioning Phobos would free us from maintaining backward compatibility with a variety of decisions that did not withstand the test of time

I have a relatively simple LTS scheme in mind:

  • We agree on a scheme for long-term support Phobos versions. For the sake of example, let's say we agree on 2.096 and every sixth minor version afterwards (2.102, 2.108 and so on).

  • Any released DMD is required to compile all LTS Phobos versions older than the DMD. LTS Phobos versions may be patched to satisfy that requirement, so no need to keep around any DMD bugs or old features Phobos depends on. However, the LTS Phobos versions do not aim to support using newer language features. Phobos bug fixes may but are not required to be backported.

  • DRuntime always uses the same version as DMD, and thus must also be compatible with older Phobos LTS versions.

  • We change the language to allow relative imports, and convert Phobos to use them. This is needed to allow using two or more different versions of Phobos side-by-side.

  • After these are in place, we may proceed with relatively aggressive changes in Phobos.

These do not mean that big a maintenance burden, because the LTS versions are only required to be kept compiling - only a fraction of the effort that would be needed to backport the bug fixes or new features. Also the LTS Phobos would act as a good real-world regression test for DMD and DRuntime.

However, this plan requires that Phobos must stop calling DRuntime internals directly - otherwise it's going to cause lots of trouble with this scheme. I don't think that is a problem though - Phobos is supposed to be just another library. Surely it does not need special privileges to do its job? If it needs, DRuntime needs a redesign.

October 27, 2021

On Wednesday, 27 October 2021 at 18:01:47 UTC, Dukc wrote:

>

[snip]

  • We change the language to allow relative imports, and convert Phobos to use them. This is needed to allow using two or more different versions of Phobos side-by-side.
    [snip]

Can you expand on what you mean by relative imports?