On 8/19/20 9:35 AM, Andrei Alexandrescu wrote:
> Even the patch can reuse std.cmp if you pass std.cmp a range of code units so you undo its penchant to autodecode.

Here's what I actually did do:

https://github.com/dlang/phobos/blob/bf258e56f2dbc28cf810e9329a0696afbe114907/std/algorithm/comparison.d#L890-L895

Essentially, if you compared 2 ranges of code units which differed, equal would integer promote one to the other, and compare that.

I fixed that bug, and in the process, handled both autodecoding and nonautodecoding strings.

This brings "equal" outside of the domain of having to think about autodecoding. If you have autodecoding enabled, it works. If you have it disabled, it works. There is no need to have a separate equal function in std.v2021. There is no need to call a separate equal function if you want no-autodecoding, equal(someString, someOtherString) works no matter what you care about autodecoding.

The more of this that we can do, the smaller the "drastic" change of moving to non-autodecoding becomes.

> 
> * * *
> 
> Lastly, I should add the following. You wrote, and I quote it again because it is so important:
> 
>> autodecoding infects so many other modules, so it would be really hard to single out only some modules, we'd likely have to make a copy of nearly everything.
> 
> This is a very powerful argument AGAINST your current approach to CHANGE things. You are basically saying the semantics of the standard library will be deeply affected by your work. Put another way, the rework of the standard library will be very incompatible with its current version. So you are killing all compatibility.

Nearly ALL the changes are going to be on template constraints or imported traits, which means the code in the alternative package will be identical (but maintained separately). The end result would be a maintenance nightmare. Maintaining almost identical code bases, especially where one is used and the other isn't is just going to result in diverging code bases where the new one doesn't work and has to be fixed. See for instance how Daniel Murphy migrated the compiler from C++ to D without having to worry about other people breaking his changes. It would not have happened with a copy of everything hand-maintained.

My plea is: let's see how far I can get with an alternative view of Phobos (tested simultaneously), without actually pulling the trigger to deprecate autodecoding. At that point, we still can make a decision to do it a different way. Nothing is set in stone.

> Changing the stdlib to remove autodecoding would kill the D programming language.
> 
> You don't want to kill the D programming language, do you?

I can't agree with either of these statements. At all.

-Steve

On 19.08.20 15:10, Andrei Alexandrescu wrote:
>>
> 
> Don't copy everything, alias everything that doesn't need change and only add the declarations that should be changed.

But which declarations are those? Aliasing declarations that would otherwise be copied verbatim does not work. Consider:

module a;

auto front(R)(R r)if(...){ /* autodecode here */ }
auto algorithm(R)(R r)if(...){ ... r.front; ... }

module b;

auto front(R)(R r)if(...){ /* no autodecode here */
static import a;
alias algorithm=a.algorithm;

This will not do what you want, because aliasing is not equivalent to copying.

On Wed, Aug 19, 2020 at 09:35:34AM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
> On 8/19/20 8:23 AM, Steven Schveighoffer wrote:
[...]
> > autodecoding infects so many other modules, so it would be really hard to single out only some modules, we'd likely have to make a copy of nearly everything.
> 
> This is a very powerful argument AGAINST your current approach to CHANGE things. You are basically saying the semantics of the standard library will be deeply affected by your work. Put another way, the rework of the standard library will be very incompatible with its current version. So you are killing all compatibility.
[...]

Whoa, hold your guns there!  We're not actually changing semantics -- yet.  Steven's changes are controlled by version directives; even after all changes have been made *current semantics remain unchanged*. Unless you deliberately compile with the no-autodecoding version set. Which users won't have to.

When we get to the point where the entire Phobos can be compiled with no-autodecoding, *then* we can gauge the true extent of breakage (by compiling public D repos with autodecoding on -- *experimentally*) to obtain some REAL DATA on which to base decisions, instead of gut feelings and other totally subjective criteria, and thereby be in a much better position to make a sane decision about where to go from there. If we absolutely do not want to break backward compatibility, then *at that time* we can freeze the current std.*, make a copy of everything in std.v2021, turn on no-autodecoding for the latter, and go from there.

Please let's not let the best be the enemy of the good yet again. Autodecoding has been our scourge for how many years now, and now that somebody's FINALLY making progress on it, please let's not gun it down prematurely.


T

-- 
What's an anagram of "BANACH-TARSKI"?  BANACH-TARSKI BANACH-TARSKI.

On 8/19/20 10:12 AM, Steven Schveighoffer wrote:
> My plea is: let's see how far I can get with an alternative view of Phobos (tested simultaneously), without actually pulling the trigger to deprecate autodecoding. At that point, we still can make a decision to do it a different way. Nothing is set in stone.

Good. Virtually all I worry about applies to after that point. Making mnst of the code decoding-neutral is a good endeavor.

On 8/19/20 10:27 AM, Timon Gehr wrote:
> On 19.08.20 15:10, Andrei Alexandrescu wrote:
>>>
>>
>> Don't copy everything, alias everything that doesn't need change and only add the declarations that should be changed.
> 
> But which declarations are those? Aliasing declarations that would otherwise be copied verbatim does not work. Consider:
> 
> module a;
> 
> auto front(R)(R r)if(...){ /* autodecode here */ }
> auto algorithm(R)(R r)if(...){ ... r.front; ... }
> 
> module b;
> 
> auto front(R)(R r)if(...){ /* no autodecode here */
> static import a;
> alias algorithm=a.algorithm;
> 
> This will not do what you want, because aliasing is not equivalent to copying.

Of course, that was implied. The difficulty is to figure whether said algorithm (and others) would be affected by autodecoding, even if it doesn't in and of itself do anything about it specifically. Those that are would need to be forked across the two versions. Ideally instead of copying them, they'd be separated to minimize duplication.

On Wednesday, 19 August 2020 at 15:39:58 UTC, H. S. Teoh wrote:
> On Wed, Aug 19, 2020 at 09:35:34AM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
>> On 8/19/20 8:23 AM, Steven Schveighoffer wrote:
> [...]

> When we get to the point where the entire Phobos can be compiled with no-autodecoding, *then* we can gauge the true extent of breakage (by compiling public D repos with autodecoding on -- *experimentally*) to obtain some REAL DATA on which to base decisions, instead of gut feelings and other totally subjective criteria, and thereby be in a much better position to make a sane decision about where to go from there.
+ 1

> If we absolutely do not want to break backward compatibility, then *at that time* we can freeze the current std.*, make a copy of everything in std.v2021, turn on no-autodecoding for the latter, and go from there.

At that time, it might not be a bad idea either. Eventually D will need something like this to ensure it's not held behind by future mistakes (which definitely will happen)...as long as it can be corrected in the next version of std. Guess we'll see when that time to comes.

On Wednesday, 19 August 2020 at 15:39:58 UTC, H. S. Teoh wrote:
> On Wed, Aug 19, 2020 at 09:35:34AM -0400, Andrei Alexandrescu via Digitalmars-d wrote:
>> This is a very powerful argument AGAINST your current approach to CHANGE things. You are basically saying the semantics of the standard library will be deeply affected by your work. Put another way, the rework of the standard library will be very incompatible with its current version. So you are killing all compatibility.
> [...]
> When we get to the point where the entire Phobos can be compiled with no-autodecoding, *then* we can gauge the true extent of breakage (by compiling public D repos with autodecoding on -- *experimentally*) to obtain some REAL DATA on which to base decisions, instead of gut feelings and other totally subjective criteria, and thereby be in a much better position to make a sane decision about where to go from there.

No, we won't ever know.  Unicode bugs are subtle, so we won't know how much is broken unless someone goes and writes regression tests for a good chunk of the code in the Dub registry.

On 8/19/20 10:12 AM, Steven Schveighoffer wrote:
> On 8/19/20 9:35 AM, Andrei Alexandrescu wrote:
>> Even the patch can reuse std.cmp if you pass std.cmp a range of code units so you undo its penchant to autodecode.
> 
> Here's what I actually did do:
> 
> https://github.com/dlang/phobos/blob/bf258e56f2dbc28cf810e9329a0696afbe114907/std/algorithm/comparison.d#L890-L895 

I have a number of strong objections to that, starting with the obvious fact that you add an undocumented symbol that the user interface then depends on. But that's not the main problem:

https://github.com/dlang/phobos/pull/7595#pullrequestreview-471042128

The main problem is we're butchering the entire standard library based on a `version` thing that essentially forks the entire library. THE ENTIRE LIBRARY. To change its semantics in ever so daintly ways. I have yet to see this anywhere else, ever. And THEN that stupid version flag gives carte blanche to anyone with a scalpel to keep on butchering, like this PR copiusly does:

https://github.com/dlang/phobos/pull/7595

This is a hack job. The version idea is awful. I told so Walter when he introduced it, he wouldn't listen.

Literally this is why we can't have good things.

IN THE NAME OF EVERYTHING GOOD, PLEASE STOP CHANGING.

PLEASE START ADDING.

> I fixed that bug, and in the process, handled both autodecoding and nonautodecoding strings.
> 
> This brings "equal" outside of the domain of having to think about autodecoding. If you have autodecoding enabled, it works. If you have it disabled, it works. There is no need to have a separate equal function in std.v2021. There is no need to call a separate equal function if you want no-autodecoding, equal(someString, someOtherString) works no matter what you care about autodecoding.
> 
> The more of this that we can do, the smaller the "drastic" change of moving to non-autodecoding becomes.

This sounded nice this morning when I read it. Then during the day I had this growing feeling that the reality is much worse than that. I went to

https://github.com/dlang/phobos/pull/7595

which confirms my worst fear. This is a hack job that will butcher the standard library beyond recognition.

IN THE NAME OF EVERYTHING GOOD, PLEASE STOP CHANGING.

PLEASE START ADDING.

On Sunday, 16 August 2020 at 17:00:52 UTC, Steven Schveighoffer wrote:
> The autodecode test is run in buildkite. PLEASE don't merge any PRs for Phobos if this test is failing! instead, ping me.

This test breaks CI for pull requests to dmd/druntime/phobos stable branches. For example:

https://github.com/dlang/dmd/pull/11633
https://github.com/dlang/dmd/pull/11634

I think it would be best to disable this until autodecode-test is available in phobos/stable.

On Friday, 28 August 2020 at 13:27:33 UTC, Paul Backus wrote:
> On Sunday, 16 August 2020 at 17:00:52 UTC, Steven Schveighoffer wrote:
>> The autodecode test is run in buildkite. PLEASE don't merge any PRs for Phobos if this test is failing! instead, ping me.
>
> This test breaks CI for pull requests to dmd/druntime/phobos stable branches. For example:
>
> https://github.com/dlang/dmd/pull/11633
> https://github.com/dlang/dmd/pull/11634
>
> I think it would be best to disable this until autodecode-test is available in phobos/stable.

Cherry-pick of the Makefile changes to stable: https://github.com/dlang/phobos/pull/7610