On Fri, Mar 27, 2020 at 11:56:40AM -0400, Steven Schveighoffer via Digitalmars-d wrote:
> There have been a lot of this pattern happening:
> 
> 1. We need to add feature X, to fix problem Y.
> 2. This will break ALL CODE IN EXISTENCE
> 3. OK, cancel the fix, we'll just live with it.
> 
> Having a new branch of the compiler will provide a way to keep D2 development alive while giving a playground to add new mechanisms, fix long-existing design issues, and provide an opt-in for code breakage.

What about supporting multiple versions of the language simultaneously, using some kind of version directive at the top of each file?  It will mean more work to develop and maintain the compiler, but the benefit is that *no* code will break, and users can migrate old code incrementally and at their leisure by bumping the version directive on a file and fixing any subsequent errors.

As Andrei said once, one solution to the problem of not breaking old code while improving new code is additive enhancements rather than replacing old things outright.  If there's a breaking language change, maybe there's a way to keep the old behaviour for old code while changing it for new code. Versioned source files is one way to do this, albeit at the cost of greater complexity in the compiler.


> Some issues I can think of:
> 
> 1. The safe by default debate
> 2. pure by default
> 3. nothrow by default
> 4. String interpolation DIP
> 5. auto-decoding
> 6. range.save
> 7. virtual by default
> 8. ProtoObject

Yeah, pretty much all of these items would be nice to have in my book.


[...]
> The biggest drawback is that we aren't a huge language, with lots of manpower to keep x branches going at once.

Yeah, forking the compiler now may have major unforeseen consequences.

Supporting multiple versions of the language simultaneously will also put a drain on our resources, and there will be tricky issues about how to interoperate code written in different versions of D, but at least it still keeps it all in one place.


> I just wanted to throw it out as a discussion point. We spend an awful lot of newsgroup server bytes debating things that to me seem obvious, but have legitimate downsides for not breaking them in a "stable" language.
[...]

Yes, we're gradually becoming everything we hated about C++. Perhaps it's not as simple to do better, as we once thought! ;-)


T

-- 
Do not reason with the unreasonable; you lose by definition.

Further thoughts on this:

On Fri, Mar 27, 2020 at 10:32:44AM -0700, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Mar 27, 2020 at 11:56:40AM -0400, Steven Schveighoffer via Digitalmars-d wrote:
[...]
> > 4. String interpolation DIP

IMO this is a purely additive change, and it was rejected mainly because we couldn't come to an agreement with Walter, I don't think it needs a whole new language version just to implement.


> > 5. auto-decoding
> > 6. range.save

These two could be done as Phobos v2, and can be an additive change that doesn't need a whole new language version.


[...]
> > 8. ProtoObject
[...]

If we go with a new language version, ProtoObject could just replace Object altogether.  We could also have other ways of dealing with the current issues with Object, such as having the compiler automatically insert things like the Monitor field on-demand.


T

-- 
Without geometry, life would be pointless. -- VS

On 3/27/20 2:33 PM, H. S. Teoh wrote:
> Further thoughts on this:

Yeah, I just listed things that have had roadblocks that could be alleviated by saying it's a new major language version.

Certainly I would consider Phobos part of the language at the moment. In D3, it could be separate if needed.

Note that I don't think we need DRASTIC changes to the language. I don't think we need to rename it or change core things. But we could just free up some of the roadblocks so we can break some eggs here.

Probably there are some more things that make sense for a breaking change. Exceptions might be redone as well. Maybe review keyword usage/consistency. DIP1000 features might work better as a type constructor. Etc.

-Steve

On Friday, 27 March 2020 at 15:56:40 UTC, Steven Schveighoffer wrote:
> Other languages evolve much quicker than D, but break things only in major updates. D seems to "sort of" break things, there's always a risk in every release. We try to be conservative, but we have this horrible mix of deciding some features can break things, while others are not allowed to, and there's no clear guide as to which breakage fits in which category.

I think there are two things that would need to be in place that would greatly reduce the breakage amount.

First one is testing top N dub packages for breakages with new versions/features. Breakages and deprecations will happen. But if something breaks vibe-d or GtkD it's a big deal and needs additional investigation effort (easy to fix or not). But if something breaks someone's personal project... well, tough luck. As far as I know the new D versions are tested on the newer evrsions so that's good. It's not just about checking if everything compiles, but assessing the scope of breakages if they happen.

Second thing is having a tool like gofix - https://blog.golang.org/introducing-gofix - gofix is an official Golang tool which automatically applies fixes to code, whether it's converting deprecated code or syntax/stdlib changes. Would also be a good test for a D compiler as a library project.

On Friday, 27 March 2020 at 16:03:47 UTC, 12345swordy wrote:
>
> Didn't the 1.0 to 2.0 conversion nearly kill the language?
>
> -Alex

I don't remember it as such. However, there seems to be a bit of lack of consensus in the community what direction should D take. It's hard to unify an effort, especially in open source (unpaid) community if everyone has their own idea of the direction it should go and doesn't want to compromise.

On Friday, 27 March 2020 at 16:55:14 UTC, Meta wrote:
> D has been around for 20 years and hasn't gained the traction that younger languages like Rust or Go have (though as we all know, the main reason for this is D's lack of a big corporate patron a la Mozilla or Google).

Possibly, or maybe not. While Rust and Go surpassed D in popularity, it's important not only to look at the competition ahead, but look behind every now and then. If it stays on course, I wouldn't be surprised Zig becoming more popular than D, and the corporate backing excuse won't work there.

On Friday, 27 March 2020 at 17:17:44 UTC, Steven Schveighoffer wrote:
>
> There was an issue with an alternative standard library (Tango), which divided the community. That shouldn't be a problem for a D3.

I'm probably in a minority here, but I feel like D would be better off if it went more in the Tango direction and being more of an OOP language, rather than moving into the STL/boost direction with templates everywhere. But that's just me, an OOP lover and not a fan of templates.

On Friday, 27 March 2020 at 17:22:30 UTC, Russel Winder wrote:
> If D has a future it is in terms of v3, v4, etc. with a strong
> technical evolution (cf. Groovy) and good marketing. Clearly D
> remaining at v2 for ever more would, I feel,  be a Very Bad Idea™ since
> it advertises no changes to the language, i.e. a language with a
> stalled evolution.

D1, D2 is only meaningful within the D community. It doesn't really mean anything outside of it. Outside of the community, people just refer it to D. Whether it's D 2.0 being improved, or D 3.0 which is just a few breaking changes, I doubt it would make any difference marketing wise.

On Friday, 27 March 2020 at 15:56:40 UTC, Steven Schveighoffer wrote:
> Some issues I can think of:
>
> 1. The safe by default debate
> 2. pure by default
> 3. nothrow by default
> 4. String interpolation DIP
> 5. auto-decoding
> 6. range.save
> 7. virtual by default
> 8. ProtoObject

My main concern here is that while it seems to be fixes to a longstanding D issues, it's also very low-level in the long run. Anyone who was turned off from D before would look at it and be like "hmm, ok, well, so what?". No one will ask "have they fixed auto decoding?". People would ask "does it work without GC now?" "can it compile to WASM?".

On Fri, Mar 27, 2020 at 03:08:51PM -0400, Steven Schveighoffer via Digitalmars-d wrote: [...]
> Certainly I would consider Phobos part of the language at the moment. In D3, it could be separate if needed.

It's arguable, since some of the avid D users here actively avoid Phobos, but OK.


> Note that I don't think we need DRASTIC changes to the language. I don't think we need to rename it or change core things. But we could just free up some of the roadblocks so we can break some eggs here.

I think for these smaller but still breaking things to be implemented as a simultaneous new version like I suggested.  Have some kind of version directive at the top of your file to indicate which language version the source was written for, then everything below version X will support the old semantics, but version X and above will use the new semantics.  As long as the change isn't too drastic, source files with different versions in the same project should still be compatible with each other.


> Probably there are some more things that make sense for a breaking change.  Exceptions might be redone as well. Maybe review keyword usage/consistency.  DIP1000 features might work better as a type constructor. Etc.
[...]

Honestly, I think cheap Exceptions could be done transparently of user code, for the most part.  Instead of using libunwind or equivalent, for example, just change the ABI to propagate an out-of-band error indicator that, say in a register or something that gets checked by the caller and branches to the function's exit block if set.


T

-- 
Give a man a fish, and he eats once. Teach a man to fish, and he will sit forever.

On Fri, Mar 27, 2020 at 07:50:44PM +0000, JN via Digitalmars-d wrote: [...]
> First one is testing top N dub packages for breakages with new versions/features.

Isn't the current CI already doing this?


[...]
> Second thing is having a tool like gofix - https://blog.golang.org/introducing-gofix - gofix is an official Golang tool which automatically applies fixes to code, whether it's converting deprecated code or syntax/stdlib changes. Would also be a good test for a D compiler as a library project.

There's already dfix.  Does it not work well enough?  What are the issues the prevent us from using it in general?


--T

[...]
> On Friday, 27 March 2020 at 17:17:44 UTC, Steven Schveighoffer wrote:
> > 
> > There was an issue with an alternative standard library (Tango), which divided the community. That shouldn't be a problem for a D3.
> 
> I'm probably in a minority here, but I feel like D would be better off if it went more in the Tango direction and being more of an OOP language, rather than moving into the STL/boost direction with templates everywhere. But that's just me, an OOP lover and not a fan of templates.

Why can't we have both?  I'm a big fan of templates and compile-time introspection, but OOP is still useful sometimes and I do still pull it out for when I really need runtime polymorphism.

In fact, combine the two together, and you get something amazing like Adam's jni.d, which makes Java interop so nice I'm tempted to stop hating Java so much ('cos I can now write Java in D :-P).


[...]
> On Friday, 27 March 2020 at 15:56:40 UTC, Steven Schveighoffer wrote:
> > Some issues I can think of:
> > 
> > 1. The safe by default debate
> > 2. pure by default
> > 3. nothrow by default
> > 4. String interpolation DIP
> > 5. auto-decoding
> > 6. range.save
> > 7. virtual by default
> > 8. ProtoObject
> 
> My main concern here is that while it seems to be fixes to a longstanding D issues, it's also very low-level in the long run. Anyone who was turned off from D before would look at it and be like "hmm, ok, well, so what?". No one will ask "have they fixed auto decoding?". People would ask "does it work without GC now?" "can it compile to WASM?".

Honestly, I think this whole no-GC thing is just barking up the wrong tree. People who have GC phobia will turn off as soon as they see "GC", doesn't matter if you can actually write D without GC or not.  They won't even look that far before walking away.  And frankly, IMO D should just embrace the GC and relish in it instead of trying to win over the no-GC crowd.  Trying to be too many things at once is stretching us too thin; D should just make a danged decision already about these issues, and stick to it instead of trying to be everything to everyone.

As for compiling to WASM, isn't LDC already doing that, or on the way to doing that?


T

-- 
Three out of two people have difficulties with fractions. -- Dirk Eddelbuettel

On Friday, 27 March 2020 at 20:25:14 UTC, H. S. Teoh wrote:
> There's already dfix.  Does it not work well enough?  What are the issues the prevent us > from using it in general?

I didn't even know it exists. Guess you learn something every day. I guess the question would be, why can't we use dfix to prevent breakages in case of simple deprecations.

> Honestly, I think this whole no-GC thing is just barking up the wrong tree. People who have GC phobia will turn off as soon as they see "GC", doesn't matter if you can actually write D without GC or not.

I agree.
>
> As for compiling to WASM, isn't LDC already doing that, or on the way to doing that?
>
>
> T

Well, just as with nogc. Many questions in D are answered with "Yeah, but...". My understanding is that LDC can compile to WASM, but only in betterC mode, runtime (and GC) don't work under WASM (this might change in the future with the Spasm effort). But from a bystander perspective betterC isn't acceptable enough of a compromise.

https://github.com/dlang-community/dfix

On Friday, 27 March 2020 at 20:16:19 UTC, H. S. Teoh wrote:
> On Fri, Mar 27, 2020 at 03:08:51PM -0400, Steven Schveighoffer
>
>
> Honestly, I think cheap Exceptions could be done transparently of user code, for the most part.  Instead of using libunwind or equivalent, for example, just change the ABI to propagate an out-of-band error indicator that, say in a register or something that gets checked by the caller and branches to the function's exit block if set.

Why not just pass the address of the error handler into the callee? So instead of

foo()
if (R12) goto i_see_dead_people;

void foo()
{
    if (zombies) { R12 = err; return; }
}

you do...

R12 = &i_see_dead_people
foo();

void foo()
{
    if (zombies) { return to R12; }
}

On x86 at least you could just poke the alternate return address into the stack and do RET as you would for normal return. So in the non-error path it's a single LEA instruction, vs a compare and conditional branch if using an extra return value.

On Friday, 27 March 2020 at 17:17:44 UTC, Steven Schveighoffer wrote:
> On 3/27/20 12:58 PM, 12345swordy wrote:
>> On Friday, 27 March 2020 at 16:54:28 UTC, Steven Schveighoffer wrote:
>>> On 3/27/20 12:03 PM, 12345swordy wrote:
>>>> Didn't the 1.0 to 2.0 conversion nearly kill the language?
>>>
>>> No.
>>>
>> 
>> Oh, must had conflict it with something else.
>
> There was an issue with an alternative standard library (Tango), which divided the community. That shouldn't be a problem for a D3.
>

I remembered Tango being the bigger issue, too. But while working on the HOPL IV paper, digging through the forum archives, and recollecting with a few people, I realized that wasn't the whole story. The Tango split was real, but it almost certainly would have been much less of an issue without the move to D2. The D1/D2 split was much more impactful. Some of the changes and new features required a paradigm shift (e.g., transitive const/immutable, ranges & algorithms). Tango with D1 was an escape hatch for those who were resistant to the changes (I was pretty resistant myself; wrote a big rant about it on my old blog). Some (like me) eventually came around. I'm now of the opinion that if Tango hadn't been around at the time, we may well have lost more people than we did.

There was also this quote from Walter a few years back:

https://forum.dlang.org/post/nmf48b$1ckm$1@digitalmars.com

"There are no plans for D3 at the moment. All plans for improvement are
backwards compatible as much as possible. D had its wrenching change with
D1->D2, and it nearly destroyed us."

I'm saying this just to point out the historical context, not to take sides on a potential D3. If we do move in that direction, IMO we need to do it in a way that's methodical and clearly mapped out.