On 4/1/20 8:43 AM, Joseph Rushton Wakeling wrote:
> On Wednesday, 1 April 2020 at 09:40:34 UTC, John Colvin wrote:
>> So many things in D are breaking changes. Because of introspection and __traits(compiler,...) etc. almost any bug fix can break someone's code. Even introducing an entirely disjoint overload for a function can be a breaking change.
>>
>> It would be an interesting but daunting (impractical?) task to characterise what is considered "API" and what is "implementation detail" in phobos.
> 
> Has there ever been any kind of systematic effort to record and categorize those sorts of introspection-based breakages?  I'm wondering if there are any regularities to what breaks that would help us avoid them more rigorously.

I think what John is saying is that it's nearly impossible. With introspection, literally EVERYTHING becomes part of the API, including all names of parameters to functions.

It doesn't mean that the API is defined as that, however. But it does mean that code that uses funky techniques might break when something is changed that is not expected to cause a problem.

However, I think we can define what the API does include, and basically warn people "don't depend on this". I just don't think we do that anywhere.

With a more robust versioning system, we could also provide a "breaking" branch which is expected to change, and a "non-breaking" branch which is expected never to break anything. Right now we have a grey area with both.

Your point about deprecations is a very good one. I think we should switch to that (only one deprecation removal release per year). We could probably engineer a versioning system that makes this detectable based on the version number.

-Steve

On Wednesday, 1 April 2020 at 13:26:20 UTC, Steven Schveighoffer wrote:
> I think what John is saying is that it's nearly impossible. With introspection, literally EVERYTHING becomes part of the API, including all names of parameters to functions.
>
> It doesn't mean that the API is defined as that, however. But it does mean that code that uses funky techniques might break when something is changed that is not expected to cause a problem.
>
> However, I think we can define what the API does include, and basically warn people "don't depend on this". I just don't think we do that anywhere.

Ah, OK.  That does make sense.  And yes, I agree that strong clarity on what can robustly be introspected in upstream code, and what not, would be a very good thing.  Is this in principle something that we could automate warnings about?

(I say warnings because I don't imagine banning such introspection would be helpful.  After all one could probably do these things internally in a codebase -- on entities you control and hence where you can prevent breakage -- and get benefit out of them.  So there would have to be some way to indicate intent to do the risky thing.)

> Your point about deprecations is a very good one. I think we should switch to that (only one deprecation removal release per year). We could probably engineer a versioning system that makes this detectable based on the version number.

Well, for example, if we made the breaking changes happen in the first release of the year, we could version D in annual epochs.  D 2020 anyone? :-)

[Bad pun, but: did anyone write up a D 2020 vision? ;-)]

On the broader original topic of your first post -- I do agree that it's really starting to feel like it's time for another major revision to the language, as that will allow some very desirable features to be done much more solidly (or be done at all).  As someone who still wants to get a lot of production use out of D in the next years, I'd MUCH rather have a breaking change that gets us to even more awesome places, than stability and less powerful features.

I'm not sure I would agree with all your listed examples -- e.g. I'm reluctant to endorse pure by default as I think that would clash too much with the ability to just script easily -- but that sort of stuff is for future discussion anyway.  The core idea of wanting to make another major language revision seems sound.

On Thursday, 2 April 2020 at 22:51:18 UTC, Joseph Rushton Wakeling wrote:
> On Wednesday, 1 April 2020 at 13:26:20 UTC, Steven Schveighoffer wrote:
>> I think what John is saying is that it's nearly impossible. With introspection, literally EVERYTHING becomes part of the API, including all names of parameters to functions.
>>
>> It doesn't mean that the API is defined as that, however. But it does mean that code that uses funky techniques might break when something is changed that is not expected to cause a problem.
>>
>> However, I think we can define what the API does include, and basically warn people "don't depend on this". I just don't think we do that anywhere.
>
> Ah, OK.  That does make sense.  And yes, I agree that strong clarity on what can robustly be introspected in upstream code, and what not, would be a very good thing.  Is this in principle something that we could automate warnings about?

That's actually a topic I was very interested in when I was working at Sociomantic. We had a pretty good versioning thing going on with libraries, but it was never formally defined what was an acceptable change. As mentioned, the ability to do `static if (!__traits(compiles, ...))` (notice the negation: checking that something does not compile) means that, except for introducing a new module (but that's due to a compiler limitation that we'd eventually fix), everything can be considered a breaking change, if you're a purist.

I've always been very fond of https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B as it describes the techniques one can use to ensure binary compatibility in C++ program, and wish we had the same in D.

> (I say warnings because I don't imagine banning such introspection would be helpful.  After all one could probably do these things internally in a codebase -- on entities you control and hence where you can prevent breakage -- and get benefit out of them.  So there would have to be some way to indicate intent to do the risky thing.)

We don't do warnings. We have some, but they are an historical bagage. I know Walter oppose to it, it was brought up multiple times.

>> Your point about deprecations is a very good one. I think we should switch to that (only one deprecation removal release per year). We could probably engineer a versioning system that makes this detectable based on the version number.
>
> Well, for example, if we made the breaking changes happen in the first release of the year, we could version D in annual epochs.  D 2020 anyone? :-)

I still think the deprecation scheme should take a few more things into account, e.g. how long has the replacement feature been available for ? What bugs are triggered by the feature that is to be removed ? How much is it used ?
For example, the body deprecation: Most of the gains come from making `body` a context-dependent keyword, and allowing `do`. Removing `body` as a keyword doesn't actually help the user. It doesn't bring them value. Additionally, TONS of code out there use `body`. Using `body` does not cause bugs at all. And the replacement was introduced only shortly before deprecation.
For this reason, we could keep it in the language for a very long time (5 years at least), and just undocument  it /  replace usages as we can.

On the other hands, D1 operator overloads: The replacement (D2) have been around for over a  decade. They have different priorities, so they don't mix well with their replacement, leading to subtle bugs (which was actually what triggered their deprecation). However there is A LOT of code out there that still uses them, so that's why I would want us to go with at least a 2 year deprecation period.

> I'm not sure I would agree with all your listed examples -- e.g. I'm reluctant to endorse pure by default as I think that would clash too much with the ability to just script easily -- but that sort of stuff is for future discussion anyway.  The core idea of wanting to make another major language revision seems sound.

That's a good point. There's no escape hatch for `pure`. How do I call `writeln` from main ?

On 4/3/20 1:00 AM, Mathias LANG wrote:
> On Thursday, 2 April 2020 at 22:51:18 UTC, Joseph Rushton Wakeling wrote:
>> On Wednesday, 1 April 2020 at 13:26:20 UTC, Steven Schveighoffer wrote:
>>> I think what John is saying is that it's nearly impossible. With introspection, literally EVERYTHING becomes part of the API, including all names of parameters to functions.
>>>
>>> It doesn't mean that the API is defined as that, however. But it does mean that code that uses funky techniques might break when something is changed that is not expected to cause a problem.
>>>
>>> However, I think we can define what the API does include, and basically warn people "don't depend on this". I just don't think we do that anywhere.
>>
>> Ah, OK.  That does make sense.  And yes, I agree that strong clarity on what can robustly be introspected in upstream code, and what not, would be a very good thing.  Is this in principle something that we could automate warnings about?

I don't think warnings are possible. But I'd say we can at least have a list of changes that the D library considers to be "non-breaking changes", even if they will break introspective code. What that list is, I have not thought of. But it should be something discussed, especially if we are to have a more formal versioning system.

> 
> That's actually a topic I was very interested in when I was working at Sociomantic. We had a pretty good versioning thing going on with libraries, but it was never formally defined what was an acceptable change. As mentioned, the ability to do `static if (!__traits(compiles, ...))` (notice the negation: checking that something does not compile) means that, except for introducing a new module (but that's due to a compiler limitation that we'd eventually fix), everything can be considered a breaking change, if you're a purist.

Yeah, that just leads to breaking changes being OK, since you can't avoid breaking changes.

> I've always been very fond of https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B as it describes the techniques one can use to ensure binary compatibility in C++ program, and wish we had the same in D.

D uses source compatibility almost exclusively, because of the vast metaprogramming capabilities -- you have to recompile because any slight change in a function or type is going to require a rebuild.

Only a library that is focused on binary compatibility could specify such a thing. Druntime and Phobos can't be in that boat.

>>> Your point about deprecations is a very good one. I think we should switch to that (only one deprecation removal release per year). We could probably engineer a versioning system that makes this detectable based on the version number.
>>
>> Well, for example, if we made the breaking changes happen in the first release of the year, we could version D in annual epochs.  D 2020 anyone? :-)

I think deprecations should have a time requirement (at least X years/months), but also only be done periodically.

> I still think the deprecation scheme should take a few more things into account, e.g. how long has the replacement feature been available for ? What bugs are triggered by the feature that is to be removed ? How much is it used ?

Yes, it's simply the maximum of X time and the next "breaking" release.

> For example, the body deprecation: Most of the gains come from making `body` a context-dependent keyword, and allowing `do`. Removing `body` as a keyword doesn't actually help the user. It doesn't bring them value. Additionally, TONS of code out there use `body`. Using `body` does not cause bugs at all. And the replacement was introduced only shortly before deprecation.
> For this reason, we could keep it in the language for a very long time (5 years at least), and just undocument  it /  replace usages as we can.

I think you mean removing body as a contextual keyword doesn't help the user because he can use `body` today as a normal variable name, right?

I agree with that, it's not critical to remove the "alternate" keyword for `do` here.

> On the other hands, D1 operator overloads: The replacement (D2) have been around for over a  decade. They have different priorities, so they don't mix well with their replacement, leading to subtle bugs (which was actually what triggered their deprecation). However there is A LOT of code out there that still uses them, so that's why I would want us to go with at least a 2 year deprecation period.

For sure. opInc or opAdd is a lot easier and quicker to write than opBinary(string op: "+"). So there is going to be a lot of code that just does the former.

We still have yet to provide a mixin that provides the forwarding system (i.e. if you have opAdd or opInc, it will define appropriate forwarding). Something like this I think should be a good library or object.d inclusion.

>> I'm not sure I would agree with all your listed examples -- e.g. I'm reluctant to endorse pure by default as I think that would clash too much with the ability to just script easily -- but that sort of stuff is for future discussion anyway.  The core idea of wanting to make another major language revision seems sound.
> 
> That's a good point. There's no escape hatch for `pure`. How do I call `writeln` from main ?

That was just a list of wishes, not necessarily what I want.

All the "by default" wishes stem from the fact that a) much non-template code is written that is already pure/safe/nothrow, but unless you tag it, they are not usable from that realm, and b) tagging everything in all the ways they are supposed to be tagged is tedious and verbose.

Honestly, I'd say a mechanism to affect the defaults:

pragma(default_attributes, pure, @safe, nothrow);

or something like that would go a long way. And you would need opposites for all to make that work (i.e. throw, impure, @system).

Then you can alter the defaults, and there truly is an escape hatch to go back to the way it was. I too would list pure as one that is too difficult to workaround to be a default default, especially since the standard hello world would fail to compile!

-Steve

The regular versioning of a language and even standard libraries with new features and/or removal of features seems to be the norm. I brought this up some years ago and there appeared to be unanimous support for remaining with what I would call “gcc-style versioning”.

On Friday, 27 March 2020 at 15:56:40 UTC, Steven Schveighoffer wrote:
> There have been a lot of this pattern happening:
>
> 1. We need to add feature X, to fix problem Y.
> 2. This will break ALL CODE IN EXISTENCE
> 3. OK, cancel the fix, we'll just live with it.
>
> Having a new branch of the compiler will provide a way to keep D2 development alive while giving a playground to add new mechanisms, fix long-existing design issues, and provide an opt-in for code breakage.
>
> Some issues I can think of:
>
> 1. The safe by default debate
> 2. pure by default
> 3. nothrow by default
> 4. String interpolation DIP
> 5. auto-decoding
> 6. range.save
> 7. virtual by default
> 8. ProtoObject
>
> Other languages evolve much quicker than D, but break things only in major updates. D seems to "sort of" break things, there's always a risk in every release. We try to be conservative, but we have this horrible mix of deciding some features can break things, while others are not allowed to, and there's no clear guide as to which breakage fits in which category.
>
> If we went to a more regular major release schedule, and decided for a roadmap for each major release what features would be included, it would allow much better planning, and much more defensible breakage of code. If you know that your code will only compile with D2.x, and you're fine with that, then great, don't upgrade to D3.x. If you desperately want a feature, you may have to upgrade to D3.x, but once you get there, you know your code is going to build for a while.
>
> We could also not plan for many major releases, but at least move to D3 for some major TLC to the language that is held back to prevent breakage.
>
> I work occasionally with Swift, and they move very fast, and break a lot of stuff, but only in major versions. It's a bit fast for my taste, but it seems to work for them. And they get to fix issues that languages like C++ might have been stuck with forever.
>
> The biggest drawback is that we aren't a huge language, with lots of manpower to keep x branches going at once.
>
> I just wanted to throw it out as a discussion point. We spend an awful lot of newsgroup server bytes debating things that to me seem obvious, but have legitimate downsides for not breaking them in a "stable" language.
>
> -Steve

I agree with the general idea of introducing breaking changes in an organised way.
I didn't write a massive amount of D, but in light of its brilliant features, sticking so rigidly to C and C++ seems like an idea that inherited difficulties from them meanwhile failing to reap the benefits. The great features which make you all gather here deserve to break free!
Fortunately now seems to be a historical moment when C syntax/semantics does no longer need to be a given, as more languages gain traction that purposefully break away from them. (For the better, I think.)
It does not matter if D only followed in these languages' footsteps, however humiliating that might feel, because it'd have the potential to give the language a new life. In its current form, its advantages over C++ seem to diminish with every new standard.

Whether that would happen or not, the new language would definitely need a solidly defined identity / value proposition so it'd be clear why to pick it up. It would also need a big break from the current design principles, which remain essentially unchallenged for a long time now. That might also go against some current mindset and eventually could be seem just too big of a deviation.

All in all, not sure if any of that would happen but it'd be really exciting!

On Wednesday, 8 April 2020 at 16:44:57 UTC, Istvan Dobos wrote:
> It does not matter if D only followed in these languages' footsteps, however humiliating that might feel, because it'd have the potential to give the language a new life. In its current form, its advantages over C++ seem to diminish with every new standard.

Funny, so called "modern C++" (C++17 and beyond) is one of the reasons why I got interested in D. In my opinion "modern C++" is absolutely awful.

The success of Java can be linked to its similarities in syntax with C and C++ so inheriting the C/C++ doesn't have to be a disadvantage. D did not become as successful as Java, for a few reasons but it feels OT in this thread.

On Wed, Apr 08, 2020 at 04:55:56PM +0000, IGotD- via Digitalmars-d wrote:
> On Wednesday, 8 April 2020 at 16:44:57 UTC, Istvan Dobos wrote:
> > It does not matter if D only followed in these languages' footsteps, however humiliating that might feel, because it'd have the potential to give the language a new life. In its current form, its advantages over C++ seem to diminish with every new standard.
> 
> Funny, so called "modern C++" (C++17 and beyond) is one of the reasons why I got interested in D. In my opinion "modern C++" is absolutely awful.
[...]

Yeah, with every new C++ revision, the language becomes more baroque, more overly complex, and introduces yet more strange exceptional cases that nobody can possibly remember (except maybe for Scott Meyers :-P). No thanks, despite whatever warts D has, it's still light-years better than C++.


T

-- 
Жил-был король когда-то, при нём блоха жила.

On Wednesday, 8 April 2020 at 17:17:07 UTC, H. S. Teoh wrote:
> On Wed, Apr 08, 2020 at 04:55:56PM +0000, IGotD- via Digitalmars-d wrote:
>> On Wednesday, 8 April 2020 at 16:44:57 UTC, Istvan Dobos wrote:
>> > It does not matter if D only followed in these languages' footsteps, however humiliating that might feel, because it'd have the potential to give the language a new life. In its current form, its advantages over C++ seem to diminish with every new standard.
>> 
>> Funny, so called "modern C++" (C++17 and beyond) is one of the reasons why I got interested in D. In my opinion "modern C++" is absolutely awful.
> [...]
>
> Yeah, with every new C++ revision, the language becomes more baroque, more overly complex, and introduces yet more strange exceptional cases that nobody can possibly remember (except maybe for Scott Meyers :-P). No thanks, despite whatever warts D has, it's still light-years better than C++.
>
>
> T

Yeah, not the first time I hear that opinion. Myself have a very little exposure to that beast and I intend to keep it that way.
My point was more like, if the idea of breaking changes + new major version settles in, it could be good to aim higher!

On Saturday, 28 March 2020 at 17:09:34 UTC, Denis Feklushkin wrote:

>
> int -> int32
> ulong -> uint64
> float -> float32
> double -> float64
> byte -> octet

if ulong is uint64.. byte should be uint8