On Monday, 24 July 2023 at 07:22:57 UTC, Paolo Invernizzi wrote:
> On Sunday, 23 July 2023 at 17:14:39 UTC, Walter Bright wrote:
>> On 7/23/2023 2:12 AM, Paolo Invernizzi wrote:
>>> On Saturday, 22 July 2023 at 01:23:54 UTC, Walter Bright wrote:
>>>> Going to revisit this:
>>>>
>>>> https://github.com/dlang/dmd/pull/15443
>>> 
>>> *sigh*
>>
>> ??
>
> You know, I'm in the "deprecate and clean-up the language" camp since forever ...

I am glad there is a deprecate and clean-up the language camp and it's gotten D to where it is today. I like to have improvements like that, but for about 7 years I have been in the stabilize the language camp and I'd like to explain why.

I have java code from 2006 that still compiles today. I have tens of thousands of lines of c/assembly from 1998 that compiles today with two errors and I would bet there's a flag to make it work if I didn't want to change it. In college (about 15 years ago) I helped someone with some code for their PHD dissertation. It was Fortran 57 code copied out of a textbook and it also compiled without issue. IMO, this is the norm for a project.

In contrast, I don't know that I have ever come back to an old D project and had it compile. Just the other day, I went to compile some code I wrote with dmd 2.100 with a 2.104 compiler and it would not compile. Sure, it's an easy fix, but that's not always the case and I am tired of tools that nickel-and-dime me. That is the whole point of picking a toolset, which includes the programming language: to have something that doesn't nickel and dime me - something that is fun and productive. When I'm in the flow, I can write code as fast as I can type forum posts. I can do that in C# for most things (and I just got a job in C# to boot). However, D is pretty much as good as C# for most things, but really shines in some other cases. It's 90% of the way to a solid choice for just about anything.

It is very demotivating to spend 100 hours on a project and then have to put it away, then come back in 2 years and have innumerable compilation errors or have all your libraries become unusable. It is very hard to want to commit to that kind of language where the first step of picking up an old project is replacing libraries - if there are even replacements - and fixing compilation errors and THEN trying to remember what you were doing and where you were at and only then can you actually make progress. This isn't necessarily a dealbreaker for people who work on it full time and keep their code up to date nor for single people who have lots of free time anyway, but it's a dealbreaker for many who are investigating the language or even for a part-time hobbyist.

So that is why I am a proponent of stabilizing the language. So I can work on solving my problem of interest and not working on rewriting libraries or compilation errors. So I can spend the time to get really good and write my own tools so it's even more out of the way of solving my problem.

Obviously, I would prefer the team to keep cleaning up the language, given it doesn't break all my existing code. Someone mentioned that gcc autocorrects code. That would facilitate clean up. Having a tool that would go through and make the minor automatic changes to upgrade to the latest standard would be a solution, if that is practical. Having a simple and unambiguous guide to changes you need to make between language versions could work if people knew to look for it and could find it easily. I also think distributing libraries as binaries could solve most of the big issues, tbh, although that idea doesn't seem to get a lot of mileage for reasons I can't understand.

I believe there is a plan that can work for everyone. I'm not an expert on compilers or build chains and I'm not super clear on where the language is going, so I'm not a great person to explain what that plan is. I am very happy to see that stability is being addressed, and I also understand I may not reflect the target audience of the language. I'm trying to find a way to get into the compiler to get some experience in my free time, but mostly I'm hoping to understand the different perspectives of people on this issue so I can draft a meaningful problem statement. I would like to be in a position to provide some good advice or even come up with a great idea to solve the stability issues without sacrificing moving the language forward. If you are open to it, I would appreciate hearing more about your perspective on this via a phone call or chat. If you're interested you can hit me up here or at my gmail address that is the same as my username.

On Tue, Jul 25, 2023 at 07:28:46AM +0000, harakim via Digitalmars-d wrote:
> On Monday, 24 July 2023 at 07:22:57 UTC, Paolo Invernizzi wrote:
[...]
> > You know, I'm in the "deprecate and clean-up the language" camp since forever ...
> 
> I am glad there is a deprecate and clean-up the language camp and it's gotten D to where it is today. I like to have improvements like that, but for about 7 years I have been in the stabilize the language camp and I'd like to explain why.
> 
> I have java code from 2006 that still compiles today. I have tens of thousands of lines of c/assembly from 1998 that compiles today with two errors and I would bet there's a flag to make it work if I didn't want to change it. In college (about 15 years ago) I helped someone with some code for their PHD dissertation. It was Fortran 57 code copied out of a textbook and it also compiled without issue. IMO, this is the norm for a project.
> 
> In contrast, I don't know that I have ever come back to an old D project and had it compile. Just the other day, I went to compile some code I wrote with dmd 2.100 with a 2.104 compiler and it would not compile. Sure, it's an easy fix, but that's not always the case and I am tired of tools that nickel-and-dime me.

I'm actually in both camps, if that even makes sense. :-D

One of the main reasons I liked D so much is that it got rid of a lot of baggage in C/C++ that it inherits from, resulting in a much cleaner language with fewer dark corners and gotchas. Over time, however, it has started accumulating its own cruft and share of dark corners. Sometimes I wish those corners could be straightened out, but each time my hopes are dashed by the spectre of "we don't want to break user code".

Ironically, at the same time user code continues to break, including my own projects, and that's also frustrating.  I remember several D projects where, coming back to it after a couple of years hoping to make a quick 5-minute fix for a small issue, I end up spending hours or even days to update it because something got deprecated, or plain broke, due to some language changes. There was one project that I couldn't even compile for *months* before my loud complaining pushed some kind soul to fix it. So in that sense I'm also in the "please don't break my existing code" camp.

These two camps seem fundamentally incompatible with each other, but I've been thinking there must be a way to have the best of both worlds. I've suggested this before, but versioning source code seems to be a potential approach. Each source file should have a version declaration to declare which version of the language/compiler it's based on, and when the compiler sees this, it enables "compatibility mode", effectively compile the file as much as possible with the old semantics of the language. Breaking changes are suppressed so that it behaves as if it was an older version of the language. Optionally when an old feature is used a warning is issued, but compilation should not abort because of that -- it should still compile and work as before. Give the user the option of using the old semantics until he's ready to upgrade it.

Of course, some changes are fundamentally incompatible with older versions of the language; in those cases we'd have to issue an error. In the ideal world, this shouldn't happen.  New code added to an old project can declare a newer version and use new features, but the old code will continue to use the older version until the programmer manually changes the version declaration. This shouldn't matter, because if new code X uses a new feature that causes a compile error due to bad interaction with old module M, it wouldn't have been checked in, in the first place. If it ever compiled, that means X must be compatible with M in spite of the version difference, so the compiler should just continue to compile it as before.  If X becomes incompatible with M and the author updates M to be compatible, then so much the better. But if not, things should still work as before.

Libraries should also have access to the declared version of the caller, so that when the library API changes, the old API can be kept present to keep existing callers compilable; user code can optionally declare a version to use the latest and greatest API instead, and update his code to be compatible. As long as the old version is in effect, there should be no API breakages.

Essentially, what I'm proposing is in effect the equivalent of having a virtual dub package whose versions reflect the language version, and different modules will depend on different versions. If there's a way to resolve the dependencies, then the code should compile. If not, it won't compile, and wouldn't have compiled in the first place, so users should have already noticed this and did something about it.  But a project that used to compile should continue to do so no matter what.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:

On Monday, 24 July 2023 at 19:24:56 UTC, max haughton wrote:
> On Monday, 24 July 2023 at 18:43:02 UTC, Walter Bright wrote:
>> On 7/24/2023 12:22 AM, Paolo Invernizzi wrote:
>>> You know, I'm in the "deprecate and clean-up the language" camp since forever ...
>>
>> I am, too, but it clearly does not work for us.
>
> Don't mistake the destination with the method of getting there e.g. a few deprecations aren't actually stopping people from using D, it's just discouraging having a violent torrent of dip1000 deprecations which would (for us) require days or weeks of engineering-time to get around (if even possible) because the underlying model just isn't able to express the patterns that make D productive (as opposed to merely expressive).
>
> For deprecations that are either syntactic or a direct substitution, the compiler should just fix it for you - clang and gcc (to a lesser extent) can both do this, e.g. the body=>do change - regardless of the justification the compiler should just parse it and fix it (or emit a diff, etc etc).

I agree that a tool like python 2to3 could be useful, but the python experience seems to point out that it was pretty a failure overall.

That remind me of the Phobos v2/v3 work about the removing of auto decoding: the party proposing an "let's go with copy/past" loose the race, but the party proposing "let's go with a cleaver super-engineered way of sharing code" stalled ... sometime the best way to move forward is just the simplest one.

I sympathize with you about the torrent of dip1000 deprecations, our code was also hit by them. But if dip1000 is an added value to the codebase, that engineering-time is a good spent time I guess. The point is, aren't the actual compiler switches enough to let you choose when to do the transition of the codebase? What's the problem about that?

We have much more difficulties in reading some "modern" D code, full of scope/ref/in/const/auto ... plus @live ... plus care if scope is before or after, plus yes here scope is doing nothing, etc etc. Maybe because we use python a lot also (you know, machine learning ....) but simplicity, and "explicit is better than implicit" is a value.

I understand your point, but to me the need to explain a programmer, "yes dude, 'in ref' was an ancient way of doing 'in', just keep ALSO that in mind" is a strategic error. Stepping back from the current deprecation mechanics is a pity.

/Paolo

On Tuesday, 25 July 2023 at 09:55:31 UTC, H. S. Teoh wrote:
> But a project that used to compile should continue to do so no matter what.
I agree with this, and I guess technically if you had every version of dmd lying around it could do it today. You wouldn't be able to necessarily do it with one version of the compiler... you would have to look at each library separately. That's where your version comes in or using binaries.

> Essentially, what I'm proposing is in effect the equivalent of having a virtual dub package whose versions reflect the language version, and different modules will depend on different versions. If there's a way to resolve the dependencies, then the code should compile. If not, it won't compile, and wouldn't have compiled in the first place, so users should have already noticed this and did something about it.

I have had similar thoughts about versioning but I think it could be difficult and also it could make the compiler extremely complex. That's why I was thinking about the one time upgrade tool. However, if you had multiple versions of the compiler installed and the latest version would just defer to an older version, if available, then this would actually be a simpler solution and require very little re-work.

On Tuesday, 25 July 2023 at 10:21:30 UTC, Paolo Invernizzi wrote:
> I agree that a tool like python 2to3 could be useful, but the python experience seems to point out that it was pretty a failure overall.
I actually think python is an excellent example of a language that wanted to make major changes and pulled it off successfully. Being arguable the most popular language in the world is a pretty good proof! I don't mind if there is a D2 and D3 like Python, so long as it's something predictable and I can compile old projects. I think the versioning idea mentioned by H. S. Teoh would be a slightly better version of what they did because you wouldn't have to have virtualenvs for each project. To be fair, I don't know why you consider 2to3 a failure because I rarely do anything in Python, but maybe the 2to3 tool specifically is not part of the solution.

> sometimes the best way to move forward is just the simplest one.
I feel like this makes sense, but simplest for whom?

In the end, I would like an ecosystem of IDEs, libraries and tools that don't even exist in other languages. Until there is a way to take a 2-year-old abandoned project and pick it up and build it, it will be starting from scratch each time and a lot of the tools will not get built and stay built. It has to be something where work done is done and not lost. I would like a roadmap to that before I personally commit a lot of my time to it.

> The point is, aren't the actual compiler switches enough to let you choose when to do the transition of the codebase? What's the problem about that?
I would like to know more about this, can you give an example? Does the compiler already let you specify you want to keep using in ref (for example) or something like that?

On Tue, Jul 25, 2023 at 03:06:22PM +0000, harakim via Digitalmars-d wrote: [...]
> In the end, I would like an ecosystem of IDEs, libraries and tools that don't even exist in other languages. Until there is a way to take a 2-year-old abandoned project and pick it up and build it, it will be starting from scratch each time and a lot of the tools will not get built and stay built. It has to be something where work done is done and not lost.  I would like a roadmap to that before I personally commit a lot of my time to it.
[...]

This is why I came up with the versioning idea.  Basically, we need to face the reality that a lot of code, once written, is going to stay that way.  Once in a while you might refactor a bit of code here and there, maybe even rewrite a module or two, but 90% of the code is going to stay untouched for a long time, many years even. This is especially true if we want D's library ecosystem to grow.  The bigger the ecosystem becomes, the more code you'd have to rewrite to keep up-to-date with the language if we continue updating it.  This is inescapable.  The result is that more and more D code will accumulate in the ecosystem, and the more it accumulates, the less likely someone will have the time / energy / motivation to rewrite it all when the language undergoes a fundamental change.

The only solution, to avoid having to do a massive rewrite of 90% of D's entire ecosystem every time there's a language change, is to have the compiler understand old code and compile old code in the way it was supposed to be compiled years ago.

Just think about it.  In human language, the same thing happens. Most literary works in any language, including English, consists of a large body of works written in an older dialect of the language -- think Chaucer, Shakespeare, and more recent, but not contemporary authors -- in the 1900's, in the 1950's, etc.. When you read them, you get the distinct sense that they used certain words differently from how we use them today.  Yet our human brain automatically adapts itself, and parses the older language *according to the older language*, so that we don't have to rewrite Shakespeare in 2023's modern English just to be able to read it.  Nobody's gonna rewrite the entire body of English literature just to keep up with the modern language, that's totally impractical. And nobody's gonna throw away older literature just because it's "not compatible" with the modern language.  No, we keep the older stuff around, and the contemporary language continues evolving like it always has.

How do we reconcile the two?  It's not that hard, you just spontaneously understand when you're reading a 1950's novel, for example, that it's older literature, and they used certain expressions (language constructs) we don't use today, but by looking at the context you can more-or-less figure out what it means. And you just adapt your reading accordingly to understand the text.

In compiler terms, this just means that the compiler has to (1) understand that it's dealing with code meant for an older version of the language, and (2) based on that knowledge adapt the way it processes the code in a way that preserves its original meaning.

Tagging code with versions takes care of (1); that leaves the compiler
to implement (2), and it would work.

Something along these lines has to be done, otherwise we're never gonna move forward.  The ecosystem is just gonna stagnate because the more the language advances, the more old code and old libraries will stop compiling.  But not changing the language doesn't work either -- English in 2023 is different from English in 1823 for a reason.  It has to keep up with the times.  Imagine if we still spoke Shakespearean English today just because we feared that change would obsolete existing literature.  That'd be ridiculous.  We need both progress and compatibility with older literature (i.e., existing code).


T

-- 
Тише едешь, дальше будешь.