On 1/11/24 22:45, Jonathan M Davis wrote:
> On Thursday, January 11, 2024 12:33:48 PM MST Walter Bright via Digitalmars-d
> wrote:
>> On 1/11/2024 5:07 AM, deadalnix wrote:
>>> These are just simple thing that I have on top of my mind, but there are a
>>> ton more.
>>
>> Thanks for taking the time to write this list. I would like to see all of
>> your issues, though! Let's make a list and then have a bugzilla entry for
>> each of them, so we can start picking them off.
>>
>>   > such as @nogc, has been a productivity disaster
>>
>> I don't really understand this. Just don't use @nogc?
> 
> ...
> I'm increasingly inclined to think that most attributes are primarily of
> theoretical benefit

No, theory predicts many of those issues with the way attributes are designed in D currently. x)

> ... Trying to use as many attributes as possibly (which many D developers
> think is best practice) really does seem to be like you're putting concrete
> on your code, making refactoring far, far harder than it would be in most
> languages - or if you'd just minimize the attributes that you're using.
> ...

Yes. Unfortunately it seems to be very hard to make new D developers understand this point until they run into the problem themself.

You put an attribute if the code is wrong if it does not conform to the attribute, otherwise you should not put it. But in D, often both choices are incorrect if you want to interact with code that does have attributes applied to it.

On Thursday, 11 January 2024 at 22:49:51 UTC, Guillaume Piolat wrote:
> On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright wrote:
>>
>> This subthread suggests to me that the problem with @nogc is it works.
>
>
> @nogc could get an escape hatch maybe?
> pure is especially annoying because no escape hatch.
>
> Perhaps we can live with their UB in a per-attribute basis.
> UB of @nogc is allocating  with GC, it only annoys in cases without GC, else well, type system was broken.
>
> It's pretty nice to enforce statically no accidental allocation in a time critical callback, in short the reasons that made @nogc invented still apply.

+1, I was posting the same. If @nogc code _could_ call _explicit_ `@gc` functions that could be a solution.

On Thursday, 11 January 2024 at 22:49:51 UTC, Guillaume Piolat wrote:
>
> @nogc could get an escape hatch maybe?
> pure is especially annoying because no escape hatch.
>
> Perhaps we can live with their UB in a per-attribute basis.
> UB of @nogc is allocating  with GC, it only annoys in cases without GC, else well, type system was broken.
>
> It's pretty nice to enforce statically no accidental allocation in a time critical callback, in short the reasons that made @nogc invented still apply.

I honestly would rather have nothrow and pure go than @nogc, which is going to be used on no-runtime, minimal runtimes (such as those that go in limited environments), runtimes with a GC that.

It's not avoiding the new that is difficult, it's the array literals, the accidental closures, and whatever also allocates quietly.

On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:
> On 12/01/2024 11:33 AM, Timon Gehr wrote:
>> If you do that D can no longer correctly implement a higher-order function that forms the composed function from its two arguments. It looks good on paper for a few minutes, but this is not a good solution to apply by default.
> 
> Unless we get something different that solves the same premise, opt-in is fine.
> 
> I'm thinking explicitly for things like ``opApply``. Having giant mixin templates that handle all combinations is far worse.

Yes, attribute polymorphism is needed for that case, but currently there is no way to abstract over attributes and this is invariably the point where language features become a bit too esoteric for people (both language designers and users) who are not type system experts and then ad-hoc hacks proliferate.

On 12/01/2024 11:56 AM, Timon Gehr wrote:
> On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:
>> On 12/01/2024 11:33 AM, Timon Gehr wrote:
>>> If you do that D can no longer correctly implement a higher-order function that forms the composed function from its two arguments. It looks good on paper for a few minutes, but this is not a good solution to apply by default.
>>
>> Unless we get something different that solves the same premise, opt-in is fine.
>>
>> I'm thinking explicitly for things like ``opApply``. Having giant mixin templates that handle all combinations is far worse.
> 
> Yes, attribute polymorphism is needed for that case, but currently there is no way to abstract over attributes and this is invariably the point where language features become a bit too esoteric for people (both language designers and users) who are not type system experts and then ad-hoc hacks proliferate.

I am very open to a counter proposal that covers the points you have outlined.

Right now, contract invalidation + finely grained template solution for when you need more control is the bar to beat, at least for me.

Either way, right now this problem is dividing the community and causing significant issues with attributes in practice. It needs solving. It would be a major pressure release for this subject for many people.

On 1/11/24 23:41, Walter Bright wrote:
> 
> Any piece of code that accepts lambdas, callbacks, etc., should be very clear about attribute use and how that will apply to the lambdas, etc.

The issue is, sometimes the piece of code that accepts lambdas is entirely unopinionated about what the right attributes are, but the callback and the calling code have a very strong opinion. There is no way to annotate this now that makes sense, you have to duplicate the code with different attribute annotations. I am not sure exactly how many overloads of `opApply` you need to cater to all use cases, but I think it is more than ten.

This is a fundamental issue with D's type system, and it is a common issue programming languages run into. Type theorists solve it with attribute polymorphism. `inout` was an attempt at that, albeit not an extremely successful one.

On Thursday, January 11, 2024 3:02:16 PM MST Adam Wilson via Digitalmars-d wrote:
> On Thursday, 11 January 2024 at 21:37:01 UTC, Walter Bright wrote:
> > On 1/11/2024 1:18 PM, DrDread wrote:
> >> this is discarding the use case that you want to mark only parts of your codebase nogc, but enforce it'S really nogc. we do use some realtime threads, but the rest off the app is @gc.
> >
> > This subthread suggests to me that the problem with @nogc is it works.
>
> @nogc works just fine. We recently spent a good chunk of time in Discord educating a newbie on what it *actually* does.
>
> What @nogc is specified to do: Prevent GC allocations from
> occurring. Fantastic.
> What people actually do with @nogc: Use it to selectively disable
> the GC without using GC.disable().
>
> The reason for this stems from a side-effect of how the *current* GC operates. Because allocations are the trigger for collections, by preventing allocations, collections are also prevented. And what people really want to do is disable collections because they don't like the collection pauses. They don't *actually* care about the allocations per se because that is generally as fast as a malloc and they are going to have to allocate at some point anyways.
>
> So @nogc works exactly as specified, but because of an unspecified implementation side-effect, that is not guaranteed to hold true in the future, the @nogc crowd writes their code as if @nogc does something else entirely.  And we end up here in this thread.

@nogc doesn't even prevent collections running while a function is called. It just prevents that specific function from triggering the collection (thanks to no allocations triggering it). Another thread could trigger a collection while that function is running. So, yes, in a single-threaded program, it's equivalent to calling GC.disable, but in the general case, it isn't.

So, anyone actually relying on @nogc preventing collections is arguably asking for trouble even with the current GC - though it seems like the kind of folks who like to use @nogc are often the kind of folks who are likely to avoid the GC entirely, so if @nogc is used everywhere in their code, then they won't end up with either GC allocations or collections. However, anyone using it selectively really can't rely on it to avoid collections unless their program is single-threaded and will always remain so.

- Jonathan M Davis

On 12/01/2024 12:05 PM, Timon Gehr wrote:
> On 1/11/24 23:41, Walter Bright wrote:
> 
>     Any piece of code that accepts lambdas, callbacks, etc., should be
>     very clear about attribute use and how that will apply to the
>     lambdas, etc.
> 
> The issue is, sometimes the piece of code that accepts lambdas is entirely unopinionated about what the right attributes are, but the callback and the calling code have a very strong opinion. There is no way to annotate this now that makes sense, you have to duplicate the code with different attribute annotations. I am not sure exactly how many overloads of |opApply| you need to cater to all use cases, but I think it is more than ten.

I have it implemented as a permutation...

This is awful for exports. It should be one, not 24 on top of a template opApply that accepts anything internally.

https://github.com/Project-Sidero/basic_memory/blob/main/source/sidero/base/internal/meta.d#L4

On 1/12/24 00:04, Richard (Rikki) Andrew Cattermole wrote:
> On 12/01/2024 11:56 AM, Timon Gehr wrote:
>> On 1/11/24 23:37, Richard (Rikki) Andrew Cattermole wrote:
>>> On 12/01/2024 11:33 AM, Timon Gehr wrote:
>>>> If you do that D can no longer correctly implement a higher-order function that forms the composed function from its two arguments. It looks good on paper for a few minutes, but this is not a good solution to apply by default.
>>>
>>> Unless we get something different that solves the same premise, opt-in is fine.
>>>
>>> I'm thinking explicitly for things like ``opApply``. Having giant mixin templates that handle all combinations is far worse.
>>
>> Yes, attribute polymorphism is needed for that case, but currently there is no way to abstract over attributes and this is invariably the point where language features become a bit too esoteric for people (both language designers and users) who are not type system experts and then ad-hoc hacks proliferate.
> 
> I am very open to a counter proposal that covers the points you have outlined.
> ...

The way this is usually handled is via generic parameters.

First, make attributes first class, so stuff like this just works:

alias nice=pure @nogc @safe nothrow;

void foo()@nice{}

Then, you need generic parameters, I'll indicate them with `[]`:

```d
void opApply[functionAttribute a](scope int delegate()a dg)a{
    ....
}
```

Done. Here, everything in `[]` exists only at compile time (think of it as ghost variables), and there is only one `opApply` function at runtime.

> Right now, contract invalidation + finely grained template solution for when you need more control is the bar to beat, at least for me.
> 
> Either way, right now this problem is dividing the community and causing significant issues with attributes in practice. It needs solving. It would be a major pressure release for this subject for many people.
> 

I am not saying don't do the ad-hoc hacks, but understand that this is what you are doing. `inout` was a decent hack, until you need to do something a bit more involved, e.g. `opApply`. Opt-in invalidation is a decent option, until you run into the same issue again at a slightly higher level of abstraction, necessitating the next hack. The mess that will be left behind will be more complicated than the proper solution would have been, even though now it may appears that what I am suggesting is more complicated.

On Thursday, January 11, 2024 3:17:05 PM MST Timon Gehr via Digitalmars-d wrote:
> On 1/11/24 22:37, Walter Bright wrote:
> > On 1/11/2024 1:18 PM, DrDread wrote:
> >> this is discarding the use case that you want to mark only parts of your codebase nogc, but enforce it'S really nogc. we do use some realtime threads, but the rest off the app is @gc.
> >
> > This subthread suggests to me that the problem with @nogc is it works. ...
>
> It does not work. There are two options:
>
> - Mark the function and callback `@nogc`, then GC users cannot use your
> library.
> - Mark the function and callback not `@nogc`, then `@nogc` users cannot
> use your library.
>
> There are two options and both are wrong. There is no right answer except to accept that `@nogc` actually does not work for library writers, and neither do the other attributes.

Yeah, to an extent, the problem can be solved with templates, but that only works if the attributes of the library code depend entirely on the template arguments, and it becomes _very_ easy to break other people's code when changing the implementation by accidentally changing which attributes are inferred.

Once the code isn't templated, you're either restricting what the library can do with its implementation by using attributes on it which then prevent you from being able to refactor the code in a way that doesn't work with those attributes - or you're restricting what the user code can do with your library, because they don't support one or more attributes that they require. Either way, the existance of those attributes restricts what can be done with your library regardless of which set of attributes you end up using.

Another way to look at the problem is that many of these attributes are effectively leaking the implementation details of your library. If you can mark something as const or pure or @nogc, etc. then that says something about what your library is currently doing. And because the code using your library then potentially relies on those attributes being there, you then can't change your implementation, because those details leaked through the API.

- Jonathan M Davis