On Thursday, 11 January 2024 at 19:58:43 UTC, Walter Bright wrote:
> On 1/11/2024 9:56 AM, deadalnix wrote:
>> But that doesn't work. If a GC cycle kicks in in between the thread starting and you having the chance to attach it, you may end up collecting live objects. This is what killed the project, BTW.
>> 
>> Having thread that are not managed by the GC is a fine thing to want, but you got to do it the other way around by detaching the thread from the GC.
>
> If you could pseudocode a solution, we can add it to the bugzilla issue. Threading issues are not something I have much expertise in.

I've no ideas how you can have a list or external created pthreads without horrible hack like hooking pthread_create.

/P

On 1/11/24 20:33, Walter Bright wrote:
> On 1/11/2024 5:07 AM, deadalnix wrote:
>> These are just simple thing that I have on top of my mind, but there are a ton more.
> Thanks for taking the time to write this list. I would like to see all of your issues, though! Let's make a list and then have a bugzilla entry for each of them, so we can start picking them off.
> 
>  > such as @nogc, has been a productivity disaster
> 
> I don't really understand this. Just don't use @nogc?

Perhaps a library you want to use is `@nogc` and you have to provide a callback, override a class method, or fork it to add a new feature you need.

Maybe you want to avoid implicit GC allocations in some part of the code, but now you have to mark the entire function `@nogc` to get the checking, and then you can no longer use exceptions in that function.

etc.

On 1/11/24 22:06, Dennis wrote:
> On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:
>> On 1/11/24 20:33, Walter Bright wrote:
>> Maybe you want to avoid implicit GC allocations in some part of the code, but now you have to mark the entire function `@nogc` to get the checking, and then you can no longer use exceptions in that function.
>>
>> etc.
> 
> Considering the compiler doesn't use @nogc for anything that matters (unlike nothrow, which changes code generation), I wonder if we could degrade @nogc from a type attribute to a linting tool. It would still point out accidental array literals or closures, but not complain when calling a function that hasn't been proven to be @nogc.
> ...

Maybe, but sometimes you really don't want any allocations, as they would lead to a memory leak. The exception case in this example is special exactly because of its implications on control flow.

> It can still complain when calling a function that has been inferred @gc,

Well, but then I can't allocate an exception again.

> and if you want to be GC free 100% certain, don't link it and the linker will complain about missing symbols.

Personally, I want to use GC, perhaps just not in the innermost loop performing latency-critical operations behind a `GC.disable()` and not for the sole reason that the compiler's escape analysis was overly conservative.

On 1/11/2024 1:18 PM, DrDread wrote:
> this is discarding the use case that you want to mark only parts of your codebase nogc, but enforce it'S really nogc. we do use some realtime threads, but the rest off the app is @gc.

This subthread suggests to me that the problem with @nogc is it works.

It reminds me of a similar debate when we added `const` to the type system. Unlike C++, const in D is ruthlessly enforced. People wanted something called "logical const" where some construct pretends to be const when it was actually being mutated.

Logical const code cannot take advantage of it being const because it isn't constant.

I don't see much utility in a "logical @nogc" that allows using the gc anyway.

If you want to use the gc, don't use @nogc.

On 12/01/2024 10:06 AM, Dennis wrote:
> On Thursday, 11 January 2024 at 20:55:52 UTC, Timon Gehr wrote:
>> On 1/11/24 20:33, Walter Bright wrote:
>> Maybe you want to avoid implicit GC allocations in some part of the code, but now you have to mark the entire function `@nogc` to get the checking, and then you can no longer use exceptions in that function.
>>
>> etc.
> 
> Considering the compiler doesn't use @nogc for anything that matters (unlike nothrow, which changes code generation), I wonder if we could degrade @nogc from a type attribute to a linting tool. It would still point out accidental array literals or closures, but not complain when calling a function that hasn't been proven to be @nogc.
> 
> It can still complain when calling a function that has been inferred @gc, and if you want to be GC free 100% certain, don't link it and the linker will complain about missing symbols.

@localnogc is what I want.

Check everything but function calls.

Easy to implement, and helps make sure if you care that a closure is stack allocated, it won't heap allocate.

On Thursday, January 11, 2024 12:33:48 PM MST Walter Bright via Digitalmars-d wrote:
> On 1/11/2024 5:07 AM, deadalnix wrote:
> > These are just simple thing that I have on top of my mind, but there are a ton more.
>
> Thanks for taking the time to write this list. I would like to see all of your issues, though! Let's make a list and then have a bugzilla entry for each of them, so we can start picking them off.
>
>  > such as @nogc, has been a productivity disaster
>
> I don't really understand this. Just don't use @nogc?

If he's talking about what I've discussed with him previously, the issue really relates to attributes in general rather than @nogc specifically. The big problem comes with refactoring code in a large codebase. The analogy that John Colvin came up with for the issue is that putting attributes on your codebase ends up being like putting concrete on it, making it incredibly difficult to refactor anything.

For instance, because function doX requires some set of attributes, everything it calls even indirectly then requires a compatible set of attributes. So, when you go to make a change to anything that doX ends up calling (even indirectly), you're then restricted by the (often arbitrary) attribute requirements that doX has. And of course, since you're dealing with a large codebase, it's not just one function that's the issue. It's a whole bunch of functions, many of which call each other. So, making changes that seem like they should be simple can then become incredibly difficult because of all of the compilation errors you get related to attributes - most of which you really don't care about in practice, but you need to make the compiler happy.

Attributes in general can be useful in a fairly restricted set of code, but once a lot of code is involved, they can become quite problematic - to the point that we're trying to rip out many of the attributes on the codebase I work on at Symmetry. And in some cases, the fact that some third party library developer decided that a function should require a specific set of attributes then becomes problematic when making code changes. Third party code that we could use just fine before suddenly becomes problematic, because we need to make a change to a type which in most languages would have been localized and encapsulated, but attributes cause the changes to cascade out all over the place and potentially make it so that we can't use third party code that worked just fine before. And even if the problems are in our own code, it becomes a time sink to figure out how to rework it all so that it works with the changed attributes.

So, for instance, a piece of code could require that the types and functions used with it be pure. The developer decided at the time that that's what they thought would make sense with what they were trying to do, and the code that was written at the time worked that way. But some time later, a change needs to happen in some other part of the codebase, and something that was pure before can no longer be pure. Suddenly, that part of the code that required pure doesn't compile anymore, and the change could be in a completely different part of the codebase to a type that just so happens to end up being used (maybe even indirectly) by the code requiring pure. So, suddenly, a bunch of code has to be refactored just because of an attribute that really doesn't do much in practice. Realistically, no optimizations were being done based on pure, and at best, they told the developers that nothing like global variables were being used (which realistically weren't being used anyway, because almost no one ever does that, because it's not maintainable). But because some piece of code was doing something simple that didn't work wtih pure but which would have been a perfectly fine and small change in C++, a bunch of code has to be changed. So, the attribute was likely adding zero value, and it just cost a ton of time to make changes because of it.

If @nogc had been used, the situation would have largely been the same. Some other attributes (like @safe) would also cause issues, but would be easier to solve, because there are backdoors. But too often, refactoring the code is then going to end up with stuff like @trusted being slapped on code just to shut the compiler up. And for the ones without easy backdoors, casts will sometimes end up being used to shut the compiler up - maybe with the idea that it's a temporary solution, but obviously, that kind of thing can end up sticking around long term. Depending on the situation and the attribute, it could then be a time bomb in the making, or it could actually be fine. It's certainly not great practice, but in practice, it's the sort of thing that the language's design encourages with larger codebases, because it sometimes becomes by far the simplest way to deal with a change in attributes.

Attributes in general simply don't seem to pay for themselves and come at too high a cost once the codebase gets large enough. They often place what are essentially arbitrary requirements on code which may have seemed reasonable to the developer working on that code at the time but which can become incredibly difficult to change later when you need to. And the benefits that they provide are often minimal in practice, making the whole thing that much worse.

I'm increasingly inclined to think that most attributes are primarily of theoretical benefit rather than of actual, practical benefit, and even if they are of practical benefit, if the codebase gets large enough, the way that they make refactoring difficult tends to make their cost too high to be worth it - and unfortunately, by the point that that becomes clear, you're already using them all over the place and wasting a ton of time because of decisions made months or years ago.

Often, attributes are used because they make sense for how a piece of code is currently written, and they work with that version of the code. But as the needs of the code change, the situation changes, and those attributes might not work anymore, which can ultimately cost a _lot_ of time or even make certain refactorings impossible, particularly as the amount of code involved grows. Really, that's the biggest problem here. We can discuss individual attributes and why they do or don't make sense (and @nogc is particularly bad), but to an extent, _all_ attributes are a problem here.

If we were to be designing D from scratch, I would be arguing very strongly that we should be much, _much_ pickier about the attributes that we have in the language (e.g. I would argue strongly against both pure and @nogc), but as it is, it's going to tend to mean that the advice for anyone working on a codebase of any significant size is going to be to minimize the list of attributes they use to those where there's going to be clear and obvious benefit and which will be much less likely to cause issues with refactoring later. Trying to use as many attributes as possibly (which many D developers think is best practice) really does seem to be like you're putting concrete on your code, making refactoring far, far harder than it would be in most languages - or if you'd just minimize the attributes that you're using.

- Jonathan M Davis