When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.

Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.

On Saturday, June 16, 2018 06:08:17 DigitalDesigns via Digitalmars-d wrote:
> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.
>
> Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.

I would point out that identifiers that start with two underscores are supposed to be reserved for the compiler. Declaring them yourself is begging for trouble in general.

- Jonathan M Davis

On 16/06/2018 6:47 PM, Jonathan M Davis wrote:
> On Saturday, June 16, 2018 06:08:17 DigitalDesigns via Digitalmars-d wrote:
>> When an identifier starts with __, allMembers does not return it.
>> This is very bad behavior! It can silently create huge problems
>> if the programmer is not aware of the bug.
>>
>> Instead, if you want to keep the current behavior at least create
>> an error or warning rather than silently create a bug to waste
>> the users time tracking it down.
> 
> I would point out that identifiers that start with two underscores are
> supposed to be reserved for the compiler. Declaring them yourself is begging
> for trouble in general.
> 
> - Jonathan M Davis

We discussed this on IRC yesterday.

What we decided that we thought was correct was to emit a warning if __* was used for an identifier outside of core.*. But make things like allMembers not check for it.

Simple, but effective.

On Saturday, 16 June 2018 at 06:47:59 UTC, Jonathan M Davis wrote:
> On Saturday, June 16, 2018 06:08:17 DigitalDesigns via Digitalmars-d wrote:
>> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.
>>
>> Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.
>
> I would point out that identifiers that start with two underscores are supposed to be reserved for the compiler. Declaring them yourself is begging for trouble in general.
>
> - Jonathan M Davis

So, you are saying that it's ok to beat me over the head and cause me trouble simply because I used __? Seriously? I thought D was all about making the programmers life easier and not more difficult! Thanks for letting me know! I guess it's time to go back to C#.

On Saturday, June 16, 2018 22:59:24 DigitalDesigns via Digitalmars-d wrote:
> On Saturday, 16 June 2018 at 06:47:59 UTC, Jonathan M Davis wrote:
> > On Saturday, June 16, 2018 06:08:17 DigitalDesigns via
> >
> > Digitalmars-d wrote:
> >> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.
> >>
> >> Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.
> >
> > I would point out that identifiers that start with two underscores are supposed to be reserved for the compiler. Declaring them yourself is begging for trouble in general.
> >
> > - Jonathan M Davis
>
> So, you are saying that it's ok to beat me over the head and cause me trouble simply because I used __? Seriously? I thought D was all about making the programmers life easier and not more difficult! Thanks for letting me know! I guess it's time to go back to C#.

There's no need to be melodramatic. All I'm saying is that identifiers starting with __ are reserved by the language - just like they are in many C-derived languages (I would expect them to be reserved in C# as well, though I don't know for sure if they are). Whether __traits stuff should ignore such attributes, I don't know (I wouldn't have expected them to be ignored, and that does seem weird), but either way, if you have any variable names that start with two underscores (be they member variables or local variables or whatever), you're potentially declaring variables that are in conflict with stuff that druntime or the compiler itself declare, and you risk bugs by doing so. Even if __traits shouldn't just ignore those identifiers, you shouldn't be declaring any variables that start with two underscores. If you didn't know that before, then I'm sorry, but the spec mentions it, and now you know.

https://dlang.org/spec/lex.html#identifiers

- Jonathan M Davis

On Saturday, 16 June 2018 at 23:33:13 UTC, Jonathan M Davis wrote:
> On Saturday, June 16, 2018 22:59:24 DigitalDesigns via Digitalmars-d wrote:
>> On Saturday, 16 June 2018 at 06:47:59 UTC, Jonathan M Davis wrote:
>> > On Saturday, June 16, 2018 06:08:17 DigitalDesigns via
>> >
>> > Digitalmars-d wrote:
>> >> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.
>> >>
>> >> Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.
>> >
>> > I would point out that identifiers that start with two underscores are supposed to be reserved for the compiler. Declaring them yourself is begging for trouble in general.
>> >
>> > - Jonathan M Davis
>>
>> So, you are saying that it's ok to beat me over the head and cause me trouble simply because I used __? Seriously? I thought D was all about making the programmers life easier and not more difficult! Thanks for letting me know! I guess it's time to go back to C#.
>
> There's no need to be melodramatic. All I'm saying is that identifiers starting with __ are reserved by the language - just like they are in many C-derived languages (I would expect them to be reserved in C# as well, though I don't know for sure if they are). Whether __traits stuff should ignore such attributes, I don't know (I wouldn't have expected them to be ignored, and that does seem weird), but either way, if you have any variable names that start with two underscores (be they member variables or local variables or whatever), you're potentially declaring variables that are in conflict with stuff that druntime or the compiler itself declare, and you risk bugs by doing so. Even if __traits shouldn't just ignore those identifiers, you shouldn't be declaring any variables that start with two underscores. If you didn't know that before, then I'm sorry, but the spec mentions it, and now you know.
>
> https://dlang.org/spec/lex.html#identifiers
>
> - Jonathan M Davis


If this was a sane language constraint then any identifiers starting with __ that were not reserved would at least give a warning but particularly give an error! Not fail silently and break code in ways that cannot be determined otherwise. It's bad enough that the whole template system cannot be properly debugged but to silently ignore user identities that start with __ just because is ignorant, and it's not being melodramatic.

I've seen you state several times that D does certain things to "protect" the user... and now you are justifying it attacking the user. You can't have it both ways, which is it?

__ is not an uncommon identifier prefix and to reserve it then have code break in ways that are nearly undetectable and could cause major bugs in complex programs is not sane. How about reserve a special character for special identifiers and do a full rename on the master branch and simply do away with the arbitrary __ constraint?

There are correct ways to solve the problem and incorrect. Justifying incorrect ways when they are clearly incorrect is not sane(but it is typical practice).

On Sunday, 17 June 2018 at 01:02:17 UTC, DigitalDesigns wrote:
> If this was a sane language constraint then any identifiers starting with __ that were not reserved would at least give a warning but particularly give an error! Not fail silently and break code in ways that cannot be determined otherwise.

It is undefined behavior to use ANY identifier with __ leading. C works exactly the same way.

On Sunday, June 17, 2018 01:02:17 DigitalDesigns via Digitalmars-d wrote:
> On Saturday, 16 June 2018 at 23:33:13 UTC, Jonathan M Davis wrote:
> > On Saturday, June 16, 2018 22:59:24 DigitalDesigns via
> >
> > Digitalmars-d wrote:
> >> On Saturday, 16 June 2018 at 06:47:59 UTC, Jonathan M Davis
> >>
> >> wrote:
> >> > On Saturday, June 16, 2018 06:08:17 DigitalDesigns via
> >> >
> >> > Digitalmars-d wrote:
> >> >> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.
> >> >>
> >> >> Instead, if you want to keep the current behavior at least create an error or warning rather than silently create a bug to waste the users time tracking it down.
> >> >
> >> > I would point out that identifiers that start with two underscores are supposed to be reserved for the compiler. Declaring them yourself is begging for trouble in general.
> >> >
> >> > - Jonathan M Davis
> >>
> >> So, you are saying that it's ok to beat me over the head and cause me trouble simply because I used __? Seriously? I thought D was all about making the programmers life easier and not more difficult! Thanks for letting me know! I guess it's time to go back to C#.
> >
> > There's no need to be melodramatic. All I'm saying is that identifiers starting with __ are reserved by the language - just like they are in many C-derived languages (I would expect them to be reserved in C# as well, though I don't know for sure if they are). Whether __traits stuff should ignore such attributes, I don't know (I wouldn't have expected them to be ignored, and that does seem weird), but either way, if you have any variable names that start with two underscores (be they member variables or local variables or whatever), you're potentially declaring variables that are in conflict with stuff that druntime or the compiler itself declare, and you risk bugs by doing so. Even if __traits shouldn't just ignore those identifiers, you shouldn't be declaring any variables that start with two underscores. If you didn't know that before, then I'm sorry, but the spec mentions it, and now you know.
> >
> > https://dlang.org/spec/lex.html#identifiers
> >
> > - Jonathan M Davis
>
> If this was a sane language constraint then any identifiers starting with __ that were not reserved would at least give a warning but particularly give an error! Not fail silently and break code in ways that cannot be determined otherwise. It's bad enough that the whole template system cannot be properly debugged but to silently ignore user identities that start with __ just because is ignorant, and it's not being melodramatic.
>
> I've seen you state several times that D does certain things to "protect" the user... and now you are justifying it attacking the user. You can't have it both ways, which is it?
>
> __ is not an uncommon identifier prefix and to reserve it then have code break in ways that are nearly undetectable and could cause major bugs in complex programs is not sane. How about reserve a special character for special identifiers and do a full rename on the master branch and simply do away with the arbitrary __ constraint?
>
> There are correct ways to solve the problem and incorrect. Justifying incorrect ways when they are clearly incorrect is not sane(but it is typical practice).

Identifiers that start with two underscores should be very uncommon, because they're usually reserved in C-derived languages (though for better or worse, most such languages don't actually flag them as errors even though they're technically illegal). There's really nothing unique about D in that respect. I suspect the reason that it's not a compiler error in D to start an identifier with two underscores is because it would make it a pain for druntime, because - being the runtime for the language - there are a number of identifiers that it declares which start with two underscores, and if the compiler tried to make that an error in general, it would either have to know all of the identifiers that druntime used or somehow know that it was compiling druntime (and at least some of those identifiers are extern(C) so that they're not mangled based on the module they're in, making detecting that something is part of druntime more complicated). It also may be that it needs to be possible to declare identifiers that start with two underscores in order to link against code from C's runtime. I don't know.

I don't disagree that it would be nice if the compiler made it an error if anyone tries to do that in their program, but given how that would affect druntime, I'm not sure how reasonable that really would be in practice even if it really should happen in principle. And in practice, programs that have identifiers that start with two underscores should be pretty rare anyway. Outside of something like druntime or the compiler, I don't think that I've ever seen it. So, I honestly would have expected this to be a completely theoretical problem, though clearly, you weren't aware that identifiers that start with two underscores are typically reserved by the language and used them anyway and ran into problems as a result, so apparently, the problem isn't completely theoretical. I do expect it to be pretty uncommon though.

Regardless, unless there's a reasonable way to have the compiler detect when it's druntime that's declaring the variable that starts with two underscores, it's going to be difficult to make it an error for anyone doing it outside of druntime, and I have no clue how easy that is to do. It could be that there are technical hurdles that prevent it, and it could be simply that it was going to be enough of a pain that no one bothered on the principle that it was highly unlikely to come up in practice. I don't know. Feel free to create a bug report for it, and maybe the compiler can be improved to actually treat it as illegal in the cases where it should be illegal while letting druntime do its thing.

Now, whether __traits should be ignoring identifiers that start with two underscores is another matter entirely, and I have no idea why that's happening, but no user code should be being written that has identifiers that start with two underscores (and that's far from unique to D).

- Jonathan M Davis

On Sunday, 17 June 2018 at 02:25:59 UTC, Adam D. Ruppe wrote:
> On Sunday, 17 June 2018 at 01:02:17 UTC, DigitalDesigns wrote:
>> If this was a sane language constraint then any identifiers starting with __ that were not reserved would at least give a warning but particularly give an error! Not fail silently and break code in ways that cannot be determined otherwise.
>
> It is undefined behavior to use ANY identifier with __ leading. C works exactly the same way.

I'm sure there are naive C programmers who use "__" prefix for their own purposes.

Maybe it would be cleaner to create more special syntax for "internal" functions/methods, e.g require some kind of UDA in addition to the "__" prefix:

@Dinternal
void __someMethod() ...

Then disallow "__" prefix altogether (with the exception above).

Now, if someone still uses both the UDA and the prefix, he really cannot say he didn't know.

On 6/15/2018 11:08 PM, DigitalDesigns wrote:
> When an identifier starts with __, allMembers does not return it. This is very bad behavior! It can silently create huge problems if the programmer is not aware of the bug.


It's not a bug, it's quite deliberate:

  https://github.com/dlang/dmd/blob/master/src/dmd/traits.d#L1385


Identifiers starting with __ are reserved for the implementation:

  https://dlang.org/spec/lex.html#identifiers

They have implementation-defined behavior. Do not use them in user code.