On Monday, 26 January 2015 at 19:44:18 UTC, Walter Bright wrote:
> On 1/26/2015 8:13 AM, Foo wrote:
>> You could do the same as C++ with override and final: they are only valid
>> attributes if they appear _after_ the function/method. Elsewhere they are still
>> valid as identifiers for e.g. variables.
>
> Just 'no' on context-sensitive tokens.
>
> For context,
>
>   TV vocabulary - 2,000 words
>   High school vocabulary - 10,000 words
>   College vocabulary - 30,000 words
>   English - 1,000,000 words
>
> i.e. words are NOT a scarce resource (quite unlike punctuation).

I'm not sure what you mean by "context-sensitive tokens".  It sounds like you might be saying that this requires the tokenizer to be context sensitive, but this isn't the case.  Instead, allowing a function to be decorated with any number of "id"  or "keyword" tokens would allow the language to add new attributes (without a '@' character) by restricting the usage of attributes in the semantics instead of the syntax.

void myfunc() foo bar safe nogc nothrow {
}

'foo' and 'bar' don't break the syntax but are caught by the semantic analysis.

Furthermore, I'm sure this type of thing is already done all over the place in the grammar.  The following example comes to mind:

struct St { }
void main()
{
  St St;
}

The "St" token is a decl type the first time and a variable name the second.  I'm sure they are implemented as the same token but the grammar results in a different semantic meaning for both tokens.

On 1/26/2015 12:34 PM, Jonathan Marler wrote:
> I'm not sure what you mean by "context-sensitive tokens".

A keyword in one context and an identifier in another.

On Monday, 26 January 2015 at 20:38:43 UTC, Walter Bright wrote:
> On 1/26/2015 12:34 PM, Jonathan Marler wrote:
>> I'm not sure what you mean by "context-sensitive tokens".
>
> A keyword in one context and an identifier in another.

Then I'm a little confused.  What does this have to do with adding new attributes to a function?  Just because a word is used as a function attribute doesn't mean it has to be a keyword.  If the grammar supported decorating a function with a list of id tokens (not just keywords), then you could implement a variation on the c++ solution of allowing "override" and "final" after a function signature.  Just because they are function attributes does not mean they were tokenized as "keywords".

On 1/26/2015 12:26 PM, Dicebot wrote:
> It was sad that calls for more breakage were mostly ignored. But there is one
> thing now that is even worse - referring to #pleasebreakmycode as an excuse to
> introduce random changes based on random reddit comment - and completely
> dismissing everything that was said on topic initially. Resulting in exactly the
> opposite action that was asked to.


I can't find Don's original post on the topic.

On Monday, 26 January 2015 at 19:51:08 UTC, Walter Bright wrote:
> Frankly, I think that is a great bikeshedding non-issue that distracts us from what is important. I hope that by doing this PR, we can actually decide that it isn't worth it, i.e. I'd be happy to get consensus and revert it.

For my part, a 'dfix' solution is worth it. Anything else, not.

On Monday, 26 January 2015 at 20:56:36 UTC, Walter Bright wrote:
> On 1/26/2015 12:26 PM, Dicebot wrote:
>> It was sad that calls for more breakage were mostly ignored. But there is one
>> thing now that is even worse - referring to #pleasebreakmycode as an excuse to
>> introduce random changes based on random reddit comment - and completely
>> dismissing everything that was said on topic initially. Resulting in exactly the
>> opposite action that was asked to.
>
>
> I can't find Don's original post on the topic.

Donno If you are referring to this [1], but at least it's about this pull...

---
Paolo

[1] http://forum.dlang.org/thread/bug-13388-3@https.issues.dlang.org%2F?page=3#post-mailman.312.1409643575.5783.digitalmars-d-bugs:40puremagic.com

On 1/26/2015 12:45 PM, Jonathan Marler wrote:
> Just because they are function attributes does not mean
> they were tokenized as "keywords".

The lexer has no idea what a function attribute is or that now it should be looking for attributes and then it should not be.

On 1/26/2015 1:07 PM, Paolo Invernizzi wrote:
> On Monday, 26 January 2015 at 20:56:36 UTC, Walter Bright wrote:
>> On 1/26/2015 12:26 PM, Dicebot wrote:
>>> It was sad that calls for more breakage were mostly ignored. But there is one
>>> thing now that is even worse - referring to #pleasebreakmycode as an excuse to
>>> introduce random changes based on random reddit comment - and completely
>>> dismissing everything that was said on topic initially. Resulting in exactly the
>>> opposite action that was asked to.
>>
>>
>> I can't find Don's original post on the topic.
>
> Donno If you are referring to this [1], but at least it's about this pull...
> [1]
> http://forum.dlang.org/thread/bug-13388-3@https.issues.dlang.org%2F?page=3#post-mailman.312.1409643575.5783.digitalmars-d-bugs:40puremagic.com

Yup, that's it. Your search-foo is better than mine!

A more direct link:

https://issues.dlang.org/show_bug.cgi?id=13388#c27

On Monday, 26 January 2015 at 21:12:50 UTC, Walter Bright wrote:
> On 1/26/2015 12:45 PM, Jonathan Marler wrote:
>> Just because they are function attributes does not mean
>> they were tokenized as "keywords".
>
> The lexer has no idea what a function attribute is or that now it should be looking for attributes and then it should not be.

Not a problem, the parser can turn the token it back into a string. All you need is a LUT.

( Besides, parser.c is screaming for a rewrite anyway... )

Walter Bright:

> Previously, it's all been advocacy and "break my code"

Breaking the code should be justified by a worth gain. This patch is of negative value.

Bye,
bearophile