This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":

https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d1/DIPs/DIP1026.md

All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on December 17, or when I make a post declaring it complete.

At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round of Community Review. Otherwise, it will be queued for the Final Review and Formal Assessment.

Anyone intending to post feedback in this thread is expected to be familiar with the reviewer guidelines:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

*Please stay on topic!*

Thanks in advance to all who participate.

On Tuesday, 3 December 2019 at 09:03:44 UTC, Mike Parker wrote:
> This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":
>
> https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d1/DIPs/DIP1026.md
>
> All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on December 17, or when I make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round of Community Review. Otherwise, it will be queued for the Final Review and Formal Assessment.
>
> Anyone intending to post feedback in this thread is expected to be familiar with the reviewer guidelines:
>
> https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md
>
> *Please stay on topic!*
>
> Thanks in advance to all who participate.

I think there's a problem with this analysis.

The package registry contains a lot of libraries and just a few other projects.
I wonder if libraries represent the real usage by "final" users.

Maybe those stats should be run over github D projects, at least.

Andrea

On Tuesday, 3 December 2019 at 09:03:44 UTC, Mike Parker wrote:
> This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":
>
> https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d1/DIPs/DIP1026.md
>
> All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on December 17, or when I make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round of Community Review. Otherwise, it will be queued for the Final Review and Formal Assessment.
>
> Anyone intending to post feedback in this thread is expected to be familiar with the reviewer guidelines:
>
> https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md
>
> *Please stay on topic!*
>
> Thanks in advance to all who participate.

Is there much point being almost context-free? Seems like we should know for sure whether there are any other parts of the grammar that are context dependent before we use it as motivation. The DIP is somewhat vague about whether this has been properly established

On 12/3/19 4:03 AM, Mike Parker wrote:
> This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":
> 
> https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d1/DIPs/DIP1026.md 

This DIP is a non-starter. Here documents are easily and effectively handled during lexing and have no impact on the language grammar.

Waste of labor is sadly a common theme in our community. We should have a mechanism to direct such investment of work toward productive outcome.

On Tuesday, 3 December 2019 at 09:03:44 UTC, Mike Parker wrote:
> This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":
>
> https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d1/DIPs/DIP1026.md
>


YES

I'm generally in favor of removing things. I think I've used token strings at times, but not the other variety discussed in the DIP and that I didn't know of.

It will break some DUB package, and that's OK since we have SemVer.

On Tuesday, 3 December 2019 at 12:38:29 UTC, Andrei Alexandrescu wrote:
> Waste of labor is sadly a common theme in our community.

I consider this low-hanging fruit: just deprecating a token takes little implementation effort, and reduction in language complexity is (as far as I know) always welcome for the usual reasons:
- less code in dmd
- less specification text
- less didactic material / stuff to learn for new D programmers
- less bug/enhancement reports
- any tool that re-implements some part of the compiler is easier to make

In this case, such tools would be syntax highlighters. There are lots of syntax highlighting implementations for D, just a few off the top off my head:
- GitHub
- Code-d
- Kate
- Atom
- Sublime
- Chroma
- Vim
- Emacs
- Notepad++
- ...

They all tend to use their own domain specific language, and I'm pretty sure most of them are not powerful enough to express identifier-delimited strings. Here's an example of one if you're curious what they look like:

https://github.com/alecthomas/chroma/blob/master/lexers/d/d.go

Notice the:

> // TODO support delimited strings

If we don't want D support in syntax highlighters to be half-baked everywhere, keeping the lexical grammar simple is a good cause.

I can improve the rationale for this DIP with examples like in this post, though if you're absolutely adamant that this is a waste of effort then that won't help obviously.

Maybe you don't care about syntax highlighting, but please judge this DIP by its own merits and not compared to potential other DIPs that you care more about.

On Tuesday, 3 December 2019 at 12:24:09 UTC, John Colvin wrote:
> Is there much point being almost context-free? Seems like we should know for sure whether there are any other parts of the grammar that are context dependent before we use it as motivation. The DIP is somewhat vague about whether this has been properly established

I _think_ this is the only thing in the lexical grammar that is not context-free but I haven't verified this so I am intentionally a bit vague about that.
This DIP is obviously necessary for being context-free, but not sufficient per se.

I can spend some time on this if it helps, but didn't want to put too much time into this before the first review in case it got shut down immediately. (And considering Andrei's post, it is at risk of that.)

On Tuesday, 3 December 2019 at 09:27:37 UTC, Andrea Fontana wrote:
> I think there's a problem with this analysis.
>
> The package registry contains a lot of libraries and just a few other projects.
> I wonder if libraries represent the real usage by "final" users.
>
> Maybe those stats should be run over github D projects, at least.

I agree, I definitely want to expand my collection of open source D code to be more representative. If I have time I may do this before the next review round.

On Tuesday, December 3, 2019 2:03:44 AM MST Mike Parker via Digitalmars-d wrote:
> This is the feedback thread for the first round of Community Review for DIP 1026, "Deprecate Context-Sensitive String Literals":
>
> https://github.com/dlang/DIPs/blob/a7199bcec2ca39b74739b165fc7b97afff9e29d 1/DIPs/DIP1026.md
>
> All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on December 17, or when I make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round of Community Review. Otherwise, it will be queued for the Final Review and Formal Assessment.
>
> Anyone intending to post feedback in this thread is expected to be familiar with the reviewer guidelines:
>
> https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md
>
> *Please stay on topic!*
>
> Thanks in advance to all who participate.

There are definitely people who use token strings in their code when writing string mixins, because it makes it so that the code in the strings actually gets syntax highlighting like normal code does instead of being displayed as a string. I expect that a number of people would be quite unhappy to not be able to do that anymore.

Personally, I never use token strings, and I'm not sure that I'd even know about them if I hadn't worked on a D lexer several years ago. I also prefer that strings look like strings even if they contain code, but I don't care enough about that to try to get the feature removed, and I'm not sure that I care much whether the DIP is accepted or not. However, there's no question that some people think that they're very valuable when writing string mixins.

- Jonathan M Davis

On Tuesday, 3 December 2019 at 14:45:31 UTC, Dennis wrote:
> I'm pretty sure most of them are not powerful enough to express identifier-delimited strings.

The identifier ones are trivial, they are a simple regex. Heck, my vim syntax highlight file not only supports them, but uses the opening as a hint as to what language is embedded:

q"html
   <!-- highlights this as html! -->
";


that said though, I don't love them because they must end on a new line, without indentation. But still, it was easy to implement.

syn region dHTML keepend matchgroup=string start="q\"html$" end="^html\"" contains=@html


And the generic fallback for other identifiers of course is just

syn region dDelimString start=+q"\z(.\)+ end=+\z1"+ contains=@Spell
syn region dHereString  start=+q"\z(\I\i*\)\n+ end=+^\z1"+ contains=@Spell

vim manages to do it all pretty well....