Thread overview
"Comment processing conceptually happens before tokenization."
Oct 17, 2006
Stewart Gordon
Oct 18, 2006
Walter Bright
Oct 18, 2006
Bruno Medeiros
Oct 18, 2006
Walter Bright
Oct 22, 2006
Bruno Medeiros
October 17, 2006
http://www.digitalmars.com/d/lex.html

"Comment processing conceptually happens before tokenization. This means that embedded strings and comments do not prevent recognition of comment openings and closings:"

But the first sentence, the way it's written, means that comments that begin within strings will be treated as comments, which is contrary to my experience.  That is,

    a = "Here is a string with a /+ comment delimiter"; // +/ in it";

would be treated as

    a = "Here is a string with a  in it";

when, in fact, the opening "/+", being within a string, is treated as part of the string; therefore a will contain

    "Here is a string with a /+ comment delimiter"

The second sentence of the quotation is also open to interpretation. Claiming that they "do not prevent recognition of comment openings" tempts one to think that arbitrary comment openings within a string or comment will be processed.

A more accurate way to put it (assuming that what the compiler does is what was intended) is:

The contents of strings and comments are not tokenized.  Consequently, comment openings occurring within a string do not begin a comment, and string delimiters within a comment do not affect the recognition of comment closings and nested "/+" comment openings.  With the exception of "/+" occurring within a "/+" comment, comment openings within a comment are ignored.

Stewart.

-- 
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/M d- s:-@ C++@ a->--- UB@ P+ L E@ W++@ N+++ o K-@ w++@ O? M V? PS- PE- Y? PGP- t- 5? X? R b DI? D G e++++ h-- r-- !y
------END GEEK CODE BLOCK------

My e-mail is valid but not my primary mailbox.  Please keep replies on the 'group where everyone may benefit.

October 18, 2006
Yours is better. I'll make the change. Thanks.
October 18, 2006
Walter Bright wrote:
> Yours is better. I'll make the change. Thanks.

What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.


-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
October 18, 2006
Bruno Medeiros wrote:
> Walter Bright wrote:
>> Yours is better. I'll make the change. Thanks.
> 
> What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.

I think the examples in www.digitalmars.com/d/lex.html should make it clear.
October 22, 2006
Walter Bright wrote:
> Bruno Medeiros wrote:
>> Walter Bright wrote:
>>> Yours is better. I'll make the change. Thanks.
>>
>> What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.
> 
> I think the examples in www.digitalmars.com/d/lex.html should make it clear.

Huh? They make clear what the behavior is, but that I already know. Instead, I wanted to know the reasoning behind that.

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
October 23, 2006
What if you want a string literal which contains the following characters?

/* comment */

For whatever reason?  Do you propose that you should have to do the following?

char[] comment_example = "/" "* comment *" "/";

Or something?  I think it makes sense to not parse comments inside of strings, which is what every other programming language I've used does.

-[Unknown]


> Walter Bright wrote:
>> Bruno Medeiros wrote:
>>> Walter Bright wrote:
>>>> Yours is better. I'll make the change. Thanks.
>>>
>>> What's the reason for this behavior BTW? Does it make parsing/tokenizing more easy? Because it seems to me the ideal behaviour is otherwise.
>>
>> I think the examples in www.digitalmars.com/d/lex.html should make it clear.
> 
> Huh? They make clear what the behavior is, but that I already know. Instead, I wanted to know the reasoning behind that.
>