Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex, and I'm very tempted to just use std.datetime to produce the values for __DATE__, __TIME__, __TIMESTAMP__. The resulting values would look cleaner IMHO, and the code would be simpler. However, that _would_ result in a slightly different format for __DATE__ and __TIMESTAMP__.

Honestly, I didn't even know that these "macros" existed before I ran into them in the lexer - there's no mention of them on the site as far as I can determine. So, I have no idea whether they're exact format is considered important and part of the spec or not. If the format _is_ important, then I might as well leave it and have the D code call time and ctime as it does in the C++ code, but I'd just as soon use std.datetime if the format doesn't have to be absolutely exact.

So, are the formats of __DATE__, __TIME__, and __TIMESTAMP__ exact such that they should _always_ be as they currently are in the C++ lexer, or are they flexible enough that I can just simplify the code in the D lexer with std.datetime?

- Jonathan M Davis

On 3/8/11 7:36 PM, Jonathan M Davis wrote:
> Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex

What is std.lang.d.lex?

Andrei

On 03/08/2011 07:36 PM, Jonathan M Davis wrote:
> Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex, and I'm very tempted to just use std.datetime to produce the values for __DATE__, __TIME__, __TIMESTAMP__. The resulting values would look cleaner IMHO, and the code would be simpler. However, that _would_ result in a slightly different format for __DATE__ and __TIMESTAMP__.
>
> Honestly, I didn't even know that these "macros" existed before I ran into them in the lexer - there's no mention of them on the site as far as I can determine.

http://www.digitalmars.com/d/2.0/lex.html#specialtokens

That also sets the format.

> So, I have no idea whether they're exact format is considered important and part of the spec or not. If the format _is_ important, then I might as well leave it and have the D code call time and ctime as it does in the C++ code, but I'd just as soon use std.datetime if the format doesn't have to be absolutely exact.
>
> So, are the formats of __DATE__, __TIME__, and __TIMESTAMP__ exact such that they should _always_ be as they currently are in the C++ lexer, or are they flexible enough that I can just simplify the code in the D lexer with std.datetime?
>
> - Jonathan M Davis
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Tuesday 08 March 2011 19:55:28 Benjamin Shropshire wrote:
> On 03/08/2011 07:36 PM, Jonathan M Davis wrote:
> > Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex, and I'm very tempted to just use std.datetime to produce the values for __DATE__, __TIME__, __TIMESTAMP__. The resulting values would look cleaner IMHO, and the code would be simpler. However, that _would_ result in a slightly different format for __DATE__ and __TIMESTAMP__.
> > 
> > Honestly, I didn't even know that these "macros" existed before I ran into them in the lexer - there's no mention of them on the site as far as I can determine.
> 
> http://www.digitalmars.com/d/2.0/lex.html#specialtokens
> 
> That also sets the format.

Bleh. Okay. I'd have preferred to just use std.datetime's format, but such is life. And obviously the search function on the site doesn't search the language documentation, since I specifically searched for all of those special tokens, and the search didn't find any of them.

In any case, thanks for pointing that out to me.

- Jonathan M Davis

On Tuesday 08 March 2011 19:53:46 Andrei Alexandrescu wrote:
> On 3/8/11 7:36 PM, Jonathan M Davis wrote:
> > Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex
> 
> What is std.lang.d.lex?

It's the D lexer for Phobos that Walter wanted written. He wanted someone to port the compiler's C++ frontend's lexer to D. I volunteered to do it.

http://lists.puremagic.com/pipermail/digitalmars-d/2010-October/085981.html

I've slowly been working on it, but I'm a ways from being done.

- Jonathan M Davis

I understand. If I could, I'd kindly discourage you from working on it. Its existence would stifle the impetus to work on general lexing and parsing frameworks (a la highly integrated lex/yacc using D's generational capabilities). Those in turn would push the compiler forward and would help fixing bugs, and would offer the proverbial fishing rod (tools for parsing any language) instead of fish (a lexer and a parser for D itself).

Andrei

On 3/8/11 8:41 PM, Jonathan M Davis wrote:
> On Tuesday 08 March 2011 19:53:46 Andrei Alexandrescu wrote:
>> On 3/8/11 7:36 PM, Jonathan M Davis wrote:
>>> Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex
>>
>> What is std.lang.d.lex?
>
> It's the D lexer for Phobos that Walter wanted written. He wanted someone to port the compiler's C++ frontend's lexer to D. I volunteered to do it.
>
> http://lists.puremagic.com/pipermail/digitalmars-d/2010-October/085981.html
>
> I've slowly been working on it, but I'm a ways from being done.
>
> - Jonathan M Davis
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Tuesday 08 March 2011 20:46:49 Andrei Alexandrescu wrote:
> I understand. If I could, I'd kindly discourage you from working on it. Its existence would stifle the impetus to work on general lexing and parsing frameworks (a la highly integrated lex/yacc using D's generational capabilities). Those in turn would push the compiler forward and would help fixing bugs, and would offer the proverbial fishing rod (tools for parsing any language) instead of fish (a lexer and a parser for D itself).

Well, I fully support a generic lexer, but I also see value in a lexer which closely follows what the front end is doing for D. But it's not like I've been making huge progress on it and am about to submit it for review. I have too much other stuff on my plate to blow through it like that.

Though honestly, if I were to be writing a lexer and/or parser for a compiler of any kind, I'd just do it by hand. It's a lot more flexible that way. So, even if a solid, generic D lexing tool existed, I'm not sure that I'd use it for anything.

In any case, as I understood it, the result of that original discussion was that we wanted to have a generic lexing solution (and presumably a corresponding parsing solution following that) but that we still want a lexer and parser for D which are close to what the C++ front end is doing. So, as requested, I've slowly been working on that.

- Jonathan M Davis

On 9 mar 2011, at 05:52, Jonathan M Davis wrote:

> On Tuesday 08 March 2011 20:46:49 Andrei Alexandrescu wrote:
>> I understand. If I could, I'd kindly discourage you from working on it. Its existence would stifle the impetus to work on general lexing and parsing frameworks (a la highly integrated lex/yacc using D's generational capabilities). Those in turn would push the compiler forward and would help fixing bugs, and would offer the proverbial fishing rod (tools for parsing any language) instead of fish (a lexer and a parser for D itself).
> 
> Well, I fully support a generic lexer, but I also see value in a lexer which closely follows what the front end is doing for D. But it's not like I've been making huge progress on it and am about to submit it for review. I have too much other stuff on my plate to blow through it like that.
> 
> Though honestly, if I were to be writing a lexer and/or parser for a compiler of any kind, I'd just do it by hand. It's a lot more flexible that way. So, even if a solid, generic D lexing tool existed, I'm not sure that I'd use it for anything.
> 
> In any case, as I understood it, the result of that original discussion was that we wanted to have a generic lexing solution (and presumably a corresponding parsing solution following that) but that we still want a lexer and parser for D which are close to what the C++ front end is doing. So, as requested, I've slowly been working on that.
> 
> - Jonathan M Davis
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals


I completely agree with this, that we need a D lexer (preferably a whole frontend) written in D, included in the standard library. I even hope that one day DMD will use that lexer.

BTW, are basing your work on DDMD?

-- 
/Jacob Carlborg

Doesn't std.datetime have function for easily formatting dates and times?

-- 
/Jacob Carlborg

On 9 mar 2011, at 04:36, Jonathan M Davis wrote:

> Do they _have_ to be in a particular format? I ask because I've slowly been going through the lexer code, translating it to D for std.lang.d.lex, and I'm very tempted to just use std.datetime to produce the values for __DATE__, __TIME__, __TIMESTAMP__. The resulting values would look cleaner IMHO, and the code would be simpler. However, that _would_ result in a slightly different format for __DATE__ and __TIMESTAMP__.
> 
> Honestly, I didn't even know that these "macros" existed before I ran into them in the lexer - there's no mention of them on the site as far as I can determine. So, I have no idea whether they're exact format is considered important and part of the spec or not. If the format _is_ important, then I might as well leave it and have the D code call time and ctime as it does in the C++ code, but I'd just as soon use std.datetime if the format doesn't have to be absolutely exact.
> 
> So, are the formats of __DATE__, __TIME__, and __TIMESTAMP__ exact such that they should _always_ be as they currently are in the C++ lexer, or are they flexible enough that I can just simplify the code in the D lexer with std.datetime?
> 
> - Jonathan M Davis
> _______________________________________________
> dmd-internals mailing list
> dmd-internals at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-internals

On Wednesday, March 09, 2011 09:28:05 Jacob Carlborg wrote:
> On 9 mar 2011, at 05:52, Jonathan M Davis wrote:
> > On Tuesday 08 March 2011 20:46:49 Andrei Alexandrescu wrote:
> >> I understand. If I could, I'd kindly discourage you from working on it. Its existence would stifle the impetus to work on general lexing and parsing frameworks (a la highly integrated lex/yacc using D's generational capabilities). Those in turn would push the compiler forward and would help fixing bugs, and would offer the proverbial fishing rod (tools for parsing any language) instead of fish (a lexer and a parser for D itself).
> > 
> > Well, I fully support a generic lexer, but I also see value in a lexer which closely follows what the front end is doing for D. But it's not like I've been making huge progress on it and am about to submit it for review. I have too much other stuff on my plate to blow through it like that.
> > 
> > Though honestly, if I were to be writing a lexer and/or parser for a compiler of any kind, I'd just do it by hand. It's a lot more flexible that way. So, even if a solid, generic D lexing tool existed, I'm not sure that I'd use it for anything.
> > 
> > In any case, as I understood it, the result of that original discussion was that we wanted to have a generic lexing solution (and presumably a corresponding parsing solution following that) but that we still want a lexer and parser for D which are close to what the C++ front end is doing. So, as requested, I've slowly been working on that.
> > 
> > - Jonathan M Davis
> > _______________________________________________
> > dmd-internals mailing list
> > dmd-internals at puremagic.com
> > http://lists.puremagic.com/mailman/listinfo/dmd-internals
> 
> I completely agree with this, that we need a D lexer (preferably a whole frontend) written in D, included in the standard library. I even hope that one day DMD will use that lexer.
> 
> BTW, are basing your work on DDMD?

No. I haven't looked at DDMD. Walter wanted a fairly direct port of the C++ frontend (with appropriate tweaks for D, but he wants to make sure that it's straightforward to port fixes from one to the other). So, I've just been directly, porting the C++ frontend, adjusting it to be somewhat more idiomatic D as appropriate but generally keeping it fairly close to the C++ implementation.

- Jonathan M davis