Thread overview | |||||||
---|---|---|---|---|---|---|---|
|
September 04, 2005 A Simple D Lexer (written in D) | ||||
---|---|---|---|---|
| ||||
http://aljudy.org/dstuff/simple.d.lexer.zip This is something I'm working on -lazily-, as a part of a bigger project. It's not heavliy tested or anything .. read the doc.txt that comes with it .. The code uses several Objects, but it's not entirely OO (unless you think of modules as singleton classes!) I tried to make the code modular and cohesive. Hopefully, most bugs would be easily fixable. I'll quote myself here (from doc.txt) to give an example of what this lexer can do: The following code segment <code> void main() { //comment /+ /* */ /+ +/ +/ printf("hi"); } </code> produces the folloing token list: type text ---------------------------------------- TOK.TOKvoid void TOK.TOKidentifier main TOK.TOKopenparn ( TOK.TOKcloseparen ) TOK.TOKopencurly { TOK.TOKidentifier printf TOK.TOKopenparen ( TOK.TOKstring "hi" TOK.TOKcloseparen ) TOK.TOKsemicolon ; TOK.TOKclosecurly } The code ships with this example, to compile it, cd to the directory where you unzipped it, and invoke build against it: >build main.d -clean and run it >main For details, see doc.txt in the zip file. As you can see, the lexer handles nesting comments pretty well. I am also working on a parser, but do not expect anything. I'm a kind of guy who works on alot of projects but doesn't finish most of them, which is why I'm releasing this lexer now. Basically I don't know whether or not I will continue working on this baby, and I don't want it to be wasted for nothing. I hope people will find it useful. Enjoy. |
September 04, 2005 Re: A Simple D Lexer (written in D) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Hasan Aljudy | On Sun, 04 Sep 2005 00:36:52 -0600, Hasan Aljudy wrote: > http://aljudy.org/dstuff/simple.d.lexer.zip > [snip] > Basically I don't know whether or not I will continue working on this baby, and I don't want it to be wasted for nothing. It compiled and worked well. > I hope people will find it useful. I have a few uses for such a beastie. Thanks. -- Derek Parnell Melbourne, Australia 4/09/2005 6:43:44 PM |
September 04, 2005 Re: A Simple D Lexer (written in D) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Hasan Aljudy | I just noticed a slight problem in lex/string_scanner.d, so I fixed it and uploaded a new `version`.
also, the
"string"\escape"string2"
issue is fixed too.
Hasan Aljudy wrote:
> http://aljudy.org/dstuff/simple.d.lexer.zip
>
> This is something I'm working on -lazily-, as a part of a bigger project.
> It's not heavliy tested or anything .. read the doc.txt that comes with it ..
> The code uses several Objects, but it's not entirely OO (unless you think of modules as singleton classes!)
>
> I tried to make the code modular and cohesive. Hopefully, most bugs would be easily fixable.
>
> I'll quote myself here (from doc.txt) to give an example of what this lexer can do:
> The following code segment
> <code> void main()
> {
> //comment
> /+
> /* */
> /+
> +/
> +/
> printf("hi");
> }
> </code>
> produces the folloing token list:
> type text
> ----------------------------------------
> TOK.TOKvoid void
> TOK.TOKidentifier main
> TOK.TOKopenparn (
> TOK.TOKcloseparen )
> TOK.TOKopencurly {
> TOK.TOKidentifier printf
> TOK.TOKopenparen (
> TOK.TOKstring "hi"
> TOK.TOKcloseparen )
> TOK.TOKsemicolon ;
> TOK.TOKclosecurly }
>
>
> The code ships with this example, to compile it, cd to the directory where you unzipped it, and invoke build against it:
> >build main.d -clean
>
> and run it
> >main
>
>
> For details, see doc.txt in the zip file. As you can see, the lexer handles nesting comments pretty well.
>
> I am also working on a parser, but do not expect anything.
> I'm a kind of guy who works on alot of projects but doesn't finish most of them, which is why I'm releasing this lexer now. Basically I don't know whether or not I will continue working on this baby, and I don't want it to be wasted for nothing.
>
> I hope people will find it useful.
> Enjoy.
|
September 04, 2005 Re: A Simple D Lexer (written in D) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Derek Parnell | Derek Parnell wrote:
> On Sun, 04 Sep 2005 00:36:52 -0600, Hasan Aljudy wrote:
>
>
>>http://aljudy.org/dstuff/simple.d.lexer.zip
>>
>
> [snip]
>
>>Basically I don't know whether or not I will continue working on this baby, and I don't want it to be wasted for nothing.
>
>
> It compiled and worked well.
>
>
>>I hope people will find it useful.
>
>
> I have a few uses for such a beastie. Thanks.
>
Thank you. It's good to hear that.
|
January 20, 2006 Re: A Simple D Lexer (written in D) | ||||
---|---|---|---|---|
| ||||
Posted in reply to Hasan Aljudy | Hasan Aljudy schrieb:
> http://aljudy.org/dstuff/simple.d.lexer.zip
Link doesn't work.
(
Not Found
The requested URL http:// was not found on this server.
)
|
Copyright © 1999-2021 by the D Language Foundation