View mode: basic / threaded / horizontal-split · Log in · Help
May 22, 2012
Re: DCT use cases - draft
On 22.05.2012 19:36, Roman D. Boiko wrote:
> On Tuesday, 22 May 2012 at 15:15:49 UTC, Roman D. Boiko wrote:
>> On Tuesday, 22 May 2012 at 14:56:42 UTC, Dmitry Olshansky wrote:
>>> I suggest to go with such kind of improvised notation with more
>>> examples until you fell that semantics are crystal clear.
>>
>> This is a possible, although not the only, option. Anyway I'm not
>> ready for designing the details yet. There is a lot to do before
>> that.
>
> After thinking a bit more, I decided to investigate the topic
> early, although not immediately :) It should help deciding which
> indices are needed, where are rough edges of API, etc.

Yeah I still think you tried to dismiss it too early.

In my mind it's encountered rather soon:

lex/scan -> postprocess (may be combined with lex) -> populate symbol 
tables (ditto  - with next step/previous step) -> parse to AST -> ...

That symbol table should contain all of the rich hierarchy of modules. 
That is the actual compiler is able to have a single stack of scopes 
that it pushes/pops as it processes code. Your DCT on the other hand 
should have all of local scopes (of every entity) at once.

It may be possible to simulate it with some deal of laziness, but I 
guess keeping the whole symbol table is the easiest and the fastest way 
still. Watch out for some mind-bending data structures :)

-- 
Dmitry Olshansky
May 22, 2012
Re: DCT use cases - draft
On Tuesday, 22 May 2012 at 16:03:49 UTC, Dmitry Olshansky wrote:
> In my mind it's encountered rather soon:
>
> lex/scan -> postprocess (may be combined with lex) -> populate 
> symbol tables (ditto  - with next step/previous step) -> parse 
> to AST -> ...
> That symbol table should contain all of the rich hierarchy of 
> modules. That is the actual compiler is able to have a single 
> stack of scopes that it pushes/pops as it processes code. Your 
> DCT on the other hand should have all of local scopes (of every 
> entity) at once.
>
> It may be possible to simulate it with some deal of laziness, 
> but I guess keeping the whole symbol table is the easiest and 
> the fastest way still.
And likely the only feasible in D. It doesn't support lazy 
evaluation for immutable data structures, and immutability is 
necessary for most use cases.

> Watch out for some mind-bending data structures :)
Do you mean not to overcomplicate? Or use classic data 
structures? Or something else?

So far I think immutable red-black trees will be central in DCT 
architecture, as well as some others.
May 22, 2012
Re: DCT use cases - draft
Le 22/05/2012 16:04, Roman D. Boiko a écrit :
> http://d-coding.com/2012/05/22/dct-use-cases.html

Nice idea but I'll not be nice.

What is needed here is design. The needs are not really news nor 
specific to D.

So far, you have ignored all existing project on the subject, and shown 
that you tend to overcomplicate design when it isn't required. I don't 
see here the right attitude to come up with a tool that will :
1/ Not repeat error previously done.
2/ Get resused.
May 22, 2012
Re: DCT use cases - draft
On 22.05.2012 20:47, Roman D. Boiko wrote:
> On Tuesday, 22 May 2012 at 16:03:49 UTC, Dmitry Olshansky wrote:
>> In my mind it's encountered rather soon:
>>
>> lex/scan -> postprocess (may be combined with lex) -> populate symbol
>> tables (ditto - with next step/previous step) -> parse to AST -> ...
>> That symbol table should contain all of the rich hierarchy of modules.
>> That is the actual compiler is able to have a single stack of scopes
>> that it pushes/pops as it processes code. Your DCT on the other hand
>> should have all of local scopes (of every entity) at once.
>>
>> It may be possible to simulate it with some deal of laziness, but I
>> guess keeping the whole symbol table is the easiest and the fastest
>> way still.
> And likely the only feasible in D. It doesn't support lazy evaluation
> for immutable data structures, and immutability is necessary for most
> use cases.
>
>> Watch out for some mind-bending data structures :)
> Do you mean not to overcomplicate? Or use classic data structures? Or
> something else?

Prefer simple for prototype. Good old nested hash table is fine for 
starters.

>
> So far I think immutable red-black trees will be central in DCT
> architecture, as well as some others.

Cool ;) Though why not Tries then if the data is immutable?

-- 
Dmitry Olshansky
May 22, 2012
Re: DCT use cases - draft
On Tuesday, 22 May 2012 at 14:04:18 UTC, Roman D. Boiko wrote:
> http://d-coding.com/2012/05/22/dct-use-cases.html

I'm really not sure I understand what this is. Is this use cases 
for a lexer or some other more high level too?. Because I don't 
see what projects and workspaces have to do with a lexer.
May 22, 2012
Re: DCT use cases - draft
On Tuesday, 22 May 2012 at 16:55:46 UTC, Dmitry Olshansky wrote:
> On 22.05.2012 20:47, Roman D. Boiko wrote:
>> On Tuesday, 22 May 2012 at 16:03:49 UTC, Dmitry Olshansky 
>> wrote:
>>> In my mind it's encountered rather soon:
>>>
>>> lex/scan -> postprocess (may be combined with lex) -> 
>>> populate symbol
>>> tables (ditto - with next step/previous step) -> parse to AST 
>>> -> ...
>>> That symbol table should contain all of the rich hierarchy of 
>>> modules.
>>> That is the actual compiler is able to have a single stack of 
>>> scopes
>>> that it pushes/pops as it processes code. Your DCT on the 
>>> other hand
>>> should have all of local scopes (of every entity) at once.
>>>
>>> It may be possible to simulate it with some deal of laziness, 
>>> but I
>>> guess keeping the whole symbol table is the easiest and the 
>>> fastest
>>> way still.
>> And likely the only feasible in D. It doesn't support lazy 
>> evaluation
>> for immutable data structures, and immutability is necessary 
>> for most
>> use cases.
>>
>>> Watch out for some mind-bending data structures :)
>> Do you mean not to overcomplicate? Or use classic data 
>> structures? Or
>> something else?
>
> Prefer simple for prototype. Good old nested hash table is fine 
> for starters.
Unfortunately, hash tables don't play nicely with immutability of 
data - although interface could be restricted to prevent 
mutation, they would not be able to reuse memory space, and thus 
would require copying.

>>
>> So far I think immutable red-black trees will be central in DCT
>> architecture, as well as some others.
>
> Cool ;) Though why not Tries then if the data is immutable?

Those too :) For different purposes than the former.
May 22, 2012
Re: DCT use cases - draft
On Tuesday, 22 May 2012 at 17:06:43 UTC, Jacob Carlborg wrote:
> On Tuesday, 22 May 2012 at 14:04:18 UTC, Roman D. Boiko wrote:
>> http://d-coding.com/2012/05/22/dct-use-cases.html
>
> I'm really not sure I understand what this is. Is this use 
> cases for a lexer or some other more high level too?. Because I 
> don't see what projects and workspaces have to do with a lexer.

This is a draft of use cases for all DCT libraries combined.

Scope for DCT is to provide semantic analysis, but not code 
generation (that may become another project some time). 
Information about projects, etc., is useful for e.g., analysing 
dependencies.

I'll improve overall structure and add some explanations + 
examples tomorrow. Could you elaborate on specific points which 
are vague?
May 22, 2012
Re: DCT use cases - draft
On 2012-05-22 19:14, Roman D. Boiko wrote:

> This is a draft of use cases for all DCT libraries combined.

This seems to be mostly focused on lexing? See below for some ideas.

> Scope for DCT is to provide semantic analysis, but not code generation
> (that may become another project some time). Information about projects,
> etc., is useful for e.g., analysing dependencies.

That's a good point.

> I'll improve overall structure and add some explanations + examples
> tomorrow. Could you elaborate on specific points which are vague?

I would probably have specified some high level use cases first, like:

* IDE integration
* Refactoring tool
* Static analysis
* Compiler
* Doc generating
* Build tool

In general, use cases that can span several compile phases, i.e. lexing, 
parsing, semantic analysis and so on. Some of these use cases can be 
broken in to several new use cases at a lower level. Some examples:

IDE integration:

* Syntax highlighting
* Code completion
* Showing lex, syntax and semantic errors

Refactoring:

* Cross-referencing symbols

Build tool:

* Tracking module dependencies

Doc generating:

* Associate a declaration and its documentation

Some of these "sub" use cases are needed by several tools, then you can 
either repeat them or pick unique sub use cases for each high level use 
case.

Then you can get into more detail over lower level use cases for the 
different compile phases. If you have enough to write you could probably 
have a post about the use cases for each phase.

It seems some of your use cases are implementation details or design 
goals, like "Store text efficiently".

It would not be necessary to start with the high level goals, but it 
would be nice. The next best thing would probably be to start with the 
use cases compiler phase you already have started on, that is lexing, if 
I have understood everything correctly.

-- 
/Jacob Carlborg
May 22, 2012
Re: DCT use cases - draft
On Tuesday, 22 May 2012 at 18:10:59 UTC, Jacob Carlborg wrote:
> On 2012-05-22 19:14, Roman D. Boiko wrote:
>
>> This is a draft of use cases for all DCT libraries combined.
>
> This seems to be mostly focused on lexing? See below for some 
> ideas.
Yeah, about 50% is lexing. I pay more attention to it because 
lexer alone is enough for several uses. I would like to have at 
least some functionality used as early as possible, this would 
provide me great feedback.

>> Scope for DCT is to provide semantic analysis, but not code 
>> generation (that may become another project some time). 
>> Information about projects, etc., is useful for e.g., 
>> analysing dependencies.
>
> That's a good point.
>
>> I'll improve overall structure and add some explanations + 
>> examples
>> tomorrow. Could you elaborate on specific points which are 
>> vague?
>
> I would probably have specified some high level use cases 
> first, like:
>
> * IDE integration
> * Refactoring tool
> * Static analysis
> * Compiler
> * Doc generating
> * Build tool
Thanks! I didn't think about build tool, for exapmle.

I started this way, but after your comment on my previous post 
that there is nothing new I reconsidered my approach and decided 
to start from concrete (low-lewel), then improve it according to 
feedback, and then split into areas (which roughly correspond to 
your hi-level use cases).

> In general, use cases that can span several compile phases, 
> i.e. lexing, parsing, semantic analysis and so on. Some of 
> these use cases can be broken in to several new use cases at a 
> lower level. Some examples:
>
> IDE integration:
>
> * Syntax highlighting
> * Code completion
> * Showing lex, syntax and semantic errors
>
> Refactoring:
>
> * Cross-referencing symbols
>
> Build tool:
>
> * Tracking module dependencies
>
> Doc generating:
>
> * Associate a declaration and its documentation
>
> Some of these "sub" use cases are needed by several tools, then 
> you can either repeat them or pick unique sub use cases for 
> each high level use case.
>
> Then you can get into more detail over lower level use cases 
> for the different compile phases. If you have enough to write 
> you could probably have a post about the use cases for each 
> phase.

Thanks for examples.

> It seems some of your use cases are implementation details or 
> design goals, like "Store text efficiently".
Actually, many of those are architectural (although low-level), 
because they are key to achieve the project goals, and failing in 
this area could cause overall failure. I intend to move any 
non-architectural information into a separate series of posts, 
feel free commenting what you don't consider important for the 
architecture (probably don't start yet, I'm reviewing text right 
now).

> It would not be necessary to start with the high level goals, 
> but it would be nice. The next best thing would probably be to 
> start with the use cases compiler phase you already have 
> started on, that is lexing, if I have understood everything 
> correctly.

Yes, and even before that I'm going to document some fundamental 
primitives, like immutability and core data structures.
May 22, 2012
Re: DCT use cases - draft
On 2012-05-22 20:33, Roman D. Boiko wrote:

> Yes, and even before that I'm going to document some fundamental
> primitives, like immutability and core data structures.

Wouldn't it be better to start with the use cases? You probably already 
have a fairly good idea about the use cases, but in theory the use cases 
could change how the data structure might look like.

-- 
/Jacob Carlborg
1 2 3
Top | Discussion index | About this forum | D home