Hello

I have just uploaded my language machine, toolkit for language and grammar to sourceforge. The  url is http://languagemachine.sourceforge.net.

Here's the summary as posted to freshmeat:

The Language Machine is a free software toolkit for language and grammar: a library, main program, and self-hosted rule compilers with one frontend. The system is easy to use. It directly implements unrestricted rule-based grammars with actions and external interfaces. In its present form the language machine consists of a shared library, a minimal main program, and a metalanguage (LMN - language machine/meta notation). The shared library is written in the Digital Mars D language using gdc, the D language front end for the GNU gcc compiler collection. The LMN compilers share a common frontend and translate into C and D as well as producing C, D, and shell script wrappers for an internal format. There is a proof-of-concept interface to Fabrice Bellard's tiny C compiler, with a working example. LMN source is compatible with the wikipedia text format - the project's web pages are an example of how the language machine can be used. Diagnostics include the lm-diagram, a diagram which show what happens when unrestricted grammar rules are applied.

The D-to-D translator example is on the web pages and is included in the distribution - without much annotation. It has been used to translate the whole of the language machine, and the result used to repeat the whole process. It has also been used to translate DMDscript. The translator is a ruleset with a frontend and a backend, and it does a fairly thorough analysis, but it may well be inaccurate in its idea of D grammar.

C is used only for external interfaces - the whole system is written entirely in D with a small metalanguage bootstrap.

Your comments and ideas are welcome.

Peri

In article <dau3bc$1hj0$1@digitaldaemon.com>, Peri Hankey says...
>
>Hello
>
>I have just uploaded my language machine, toolkit for language and grammar to sourceforge. The  url is http://languagemachine.sourceforge.net.
>
>Here's the summary as posted to freshmeat:
>
>The Language Machine is a free software toolkit for language and grammar: a library, main program, and self-hosted rule compilers with one frontend. The system is easy to use. It directly implements unrestricted rule-based grammars with actions and external interfaces. In its present form the language machine consists of a shared library, a minimal main program, and a metalanguage (LMN - language machine/meta notation). The shared library is written in the Digital Mars D language using gdc, the D language front end for the GNU gcc compiler collection. The LMN compilers share a common frontend and translate into C and D as well as producing C, D, and shell script wrappers for an internal format. There is a proof-of-concept interface to Fabrice Bellard's tiny C compiler, with a working example. LMN source is compatible with the wikipedia text format - the project's web pages are an example of how the language machine can be used. Diagnostics include the lm-diagram, a diagram which show what happens when unrestricted grammar rules are applied.
>
>The D-to-D translator example is on the web pages and is included in the distribution - without much annotation. It has been used to translate the whole of the language machine, and the result used to repeat the whole process. It has also been used to translate DMDscript. The translator is a ruleset with a frontend and a backend, and it does a fairly thorough analysis, but it may well be inaccurate in its idea of D grammar.
>
>C is used only for external interfaces - the whole system is written entirely in D with a small metalanguage bootstrap.
>
>Your comments and ideas are welcome.
>
>Peri

On first glance, the source looks really clean, and the meta-language well thought out. I'm not too fond of the syntax, but I guess one could change that quite easily, no? The information on the webpage is really good too; I was glued to the guide for quite a while. Me likes meta-languages :)

A few questions though:
* What exactly does the D-to-D translator do? Have I understood correctly if I
say that it lexes D source, verifying syntax correctness, and outputs D source
in a standard format?

* And what is the work division between the frontend/backend?

* Is the language machine meant primarily for interpreted languages, or could it be used as, say, a frontend for GCC?

I'll post more thorough impressions when I have had time to play with it a bit. As of currently, I really need to sleep.

Good work!

-Nod-

Nod wrote:
> In article <dau3bc$1hj0$1@digitaldaemon.com>, Peri Hankey says...
> 
>>Hello
>>
>>I have just uploaded my language machine, toolkit for language and grammar to sourceforge. The  url is http://languagemachine.sourceforge.net.
>>
>>Here's the summary as posted to freshmeat:
>>
>>The Language Machine is a free software toolkit for language and grammar: a library, main program, and self-hosted rule compilers with one frontend. The system is easy to use. It directly implements unrestricted rule-based grammars with actions and external interfaces. In its present form the language machine consists of a shared library, a minimal main program, and a metalanguage (LMN - language machine/meta notation). The shared library is written in the Digital Mars D language using gdc, the D language front end for the GNU gcc compiler collection. The LMN compilers share a common frontend and translate into C and D as well as producing C, D, and shell script wrappers for an internal format. There is a proof-of-concept interface to Fabrice Bellard's tiny C compiler, with a working example. LMN source is compatible with the wikipedia text format - the project's web pages are an example of how the language machine can be used. Diagnostics include the lm-diagram, a diagram which show what happens when unrestricted grammar rules are applied.
>>
>>The D-to-D translator example is on the web pages and is included in the distribution - without much annotation. It has been used to translate the whole of the language machine, and the result used to repeat the whole process. It has also been used to translate DMDscript. The translator is a ruleset with a frontend and a backend, and it does a fairly thorough analysis, but it may well be inaccurate in its idea of D grammar.
>>
>>C is used only for external interfaces - the whole system is written entirely in D with a small metalanguage bootstrap.
>>
>>Your comments and ideas are welcome.
>>
>>Peri
> 
> 
> On first glance, the source looks really clean, and the meta-language well
> thought out. I'm not too fond of the syntax, but I guess one could change that
> quite easily, no? The information on the webpage is really good too; I was glued
> to the guide for quite a while. Me likes meta-languages :)

That's terrific - most of the people I have explained it to fall asleep instantly - a good cure for insomnia.

It's easy to change the syntax, but hard to find the right one. I was looking for a minimal syntax to get the job done.

> 
> A few questions though:
> * What exactly does the D-to-D translator do? Have I understood correctly if I
> say that it lexes D source, verifying syntax correctness, and outputs D source
> in a standard format?

The D-to-D translator started because I was looking for ways to integrate grammar rules directly as an extension of D (this would probably have been a mistake). The best way to prove to myself that my rules were understanding D source was to produce something could be compiled and tested. Also there was a lot of talk here about experimenting with things that could be done with D, so I thought there might be some interest

> * And what is the work division between the frontend/backend?

Both frontend and backend are just rules: the LMN notation allows you to produce prepackaged analyses such as

    '?{' whatnot :F expression :X '}' <- thing :{ whatnotkindofthing :{ operator :F :X }};

This means: '?{' then a  "whatnot" (with a value we call F) followed by an "expression" (with a value we call X) and by '}' can be treated as a "thing" which produces a bag containing a "whatnotkindofthing" which produces a bag containing an "operator" with the F and the X as its values.

A backend rule to handle this would receive a "whatnotkindofthing" in the first place. It can strip off some of the packaging:

    whatnotkindofthing :Q  <- code - 'hoho it\'s a ' Q eoc;

Or it can look inside the bag:

    whatnotkindofthing : thiskindofthing :T :F :X  <- code - .....

Here it looks in the bag and forces the "operator" to be treated as a "thiskindofthing".

So the frontend rules produce prepackaged stuff that is recognised and translated into some external representation by the backend.

There's a hypersimple example of how this is done in the rules for converting mediawiki text to html: http://languagemachine.sourceforge.net/mediawiki.html.

The prepackaged stuff is in a sense the symbolic representation of a parse tree that can be treated as input by backend rules. But the same kind of rule is used at the lexical level, for syntax, and in the back end.

> * Is the language machine meant primarily for interpreted languages, or could it
> be used as, say, a frontend for GCC?

It could be used in either way: in particular, you could use the interface with tcc to compile via C into machine code im memory for immediate execution.

> I'll post more thorough impressions when I have had time to play with it a bit.
> As of currently, I really need to sleep. 
> 
> Good work!
> 
> -Nod-
> 
> 

In article <dau3bc$1hj0$1@digitaldaemon.com>, Peri Hankey says...

>The D-to-D translator example is on the web pages and is included in the distribution - without much annotation. It has been used to translate the whole of the language machine, and the result used to repeat the whole process.

How would this thing do for a d-to-HTML converter. Maybe generate a syntax highlighted HTML file from a d file, maybe even parse out the comments into something nicer to look at.

B Shropshire wrote:
> In article <dau3bc$1hj0$1@digitaldaemon.com>, Peri Hankey says...
> 
> 
>>The D-to-D translator example is on the web pages and is included in the distribution - without much annotation. It has been used to translate the whole of the language machine, and the result used to repeat the whole process.
> 
> 
> How would this thing do for a d-to-HTML converter. Maybe generate a syntax
> highlighted HTML file from a d file, maybe even parse out the comments into
> something nicer to look at.
> 
> 
Yes - it would be a matter of making an adapted backend ruleset. At present the translator consists of

d2xfe.lmn - the frontend ruleset
d2dbe.lmn - the d2d backend ruleset

But if you look at the metalanguage compilers, these are constructed with (frontend, backend) pairs:

lmn2xfe.lmn, lmn2mbe.lmn - compile to own loader format
lmn2xfe.lmn, lmn2dbe.lmn - wrap loader format as D
lmn2xfe.lmn, lmn2cbe.lmn - wrap loader format as C
lmn2xfe.lmn, lmn2Dbe.lmn - compile to D
lmn2xfe.lmn, lmn2Cbe.lmn - compile to C

The ruleset pairs can either be combined as source or concatenated at runtime.

Peri

In article <davvoc$8h9$1@digitaldaemon.com>, Peri Hankey says...
>
> <snip>
>

Whoa! The Language Machine is way cool, indeed!

I have a ton of questions, but I feel it is rather OT for this newsgroup... Would it be possible for you to set up a dedicated communications forum for TLM?

A mailing-list on sourceforge would suffice, but might I suggest requesting an account on www.dsource.org? In addition to providing a nice forum, it also gives access to an SVN repository, rather than the ancient CVS used on sf.net.

And of course, www.dsource.org is a lot more responsive to feature requests than sf.net :)

-Nod-

Nod wrote:
> In article <davvoc$8h9$1@digitaldaemon.com>, Peri Hankey says...
> 
>><snip>
>>
> 
> Whoa! The Language Machine is way cool, indeed!
> 
> I have a ton of questions, but I feel it is rather OT for this newsgroup...
> Would it be possible for you to set up a dedicated communications forum for TLM?
> 
> A mailing-list on sourceforge would suffice, but might I suggest requesting an
> account on www.dsource.org? In addition to providing a nice forum, it also gives
> access to an SVN repository, rather than the ancient CVS used on sf.net.
> 
> And of course, www.dsource.org is a lot more responsive to feature requests than
> sf.net :)
> 
> -Nod-
> 
> 

Nod - thanks - I agree that this isn't the place. I have emailed dsource.org, and we'll see what happens.

In the meantime there are 2 forums at the sourceforge project pages https://sourceforge.net/forum/?group_id=143185 - we can later distill and extract useful stuff to a faq or to wherever the forum finally goes.

Peri

Peri Hankey wrote:
> Nod wrote:
> 
>> In article <davvoc$8h9$1@digitaldaemon.com>, Peri Hankey says...
>>
>>> <snip>
>>>
>>
>> Whoa! The Language Machine is way cool, indeed!
>>
>> I have a ton of questions, but I feel it is rather OT for this
>> newsgroup...
>> Would it be possible for you to set up a dedicated communications
>> forum for TLM?
>>
>> A mailing-list on sourceforge would suffice, but might I suggest
>> requesting an
>> account on www.dsource.org? In addition to providing a nice forum, it
>> also gives
>> access to an SVN repository, rather than the ancient CVS used on sf.net.
>>
>> And of course, www.dsource.org is a lot more responsive to feature
>> requests than
>> sf.net :)
>>
>> -Nod-
>>
>>
> 
> Nod - thanks - I agree that this isn't the place. I have emailed dsource.org, and we'll see what happens.

Been on vacation but I'm back now.  I've sent Peri an email...

BA

> 
> In the meantime there are 2 forums at the sourceforge project pages https://sourceforge.net/forum/?group_id=143185 - we can later distill and extract useful stuff to a faq or to wherever the forum finally goes.
> 
> Peri
>