Charlie Patterson wrote:
> I'm not sure where you want to go with this meta-language but I'm here to burst your bubble.  (-:  

Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.

> I always wanted a meta-langauge... until I saw one. There is one for C++ and it is indecipherable.  It seems the movement in langagues is towards cleaning up and pairing down, not making them more "tricky."

True. And the C++ one is so gross, it makes everybody give up.

> Lots of people still claim C++ is too hard precisely because you don't know what it will do under the covers for a simple assignment statement.  And keep in mind C++ and even Java and D have some basic "meta programming" with operator= (opAssign) type options.

Right. Now, if Java and D already have some of this, then I think it just proves we can't live without one.

> If you really, really want to add whole new calculus to the language, well that is what I thought I wanted until I saw an example.  Most people just get disturbed by creating it and even just being forced to use the results.

Naaw, a new calculus is not the target. What I want, is a way to, at one end express repetitions and patterns in source with something smarter than just plain writing it all out (c.f. the DMD #define example), and at the other, a way to increase the expressivenes of the, er, "combined" language.

And it ABSOLUTELY has to be easy to write and understand. For example, simple #defines in cpp are pretty easy to understand, even for those who've never written one. But C++ templates get just unreadable pretty fast. And I blame their specific template language for it. I don't believe it _has_ to be so difficult.

The way to express a solution should never have to be more complicated than the solution itself.

> Finally, I've been on a tear for the last few years that people over-use languages anyway.  I think I got the idea from a Myer's C++ book (and also that Andrescu template book).  I was following along with all the insanely detailed examples of using private inheritance with a customized new operator, etc, etc, when they started adjusting the C++ vtable in order to create some lisp-like capabilities (non-portable obviously).  That was the last straw for me.  If you have to *beat* the language, then you need to simplify your app... or suffer the ugly consequences of maintaing it.  Or writing a new language!  But at some level of power, people just drop off.

True. D is a "practical language for practical programmers". We need a practical meta language, that is practical for practical purposes. And if we can give it expressive power that does not run out of stream right after the first block, without making either simple or hard things harder than they _inherently_ are, then we've succeeded.

"Georg Wrede" <georg.wrede@nospam.org> wrote in message news:42300F90.40600@nospam.org...
> Norbert Nemec wrote:
>> Georg Wrede schrieb:
>>
>>> I've been thinkin hard on a meta language.
>>>
>>> I've talked with folks, amongst them prof. Jaakko Järvi, who actually met Walter at the C++ meeting last fall.
>>>
>>> Bad news: I seem to have spilled the beans, big time. After asking his opinion about some of the core concepts I'd been cooking, it occurred to me that -- this guy's office is down the hall from Bjarne Stroustrup!
>>>
>>> If these guys stop dwelling _within_ the C++ template jungle, and start looking at it from above, then it'll be just 18 months before one of them holds a ground breaking lecture.   :-(   :-(
>>
>>
>> Don't worry. The basic idea has been around for some time already. Within C++, it will be hard to go far beyond the current state without seriously breaking compatibility. It can only be new languages that can take the next step.
>
> Of course I worry. Guys of this caliber do things "right". And once they've figured out a replacement to their current pp/tl system, the new one will take care of the old pp/tl syntax just with a snap of fingers.
>
> The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap.
>
> So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle.
>
> One bad (for us, that is) thing that follows from this, is that once they've done this, then, given time, folks will rewrite the current STL, BOOST, etc. with the new syntax (possibly even skipping the bla<foo<bar>> notation), and this will inevitably result in a vast speed-up of compilation!!!!
>
> So, I'm worried as hell. For D. But for Human Kind this is of course a Good Thing(TM).
>
> -----------

I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations.

But maybe I'm a starry-eyed optimist. :-)

"Georg Wrede" <georg.wrede@nospam.org> wrote in message news:4230C679.40509@nospam.org...
> Charlie Patterson wrote:
>> I'm not sure where you want to go with this meta-language but I'm here to burst your bubble.  (-:
>
> Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.

Being the dullard that I am, I have a real hard time imagining what you're talking about absent examples.

Can you cook up a couple - just small one's be ok - and that'd help me get into it?

Cheers

Matthew

Matthew wrote:
> "Georg Wrede" <georg.wrede@nospam.org> wrote in message news:42300F90.40600@nospam.org...
> 
>>Norbert Nemec wrote:
>>
>>>Georg Wrede schrieb:
>>>
>>>
>>>>I've been thinkin hard on a meta language.
>>>>
>>>>I've talked with folks, amongst them prof. Jaakko Järvi, who actually met Walter at the C++ meeting last fall.
>>>>
>>>>Bad news: I seem to have spilled the beans, big time. After asking his opinion about some of the core concepts I'd been cooking, it occurred to me that -- this guy's office is down the hall from Bjarne Stroustrup!
>>>>
>>>>If these guys stop dwelling _within_ the C++ template jungle, and start looking at it from above, then it'll be just 18 months before one of them holds a ground breaking lecture.   :-(   :-(
>>>
>>>
>>>Don't worry. The basic idea has been around for some time already. Within C++, it will be hard to go far beyond the current state without seriously breaking compatibility. It can only be new languages that can take the next step.
>>
>>Of course I worry. Guys of this caliber do things "right". And once they've figured out a replacement to their current pp/tl system, the new one will take care of the old pp/tl syntax just with a snap of fingers.
>>
>>The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap.
>>
>>So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle.
>>
>>One bad (for us, that is) thing that follows from this, is that once they've done this, then, given time, folks will rewrite the current STL, BOOST, etc. with the new syntax (possibly even skipping the bla<foo<bar>> notation), and this will inevitably result in a vast speed-up of compilation!!!!
>>
>>So, I'm worried as hell. For D. But for Human Kind this is of course a Good Thing(TM).
>>
>>-----------
> 
> 
> I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations.
> 
> But maybe I'm a starry-eyed optimist. :-)
> 
> 

I remember Walter saying something about using '$' for something big, like meta language.

Besides, what would be so bad if D sat back, watched it happened, then found a better way to add it when it is ready?

Matthew wrote:
> "Georg Wrede" <georg.wrede@nospam.org> wrote:
>>Charlie Patterson wrote:
>>
>>>I'm not sure where you want to go with this meta-language but I'm here to burst your bubble.  (-:
>>
>>Thanks! It NEEDS to be stabbed at, wrought, wrinkled, etc. It's the only way we'll prune the chaff off. Or maybe prune the whole deal if it doesn't deserve to live.
> 
> Being the dullard that I am, I have a real hard time imagining what you're talking about absent examples.
> 
> Can you cook up a couple - just small one's be ok - and that'd help me get into it?

OK. But I will not do examples on the language, only how it could be used. We need to have that covered, before we even start getting into the meta syntax.

Say you need to do a long case statement, like the C++ one:
---------------------
        switch (token.value)
        {
#define X(tok,ector) case tok:  nextToken(); \
  e2 = parseAssignExp(); e = new ector(loc,e,e2); continue;

            X(TOKassign,    AssignExp);
            X(TOKaddass,    AddAssignExp);
//etc......
            X(TOKushrass,   UshrAssignExp);
            X(TOKcatass,    CatAssignExp);

#undef X
            default:                 break;
        }
---------------------
One way to write it could be:
---------------------
switch (token.value)
{
    lkjsdflkjlklasldkjlasd    //  <-- the meta def
    TOKassign    AssignExp
    TOKaddass    AddAssignExp
//etc......
    TOKushrass   UshrAssignExp
    TOKcatass    CatAssignExp
    lksdjflksjdlf            //   <-- undo meta def
    default: break;
}
---------------------
Or, if we wanted to make it more readable to the C++ people, we'd do the meta def so that we could write the TOKassign lines just like they'd write:
---------------------
switch (token.value)
{
    lkjsdflkjlklasldkjlasd    //  <-- another meta def
            X(TOKassign,    AssignExp);
            X(TOKaddass,    AddAssignExp);
//etc......
            X(TOKushrass,   UshrAssignExp);
            X(TOKcatass,    CatAssignExp);
    lksdjflksjdlf            //   <-- undo meta def
    default: break;
}
---------------------


At this point everyting is open. For example: do we want the meta syntax to be "invisible"? (That means, an unconspicuous part of D.) Or do we want it to stand out like heck?

Unconspicuous:
---------------------
char[] foo = "lksdf";
int number = 8;
meta(create, kjdslfkjlskdjlsk);   // <-- a meta def here
tree appletree
   trunk
      mainBranch strong internetConnected
         hangar branch filesystemAware
      auxBranch temporary stdErrOutput
   roots
      mainRoot suction localNetConnected
         hardwareBranch tapesystem rw
            hwType HP9245
            hwType MaxstoreUnidriver
unmeta(last);                     // <-- undo meta def
int errlevel = 0;
---------------------

Conspicuous:
---------------------
char[] foo = "lksdf";
int number = 8;

####!!!!! META-LANGUAGE-DEFINITION-SECTION-BEGIN
##!!! METADEF (CREATE, lksjdflksjdlks)  <-- a meta def here

tree appletree
   trunk
      mainBranch strong internetConnected
         hangar branch filesystemAware
      auxBranch temporary stdErrOutput
   roots
      mainRoot suction localNetConnected
         hardwareBranch tapesystem rw
            hwType HP9245
            hwType MaxstoreUnidriver

##!!! DESTROY-METADEF (lksjdflksjdlks)  <-- undo meta def
####!!!!! META-LANGUAGE-DEFINITION-SECTION-END

int errlevel = 0;

---------------------

The time for meta syntax is not yet. Issues like the above have to be thought out before that.


In the mean time:
We need more use cases!!

Tell me a few stumbling blocks you run into when writing STL for D!

Or other things that are currently hard in D and not in C++.


Another example:

---------------------
// excerpt from the Aaron Spelling far-out math library
// some formulas from the text book, compiled here
// these take and return floats
lksjdflksajdlfksadjlf // the metadef
   doubleSin(x) := sin(x) + sin(x);
   antiSin(x)   := sin(x + pi);
lksjdf // undo metadef
---------------------

Sorry, that example looks disgustingly trivial. Totally worthless. Let's write something worse, this time actually from a math book.  :-(

[[ Two hours later: Crap, I'll do this example later. I will, honest. In the meantime, let's see something else: ]]

Say, we need to do a lot of matrix literals. This might be part of some library code:

---------------------
LKAJSDLFKAJSLDKFJLSADKJF   // the metadef

eigen_schmeigen =
  2  3  2  0  1  0
 -3  0  0  0  1 -3
;

//... snip, a lot of them here

wamon_schmamon =
  0 -1  0 -1  1  0
  0  1  1  1  1  1
  1 -1 -1  0  0  1
;

LKAJSDF // end metadef
---------------------

So what's the use, you might ask. Point is, for a lot of things there are established ways of expressing them. Often, these are clearer than doing the same in regular programming language syntax. So, i bet some corporations, universities, oss, and others will contribute to our meta language library, and later the programmer only has to write:

---------------------
import uni_groeningen.matrix;
alias uni_groeningen.matrix.free_mat_notation.begin ug.begin;
alias uni_groeningen.matrix.free_mat_notation.end ug.end;
// ...
ug.begin;
   eigen_schmeigen =
     2  3  2  0  1  0
    -3  0  0  0  1 -3
   ;
   wamon_schmanon =
     0 -1  0 -1  1  0
     0  1  1  1  1  1
     1 -1 -1  0  0  1
   ;
ug.end;
---------------------

Sure, this opens the door for obfuscation C++ has never even seen. But hey, even if I have a kitchen knife in my kitchen, I don't stab everyone who comes there. (I'll leave NRA out of this. :-(  )

#########

Somehow, though, I feel that examples from template programming (D, C++) might be needed here. What's something you currently wanted to do but can't?

Andy Friesen wrote:
> Georg Wrede wrote:
> 
>> My real goof was in waking up this guy, from "dwelling withing the template jungle" to looking at it "from above", "outside the box". Men of this kind don't need more than that.
> 
> 
> Don't worry, you haven't pointed out anything he didn't know already. Lisp has been doing this for years now.  The majority of the syntax of the Nemerle language <http://nemerle.org> is implemented as macros. (even stuff like 'if' statements)
> 
> Also, Daveed Vandevoorde had a quasi-working implementation of such an extension for C++ as far back as 2003: <http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2003/n1471.pdf>

Thanks!
Already peeked at 'em. Gonna take a closer look.

Matthew wrote:
> "Georg Wrede" <georg.wrede@nospam.org> wrote:
.......
>>The goal, after all, once they start thinking "outside the box" is to create a universal meta language. For such a language it is a piece of cake to handle all of the current crap.
>>
>>So, they'll get their Backward Compatibility, but this time as a _mere_ special case -- just one small thing among the tousands of grand things it'll handle.
........
>>So, I'm worried as hell.
> 
> I'd chill out. I have not the slightest doubt that Walter's smart enough to do this. And I'm confident it'll be out before any C++ implementations.

Course he is smart enough! But isn't this newsgroup here to let ideas cook, folks find the early worm, ask the Right Questions, etc.? (Not forgetting helping newbies, too.)

> But maybe I'm a starry-eyed optimist. :-)

Somehow I get the feeling there _is_ a meta language project, already well rolling forward -- behind the scenes. For some reason it's been awful quiet around here about its specifics.

Could be a lot of reasons. Maybe we'd crowd it with all these ideas, maybe he wants some peace with it, maybe Walter wants to save it for a Grand Opening, maybe it is "secret" to avoid embarrasment in case it folds?

Or, he's talking with superhumans in the academia and other Einstein/MahreshYogi look-alike hermits, the only ones who've Received the Knowledge from Above.

If this is the case, then I'd better shut up. I could always try to cook something up, and if it tastes good, then maybe try to earn some money with it? The it'd of course be a pure preprocessor.

xs0 wrote:
> As I see it, two issues are involved:
> - being able to specify new syntax, both locally (as the X example below) and globally, where the main problem is - what is the syntax for syntax, and how do you make it self-extensible; if it's not self-extensible, it's just another layer of templates, I guess..

Forgot to answer this.

Of course it has to be sef-extensible. I see no other option. The entire point of a Decent Meta Language is, to be able to cope with the unforeseen.

It would have to be self-extensible, and self-modifiable. -- Without imposing either on the down-to-earth regular user of it.

It's like, D allows you to do OO or do without OO, or mix them. The Meta shoud be the same.

-------------

We could view (with some effort and will power) D itself as an asm metalanguage.

  "D is another asm compiler, but it wraps many of the most-used things in a convenient meta language, and has an extensive (in this context anyway) library (phobos), to relieve the programmer of writing the most usual things over and over. It handles high-level I/O, the usual data structures, and conversions for you."

But because it is not self-extensible and not self-modifiable, it can only (in a broad sense) be used with the kind of problems its creators had in mind. (( A derogatory sounding way of saying `it's for stuff you'd use C, C++, Java etc. for`.))

Just for my understanding: would a metalanguage allow you to define new operators on the fly?

This way, the current D language would in fact be an implementation using (its own) metalanguage for defining the existing operators and their precedence?

Lionello.

Matthew wrote:
> Being the dullard that I am, I have a real hard time imagining what you're talking about absent examples.

I'm not talking about absent examples. You are.
SORRY, couldn'resist!    :-)



##############################################

                Disclaimer:

The stuff below is idiotic. Pleas try to
look at the language part of it, not the
stupidity of the examples themselves...  :-/
##############################################


This guy works in a company where most of the core competence is in using the Intel math coprocessor in creative ways. They do contracting for NASA, NSA, SHELL, and some even for CERN.

They have reams of code libraries to ease the use of the math coprocessor. However, the languages they've used so far (Forth, C, asm, C++, Lisp) have only been able to cater for some aspect of application development.

Currently they have separate staff groups for each language, and (the applications made being huge and complicated) most projects use the work from several of these groups.

They have tried to create a C++ template library, and a Lisp (actually Scheme) library, that'd let the programmer express his wishes both at low level as well as on the very high level this company needs. But nothing seems to work seamlessly, or efficiently.

Then they found D. An excerpt from the low level programming:

---- library code

module foo.bar;

lkajsdlkjaldskfjlsadk;

/* That was a meta definition that essentially creates the function wrapper for D function definitions that need low-level math coprosessor routines. Inside this kind of function definitions you can mix [Our Proprietary, Secret Notation] with regular D code. */

---- production code

import foo.bar;

alias foo.bar b;

float mLiff;

b.floatFunc  float aCoolLength(p1, p2, p3, p4)
{                     // all params float, unless other specified
   float m1, m2, m3;

   if ( p1 < p2 )
   {
      p1 p3 push
      sin
      p4 push
      log
      exp
      p2 inv
      mul
      m1 pop
   }

   if (m1 < p4)
      m2 = m1
   else
      m2 = m1 + p1;

   return m2;
}

void main()
{
   mLiff = aCoolLength(2.3, 4.5, 6.7, 8.9);
   writefln("The meaning of Life is: %f.2", mLiff);
}

Now, this was an example from the low level. I'd like to do one with high level too, but that'll take some more time.

#################################################

Another example: This firm manufactures and sells sugar cube size mass produced brains. But model life cycle is short, and customers keep inventing unforeseen uses for them. Essentially, it is a cpu, some rom, some ram, and pins for 16 TTL connections programmable to be in or out, and an infrared interface for field programming. It also has 3 analog inputs and 3 outputs.

Their customers include toy manufacturers, mining robotics, fire departments, the military, and some unnamed government agencies. Often a project is to create the needed software for a new gadget -- and (with some unnamed customers) it has to be ready and bug free in a matter of hours. (They still remember the hulabaloo with a project during the Gulf War.)

They could make a huge library, have a language of their own, the works. But specs change, processors change, etc. So they need to do C like code, but it has to be convenient to do the behavioral primitives.

All the gadgets, once out in the field, are programmable with a small, interpreted language. End-users do field programming with this language. This firm designs and assembles these sugar-cube brains, and ships them with firmware customized for the retail gadget (or hand-held nuclear missile guidance system). They also sell the field programming software, and of course there are different UI versions for the toy makers and the marines.

They have found out that "We really want a C, but with faster development times, and vastly less bugs in the binaries."

Since most programs are just a single file, the low level gets defined in the same file as where it is used to build behavioral primitives. Few languages have been of help here.

Then they found D. Some excerpts, not necessarily from same project:

void step()
{
       if ( ! obstacle(ahead) )
       {
           unlock_brake
           forward.slow
           while (!obstacle(ahead) && distance < STEPDISTANCE )
           {}
           stop
       }
}

void deployPayload()
{
       unlockArm
       extendArm wristRight extendFinger1
       jawOpen smallExtendElbow jawClose
}

Well, this turned out to be intermediate level code. :-(
What they really wanted was to change having to write this (pure D code)

void function foo(){
     bar();
     barf();
     bar();
     barf();
     boo();
     doo(); dee(); dii();
     lee(); loo();
     luu();
     moo();
     soo();
}

to this:

void function foo(){
     bar
     barf
     bar
     barf
     boo
     doo dee dii
     lee loo
     luu
     moo
     soo
}

Most of their programming is just invoking tiny functions that neither take or return anything. Slapping lots of them in your code day in and day out gets tedious with all the "();" all over the place. Also, they wanted the code to look cleaner, since they discovered that it reduces bugs, and helps find errors, too.

So, their standard library merely defines that inside
  void fname()
you don't have to use either semicolons or parentheses.

They want, however to remain flexible. That's why they were so interested in D, with its superior meta code facilities. Now they are considering automatic creation of function sets.

The programmer would create all of the functions like:
int fname(int a = 1, int b = -1)
and the meta system would automatically create a set of these. This would be used where functions need to return, say, a success code, or length traversed. Most of the time that is not needed (well, in this example company, anyway), but sometimes it is needed.

For example the programmer creates:

// walk on water
int wow (int distance = 1, int careful = -1)
{
    // let's say careful == -1 means keep last level
    // The code here would use the params, and return
    // a value, in this case the distance traversed.
    // But most often it is not used.
}

Then the meta would create the following versions:

void  wow (int distance = 1, int careful = -1)
{
   // take none, return none
   // just walk one unit with default or
   // last used carefulness
}
int Wow (int distance = 1, int careful = -1)
{
   // return distance
}
int woW (int distance = 1, int careful = -1)
{
   // walk till obstacle
}
int WOW (int distance = 1, int careful = -1)
{
   // panic walk (careful at min, dist till obstacle)
}

And so on. (Since I haven't worked there, I don't know the standard permutations! But you get the drift, right?)

Those permutations would contain upper case letters in the name. The standard for which letter means what, has become plain obvious at the company, and it has been used for years, very successfully.

Problem was, that it was a constant source of bugs. People capped the wrong letter, etc. Now they see the end of those bugs. And since there is only one "copy" of actual function code, it stays in sync with itself.

Now they use this tac for most of the primitives. Walk, extendArm, drill, turn, sleep, all of them are now encoded with the three-first-letter-caps. Programmers are happy, and roll-out times smaller. This is also the default for any new kind of action primitives.

########################################

The way I see the meta language is, that it would not bee too hard to have it check and enforce the "self-made" syntaxes. Not automatically, but with writing meta code for the checking and enforcing -- and still be understandable to a normal person!