On 2010-12-14 19:13, Nick Sabalausky wrote:
> "Graham St Jack"<Graham.StJack@internode.on.net>  wrote in message
> news:ie76ig$b2v$1@digitalmars.com...
>>
>> What you are suggesting here seems to be a way to dramatically reduce the
>> complexity of code that generates source-code and mixes it in. I think
>> something like that is needed before this mind-bogglingly powerful feature
>> of D can realise its potential.
>>
>
> I think a decent string-template library could probably come very close to
> the proposal without needing any language changes at all:
>
> string get_set(T, string name)()
> {
>       return
>       q{
>           @type _@name;
>
>           @type @name ()
>           {
>               return _@name;
>           }
>
>           @type @name (@type @name)
>           {
>               return _@name = @name;
>           }
>       }.replace( ["@type": T.stringof, "@name": name] );
> }
>
> class Foo
> {
>       mixin(get_set!(int, "bar")());
> }
>
> There are definitely some things about the proposal that are better than
> with this, but I just wanted to point out that the value of the proposal
> should probably be evaluated against something roughly like the above rather
> than something that does a bunch of procedural string splicing.

The whole point of the idea was to get rid of the strings and the mixin expression (as it looks like to day).

-- 
/Jacob Carlborg

"Jacob Carlborg" <doob@me.com> wrote in message news:ie8i8c$15f0$1@digitalmars.com...
> On 2010-12-14 19:13, Nick Sabalausky wrote:
>> "Graham St Jack"<Graham.StJack@internode.on.net>  wrote in message news:ie76ig$b2v$1@digitalmars.com...
>>>
>>> What you are suggesting here seems to be a way to dramatically reduce
>>> the
>>> complexity of code that generates source-code and mixes it in. I think
>>> something like that is needed before this mind-bogglingly powerful
>>> feature
>>> of D can realise its potential.
>>>
>>
>> I think a decent string-template library could probably come very close
>> to
>> the proposal without needing any language changes at all:
>>
>> string get_set(T, string name)()
>> {
>>       return
>>       q{
>>           @type _@name;
>>
>>           @type @name ()
>>           {
>>               return _@name;
>>           }
>>
>>           @type @name (@type @name)
>>           {
>>               return _@name = @name;
>>           }
>>       }.replace( ["@type": T.stringof, "@name": name] );
>> }
>>
>> class Foo
>> {
>>       mixin(get_set!(int, "bar")());
>> }
>>
>> There are definitely some things about the proposal that are better than
>> with this, but I just wanted to point out that the value of the proposal
>> should probably be evaluated against something roughly like the above
>> rather
>> than something that does a bunch of procedural string splicing.
>
> The whole point of the idea was to get rid of the strings and the mixin expression (as it looks like to day).
>

While I'm not necessarily opposed to the idea of getting rid of the strings, I guess I don't really see what improvement your proposal provides other than not having to manually specify the mapping of "replace what identifier with what data".

Getting rid of the "mixin" I see as a separate issue. We could just as easily say that given the function "string get_set(...)":

    @get_set("int", "bar");
    or
    @get_set int bar;

...Is shorthand for, or is even the new syntax for:

    mixin(get_set("int", "bar"));

> I don't know, do you have an example ?
>
>> For example taking a classname and a bunch of field types and names, and turning it into a class definition complete with constructor from field values, constructor from an input stream, a method to write to an output stream, and const getters. Or maybe std.typecons.AutoImplement.
>
> Could you post an example of how that mixin would be used and the code it would generate then I can see if I can translate it to my syntax. AutoImplement seems to just contain template mixins which is something else.
>

I have attached my concurrency framework, which relies heavily on mixins, plus its unit test to show how it is used. I haven't included the various dependencies because I assume you just want the example code. Let me know if you want something buildable, or perhaps something more cut-down.

What the code-generating template does is to create from something like this (you can have any number of Messages in a Protocol):


alias Protocol!("Requests",    Message!("job",    string, "name")).code
jobCode;
mixin(jobCode);


this code:


class Requests {
     struct jobMsg {
         string name;
         this(string name) {
              this.name = name;
         }
         void read(InStream stream) {
              name = stream.get!string;
         }
         void write(OutStream stream) {
             stream(name);
         }
     }
     struct Message {
         uint kind;
         union {
         jobMsg job;
         }
         this(ref jobMsg msg) {
             kind = 0;
             job = msg;
         }
         this(InStream stream) {
             kind = stream.get!uint;
             switch(kind) {
             case 0: job.read(stream); break;
                 default: assert(0, "Cannot read unsupported message kind");
             }
         }
         void write(OutStream stream) {
             stream(kind);
             switch(kind) {
             case 0: job.write(stream); break;
                 default: assert(0, "Cannot write unsupported message
kind");
             }
         }
     }
     private alias Channel!(Message) _Chan;
     private alias shared _Chan Chan;

     private Chan channel;
     this() { channel = new Chan(); }
     ChannelSelectable newSelectable() { return channel.newSelectable(); }
     void finalize() { channel.finalize; }

     interface IHandler {
         void job(string name);
     }
     void job(string name) {
         channel.add(Message(jobMsg(name)));
     }
     void receive(IHandler handler) {
         auto message = channel.remove;
         switch (message.kind) {
             case 0: handler.job(message.job.name); break;
             default: assert(0, "Cannot dispatch unsupported message kind");
         }
     }
}



I use this for inter-thread communications, and I use the discriminated union to pass messages between processes. The manual mixin after the alias is a debugging aid - the compiler errors when doing mixins aren't helpful at all. I case you are wondering why I did all this, it was partly a learning experience, but mostly an attempt to do something properly thread-safe (!hasAliasing), using shared, const and immutable properly.

-- 
Graham St Jack

Jacob Carlborg wrote:
> On 2010-12-14 13:05, Don wrote:
>> Graham St Jack wrote:
>>> On 14/12/10 20:33, Vladimir Panteleev wrote:
>>>> On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack
>>>> <Graham.StJack@internode.on.net> wrote:
>>>>
>>>>> There is of course the worry that it could get so easy that everyone
>>>>> starts doing it, and we have (relatively) impenetrable code everywhere
>>>>> instead of just deep in the bowels of framework libraries.
>>>>
>>>> TBH, I'm more excited by AST macros which I understood are planned for
>>>> D3, as mentioned here:
>>>> http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf
>>>> They seem to promise the power of mixins, but without the mess.
>>>>
>>>
>>> I took a look at the pdf, but couldn't see how the AST macros could come
>>> close to the kinds of things that are possible (but difficult) with
>>> mixins.
>>
>> That fact was recognized at the conference, on the following day. As a
>> result, AST macros were dropped from D2.
> 
> Do you have an example that would work with string mixins but not with AST macros?

Well, it's a bit hard without a formal definition of AST macros.
But the stuff I talked about at the conference, I have no idea how to do with AST macros.

There's code like this, which generates an asm instruction.

------
mixin( opToSSE[operations[done+1]] ~ suffix ~ " " ~ XMM(numOnStack-1) ~ ", "  ~ indexedSSEVector(ranklist, operations[done], vectorsize));
-------

using the functions:
============
const char [][5] vectorRegister = ["ECX", "EDX", "EBX", "ESI", "EDI"];

char [] indexedSSEVector(char [] ranklist, char var, char [] vecsize)
{
    return "[" ~ vectorRegister[vectorNum(ranklist, var)] ~ " + " ~ vecsize ~"*EAX]";
}

char [] XMM(int k) { return "XMM"~ itoa(k); }

char [][char] opToSSE() {
    return ['*':"mulp"[], '+': "addp", '-': "subp", '/': "divp"]; }

int vectorNum(char [] ranklist, char var)
{
    int numVecs=0;
    for (int i=0; i<var-'A'; ++i) {
        if (ranklist[i]=='1') ++numVecs;
    }
    return numVecs;
}
============

>> They need to roughly match string mixins in power. At this stage, there
>> is no proposal for how they should work.
> 
> I think someone, Nick Sabalausky perhaps, suggested to have something like the hygiene macros in Nemerle: http://nemerle.org/wiki/index.php?title=Macros

From an implementation point of view, the differences between Nemerle macros and string mixins are mostly syntactic.
The one thing about them that I find really impressive is the IDE integration, especially that they got syntax highlighting to work. I don't know they've done that.

On 2010-12-14 21:44, Nick Sabalausky wrote:
> "Jacob Carlborg"<doob@me.com>  wrote in message
> news:ie8i8c$15f0$1@digitalmars.com...
>> On 2010-12-14 19:13, Nick Sabalausky wrote:
>>> "Graham St Jack"<Graham.StJack@internode.on.net>   wrote in message
>>> news:ie76ig$b2v$1@digitalmars.com...
>>>>
>>>> What you are suggesting here seems to be a way to dramatically reduce
>>>> the
>>>> complexity of code that generates source-code and mixes it in. I think
>>>> something like that is needed before this mind-bogglingly powerful
>>>> feature
>>>> of D can realise its potential.
>>>>
>>>
>>> I think a decent string-template library could probably come very close
>>> to
>>> the proposal without needing any language changes at all:
>>>
>>> string get_set(T, string name)()
>>> {
>>>        return
>>>        q{
>>>            @type _@name;
>>>
>>>            @type @name ()
>>>            {
>>>                return _@name;
>>>            }
>>>
>>>            @type @name (@type @name)
>>>            {
>>>                return _@name = @name;
>>>            }
>>>        }.replace( ["@type": T.stringof, "@name": name] );
>>> }
>>>
>>> class Foo
>>> {
>>>        mixin(get_set!(int, "bar")());
>>> }
>>>
>>> There are definitely some things about the proposal that are better than
>>> with this, but I just wanted to point out that the value of the proposal
>>> should probably be evaluated against something roughly like the above
>>> rather
>>> than something that does a bunch of procedural string splicing.
>>
>> The whole point of the idea was to get rid of the strings and the mixin
>> expression (as it looks like to day).
>>
>
> While I'm not necessarily opposed to the idea of getting rid of the strings,
> I guess I don't really see what improvement your proposal provides other
> than not having to manually specify the mapping of "replace what identifier
> with what data".
>
> Getting rid of the "mixin" I see as a separate issue. We could just as
> easily say that given the function "string get_set(...)":
>
>      @get_set("int", "bar");
>      or
>      @get_set int bar;
>
> ...Is shorthand for, or is even the new syntax for:
>
>      mixin(get_set("int", "bar"));

That was my idea as well, that

@get_set("int", "bar");

could be translated into

mixin(get_set("int", "bar")); just like

just like scope statements are translated into try/catch/finally.

-- 
/Jacob Carlborg

On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
> On 2010-12-14 21:44, Nick Sabalausky wrote:
> > "Jacob Carlborg"<doob@me.com>  wrote in message news:ie8i8c$15f0$1@digitalmars.com...
> > 
> >> On 2010-12-14 19:13, Nick Sabalausky wrote:
> >>> "Graham St Jack"<Graham.StJack@internode.on.net>   wrote in message news:ie76ig$b2v$1@digitalmars.com...
> >>> 
> >>>> What you are suggesting here seems to be a way to dramatically reduce
> >>>> the
> >>>> complexity of code that generates source-code and mixes it in. I think
> >>>> something like that is needed before this mind-bogglingly powerful
> >>>> feature
> >>>> of D can realise its potential.
> >>> 
> >>> I think a decent string-template library could probably come very close
> >>> to
> >>> the proposal without needing any language changes at all:
> >>> 
> >>> string get_set(T, string name)()
> >>> {
> >>> 
> >>>        return
> >>>        q{
> >>> 
> >>>            @type _@name;
> >>> 
> >>>            @type @name ()
> >>>            {
> >>> 
> >>>                return _@name;
> >>> 
> >>>            }
> >>> 
> >>>            @type @name (@type @name)
> >>>            {
> >>> 
> >>>                return _@name = @name;
> >>> 
> >>>            }
> >>> 
> >>>        }.replace( ["@type": T.stringof, "@name": name] );
> >>> 
> >>> }
> >>> 
> >>> class Foo
> >>> {
> >>> 
> >>>        mixin(get_set!(int, "bar")());
> >>> 
> >>> }
> >>> 
> >>> There are definitely some things about the proposal that are better
> >>> than with this, but I just wanted to point out that the value of the
> >>> proposal should probably be evaluated against something roughly like
> >>> the above rather
> >>> than something that does a bunch of procedural string splicing.
> >> 
> >> The whole point of the idea was to get rid of the strings and the mixin expression (as it looks like to day).
> > 
> > While I'm not necessarily opposed to the idea of getting rid of the strings, I guess I don't really see what improvement your proposal provides other than not having to manually specify the mapping of "replace what identifier with what data".
> > 
> > Getting rid of the "mixin" I see as a separate issue. We could just as
> > 
> > easily say that given the function "string get_set(...)":
> >      @get_set("int", "bar");
> >      or
> >      @get_set int bar;
> > 
> > ...Is shorthand for, or is even the new syntax for:
> >      mixin(get_set("int", "bar"));
> 
> That was my idea as well, that
> 
> @get_set("int", "bar");
> 
> could be translated into
> 
> mixin(get_set("int", "bar")); just like
> 
> just like scope statements are translated into try/catch/finally.

Honestly, I don't see much gain in using @ rather than mixin(). It's a little less typing, but that's it. And it precludes stuff like mixin("lhs " ~ op ~ " rhs") like happens all the time in overloaded operator functions.

- Jonathan M Davis

"Jonathan M Davis" <jmdavisProg@gmx.com> wrote in message news:mailman.1035.1292441722.21107.digitalmars-d@puremagic.com...
> On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
>>
>> That was my idea as well, that
>>
>> @get_set("int", "bar");
>>
>> could be translated into
>>
>> mixin(get_set("int", "bar")); just like
>>
>> just like scope statements are translated into try/catch/finally.
>
> Honestly, I don't see much gain in using @ rather than mixin(). It's a
> little
> less typing, but that's it.

It does seem like a small difference, just replacing "mixin" with "@" and removing one layer of parens. But I think that extra layer of parens, minor as it seems, makes a big difference in the readability (and "typeability") of mixin invocations. Those extra parens do get to be a real bother, major visual noise at least to my eyes.

> And it precludes stuff like mixin("lhs " ~ op ~ "
> rhs") like happens all the time in overloaded operator functions.
>

I don't see why these shouldn't work:

@"int foo;";
return @("lhs " ~ op ~ " rhs");

At least with just the "@" part of the proposal. Maybe the delegate thing might make it tricker, I dunno.

On 12/15/2010 11:00 PM, Nick Sabalausky wrote:
> "Jonathan M Davis"<jmdavisProg@gmx.com>  wrote in message
> news:mailman.1035.1292441722.21107.digitalmars-d@puremagic.com...
>> On Wednesday, December 15, 2010 11:27:47 Jacob Carlborg wrote:
>>>
>>> That was my idea as well, that
>>>
>>> @get_set("int", "bar");
>>>
>>> could be translated into
>>>
>>> mixin(get_set("int", "bar")); just like
>>>
>>> just like scope statements are translated into try/catch/finally.
>>
>> Honestly, I don't see much gain in using @ rather than mixin(). It's a
>> little
>> less typing, but that's it.
>
> It does seem like a small difference, just replacing "mixin" with "@" and
> removing one layer of parens. But I think that extra layer of parens, minor
> as it seems, makes a big difference in the readability (and "typeability")
> of mixin invocations. Those extra parens do get to be a real bother, major
> visual noise at least to my eyes.
>

I agree with this. Actually, just removing the parenthesis would be a huge gain for me.

>> And it precludes stuff like mixin("lhs " ~ op ~ "
>> rhs") like happens all the time in overloaded operator functions.
>>
>
> I don't see why these shouldn't work:
>
> @"int foo;";
> return @("lhs " ~ op ~ " rhs");
>
> At least with just the "@" part of the proposal. Maybe the delegate thing
> might make it tricker, I dunno.
>

This could work, but I don't think anyone is suggesting completely replacing the mixin. I think @ could be a function/template thing, and have strings in a more explicit mixin"";

Then again, inconsistency sucks.

On 2010-12-15 00:19, Graham St Jack wrote:
>
>> I don't know, do you have an example ?
>>
>>> For example taking a classname and a bunch of field types and names, and turning it into a class definition complete with constructor from field values, constructor from an input stream, a method to write to an output stream, and const getters. Or maybe std.typecons.AutoImplement.
>>
>> Could you post an example of how that mixin would be used and the code it would generate then I can see if I can translate it to my syntax. AutoImplement seems to just contain template mixins which is something else.
>>
>
> I have attached my concurrency framework, which relies heavily on mixins, plus its unit test to show how it is used. I haven't included the various dependencies because I assume you just want the example code. Let me know if you want something buildable, or perhaps something more cut-down.
>
> What the code-generating template does is to create from something like this (you can have any number of Messages in a Protocol):
>
>
> alias Protocol!("Requests", Message!("job", string, "name")).code jobCode;
> mixin(jobCode);
>
>
> this code:
>
>
> class Requests {
> struct jobMsg {
> string name;
> this(string name) {
> this.name = name;
> }
> void read(InStream stream) {
> name = stream.get!string;
> }
> void write(OutStream stream) {
> stream(name);
> }
> }
> struct Message {
> uint kind;
> union {
> jobMsg job;
> }
> this(ref jobMsg msg) {
> kind = 0;
> job = msg;
> }
> this(InStream stream) {
> kind = stream.get!uint;
> switch(kind) {
> case 0: job.read(stream); break;
> default: assert(0, "Cannot read unsupported message kind");
> }
> }
> void write(OutStream stream) {
> stream(kind);
> switch(kind) {
> case 0: job.write(stream); break;
> default: assert(0, "Cannot write unsupported message kind");
> }
> }
> }
> private alias Channel!(Message) _Chan;
> private alias shared _Chan Chan;
>
> private Chan channel;
> this() { channel = new Chan(); }
> ChannelSelectable newSelectable() { return channel.newSelectable(); }
> void finalize() { channel.finalize; }
>
> interface IHandler {
> void job(string name);
> }
> void job(string name) {
> channel.add(Message(jobMsg(name)));
> }
> void receive(IHandler handler) {
> auto message = channel.remove;
> switch (message.kind) {
> case 0: handler.job(message.job.name); break;
> default: assert(0, "Cannot dispatch unsupported message kind");
> }
> }
> }
>
>
>
> I use this for inter-thread communications, and I use the discriminated union to pass messages between processes. The manual mixin after the alias is a debugging aid - the compiler errors when doing mixins aren't helpful at all. I case you are wondering why I did all this, it was partly a learning experience, but mostly an attempt to do something properly thread-safe (!hasAliasing), using shared, const and immutable properly.

I've attached a part of how concurrency.d could look like translated to my suggested syntax. It probably contains a lot of errors because did a quick translation and I had some trouble understanding the mixins.

-- 
/Jacob Carlborg

On 2010-12-15 08:57, Don wrote:
> Jacob Carlborg wrote:
>> On 2010-12-14 13:05, Don wrote:
>>> Graham St Jack wrote:
>>>> On 14/12/10 20:33, Vladimir Panteleev wrote:
>>>>> On Tue, 14 Dec 2010 09:30:46 +0200, Graham St Jack
>>>>> <Graham.StJack@internode.on.net> wrote:
>>>>>
>>>>>> There is of course the worry that it could get so easy that everyone
>>>>>> starts doing it, and we have (relatively) impenetrable code
>>>>>> everywhere
>>>>>> instead of just deep in the bowels of framework libraries.
>>>>>
>>>>> TBH, I'm more excited by AST macros which I understood are planned for
>>>>> D3, as mentioned here:
>>>>> http://s3.amazonaws.com/dconf2007/WalterAndrei.pdf
>>>>> They seem to promise the power of mixins, but without the mess.
>>>>>
>>>>
>>>> I took a look at the pdf, but couldn't see how the AST macros could
>>>> come
>>>> close to the kinds of things that are possible (but difficult) with
>>>> mixins.
>>>
>>> That fact was recognized at the conference, on the following day. As a
>>> result, AST macros were dropped from D2.
>>
>> Do you have an example that would work with string mixins but not with
>> AST macros?
>
> Well, it's a bit hard without a formal definition of AST macros.
> But the stuff I talked about at the conference, I have no idea how to do
> with AST macros.
>
> There's code like this, which generates an asm instruction.
>
> ------
> mixin( opToSSE[operations[done+1]] ~ suffix ~ " " ~ XMM(numOnStack-1) ~
> ", " ~ indexedSSEVector(ranklist, operations[done], vectorsize));
> -------
>
> using the functions:
> ============
> const char [][5] vectorRegister = ["ECX", "EDX", "EBX", "ESI", "EDI"];
>
> char [] indexedSSEVector(char [] ranklist, char var, char [] vecsize)
> {
> return "[" ~ vectorRegister[vectorNum(ranklist, var)] ~ " + " ~ vecsize
> ~"*EAX]";
> }
>
> char [] XMM(int k) { return "XMM"~ itoa(k); }
>
> char [][char] opToSSE() {
> return ['*':"mulp"[], '+': "addp", '-': "subp", '/': "divp"]; }
>
> int vectorNum(char [] ranklist, char var)
> {
> int numVecs=0;
> for (int i=0; i<var-'A'; ++i) {
> if (ranklist[i]=='1') ++numVecs;
> }
> return numVecs;
> }
> ============

I can't quite visualize how the final code will look like and as you say it's hard without a formal definition of AST macros. But I think somehow it would be possible, I mean instead of combining strings one combine expressions/syntax. But I don't know if it would be possible to combine incomplete expressions.

>>> They need to roughly match string mixins in power. At this stage, there
>>> is no proposal for how they should work.
>>
>> I think someone, Nick Sabalausky perhaps, suggested to have something
>> like the hygiene macros in Nemerle:
>> http://nemerle.org/wiki/index.php?title=Macros
>
>  From an implementation point of view, the differences between Nemerle
> macros and string mixins are mostly syntactic.
> The one thing about them that I find really impressive is the IDE
> integration, especially that they got syntax highlighting to work. I
> don't know they've done that.

As far as I can see the content of a macro in Nemerle is just code. But if you're referring to the syntax expression/statement which adds new syntax to the language then I agree.

-- 
/Jacob Carlborg