Hello Adam,

> BCS wrote:
> 
>> Hello Adam,
>> 
>>> On Sat, Apr 18, 2009 at 06:10:27PM -0700, Andrei Alexandrescu wrote:
>>> 
>>>> The point of using "." is not syntactic convenience as much as the
>>>> ability of the Dynamic structure to work out of the box with
>>>> algorithms that use the standard notation.
>>>> 
>>> What if the dot remained exactly like it is now and the -> took
>>> the place of the dot in the proposal; regular method calls when
>>> possible and forwarded to opExtension (opDotExp needs a better name)
>>> when that fails?
>>> Thus your generic algorithm can use -> and work in all cases, and
>>> the
>>> dot operator remains the same as it is now.
>> Going the other way would be better; '.' works as Andrei wants and
>> '->' is an explicit, "don't use the dynamic stuff" invocation. If it
>> went the other way virtually all template code would end up needing
>> to use '->' for everything just in cases someone wants to pass in a
>> type that uses opDotExp.
>> 
> Yea and that would be bad, since then as far as I am concerned you
> have destroyed the purpose of templates. Seems to me templates and
> dynamic calls are sorta there to solve the same problem, how do you
> write code that is independent of type? Dynamic takes it to extreme
> and makes the evaluation of methods (or public types variables I don't
> see how that would be different) and leave it all to runtime which is
> far more flexible but potentially has holes if the type doesn't meet
> the requirements. Templates take a more conservative route by still
> letting you write independent of type but then using the information
> provided at call time it can still perform static code checks to make
> sure the type meets its needs, making it a little more safe than
> dynamic. However soon as you throw some dynamic calls into a template
> the guarantees usually provided go out the window since the template
> is letting that none existent member slip by.

I see your point but disagree. I see it more as a "what to do if the code author doesn't have full API knowledge?" problem. Templates allow the author of the consuming code to go with a "I'll just assume this can be done and let the compiler check it" approach and the dynamic option allows the author of the producer to go with a "Do whatever you want and if I didn't say what to do with it use this code to figure it out" approach. 

(In the above, you seeme to be working with the assumption of the non static opDotExp form. I, BTW, see no use for it as it adds no new functionality to D where as the static opDotExp(char[],T...)(T t) form adds a new ability) 

[ the rest of the post dealt with implications of non-static forms of opDotExp ]

Hello Yigal,

> On 19/04/2009 01:22, BCS wrote:
> 
>> Hello Yigal,
>> 
>>> On 18/04/2009 21:16, Andrei Alexandrescu wrote:
>>> 
>>>> In the syntax
>>>> 
>>>> a.b
>>>> 
>>>> how would either of a and b be identified at runtime? I mean, you
>>>> write the code somewhere and it gets compiled. It's not like you're
>>>> reading "a.b" from the console and then call some eval() function
>>>> against them.
>>>> 
>>>> Andrei
>>>> 
>>> what prevents D from having an eval function?
>>> suppose someone modifies the DMD front-end to compile a string with
>>> the
>>> source code of a function in-memory, than this is processed by
>>> something
>>> based on DDL and what you get is an API call that takes source code
>>> in
>>> a
>>> string and returns a function pointer.
>> Even then it is *still* going to be compile time. Just a compile time
>> running at runtime... Ooohh, my heads's going to start hearing here
>> in a bit.
>> 
> No it won't. you will call a standard library API at *runtime* with a
> *runtime* string and get back a pointer to a function that you can
> call.

That is exactly what I was thinking of and something I have wanted for some time.

But for that to work you need to *compile* the string to a function so there is still a compile time inside that function call. That library function you refer to, for the topic at hand, is no different than DMD and the normal compile process.

To give a concrete example: suppose the eval function was done by copying the string to a file, tacking in includes as needed, shelling out to DMD to make a DLL/SO and then loading that file back into the same process. In that case there would could be a compile time resolution. In what way would puling the whole thing inside the first process be different?

> in fact Some Lisp compilers implement eval() exactly like that.

The only way I can see that making a difference is if you want that eval function to be able to take code that calls function that don't exist yet and I don't even want to think about what it would take to implement that.

> you're thinking in C++ terms. try thinking in Lisp...

(BTW I may have done more Lisp/schema than C++, saying more about by C++ history than anything else.)

On 19/04/2009 23:33, BCS wrote:
> Hello Yigal,
>
>> On 19/04/2009 01:22, BCS wrote:
>>
>>> Hello Yigal,
>>>
>>>> On 18/04/2009 21:16, Andrei Alexandrescu wrote:
>>>>
>>>>> In the syntax
>>>>>
>>>>> a.b
>>>>>
>>>>> how would either of a and b be identified at runtime? I mean, you
>>>>> write the code somewhere and it gets compiled. It's not like you're
>>>>> reading "a.b" from the console and then call some eval() function
>>>>> against them.
>>>>>
>>>>> Andrei
>>>>>
>>>> what prevents D from having an eval function?
>>>> suppose someone modifies the DMD front-end to compile a string with
>>>> the
>>>> source code of a function in-memory, than this is processed by
>>>> something
>>>> based on DDL and what you get is an API call that takes source code
>>>> in
>>>> a
>>>> string and returns a function pointer.
>>> Even then it is *still* going to be compile time. Just a compile time
>>> running at runtime... Ooohh, my heads's going to start hearing here
>>> in a bit.
>>>
>> No it won't. you will call a standard library API at *runtime* with a
>> *runtime* string and get back a pointer to a function that you can
>> call.
>
> That is exactly what I was thinking of and something I have wanted for
> some time.
>
> But for that to work you need to *compile* the string to a function so
> there is still a compile time inside that function call. That library
> function you refer to, for the topic at hand, is no different than DMD
> and the normal compile process.
>
> To give a concrete example: suppose the eval function was done by
> copying the string to a file, tacking in includes as needed, shelling
> out to DMD to make a DLL/SO and then loading that file back into the
> same process. In that case there would could be a compile time
> resolution. In what way would puling the whole thing inside the first
> process be different?
>
everything you said is true. there is some sort of a compile-time since the code is getting compiled. But in the above scheme there isn't any real difference between run-time and compile-time and this distinction has lost its meaning.
compare the following:
process A:
1) use runtime dispatch based on run-time user input
2) generate source code for the above case "on the fly"
2) compile the above code and call it
process B:
1) generete templated source code at runtime
2) call run-time compiler module on the above code
3) compiler instantiates the template based on the input at *compile-time* (i.e. compile time dispatch)
4) call the templated instance

the above are identical even if implemented differently since that compile-time above is actually part of the run-time.
the only difference is the style of coding and I'd argue that the second process is unnecessarily more complex without any gains.
if nothing else than the source code generated will be shorter since you wouldn't need to stick "static" everywhere.

>> in fact Some Lisp compilers implement eval() exactly like that.
>
> The only way I can see that making a difference is if you want that eval
> function to be able to take code that calls function that don't exist
> yet and I don't even want to think about what it would take to implement
> that.
>
>> you're thinking in C++ terms. try thinking in Lisp...
>
> (BTW I may have done more Lisp/schema than C++, saying more about by C++
> history than anything else.)
>
>

Hello Yigal,

> everything you said is true. there is some sort of a compile-time
> since
> the code is getting compiled. But in the above scheme there isn't any
> real difference between run-time and compile-time and this distinction
> has lost its meaning.
> compare the following:
> process A:
> 1) use runtime dispatch based on run-time user input
> 2) generate source code for the above case "on the fly"
> 2) compile the above code and call it
> process B:
> 1) generete templated source code at runtime
> 2) call run-time compiler module on the above code
> 3) compiler instantiates the template based on the input at
> *compile-time* (i.e. compile time dispatch)
> 4) call the templated instance
> the above are identical even if implemented differently since that
> compile-time above is actually part of the run-time.
> the only difference is the style of coding and I'd argue that the
> second
> process is unnecessarily more complex without any gains.
> if nothing else than the source code generated will be shorter since
> you
> wouldn't need to stick "static" everywhere.

I think you understand my point about what is possible but I'm not sure you are spotting what I'm saying regarding what should be done. I'll try and cleanly state what I'm asserting:


a number of red hearings seem to be floating about based on cool uses for truly runtime dynamic function lookup (that we just agreed doesn't exist for the given syntax as for that case, the binding happens at sort sort of compile time). I'm calling them red herrings because those uses can be done with other functionalties that already exist.

If we throw those out, the only new functionality remanding that any of this thread has provided is allowing the following syntax:

a.b(args);

to be used when the type of 'a' doesn't define a 'b'. What a lot of debate seems to be about is if that should be translated to:

a.opDotExp("b", args);

or

a.opDotExp!("b", typeof(args))(args);

but that's an easy question (go with option 2) if you note that the transformation is always at /some sort/ of compile time. If you want this and the non-static (a.k.a. the non compile time binding (where you thinking of static like static member and not static if/assert?)) form you can implement the non-static version using existing devices and make the above a trivial shell around.

Trying to go the other way doesn't work as there is no way to make static decisions (static if or template specialization) in a purely runtime mode using runtime values.

[ot] the annoying thing about this whole thread is that I'd bet a small amount of money that I could convince anyone here of my point in about 5 minuets if I had me, them and a chalkboard in the same room. I think almost all the confusion is not because people would disagree with what I'm trying to say but because there are about 5 different and almost unconnected things people are arguing about: a sort of "A is true"/"No, B is true" where A != B and A != !B and !A != B ...

Written language has the advantage that you can be very precise and thought out but has zero real time feedback like you get in a face-to-face and an insanely long delay for what feedback it does have.

BCS wrote:

> Hello Adam,
> 
>> BCS wrote:
>> 
>>> Hello Adam,
>>> 
>>>> On Sat, Apr 18, 2009 at 06:10:27PM -0700, Andrei Alexandrescu wrote:
>>>> 
>>>>> The point of using "." is not syntactic convenience as much as the ability of the Dynamic structure to work out of the box with algorithms that use the standard notation.
>>>>> 
>>>> What if the dot remained exactly like it is now and the -> took
>>>> the place of the dot in the proposal; regular method calls when
>>>> possible and forwarded to opExtension (opDotExp needs a better name)
>>>> when that fails?
>>>> Thus your generic algorithm can use -> and work in all cases, and
>>>> the
>>>> dot operator remains the same as it is now.
>>> Going the other way would be better; '.' works as Andrei wants and '->' is an explicit, "don't use the dynamic stuff" invocation. If it went the other way virtually all template code would end up needing to use '->' for everything just in cases someone wants to pass in a type that uses opDotExp.
>>> 
>> Yea and that would be bad, since then as far as I am concerned you have destroyed the purpose of templates. Seems to me templates and dynamic calls are sorta there to solve the same problem, how do you write code that is independent of type? Dynamic takes it to extreme and makes the evaluation of methods (or public types variables I don't see how that would be different) and leave it all to runtime which is far more flexible but potentially has holes if the type doesn't meet the requirements. Templates take a more conservative route by still letting you write independent of type but then using the information provided at call time it can still perform static code checks to make sure the type meets its needs, making it a little more safe than dynamic. However soon as you throw some dynamic calls into a template the guarantees usually provided go out the window since the template is letting that none existent member slip by.
> 
> I see your point but disagree. I see it more as a "what to do if the code author doesn't have full API knowledge?" problem. Templates allow the author of the consuming code to go with a "I'll just assume this can be done and let the compiler check it" approach and the dynamic option allows the author of the producer to go with a "Do whatever you want and if I didn't say what to do with it use this code to figure it out" approach.
That's essentially what I was trying to say. What I was adding is that when you add the dynamic calls inside the template you lose some of the compile- time checking (and as mentioned in another post how would you use interface constraint with that?) and therefore your template ends up relying on "Do whatever you want and if I didn't say what to do with it use this code to figure it out" for some portions of the code, so you get a weird hybrid that has stepped on templates more than it has dynamic.
> (In the above, you seeme to be working with the assumption of the non
> static opDotExp form. I, BTW, see no use for it as it adds no new
> functionality to D where as the static opDotExp(char[],T...)(T t) form
> adds a new ability)
When you say static opDotExp I am assuming you are talking about the example where someone writes a "struct Wrapper(T)" in just a few lines of code to wrap a type and add logging (I been bit busy to read rest of the threads so I only been replying to stuff following the trail leading up to my post for the most part, till now)? If so then yea that does look nice and seems like something I would use. It also doesn't contain the holes my proposal is attempting to solve since the function name is still evaluated at compile time, so removing 'open' from 'ServerProxy' wouldn't magically turn into a runtime error as the static assert will kick in.

I also agree that the none static version of opDotExp doesn't bring new functionality to the table that is worth having. Personally in them cases I would prefer to use dispatch(method, params) directly, it feels less obscure, non-static opDotExp just seems like a bit of syntax sugar with more issues than it is worth. My concern was if that functionality was implemented then using '.' for the dynamic runtime calls means is it would cause more problems than it is worth, so my proposal isn't a proposal to add the feature (since I doubt I would use it) it's one that says if you are going to then I think this method of implementing it is less intrusive.
> [ the rest of the post dealt with implications of non-static forms of [ opDotExp
I am sure I have seen some people still arguing for it (but this thread is so big with lots of sub-threads I am a little confused as to whether people are or whether they have settled to just non-static. Also for me the use of the word dynamic from the beginning and the subject of the thread meant runtime so I wouldn't really consider static opDotExp dynamic, just clever compile-time trickery :-P so talk about that has gone off topic from the subject imo) and if that is the case then the proposal still stands as a method of implementing it. If however people are not arguing for it and have settled on static opDotExp then I quite gladdly remove the proposal :-).

Adam

Andrei Alexandrescu wrote:
> Michel Fortin wrote:
>> On 2009-04-18 22:21:50 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
>>
>>> I did, but sorry, it doesn't make sense and does nothing but continue the terrible confusion going in this thread.
>>
>> Then let's try to remove some of that confusion.
> 
> Thanks for doing so. Given that my wits are spent with regard to this thread, I am replying out of respect for the considerable effort you have put in writing an explanation.
> 
> I agree that method names swallowed by opDotExp will not be exposed to any reflection mechanism and that that can be a drawback. But sometimes you don't even want to exhaustively generate all method names. In the Pascalize example, you wouldn't want to generate all case combinations of your methods. Also in the swizzle example, maybe you don't want to compulsively expose all of the xyzw combinations.
> 
> You completely lost me about the necessity of a standardized catch-all function. My view is that if you want to forward to someone else, you just call the runtime invoke() for the guy you want to forward to. So I see exactly zero need for a standardized catch-all, and a red herring in the discussion about the syntactic help brought about by opDotExp.

The utility is when you are looking for methods to invoke via runtime reflection, you can determine that a given function is one of these runtime opDotExp equivalents.

I don't know how useful this is, in point of fact. Unless there's a standard system in D for integrating with scripting languages, anyway.

Hello Adam,

> BCS wrote:
> 
>> (In the above, you seeme to be working with the assumption of the non
>> static opDotExp form. I, BTW, see no use for it as it adds no new
>> functionality to D where as the static opDotExp(char[],T...)(T t)
>> form adds a new ability)
>> 
> When you say static opDotExp I am assuming you are talking about the
> example where someone writes a "struct Wrapper(T)" in just a few lines
> of code to wrap a type and add logging (I been bit busy to read rest
> of the threads so I only been replying to stuff following the trail
> leading up to my post for the most part, till now)? If so then yea
> that does look nice and seems like something I would use. It also
> doesn't contain the holes my proposal is attempting to solve since the
> function name is still evaluated at compile time, so removing 'open'
> from 'ServerProxy' wouldn't magically turn into a runtime error as the
> static assert will kick in.

OK, it seems that we agree on the one point I'm still interested in running with. I must have been reading the rest of that from the wrong viewpoint. I'll leave it at that.

Christopher Wright wrote:
> Andrei Alexandrescu wrote:
>> Michel Fortin wrote:
>>> On 2009-04-18 22:21:50 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
>>>
>>>> I did, but sorry, it doesn't make sense and does nothing but continue the terrible confusion going in this thread.
>>>
>>> Then let's try to remove some of that confusion.
>>
>> Thanks for doing so. Given that my wits are spent with regard to this thread, I am replying out of respect for the considerable effort you have put in writing an explanation.
>>
>> I agree that method names swallowed by opDotExp will not be exposed to any reflection mechanism and that that can be a drawback. But sometimes you don't even want to exhaustively generate all method names. In the Pascalize example, you wouldn't want to generate all case combinations of your methods. Also in the swizzle example, maybe you don't want to compulsively expose all of the xyzw combinations.
>>
>> You completely lost me about the necessity of a standardized catch-all function. My view is that if you want to forward to someone else, you just call the runtime invoke() for the guy you want to forward to. So I see exactly zero need for a standardized catch-all, and a red herring in the discussion about the syntactic help brought about by opDotExp.
> 
> The utility is when you are looking for methods to invoke via runtime reflection, you can determine that a given function is one of these runtime opDotExp equivalents.
> 
> I don't know how useful this is, in point of fact. Unless there's a standard system in D for integrating with scripting languages, anyway.

Oddly enough, I got it now :o).

Andrei

Andrei Alexandrescu wrote:

> Christopher Wright wrote:
> 
>> Andrei Alexandrescu wrote:
>> 
>>> You completely lost me about the necessity of a standardized catch-all function. My view is that if you want to forward to someone else, you just call the runtime invoke() for the guy you want to forward to. So I see exactly zero need for a standardized catch-all, and a red herring in the discussion about the syntactic help brought about by opDotExp.
>> 
>> The utility is when you are looking for methods to invoke via runtime reflection, you can determine that a given function is one of these runtime opDotExp equivalents.
>> 
>> I don't know how useful this is, in point of fact. Unless there's a standard system in D for integrating with scripting languages, anyway.
> 
> Oddly enough, I got it now :o).

Thanks Cristopher. :-)

Another interesting use of this, beside integration with scripting languages, would be to replicate the concept of distributed objects in Cocoa.
<http://developer.apple.com/DOCUMENTATION/Cocoa/Conceptual/DistrObjects/Concepts/architecture.html>

-- 


Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Sun, 19 Apr 2009 10:42:19 -0400, Denis Koroskin <2korden@gmail.com> wrote:

> On Sun, 19 Apr 2009 18:26:11 +0400, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
>
>> On Sun, 19 Apr 2009 06:26:57 -0400, Denis Koroskin <2korden@gmail.com> wrote:
>>
>>> On Sun, 19 Apr 2009 05:40:32 +0400, Steven Schveighoffer <schveiguy@yahoo.com> wrote:
>>>
>>>> On Sat, 18 Apr 2009 21:10:27 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> wrote:
>>>>
>>>>> Adam Burton wrote:
>>>>>> Andrei Alexandrescu wrote:
>>>>>>>> What about using something like '->' for dynamic calls instead of '.'?
>>>>>>> That's absolutely useless. If I have to write anything different from
>>>>>>> "." I might as well write "bloodyMaryBloodyMaryBloodyMary".
>>>>>>>
>>>>>>> Andrei
>>>>>> You could even write 'noodles' but that doesn't really give me a reason as to why it's absolutely useless. Please clarify, I thought it seemed like a reasonable idea, if it isn't I would like to know why.
>>>>>
>>>>> I apologize for the snapping. There's no excuse really, but let me mention that this thread has been particularly meandering.
>>>>>
>>>>> The point of using "." is not syntactic convenience as much as the ability of the Dynamic structure to work out of the box with algorithms that use the standard notation.
>>>>
>>>> Hm... the thought just occurred to me.
>>>>
>>>> At what time are you going to use opDotExp so an entity be used in an algorithm rather than actually defining the functions directly?  For example, if you want to make a class/struct a range, why not just define the functions directly?  It seems odd to define them using opDotExp.
>>>>
>>>
>>> Variant variantRange = someRange();
>>> foreach (element; variantRange) {
>>>     // ...
>>> }
>>>
>>> Variant forwards all the front/back/etc methods to an underlying range.
>>
>> Doesn't the current opDot solution do this?  Forwarding all calls to a certain member is not a really compelling argument for changing opDot to allow dynamic method names.
>>
>> -Steve
>
> opDot is reprecated (use alias this instead) and will be eventually removed.
>
> But "alias this" is quite unsafe because it exposes the inner data (which is supposed to be hidden in some case, think of reference counting wrapper - you never want to give raw access to an underlying data).
>
> Besides, there are is a use-cases that you can't implement in a way other than opDotExp. For example, add some logging or profiling:
>
> struct Profile(T)
> {
>     private T inner;
>
>     auto opDotExp(string name, T...)(T args)
>     {
>         profiler.enter(name);
>         scope(exit) profiler.leave();
>
>         return <forward a call to 'inner' - how one would do this?>(args);
>     }
> }
>

Yes, there are many things that opDotExp can do that opDot or alias this (which is essentially opDot without any code).  Hooking every function call on a type seems to be one of the two killer use cases of this feature (the other being defining a large range of functions from which only a small number need to exist).  But call forwarding seems not to be one of them.  There are better ways to simply forward a call (such as in your variant example).

I'm pretty convinced that this is a useful feature, I still have qualms about how it's really easy to define a runtime black hole where the compiler happily compiles empty functions that do nothing instead of complaining about calling a function that does not exist.

Also, I don't think the requirement for this feature needs to be for the arguments to be templated, it should be sufficient to have a single string template argument.  This way, you can overload opDotExp functions via argument lists.

And BTW, the answer to your question above I think:

mixin("return inner."~name~"(args)");

Andrei demonstrated this usage in his Pascalize example.

-Steve