Andrei Alexandrescu (See Website For Email) wrote:
> kris wrote:
> 
>> Andrei Alexandrescu (See Website For Email) wrote:
>>
>>> kris wrote:
>> You're saying that a 'normal' D program is not sufficiently powerful to write other programs?  Au contraire!  There's nothing wrong with doing that at runtime, rather than turning the compiler itself into an abstract virtual machine?
> 
> 
> You don't understand. Code generation is only interesting when the generated code works together with handwritten code and lives within the same symbolic ecosystem.

Okay, I do understand that. My overall point is that such things are of  interest to only a tiny fragment of the commercial market.


> I tried to provide solid argumentation and/or evidence for my statements. The post I replied to originally, and the post I am replying to now, use rhetoric and bare statements somehow implying they are drawn from common knowledge. I will not simply agree with a bare statement claiming that this is not needed or that is not necessary.

I thought I added some evidence myself, in the paragraph below, but you seems to have missed that?

> 
>> We might as well be discussing whether the compiler should embed a GUI generator. So it can be used at compile-time. There are better ways of doing that. Ways that are more accessible, more maintainable, and have a much easier learning curve. The OSX GUI builder is one fine example.
>>
>>
>>> To survive, D must compensate for its relative lack of clout and publicity by offering above and beyond what more mainstream languages offer.
>>
>>
>> To survive, D needs to get serious about being taken seriously.
> 
> 
> Exactly what is anyone to make of this? 

Andrei, it's really no different to your claim that D must somehow "compensate for its relative lack of clout" by adding compile-time regex to the language (the topic at hand). Wouldn't you agree? D would arguably have more clout if a number of items were resolved, rather than adding more features.

Sure, I can imagine there's a lot more behind this than just Regex - but the point made elsewhere is a good one: if the DSL facilities are going to be totally awesome, one should be able to implement regex in it; efficiently.

You can do that today using templates. It's just not terribly efficient, as Walter pointed out.


> Reminds me of Jerome K. Jerome:
> "Never do something shameful, my son," the mother said, "and then
> you'll never be ashamed of what you did."


How about: "Just because you can, doesn't mean you should" ?


> 
>> Concentrating on what /might/ become a niche ideal doesn't strike me as the best approach. Compare and constrast with, say, targeting D for cell-phone devices? It's the biggest market on the planet, just sitting there /waiting/ for D to come along with the right set of features.
> 
> 
> Sure if there are abstractions of interest to embedded programs they are worth discussing. But again, leaving it at the level of Zen statements is only empty rhetoric.

Hrm ... I won't go there :)


> 
>>>> What is wrong with runtime execution anyway? It sure is easier to write and maintain clean D code than (for many ppl) complex concepts that are, what amount to, nothing more than runtime optimizations. Isn't that true?
>>>
>>>
>>>
>>> No. Accommodating DSLs and generating code has more to do with correctness and avoiding duplication of source code, than anything else.

Okay. That comes back to my original point about whether this is something that is gonna attract mainstream developers, en masse. Forgive me, but I just don't see that it does.


>> Yes and no. I will defer, of course, to your experience in the matter; but will note that there's /always/ a point of diminishing return.
> 
> 
> To this I'll insert the obligatory answer that that's a truism.
> 
>>> a "white hole" class is an implementation of A that implements all methods to throw, and a "black hole" class is an implementation of A that implements all methods to return the default value of the return type.
>>>
>>> This pattern is very useful for either quick starting points for writing true classes implementing A, or as standalone degenerate implementations.
>>>
>>> To some programmers, black and white holes might not even raise a "duplicated code" flag. They sit down and write:
>>
>>
>> That could reasonably be argued as a point of diminishing return? If it takes more effort or knowledge of how to abstract the pattern from two or more concepts, and to implement it using something unfamiliar, then 99% of developers will ignore it completely.
> 
> 
> I think any developer can write an alias statement. The point of the example was to illustrate how an advanced feature can be used by an expert to democratize efficient development.

I agree. But then how many everyday /useful/ patterns really exist in that form? Ones that ppl would actually use? Gui patterns are one of the few areas that come to mind. Componentization of software dev has long been a dream (akin to how hardware components are assembled). But the software world has never operated that way. It gets further and further away every year, as code becomes more and more throwaway.

More often than not, a 'reusable' component (perhaps written by an expert) simply does not offer what the user needs. The time it takes to understand how to 'adapt' that component is often longer than it takes to implement the required functionality directly. I don't agree with the approach, but it's what I see in both commerce and research.

It doesn't matter whether the component is hand-written or machine generated; the end result appears to be the same (IMO).


> 
> Abstraction is hard, no question about that. (Another truism. :o)) The problem is that often, even when the abstraction is understood, reflecting it in code is a complete mess.

Yes.

> 
> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that were a complete mess to deal with in concreteland.
> 

Let look at that case study, then. The /real/ power in RoR comes from being able to dynamically bind via rich reflection. What we're talking about here does not add full reflection to D. Neither does it assist in getting D modules dynamically loaded at runtime.

As it turns out, some of us are actively looking /specifically/ at the killer RoR for D; far beyond what RoR does. Oddly enough, our working name for it is - DeRailed -

We have solid notions of what's needed; and several of us have build related platforms in the past. But this topic, at face value, doesn't appear to help us in any notable fashion. Perhaps you can expain this further?



>> And I can't see how this could possibly help D get notable traction, since it can also be done using the tools already available in D: contracts via interfaces or abstract base-classes.
> 
> 
> This reflects misunderstanding of the stakes. Interfaces and abstract base classes are a recipe for _handwritten_ code. black_hole and white_hole are tools that generate code _mechanically_.

Okay. But the fundamental approach to patterns is rather similar. You were discussing facilities provided as a template-of-functionality (in a lib or somewhere). At that level it doesn't matter, to the user, whether  they were originally hand written or not. That's what drove my point, rather than a misunderstanding.

- Kris

kris wrote:
[among other things]
> Sure, I can imagine there's a lot more behind this than just Regex - but the point made elsewhere is a good one: if the DSL facilities are going to be totally awesome, one should be able to implement regex in it; efficiently.

I think this is a very valuable outcome that today's rumpus converged towards.

Andrei

Walter Bright wrote:
> Andrei Alexandrescu (See Website For Email) wrote:
>> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that
>> were a complete mess to deal with in concreteland.
> 
> I found this essay to be pivotal in piquing my interest in this:
> 
> http://www.paulgraham.com/avg.html
> 
> and a related one:
> 
> http://lib.store.yahoo.net/lib/paulgraham/bbnexcerpts.txt

Awesome. The site could have a section with "external links" pointing to these articles, as well as SICP (which is also available online at http://mitpress.mit.edu/sicp/), The View From Berkeley (http://view.eecs.berkeley.edu/wiki/Main_Page) and other texts that are important to grasp in order to understand and shape the future of D.


Andrei

Andrei Alexandrescu (See Website For Email) wrote:
> Walter Bright wrote:
>> Andrei Alexandrescu (See Website For Email) wrote:
>>> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that
>>> were a complete mess to deal with in concreteland.
>>
>> I found this essay to be pivotal in piquing my interest in this:
>>
>> http://www.paulgraham.com/avg.html
>>
>> and a related one:
>>
>> http://lib.store.yahoo.net/lib/paulgraham/bbnexcerpts.txt
> 
> Awesome. The site could have a section with "external links" pointing to these articles, as well as SICP (which is also available online at http://mitpress.mit.edu/sicp/), The View From Berkeley (http://view.eecs.berkeley.edu/wiki/Main_Page) and other texts that are important to grasp in order to understand and shape the future of D.
> 
> 
> Andrei

Videos of Abelson and Sussman lectures are also available online at MIT Courseware.  Good stuff if you have the time to watch.

--bb

Miles wrote:
> Walter Bright wrote:
>> The problem is that parsing strings using templates generates a large
>> number of template instantiations, is (relatively) very slow, and
>> consumes a lot of memory (at compile time, not runtime). For example,
>> ParseInteger would need 4 template instantiations to parse 5678,
> 
> Why instead of doing perversions with templates, don't you add a proper
> compile-time D interpreter for the purposes of generating code? Doing
> loops with template recursion sucks a lot. It really looks the wrong
> approach for the problem.
> 
> D already has static if() and tuple-foreach(). Just adding compile-time
> variables and a static for() will be great. A compile-time D interpreter
> will be awesome.

As an example of this, I recently was playing with the new mixins, and wanted to mixin an n amount of structs. I quickly realized that this would not be as trivial as

static for(int i = n; i < n; i++)
{
    mixin("my struct code");
}

but ended up being:

import std.metastrings;

template ManyStructs(int count)
{
	static if(count == 1)
	{
		const char[] ManyStructsImpl = "struct Struct" ~ ToString!(count) ~ " { int baz = "~ToString!(count)~"; }";
	}
	else
	{
		const char[] ManyStructsImpl = "struct Struct" ~ ToString!(count) ~ " { int baz = "~ToString!(count)~"; }"~ManyStructsImpl!(count - 1);
	}
}

mixin(ManyStructs!());

Clearly the former is much more succinct.

>>
>> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that were a complete mess to deal with in concreteland.
>>
> 
> Let look at that case study, then. The /real/ power in RoR comes from being able to dynamically bind via rich reflection. What we're talking about here does not add full reflection to D. Neither does it assist in getting D modules dynamically loaded at runtime.
> 
> As it turns out, some of us are actively looking /specifically/ at the killer RoR for D; far beyond what RoR does. Oddly enough, our working name for it is - DeRailed -
> 
> We have solid notions of what's needed; and several of us have build related platforms in the past. But this topic, at face value, doesn't appear to help us in any notable fashion. Perhaps you can expain this further?
> 
> - Kris

I'm having a hard time putting together the association with RoR, DSL's and the regex feature together. Perhaps they're completely separate.

RoR really doesn't express abstractions per se, Ruby does and very well actually. Actually the real power comment pertains to Ruby, instead of RoR - but I agree on it's intended meaning.

I've hacked around on a possible ActiveRecord 'wannabe'[1] from time to time over the past few weeks and must admit I constantly miss things that are just there in Ruby
-some of the things I'm aware are going to be hard to transcribe to D due to it's background - blocks
-some things that just haven't been implemented yet - true dynamic information/loading,
-somethings that just don't represent themselves well in D - symbols, *everything* is an object.

But the languages as they are are pretty close to each other (pretty painless interfacing to c code, mixins etc.

Not you directly Kris, however I've seen mentioned a few times about needing an RoR of D (an en mass application to bring in new developers.)
There are significant downsides to this situation should it happen[2] and there are issues with having such an application.

I personally came to D for a few reasons, which is probably too long to bring here, however one of the things I liked about using Ruby almost exclusively was the pure readability that came with the language by design[3]. I find D quite readable at present, I just hope it doesn't lose that edge for meta programming concepts.[4]

If I could ask for one feature.. it would be bringing the method with an array argument over to all built in types. While the built in type wouldn't be object based such as ruby's, the approach would be quite nice.

I must admit, with over 200 feeds I read throughout the day I find myself reading more and more in this set of NG's, I'm enjoying the insight, and D and its community should be proud of the community it's fostered.. it's quite nice.

And personally, IMO a compiled, c style Rebol'ish setup would be the 'one killer app', the amount of wow that little 'engine that could' does
is awesome..

Robby


[1] any central point of contact for DeRailed? I'd be willing to sling code, thoughts if there is one.
[2] http://www.oreillynet.com/ruby/blog/2005/12/ruby_is_not_a_religion.html shows the downsides to the whole utopia application issue
[3]Allowing '?' as a final character to an indentifier and the convention for representing boolean methods is a simple and pure example among others.
[4]Yeah, I've read "Beating the Averages" by Graham and I understand that the readability will come with learning.. but I'm coming from a new user point of view...

Context: I've written in Ruby for over 4 years, and have used Rails since inception so I'm not used to the compile time frame of mind (thus I'm pretty useless to a thread such as this :))

BCS wrote:
> Walter Bright wrote:
>> kenny wrote:
>>
>>> I know I'm asking for a lot, but the way templates handle string are still kinda weird to me. Would string parsing in this sort of way be absolutely impossible with templates? I have not had good luck with it. 
>>
>>
>> I just haven't thought about this enough. Certainly, however, solving the problem in a more general, D-ish way than regex would be a much bigger win. Regex works only for a subset of problems (can't do recursive descent parsing with it).
> 
> As I see it the biggest problem with compile time parsing in D is that building non linear structure is a pain. Tuples implicitly cated when passed together an this make some things really hard. Allowing a tuple to be a member of another tuple would put D template in the same class as LISP.
> 
> Another things that might make things easier is some way to mark a template as "evaluate to value and abandon". This would cause the template to be processed but none of the symbols generated by it would be kept, only the value. Of course, suitable restrictions would apply.

so it would be like writing normal D code inside of a template? Could we use phobos or do more metastring functions like find, strip, etc. need to be reinvented in D. Or can I take those functions and just put a wrapper around it? I think this is getting into the security issues again, but something like this:

auto my_text = meta trim_whitespace(import("myxml.xml")); // obviously a better keyword should be used, and can only be used in global scope too.

where trim_whitespace is an actual function that is actually defined up in the file somewhere, and will be compiled, used on the import file, then discarded and the result stored into auto my_text? A day or so ago, someone mentioned a .rc compiler. I personally would use it to parse stuff for interface elements .. to generate them dynamically. We already have libraries to parse XML (XHTML), CSS, and other config type things. It would be SUPER AWESOME to be able to just re-use them as a template without any other extra work!!! Oh man, that has me really excited, if that's possible :)

woah!

Kyle Furlong wrote:
> import std.metastrings;
> 
> template ManyStructs(int count)
> {
>     static if(count == 1)
>     {
>         const char[] ManyStructsImpl = "struct Struct" ~ ToString!(count) ~ " { int baz = "~ToString!(count)~"; }";
>     }
>     else
>     {
>         const char[] ManyStructsImpl = "struct Struct" ~ ToString!(count) ~ " { int baz = "~ToString!(count)~"; }"~ManyStructsImpl!(count - 1);
>     }
> }
> 
> mixin(ManyStructs!());
> 
> Clearly the former is much more succinct.

Ending the loop at 0 instead of 1:
---
import std.metastrings;

template ManyStructs(int count)
{
    static if(count == 0)
    {
        const char[] ManyStructsImpl = "";
    }
    else
    {
        const char[] ManyStructsImpl = "struct Struct" ~ ToString!(count) ~ " { int baz = "~ToString!(count)~"; }"~ManyStructsImpl!(count - 1);
    }
}
---
Duplicate code is bad :P.

Robby wrote:
> 
>>>
>>> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that were a complete mess to deal with in concreteland.
>>>
>>
>> Let look at that case study, then. The /real/ power in RoR comes from being able to dynamically bind via rich reflection. What we're talking about here does not add full reflection to D. Neither does it assist in getting D modules dynamically loaded at runtime.
>>
>> As it turns out, some of us are actively looking /specifically/ at the killer RoR for D; far beyond what RoR does. Oddly enough, our working name for it is - DeRailed -
>>
>> We have solid notions of what's needed; and several of us have build related platforms in the past. But this topic, at face value, doesn't appear to help us in any notable fashion. Perhaps you can expain this further?
>>
>> - Kris
> 
> 
> I'm having a hard time putting together the association with RoR, DSL's and the regex feature together. Perhaps they're completely separate.

Me too. I failed to see any connection that would measurably assist DeRailed. And the question above was sadly left unaddressed.

> 
> RoR really doesn't express abstractions per se, Ruby does and very well actually. Actually the real power comment pertains to Ruby, instead of RoR - but I agree on it's intended meaning.

You're right of course.


> I've hacked around on a possible ActiveRecord 'wannabe'[1] from time to time over the past few weeks and must admit I constantly miss things that are just there in Ruby
> -some of the things I'm aware are going to be hard to transcribe to D due to it's background - blocks

blocks can be emulated, with delegates?

> -some things that just haven't been implemented yet - true dynamic information/loading,

Absolutely. There's a number of projects currently looking into that, but 'raw' D has very little support at this time.

> -somethings that just don't represent themselves well in D - symbols, *everything* is an object.
> 
> But the languages as they are are pretty close to each other (pretty painless interfacing to c code, mixins etc.
> 
> Not you directly Kris, however I've seen mentioned a few times about needing an RoR of D (an en mass application to bring in new developers.)
> There are significant downsides to this situation should it happen[2] and there are issues with having such an application.

Yeah. We figure it's better to have it than not, and intend to address a lot of the RoR concerns bandied around the blogosphere.

> 
> I personally came to D for a few reasons, which is probably too long to bring here, however one of the things I liked about using Ruby almost exclusively was the pure readability that came with the language by design[3]. I find D quite readable at present, I just hope it doesn't lose that edge for meta programming concepts.[4]
> 
> If I could ask for one feature.. it would be bringing the method with an array argument over to all built in types. While the built in type wouldn't be object based such as ruby's, the approach would be quite nice.
> 
> I must admit, with over 200 feeds I read throughout the day I find myself reading more and more in this set of NG's, I'm enjoying the insight, and D and its community should be proud of the community it's fostered.. it's quite nice.
> 
> And personally, IMO a compiled, c style Rebol'ish setup would be the 'one killer app', the amount of wow that little 'engine that could' does
> is awesome..
> 
> Robby
> 
> 
> [1] any central point of contact for DeRailed? I'd be willing to sling code, thoughts if there is one.

Catch us via the Tango site and/or IRC? Always good to have extra pair of (willing) hands :)

> [2] http://www.oreillynet.com/ruby/blog/2005/12/ruby_is_not_a_religion.html shows the downsides to the whole utopia application issue
> [3]Allowing '?' as a final character to an indentifier and the convention for representing boolean methods is a simple and pure example among others.
> [4]Yeah, I've read "Beating the Averages" by Graham and I understand that the readability will come with learning.. but I'm coming from a new user point of view...
> 
> Context: I've written in Ruby for over 4 years, and have used Rails since inception so I'm not used to the compile time frame of mind (thus I'm pretty useless to a thread such as this :))

We'd like you on-board with DeRailed :)

kris wrote:
> Robby wrote:
>>
>>>>
>>>> Walter gave another good case study: Ruby on Rails. The success of Ruby on Rails has a lot to do with its ability to express abstractions that were a complete mess to deal with in concreteland.
>>>>
>>>
>>> Let look at that case study, then. The /real/ power in RoR comes from being able to dynamically bind via rich reflection. What we're talking about here does not add full reflection to D. Neither does it assist in getting D modules dynamically loaded at runtime.
>>>
>>> As it turns out, some of us are actively looking /specifically/ at the killer RoR for D; far beyond what RoR does. Oddly enough, our working name for it is - DeRailed -
>>>
>>> We have solid notions of what's needed; and several of us have build related platforms in the past. But this topic, at face value, doesn't appear to help us in any notable fashion. Perhaps you can expain this further?
>>>
>>> - Kris
>>
>>
>> I'm having a hard time putting together the association with RoR, DSL's and the regex feature together. Perhaps they're completely separate.
> 
> Me too. I failed to see any connection that would measurably assist DeRailed. And the question above was sadly left unaddressed.

It's very simple. A scheme based on compile-time in(tro)spection has superior and automatic means to detect, say, mismatches between an expected database schema and the runtime reality.


Andrei