I am currently working on a program to manipulate OMF import libraries.

It turns out that OMF files consist of a series of records, and as
a result, a class that handles OMF files sort of resembles a database.

I decided to use one of D's best features, delegates, to add a member whose function is analogous to SQL's "SELECT" statement. Impossible implement in C and C++, it is declared as follows:

OMF_Record[] select_records(bool delegate(OMF_Record rec) where_clause) ;

I thought I could call this method as follows:

OMF_Record[] results = library.select_records({ return (rec.type == LIB_HEADER); });

It looks something like an SQL query, and it should do something similar as well.

But it turns out that a delegate literal's type defaults to a delegate
that takes no arguments and returns an int. As a result, the above statement
does not compile, for two reasons: Wrong argument type, and 'rec' isn't
defined in that scope.

Since D does not implicitly give the delegate the correct type, it takes a ton of keywords to make the delegate have the desired type. The result is this:

OMF_Record[] results = library.select_records(delegate(OMF_Record rec) { return (rec.record_type == LIB_HEADER);});

Hideous, no matter where you place the tabs and line breaks. It is now several orders of magnitude more complicated than an SQL query, and it duplicates information: The type, name, and number of arguments that the delegate must have will be repeated every time select_records is used.

Also, it can be noticed that D actually does infer the return type of the delegate (using 'bool' in the above expression causes a compile error), but it still can't infer the number of arguments or their types.

So I tried to use D's "alias" keyword to make some syntactic sugar:

alias delegate(OMF_Record rec) where_t;

This gives an error to the effect that a basic type is required where the word "delegate" is. This compiles:

alias bool delegate(OMF_Record rec) where_t;

But the resulting definition of where_t cannot be put in front of the delegate literal (it can be used in the declaration of the member function, however).

I tried using the following mixin:

template selrec() {
	const char[] selrec = "delegate(OMF_Record rec)";
}

But this produced an error, and the invocation of the mixin had twice as many parentheses as the expression it was intended to replace.

I finally tried to use D's "lazy evaluation" feature, but I didn't really expect it to work because it uses delegates underneath it all:

class OMF {
 /* ... */

 // Change the declaration of the member
 // function...  It just happens to be that the
 // foreach loop inside this function contains
 // a variable called 'rec' that it was passing
 // to the delegate as its 'rec' argument.

  OMF_Record[] select_records(lazy bool where_clause) ;
 /* ... */
}

/* ... */

// Now let's use it:

 OMF_Record results = library.select_records(rec.type == LIB_HEADER);

But as it turns out (as I expected, which was why I tried this last), D's "lazy" evaluation isn't as lazy as it would be if this was lazy evaluation in an interpreted language. "rec" comes from the scope of the calling function, not the scope of the foreach loop inside select_records where the expression will be evaluated.

The only way I can get the result that I want is to subject the beautiful D programmming language to the ugly C preprocessor. This enables me to write this:

#define select_records(x) _select_records(delegate(OMF_Record rec) x)

...which would make it LOOK like the delegate implicitly has the type of the argument (which was what I initially assumed, and what would be better). This technique would bring with it all the perils of the C preprocessor, and even threatens to wipe out some of the benefits of D. If I released such a class as open-source software, its users would have to subject _their_ programs to the CPP to be able to use the class.

Fortunately (or not), I cannot abuse D in this way using DM's C preprocessor, because the preprocessor seems to be built into the C compiler somehow (as opposed to GCC, which has an external CPP that can be called separately).

But D is still in its infancy. Over the years, future D programmers will run into many duplicates of this problem, and they may eventually cope with it by introducing an external C preprocessor to the mix.

-- 
Delete all files?
<Y>es, <S>ure, <A>bsolutely, <W>hy not :

Hmm.... isn't this what the new mixins are for?

Example (just to fit in with your example, you'll have to adjust):

------
struct OMF_Record
{
	int record_type;
}

struct library
{
static:
	OMF_Record[] select_records (char[] clause) ()
	{
		OMF_Record[] results;

		OMF_Record rec;
		if (mixin(clause))
			results ~= rec;

		return results;
	}
}

int main()
{
	OMF_Record[] records = library.select_records!("rec.record_type == 1");

	return 0;
}
-----

That's what you want, right?  I mean, the ! isn't too bad is it?

-[Unknown]



> I am currently working on a program to manipulate OMF import libraries.
> 
> It turns out that OMF files consist of a series of records, and as
> a result, a class that handles OMF files sort of resembles a database.
> 
> I decided to use one of D's best features, delegates, to add a member
> whose function is analogous to SQL's "SELECT" statement. Impossible
> implement in C and C++, it is declared as follows: 
> 
> OMF_Record[] select_records(bool delegate(OMF_Record rec) where_clause) ;
> 
> I thought I could call this method as follows:
> 
> OMF_Record[] results = library.select_records({ return (rec.type == LIB_HEADER); });
> 
> It looks something like an SQL query, and it should do something similar as
> well.
> 
> But it turns out that a delegate literal's type defaults to a delegate
> that takes no arguments and returns an int. As a result, the above statement
> does not compile, for two reasons: Wrong argument type, and 'rec' isn't
> defined in that scope. 
> 
> Since D does not implicitly give the delegate the correct type,
> it takes a ton of keywords to make the delegate have the desired
> type. The result is this:
> 
> OMF_Record[] results = library.select_records(delegate(OMF_Record rec) { return (rec.record_type == LIB_HEADER);});
> 
> Hideous, no matter where you place the tabs and line breaks. It is now
> several orders of magnitude more complicated than an SQL query, and
> it duplicates information: The type, name, and number of arguments
> that the delegate must have will be repeated every time select_records
> is used.
> 
> Also, it can be noticed that D actually does infer the return type of the
> delegate (using 'bool' in the above expression causes a compile error),
> but it still can't infer the number of arguments or their types.
> 
> So I tried to use D's "alias" keyword to make some syntactic sugar:
> 
> alias delegate(OMF_Record rec) where_t;
> 
> This gives an error to the effect that a basic type is required where
> the word "delegate" is. This compiles:
> 
> alias bool delegate(OMF_Record rec) where_t;
> 
> But the resulting definition of where_t cannot be put in front of the
> delegate literal (it can be used in the declaration of the member function,
> however).
> 
> I tried using the following mixin:
> 
> template selrec() {
> 	const char[] selrec = "delegate(OMF_Record rec)";
> }
> 
> But this produced an error, and the invocation of the mixin had
> twice as many parentheses as the expression it was intended to
> replace.
> 
> I finally tried to use D's "lazy evaluation" feature, but I didn't
> really expect it to work because it uses delegates underneath it
> all:
> 
> class OMF {
>  /* ... */
> 
>  // Change the declaration of the member
>  // function...  It just happens to be that the
>  // foreach loop inside this function contains
>  // a variable called 'rec' that it was passing
>  // to the delegate as its 'rec' argument.
> 
>   OMF_Record[] select_records(lazy bool where_clause) ;
>  /* ... */
> }
> 
> /* ... */
> 
> // Now let's use it:
> 
>  OMF_Record results = library.select_records(rec.type == LIB_HEADER);
> 
> But as it turns out (as I expected, which was why I tried this last),
> D's "lazy" evaluation isn't as lazy as it would be if this was lazy
> evaluation in an interpreted language. "rec" comes from the scope of the
> calling function, not the scope of the foreach loop inside select_records
> where the expression will be evaluated.
> 
> The only way I can get the result that I want is to subject the beautiful
> D programmming language to the ugly C preprocessor. This enables me to
> write this:
> 
> #define select_records(x) _select_records(delegate(OMF_Record rec) x)
> 
> ...which would make it LOOK like the delegate implicitly has the type
> of the argument (which was what I initially assumed, and what would
> be better). This technique would bring with it all the perils of the
> C preprocessor, and even threatens to wipe out some of the benefits
> of D. If I released such a class as open-source software, its users
> would have to subject _their_ programs to the CPP to be able to use
> the class.
> 
> Fortunately (or not), I cannot abuse D in this way using DM's C
> preprocessor, because the preprocessor seems to be built into
> the C compiler somehow (as opposed to GCC, which has an external
> CPP that can be called separately).
> 
> But D is still in its infancy. Over the years, future D programmers
> will run into many duplicates of this problem, and they may eventually
> cope with it by introducing an external C preprocessor to the mix.
> 

nobody@nowhere.nonet Wrote:

> Since D does not implicitly give the delegate the correct type, it takes a ton of keywords to make the delegate have the desired type. The result is this:
> 
> OMF_Record[] results = library.select_records(delegate(OMF_Record rec) { return (rec.record_type == LIB_HEADER);});
> 
> Hideous, no matter where you place the tabs and line breaks.

It's not as sugary sweet as you may want, but you can leave off the 'delegate' keyword.

OMF_Record[] results = library.select_records((OMF_Record rec) { return (rec.record_type == LIB_HEADER);});

Unknown W. Brackets <unknown@simplemachines.org> spewed this unto the Network:
> Hmm.... isn't this what the new mixins are for?
> 
> Example (just to fit in with your example, you'll have to adjust):
> 
> ------
> struct OMF_Record
> {
> 	int record_type;
> }
> 
> struct library
> {
> static:
> 	OMF_Record[] select_records (char[] clause) ()
> 	{
> 		OMF_Record[] results;
> 
> 		OMF_Record rec;
> 		if (mixin(clause))
> 			results ~= rec;
> 
> 		return results;
> 	}
> }
> 
> int main()
> {
> 	OMF_Record[] records = library.select_records!("rec.record_type == 1");
> 
> 	return 0;
> }
> -----
> 
> That's what you want, right?  I mean, the ! isn't too bad is it?
> 
> -[Unknown]

Well, that isn't too bad. But why does it work?  I don't see the word "template" anywhere, but at this website:

	http://www.digitalmars.com/d/mixin.html

It never says "mixin" without saying "template" first. I was under the impression that mixins only operated on templates.

I've also never seen the ! away from the word "mixin". What does the "!" mean?

And why does select_records()() get an extra set of parentheses now?

Haha, sorry.  It's a shortcut syntax for templates.  Here's a few clearer examples:

bool in_array (T) (T item, T[] array);
T max (T) (T[] array);

Essentially, it's like creating a template, with the parameters listed after the function first, with a function inside that has the second list of parameters and same return type.

I prefer them because it makes namespacing cleaner, usually.  Also, in the above examples, you can leave off the ! because it will guess based on parameters... meaning you could do:

int[] example;
int i = max(example);

In this case, it does mean that the clause cannot be dynamic, as it can in SQL (it must be known at compile time.)  For that, you'd have to introduce parameters of some sort, e.g.:

OMF_Record[] select_records (char[] clause) (box[] parameters...);

library.select_records!("rec.record_type == unbox!(int)(parameters[1])")(1);

I'm not sure if the above is exactly right off hand, but you get the idea.  That isn't quite as clean (although you could do more advanced parsing of the string at compile time to make it better.)

The ! is how you create a template.  In this case, I'm abusing the fact that you can leave parens off the function call.

Does that make more sense?

-[Unknown]


> Unknown W. Brackets <unknown@simplemachines.org> spewed this unto the Network: 
>> Hmm.... isn't this what the new mixins are for?
>>
>> Example (just to fit in with your example, you'll have to adjust):
>>
>> ------
>> struct OMF_Record
>> {
>> 	int record_type;
>> }
>>
>> struct library
>> {
>> static:
>> 	OMF_Record[] select_records (char[] clause) ()
>> 	{
>> 		OMF_Record[] results;
>>
>> 		OMF_Record rec;
>> 		if (mixin(clause))
>> 			results ~= rec;
>>
>> 		return results;
>> 	}
>> }
>>
>> int main()
>> {
>> 	OMF_Record[] records = library.select_records!("rec.record_type == 1");
>>
>> 	return 0;
>> }
>> -----
>>
>> That's what you want, right?  I mean, the ! isn't too bad is it?
>>
>> -[Unknown]
> 
> Well, that isn't too bad. But why does it work?  I don't see the word
> "template" anywhere, but at this website:
> 
> 	http://www.digitalmars.com/d/mixin.html
> 
> It never says "mixin" without saying "template" first. I was under
> the impression that mixins only operated on templates.
> 
> I've also never seen the ! away from the word "mixin". What does the
> "!" mean?
> 
> And why does select_records()() get an extra set of parentheses now?

nobody@nowhere.nonet wrote:
> Unknown W. Brackets <unknown@simplemachines.org> spewed this unto the Network: 
>> Hmm.... isn't this what the new mixins are for?
>>
>> Example (just to fit in with your example, you'll have to adjust):
>>
>> ------
>> struct OMF_Record
>> {
>> 	int record_type;
>> }
>>
>> struct library
>> {
>> static:
>> 	OMF_Record[] select_records (char[] clause) ()
>> 	{
>> 		OMF_Record[] results;
>>
>> 		OMF_Record rec;
>> 		if (mixin(clause))
>> 			results ~= rec;
>>
>> 		return results;
>> 	}
>> }
>>
>> int main()
>> {
>> 	OMF_Record[] records = library.select_records!("rec.record_type == 1");
>>
>> 	return 0;
>> }
>> -----
>>
>> That's what you want, right?  I mean, the ! isn't too bad is it?
>>
>> -[Unknown]
> 
> Well, that isn't too bad. But why does it work?  I don't see the word
> "template" anywhere, but at this website:
> 
> 	http://www.digitalmars.com/d/mixin.html
> 
> It never says "mixin" without saying "template" first. I was under
> the impression that mixins only operated on templates.
> 

D has two distinct kinds of mixins: template mixins and the newer string literal mixins. The keyword index can help disambiguate them:

http://www.prowiki.org/wiki4d/wiki.cgi?LanguageSpecification/KeywordIndex

The spec very deliberately uses "template mixin" to unambiguously refer to one particular kind of mixin. The above code uses the other kind.

> I've also never seen the ! away from the word "mixin". What does the
> "!" mean?
> 
> And why does select_records()() get an extra set of parentheses now?

The ! denotes a template instantiation. In the above, select_records is a function template. It has one template argument (char[] clause) and no function arguments. We can elide the trailing () when calling it because of D's property syntax.

The "clause" argument must be a template argument, since string mixins can only operate on compile-time strings.

-- 
Kirk McDonald
http://kirkmcdonald.blogspot.com
Pyd: Connecting D and Python
http://pyd.dsource.org

Unknown W. Brackets <unknown@simplemachines.org> spewed this unto the Network:
> Haha, sorry.  It's a shortcut syntax for templates.  Here's a few clearer examples:
> 
> bool in_array (T) (T item, T[] array);
> T max (T) (T[] array);
> 
> Essentially, it's like creating a template, with the parameters listed after the function first, with a function inside that has the second list of parameters and same return type.
> 
> I prefer them because it makes namespacing cleaner, usually.  Also, in the above examples, you can leave off the ! because it will guess based on parameters... meaning you could do:
> 
> int[] example;
> int i = max(example);
> 
> In this case, it does mean that the clause cannot be dynamic, as it can in SQL (it must be known at compile time.)  For that, you'd have to introduce parameters of some sort, e.g.:
> 
> OMF_Record[] select_records (char[] clause) (box[] parameters...);
> 
> library.select_records!("rec.record_type == unbox!(int)(parameters[1])")(1);
> 
> I'm not sure if the above is exactly right off hand, but you get the idea.  That isn't quite as clean (although you could do more advanced parsing of the string at compile time to make it better.)
> 
> The ! is how you create a template.  In this case, I'm abusing the fact that you can leave parens off the function call.
> 
> Does that make more sense?
> 

It makes lots of sense now. Thanks.

Reply to Justin,

> nobody@nowhere.nonet Wrote:
> 
> It's not as sugary sweet as you may want, but you can leave off the
> 'delegate' keyword.
> 
> OMF_Record[] results = library.select_records((OMF_Record rec) {
> return (rec.record_type == LIB_HEADER);});
> 
I can't think of any reason that the above shouldn't be reducable to

auto results = lib.select_records((rec){return (rec.record_type == LIB_HEADER);});

as long as there is only one function that take a delegate that takes one argument, the type of the argument could be inferred.

Scary!

But probably just because

  - it's puts one in awe with it's power, right up front
  - it offer amazing economy of expression for a compiled C-family
    language
  - the documentation is scarce and scattered and obviously not
    written in an educational spirit :-(

It shouldn't be scary because it's _not_

  - intractable
  - rocket science

and it doesn't require "inhumane mental acrobatics", like the "hard-core template jujutsu" that's only accessible to the few of us. (That's not to say Walter should remove it!!)


Now, it might do D a lot of good if some of you guys got together and wrote one to two web pages, explaining things like

  - using templates with IFTI
  - combining delegates with both
  - maybe some other things related to this

If this gets well written (as in good textbook style) with good examples, then I believe most of us would happily end up using this stuff in regular code.

georg


BCS wrote:
> Reply to Justin,
>
>> nobody@nowhere.nonet Wrote:
>>
>> It's not as sugary sweet as you may want, but you can leave off the
>> 'delegate' keyword.
>>
>> OMF_Record[] results = library.select_records((OMF_Record rec) {
>> return (rec.record_type == LIB_HEADER);});
>>
> I can't think of any reason that the above shouldn't be reducable to
>
> auto results = lib.select_records((rec){return (rec.record_type ==
> LIB_HEADER);});
>
> as long as there is only one function that take a delegate that takes
> one argument, the type of the argument could be inferred.


Unknown W. Brackets wrote:
> Haha, sorry.  It's a shortcut syntax for templates.  Here's a few clearer examples:
> 
> bool in_array (T) (T item, T[] array);
> T max (T) (T[] array);
> 
> Essentially, it's like creating a template, with the parameters listed after the function first, with a function inside that has the second list of parameters and same return type.
> 
> I prefer them because it makes namespacing cleaner, usually.  Also, in the above examples, you can leave off the ! because it will guess based on parameters... meaning you could do:
> 
> int[] example;
> int i = max(example);
> 
> In this case, it does mean that the clause cannot be dynamic, as it can in SQL (it must be known at compile time.)  For that, you'd have to introduce parameters of some sort, e.g.:
> 
> OMF_Record[] select_records (char[] clause) (box[] parameters...);
> 
> library.select_records!("rec.record_type == unbox!(int)(parameters[1])")(1);
> 
> I'm not sure if the above is exactly right off hand, but you get the idea.  That isn't quite as clean (although you could do more advanced parsing of the string at compile time to make it better.)
> 
> The ! is how you create a template.  In this case, I'm abusing the fact that you can leave parens off the function call.
> 
> Does that make more sense?
> 
> -[Unknown]
> 
> 
>> Unknown W. Brackets <unknown@simplemachines.org> spewed this unto the Network:
>>
>>> Hmm.... isn't this what the new mixins are for?
>>>
>>> Example (just to fit in with your example, you'll have to adjust):
>>>
>>> ------
>>> struct OMF_Record
>>> {
>>>     int record_type;
>>> }
>>>
>>> struct library
>>> {
>>> static:
>>>     OMF_Record[] select_records (char[] clause) ()
>>>     {
>>>         OMF_Record[] results;
>>>
>>>         OMF_Record rec;
>>>         if (mixin(clause))
>>>             results ~= rec;
>>>
>>>         return results;
>>>     }
>>> }
>>>
>>> int main()
>>> {
>>>     OMF_Record[] records = library.select_records!("rec.record_type == 1");
>>>
>>>     return 0;
>>> }
>>> -----
>>>
>>> That's what you want, right?  I mean, the ! isn't too bad is it?
>>>
>>> -[Unknown]
>>
>>
>> Well, that isn't too bad. But why does it work?  I don't see the word
>> "template" anywhere, but at this website:
>>
>>     http://www.digitalmars.com/d/mixin.html
>>
>> It never says "mixin" without saying "template" first. I was under
>> the impression that mixins only operated on templates.
>>
>> I've also never seen the ! away from the word "mixin". What does the
>> "!" mean?
>>
>> And why does select_records()() get an extra set of parentheses now?

nobody@nowhere.nonet wrote:
> I am currently working on a program to manipulate OMF import libraries.
> 
> It turns out that OMF files consist of a series of records, and as
> a result, a class that handles OMF files sort of resembles a database.
> 
> I decided to use one of D's best features, delegates, to add a member
> whose function is analogous to SQL's "SELECT" statement. Impossible
> implement in C and C++, it is declared as follows: 
> 
> OMF_Record[] select_records(bool delegate(OMF_Record rec) where_clause) ;
> 
> I thought I could call this method as follows:
> 
> OMF_Record[] results = library.select_records({ return (rec.type == LIB_HEADER); });
> 
> It looks something like an SQL query, and it should do something similar as
> well.
> 
> But it turns out that a delegate literal's type defaults to a delegate
> that takes no arguments and returns an int. As a result, the above statement
> does not compile, for two reasons: Wrong argument type, and 'rec' isn't
> defined in that scope. 
> 
> Since D does not implicitly give the delegate the correct type,
> it takes a ton of keywords to make the delegate have the desired
> type. The result is this:
> 
> OMF_Record[] results = library.select_records(delegate(OMF_Record rec) { return (rec.record_type == LIB_HEADER);});
> 
> Hideous, no matter where you place the tabs and line breaks. It is now
> several orders of magnitude more complicated than an SQL query, and
> it duplicates information: The type, name, and number of arguments
> that the delegate must have will be repeated every time select_records
> is used.
> 
> Also, it can be noticed that D actually does infer the return type of the
> delegate (using 'bool' in the above expression causes a compile error),
> but it still can't infer the number of arguments or their types.
> 
> So I tried to use D's "alias" keyword to make some syntactic sugar:
> 
> alias delegate(OMF_Record rec) where_t;
> 
> This gives an error to the effect that a basic type is required where
> the word "delegate" is. This compiles:
> 
> alias bool delegate(OMF_Record rec) where_t;
> 
> But the resulting definition of where_t cannot be put in front of the
> delegate literal (it can be used in the declaration of the member function,
> however).
> 
> I tried using the following mixin:
> 
> template selrec() {
> 	const char[] selrec = "delegate(OMF_Record rec)";
> }
> 
> But this produced an error, and the invocation of the mixin had
> twice as many parentheses as the expression it was intended to
> replace.
> 
> I finally tried to use D's "lazy evaluation" feature, but I didn't
> really expect it to work because it uses delegates underneath it
> all:
> 
> class OMF {
>  /* ... */
> 
>  // Change the declaration of the member
>  // function...  It just happens to be that the
>  // foreach loop inside this function contains
>  // a variable called 'rec' that it was passing
>  // to the delegate as its 'rec' argument.
> 
>   OMF_Record[] select_records(lazy bool where_clause) ;
>  /* ... */
> }
> 
> /* ... */
> 
> // Now let's use it:
> 
>  OMF_Record results = library.select_records(rec.type == LIB_HEADER);
> 
> But as it turns out (as I expected, which was why I tried this last),
> D's "lazy" evaluation isn't as lazy as it would be if this was lazy
> evaluation in an interpreted language. "rec" comes from the scope of the
> calling function, not the scope of the foreach loop inside select_records
> where the expression will be evaluated.
> 
> The only way I can get the result that I want is to subject the beautiful
> D programmming language to the ugly C preprocessor. This enables me to
> write this:
> 
> #define select_records(x) _select_records(delegate(OMF_Record rec) x)
> 

If you don't mind using a local var:

Record rec;
Record[] select_records( rec, rec.type==LIB_HEADER );

where the function is defined:

Record[]  select_records( ref Record r, lazy bool clause )
{
  Record[] result;
  foreach(Record temp; items ) {
    r=temp; if ( clause() ) result ~= temp; }
  }
  return result;
}



-DavidM





> ...which would make it LOOK like the delegate implicitly has the type
> of the argument (which was what I initially assumed, and what would
> be better). This technique would bring with it all the perils of the
> C preprocessor, and even threatens to wipe out some of the benefits
> of D. If I released such a class as open-source software, its users
> would have to subject _their_ programs to the CPP to be able to use
> the class.
> 
> Fortunately (or not), I cannot abuse D in this way using DM's C
> preprocessor, because the preprocessor seems to be built into
> the C compiler somehow (as opposed to GCC, which has an external
> CPP that can be called separately).
> 
> But D is still in its infancy. Over the years, future D programmers
> will run into many duplicates of this problem, and they may eventually
> cope with it by introducing an external C preprocessor to the mix.
>