To date, I've been using D in much the same way I used C++, without heavy
use of templates.  Now I'm trying out a more functional style using
std.algorithm.  However I'm pretty much stuck at the first hurdle: map.
 With type inference, this works:

import std.algorithm;
import std.stdio;

void main() {
  auto start = [1,2,3,4,5];
  auto squares = map!((a) { return a * a; })(start);
  writeln(squares);
}


Without type inference (obviously necessary beyond trivial examples), it'd
be nice to do:

import std.algorithm;
import std.stdio;

void main() {
  int[] start = [1,2,3,4,5];
  int[] squares = map!((a) { return a * a; })(start);
  writeln(squares);
}


but this gives "Error: cannot implicitly convert expression (map(start)) of
type Result to int[]".  That opaque type "Result" is weird (not
parameterized on "int"?), but OK, let's try that:

import std.algorithm;
import std.stdio;

void main() {
  int[] start = [1,2,3,4,5];
  Result squares = map!((a) { return a * a; })(start);
  writeln(squares);
}


gives "undefined identifier Result".  Not sure why, but OK.  I can't see examples in the docs of explicit type declarations -- annoyingly they all use "auto".

However, they do tell me that map "returns a range".  Assuming all definitions of ranges are in std.range, there's no such thing as a "Range" interface, so it's not that.  The most general interfaces I can see are InputRange(E) and OutputRange(E).  It certainly can't be an OutputRange, so I guess it must satisfy InputRange, presumably type-parameterized with "int".  So this should work:

import std.algorithm;
import std.stdio;
import std.range;

void main() {
  int[] start = [1,2,3,4,5];
  InputRange!int squares = map!((a) { return a * a; })(start);
  writeln(squares);
}


But the compiler complains "cannot implicitly convert expression
(map(start)) of type Result to std.range.InputRange!(int).InputRange".
 That's weird, because "std.range.InputRange!(int).InputRange" doesn't even
look like a type.

I've tried all manner of combinations here, but nothing works.  Seems like there's something fundamental I'm not understanding.

James Fisher:

> To date, I've been using D in much the same way I used C++, without heavy use of templates.  Now I'm trying out a more functional style using std.algorithm.  However I'm pretty much stuck at the first hurdle: map.

Welcome to functional D :-)


> The most general interfaces I can see are InputRange(E) and OutputRange(E).

In D language there is about as much support for OOP as in Java (and more), but in practice OO interfaces are not used significantly in the D standard library. So those are a different kind of "interfaces", they are a bit more like C++0x Concepts, they are very statically typed. So while they are organized in a hierarchy, you can't assign them as OO interfaces.

You generally use type inference to assign the type of map. In functional programming a bit of type inference is not optional, it's almost needed.

map() in D is lazy, it doesn't return an array, but a lazily iterable range. If you want an eager dynamic array you may use array():
array(map!((a) { return a * a; })(start));

If you don't want to use type inference for the return result of map, you have to take a look at the source code of std.functional, read what type is exactly map, and copy it. Generally this is not done in normal user code.

Bye,
bearophile

On Thu, 07 Jul 2011 10:00:48 -0400, James Fisher <jameshfisher@gmail.com> wrote:

> To date, I've been using D in much the same way I used C++, without heavy
> use of templates.  Now I'm trying out a more functional style using
> std.algorithm.  However I'm pretty much stuck at the first hurdle: map.
>  With type inference, this works:
>
> import std.algorithm;
> import std.stdio;
>
> void main() {
>   auto start = [1,2,3,4,5];
>   auto squares = map!((a) { return a * a; })(start);
>   writeln(squares);
> }
>
>
> Without type inference (obviously necessary beyond trivial examples),

Type inference is useful everywhere, at any level of complexity.

Where it's not useful is declarations, for example, declaring the type of a struct or class member.  However, typeof should work to use type inference there.

> it'd
> be nice to do:
>
> import std.algorithm;
> import std.stdio;
>
> void main() {
>   int[] start = [1,2,3,4,5];
>   int[] squares = map!((a) { return a * a; })(start);
>   writeln(squares);
> }
>
>
> but this gives "Error: cannot implicitly convert expression (map(start)) of
> type Result to int[]".  That opaque type "Result" is weird (not
> parameterized on "int"?), but OK, let's try that:
>
> import std.algorithm;
> import std.stdio;
>
> void main() {
>   int[] start = [1,2,3,4,5];
>   Result squares = map!((a) { return a * a; })(start);
>   writeln(squares);
> }
>
>
> gives "undefined identifier Result".  Not sure why, but OK.  I can't see
> examples in the docs of explicit type declarations -- annoyingly they all
> use "auto".

OK, so this is kind of a weird case.  std.algorithm.map returns an inner struct, which means, it has no public name.  But then how can you even use it?  Well, it just doesn't have a public *name*, but it still can be used outside the function.  It's definitely an oddball.  However, what's nice about it is that you can encapsulate the struct inside the one place it is used.

You can see the definition of map here: https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm.d#L366

And a few lines down, the declaration of Result.

You can use typeof if you want to get the type, but again, auto is much better unless you are declaring something.

> However, they do tell me that map "returns a range".  Assuming all
> definitions of ranges are in std.range, there's no such thing as a "Range"
> interface, so it's not that.  The most general interfaces I can see are
> InputRange(E) and OutputRange(E).  It certainly can't be an OutputRange, so
> I guess it must satisfy InputRange, presumably type-parameterized with
> "int".

No, a range is a concept, meaning it is a compile-time interface.  Any type which satisfies the requirements of the input range can be a range, even non-polymorphic types.  Even an array is a range.

>  So this should work:
>
> import std.algorithm;
> import std.stdio;
> import std.range;
>
> void main() {
>   int[] start = [1,2,3,4,5];
>   InputRange!int squares = map!((a) { return a * a; })(start);
>   writeln(squares);
> }
>
>
> But the compiler complains "cannot implicitly convert expression
> (map(start)) of type Result to std.range.InputRange!(int).InputRange".

Right, because it's not a derivative of that interface, it's its own type, defined only inside the function.  Yeah, I know it's confusing :)

>  That's weird, because "std.range.InputRange!(int).InputRange" doesn't even
> look like a type.

std.range is the module, but I assume you already know that.

But one of the coolest things about D templates is the eponymous rule.  That is, if you declare a template, and that template has exactly one member, and that one member is named the same as the template, then x!(y) becomes the equivalent of x!(y).x.

For example:

template t(T)
{
   class t
   {
      T val;
   }
}

t!(int) is equivalent to t!(int).t

In other words, a template is a *namespace* for declarations using the template parameters.  And in this special case, you can omit the member of the namespace you are accessing.

Then what follows is that:

class t(T)
{
   T val;
}

is shorthand for the above.

But the compiler maintains the namespace.member nomenclature, which is why you see that in the error message.

-Steve

OK, good replies.  Cool.  So the two places I thought I'd need to use explicit types are in parameter and return types.  Say, a function returns the result of map, and another consumes it to print it.  The consuming function seems to work with templating:

void printSquares(T)(T squares) {
  writeln(squares);
}

void main() {
  int[] start = [1,2,3,4,5];
  auto squares = map!((a) { return a * a; })(start);
  printSquares(squares);
}


However, returning the result of map doesn't seem to work so well with the same method:

T getSquares(T)() {
  int[] start = [1,2,3,4,5];
  return map!((a) { return a * a; })(start);
}

void main() {
  auto squares = getSquares();
  writeln(squares);
}


Gives the errors:

test.d(11): Error: template test.getSquares(T) does not match any function
template declaration
test.d(11): Error: template test.getSquares(T) cannot deduce template
function from argument types !()()


But this does work with auto return type inference:

auto getSquares() {
  int[] start = [1,2,3,4,5];
  return map!((a) { return a * a; })(start);
}

void main() {
  auto squares = getSquares();
  writeln(squares);
}


So ... stuff works, but I'm not really sure why one uses function templating and the other uses return type inference.  Any answers?

On Thu, Jul 7, 2011 at 7:31 PM, Steven Schveighoffer <schveiguy@yahoo.com>wrote:

> On Thu, 07 Jul 2011 10:00:48 -0400, James Fisher <jameshfisher@gmail.com> wrote:
>
>  To date, I've been using D in much the same way I used C++, without heavy
>> use of templates.  Now I'm trying out a more functional style using
>> std.algorithm.  However I'm pretty much stuck at the first hurdle: map.
>>  With type inference, this works:
>>
>> import std.algorithm;
>> import std.stdio;
>>
>> void main() {
>>  auto start = [1,2,3,4,5];
>>  auto squares = map!((a) { return a * a; })(start);
>>  writeln(squares);
>> }
>>
>>
>> Without type inference (obviously necessary beyond trivial examples),
>>
>
> Type inference is useful everywhere, at any level of complexity.
>
> Where it's not useful is declarations, for example, declaring the type of a struct or class member.  However, typeof should work to use type inference there.
>
>
>  it'd
>> be nice to do:
>>
>> import std.algorithm;
>> import std.stdio;
>>
>> void main() {
>>  int[] start = [1,2,3,4,5];
>>  int[] squares = map!((a) { return a * a; })(start);
>>  writeln(squares);
>> }
>>
>>
>> but this gives "Error: cannot implicitly convert expression (map(start))
>> of
>> type Result to int[]".  That opaque type "Result" is weird (not
>> parameterized on "int"?), but OK, let's try that:
>>
>> import std.algorithm;
>> import std.stdio;
>>
>> void main() {
>>  int[] start = [1,2,3,4,5];
>>  Result squares = map!((a) { return a * a; })(start);
>>  writeln(squares);
>> }
>>
>>
>> gives "undefined identifier Result".  Not sure why, but OK.  I can't see examples in the docs of explicit type declarations -- annoyingly they all use "auto".
>>
>
> OK, so this is kind of a weird case.  std.algorithm.map returns an inner struct, which means, it has no public name.  But then how can you even use it?  Well, it just doesn't have a public *name*, but it still can be used outside the function.  It's definitely an oddball.  However, what's nice about it is that you can encapsulate the struct inside the one place it is used.
>
> You can see the definition of map here: https://github.com/D-** Programming-Language/phobos/**blob/master/std/algorithm.d#**L366<https://github.com/D-Programming-Language/phobos/blob/master/std/algorithm.d#L366>
>
> And a few lines down, the declaration of Result.
>
> You can use typeof if you want to get the type, but again, auto is much better unless you are declaring something.
>
>
>  However, they do tell me that map "returns a range".  Assuming all
>> definitions of ranges are in std.range, there's no such thing as a "Range"
>> interface, so it's not that.  The most general interfaces I can see are
>> InputRange(E) and OutputRange(E).  It certainly can't be an OutputRange,
>> so
>> I guess it must satisfy InputRange, presumably type-parameterized with
>> "int".
>>
>
> No, a range is a concept, meaning it is a compile-time interface.  Any type which satisfies the requirements of the input range can be a range, even non-polymorphic types.  Even an array is a range.
>
>
>   So this should work:
>>
>> import std.algorithm;
>> import std.stdio;
>> import std.range;
>>
>> void main() {
>>  int[] start = [1,2,3,4,5];
>>  InputRange!int squares = map!((a) { return a * a; })(start);
>>  writeln(squares);
>> }
>>
>>
>> But the compiler complains "cannot implicitly convert expression
>> (map(start)) of type Result to std.range.InputRange!(int).**InputRange".
>>
>
> Right, because it's not a derivative of that interface, it's its own type, defined only inside the function.  Yeah, I know it's confusing :)
>
>
>   That's weird, because "std.range.InputRange!(int).**InputRange" doesn't
>> even
>> look like a type.
>>
>
> std.range is the module, but I assume you already know that.
>
> But one of the coolest things about D templates is the eponymous rule.
>  That is, if you declare a template, and that template has exactly one
> member, and that one member is named the same as the template, then x!(y)
> becomes the equivalent of x!(y).x.
>
> For example:
>
> template t(T)
> {
>   class t
>   {
>      T val;
>   }
> }
>
> t!(int) is equivalent to t!(int).t
>
> In other words, a template is a *namespace* for declarations using the template parameters.  And in this special case, you can omit the member of the namespace you are accessing.
>
> Then what follows is that:
>
> class t(T)
> {
>   T val;
> }
>
> is shorthand for the above.
>
> But the compiler maintains the namespace.member nomenclature, which is why you see that in the error message.
>
> -Steve
>

On Fri, Jul 8, 2011 at 4:06 PM, James Fisher <jameshfisher@gmail.com> wrote:
>
> So ... stuff works, but I'm not really sure why one uses function templating and the other uses return type inference.  Any answers?


... Wait, brain malfunction; I think I do understand.  The type is inferable purely from the function definition in the latter case.

On Fri, 08 Jul 2011 11:08:14 -0400, James Fisher <jameshfisher@gmail.com> wrote:

> On Fri, Jul 8, 2011 at 4:06 PM, James Fisher <jameshfisher@gmail.com> wrote:
>>
>> So ... stuff works, but I'm not really sure why one uses function
>> templating and the other uses return type inference.  Any answers?
>
>
> ... Wait, brain malfunction; I think I do understand.  The type is inferable
> purely from the function definition in the latter case.


Yes, the return type is determined by the return statement in the function itself.  Note that it would be an error to have two different return types that compiled into the same function.  A frequent use of auto is when the return type could be different based on template parameters to the function, or version statements.  In these cases, effectively there is only one return type, because only one return statement gets compiled.

The reason your getSquares(T)() *call* doesn't work is because the compiler first must instantiate the template, and it doesn't have enough information to do so.  Remember that a function template like this:

T getSquares(T)() {...}

is equivalent to this:

template getSquares(T)
{
   T getSquares() {...}
}

So in fact, the compiler cannot evaluate the inner part of the template until it knows what T is.  However, there is a special feature called implicit function template instantiation (IFTI) which allows the compiler to infer the values of T from the function call being made.  This means the types of arguments to the function can be used to infer the template parameters.  This *only* works for template functions or templates that can be reduced to template functions.

So for example:

T foo(T)(T t) {return t;}

you can call this function via:

foo(1);

And T will implicitly be assigned int.

But since your getSquares function takes no parameters, IFTI cannot be used.  Note that the return type does NOT play a role in IFTI.

However, you can still use getSquares, just specify the template parameter explicitly:

auto sq = getSquares!int();

Another option is to give a default value for T:

T getSquares(T = int)() {...}

-Steve

On Fri, Jul 8, 2011 at 4:49 PM, Steven Schveighoffer <schveiguy@yahoo.com>wrote:

> On Fri, 08 Jul 2011 11:08:14 -0400, James Fisher <jameshfisher@gmail.com> wrote:
>
>  On Fri, Jul 8, 2011 at 4:06 PM, James Fisher <jameshfisher@gmail.com>
>> wrote:
>>
>>>
>>> So ... stuff works, but I'm not really sure why one uses function templating and the other uses return type inference.  Any answers?
>>>
>>
>>
>> ... Wait, brain malfunction; I think I do understand.  The type is
>> inferable
>> purely from the function definition in the latter case.
>>
>
>
> Yes, the return type is determined by the return statement in the function itself.  Note that it would be an error to have two different return types that compiled into the same function.  A frequent use of auto is when the return type could be different based on template parameters to the function, or version statements.  In these cases, effectively there is only one return type, because only one return statement gets compiled.
>
> The reason your getSquares(T)() *call* doesn't work is because the compiler first must instantiate the template, and it doesn't have enough information to do so.  Remember that a function template like this:
>
> T getSquares(T)() {...}
>
> is equivalent to this:
>
> template getSquares(T)
> {
>   T getSquares() {...}
> }
>
> So in fact, the compiler cannot evaluate the inner part of the template until it knows what T is.  However, there is a special feature called implicit function template instantiation (IFTI) which allows the compiler to infer the values of T from the function call being made.  This means the types of arguments to the function can be used to infer the template parameters.  This *only* works for template functions or templates that can be reduced to template functions.
>
> So for example:
>
> T foo(T)(T t) {return t;}
>
> you can call this function via:
>
> foo(1);
>
> And T will implicitly be assigned int.
>
> But since your getSquares function takes no parameters, IFTI cannot be used.  Note that the return type does NOT play a role in IFTI.
>
> However, you can still use getSquares, just specify the template parameter explicitly:
>
> auto sq = getSquares!int();
>
> Another option is to give a default value for T:
>
> T getSquares(T = int)() {...}
>
> -Steve
>

This all makes sense.  Thanks for the comprehensive explanation. :)