On Friday, 17 July 2015 at 09:07:29 UTC, Jarl André Hübenthal wrote:
> Thanks. Its a lot more cleaner and syntactically readable having .array at the end. But about laziness the same applies to clojure and scala. In clojure you must force evaluate the list, in scala you must to mostly the same as in D, put a toList or something at the end. Or loop it. But its pretty nice to know that there is laziness in D, but when I query mongo I expect all docs to be retrieved, since there are no paging in the underlying queries? Thus, having a lazy functionality on top of non lazy db queries seem a bit off dont you think?

I'm almost certain that the D database driver returns eagerly all the results that you've requested. The lazy stuff should happen when you start doing range operations after the results are returned from the database. It's not impossible to lazily query the database, but I think that the developers have chosen the eager approach, since it's more straightforward.

Currently, in D most of the laziness is a convention, rather than something directly built into the language. There are many features that enable (indirectly) effective and easy to use lazy algorithms, but these features are have many other uses (templates, auto type deduction, compile-time reflection, etc.).

The only two direct features are:
1) foreach can iterate over ranges (objects of structs or classes for which isInputRange is true. Here's an example:

import std.algorithm.iteration : map, filter;

foreach (name; persons.filter!(p => p.age > 18).map!(p => p.name))
    writeln(name);

import std.range.primitives : isInputRange;

static assert (
    isInputRange!(
        typeof(
           persons.filter!(p => p.age > 18).map!(p => p.name)
        )
    ) == true
);

See http://dlang.org/phobos/std_range_primitives.html#isInputRange

2) The lazy keyword - when you annotate function parameters with lazy they are evaluated not at the caller site, but only when needed like in other more traditional functional languages. For example:

void calculate(int[] numbers)
{
    import std.format : format;
    // ...

    logErrorIf(numbers[3] < 5,
        format("Expected value < 5, but got %s !", numbers[3]));
    //  ^~~~~~~~~~~~ this is only evaluated

    // ...
}

void logErrorIf(bool condition, lazy string error_message)
{
    if (condition)
        writeln(message);
    //          ^~~~ here, if the condition is true
}

( In D string is just an alias to immutable(char)[], so the above signature is identical to this:
void logErrorIf(bool condition, lazy immutable(char)[] error_message) )

You can think of lazy parameters as implicit lambdas that return the expression passed as argument only when called.
Here you can learn more about the lazy keyword http://dlang.org/lazy-evaluation.html

Even though we have 'lazy' built into the language, most of the lazy algorithms do not use it. I just made a quick search through druntime and phobos for 'lazy' and 'range' (don't how correct it was - I admit I'm a unix noob) and here's what I got:

// (I have DMD v2.067.1 installed)

// lazy at the head of the function parameter list or in the tail
$ find /usr/include/dmd/ -name '*.d' -exec cat {} \; | grep -c '(lazy \|, lazy '
74

// just containing lazy
$ find /usr/include/dmd/ -name '*.d' -exec cat {} \; | grep -c 'lazy'
138

// just containing range
$ find /usr/include/dmd/ -name '*.d' -exec cat {} \; | grep -c 'range'
3548

I think that this because ranges are a more generic, flexible and powerful abstraction, and are more efficient maybe because they're easier to optimize to simple loops (eg. I've seen that the ldc compiler handles them very well).
'lazy' is still useful but generally I have seen it used for more simpler stuff (like the above 'lazy' example), and not for propagating state through range pipelines (or more simply - function chaining).

So you'll see both functions that are lazy and functions that are not throughout Phobos (and most use ranges, as you can see from the results).

Generally you can distinguish range functions from others by their signatures. Since most ranges in D are templated structs and not classes inheriting some interface
(though there some, see http://dlang.org/phobos/std_range_interfaces#InputRange),
functions that operate on ranges are templated at least on one range type:

// Check if the function 'fun' is really a predicate
enum isUnaryPredicate(alias fun, T) =
    is( typeof( fun(T.init) ) : bool);

import std.range.primitives: isInputRange, ElementType;

// templated on predicate and range type
//                ~~~~v~~~~  ~~v~~
auto filter1(alias predicate, Range)(Range range)
    if (isInputRange!Range &&   //       <- some template
        isUnaryPredicate!(predicate, ElementType!Range))
    //  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  constraints
{
    return ...
}

// Even if you can't look at the function body, you
// can guess that it can't be lazy because, it must
// have the whole result, before it returns it.
T[] filter2(alias predicate, T)(const(T)[] arrayToFilter)
    if (isUnaryPredicate!(predicate, T))
    //  ^~~~~~ a bit less template constraints
{
    return ...
}

Here's a full example:
http://dpaste.dzfl.pl/9023c63f9393

On Friday, 17 July 2015 at 15:41:22 UTC, ZombineDev wrote:
> I'm almost certain that the D database driver returns eagerly all the results that you've requested. The lazy stuff should happen when you start doing range operations after the results are returned from the database. It's not impossible to lazily query the database, but I think that the developers have chosen the eager approach, since it's more straightforward.
>

Discard this. I didn't read the previous comments.

On Friday, 17 July 2015 at 15:41:22 UTC, ZombineDev wrote:
> eager approach, since it's more straightforward.

What makes you think it's always more straightforward? Sometimes (like in this case with MongoDB) you cannot write eager approach without first writing lazy one.

On Friday, 17 July 2015 at 17:56:51 UTC, sigod wrote:
> On Friday, 17 July 2015 at 15:41:22 UTC, ZombineDev wrote:
>> eager approach, since it's more straightforward.
>
> What makes you think it's always more straightforward? Sometimes (like in this case with MongoDB) you cannot write eager approach without first writing lazy one.

Well I just wrote without properly looking up what MongoDB does. I thought collection.find() returned the first n elements, not a iterator (cursor in MongoDB's terms) and that it would be additional work split those n elements.
Anyway, thanks for the correction, now I am bit more educated ;)

On Friday, 17 July 2015 at 12:59:24 UTC, Kagamin wrote:
> On Friday, 17 July 2015 at 09:07:29 UTC, Jarl André Hübenthal wrote:
>> Or loop it. But its pretty nice to know that there is laziness in D, but when I query mongo I expect all docs to be retrieved, since there are no paging in the underlying queries? Thus, having a lazy functionality on top of non lazy db queries seem a bit off dont you think?
>
> From the client point of view db is sort of lazy: data is received from server as needed. Why would you want to put all data into an array before processing it? Why can't you process it from the range directly?

Its simple. In most cases you do an advanced aggregated search in mongo, and what you get is then a mongocursor. Lets say I am retrieving all projects for a given customer where the project is started.. I really am in no interest of lazily evaluating this result, because I want to return this data to the client (browser) immediately. And lets say I am in a prototype phase where i haven't yet implemented all those nasty mongo queries, I want to be able to filter, map and reduce the result and work with arrays not some sort of non evaluated lazy MapResult. In scala luckily I have implicit converts, so that I can just stop thinking about it and have it converted automatically.

On Saturday, 18 July 2015 at 08:03:56 UTC, Jarl André Hübenthal wrote:
> Its simple. In most cases you do an advanced aggregated search in mongo, and what you get is then a mongocursor. Lets say I am retrieving all projects for a given customer where the project is started.. I really am in no interest of lazily evaluating this result, because I want to return this data to the client (browser) immediately.

How big is the slowdown you notice for lazy processing? Lazy processing is believed to be faster because it's less resource consuming.

> And lets say I am in a prototype phase where i haven't yet implemented all those nasty mongo queries, I want to be able to filter, map and reduce the result and work with arrays not some sort of non evaluated lazy MapResult.

I believe those algorithms were written to work on lazy ranges. What makes you think they can't do that?

On Saturday, 18 July 2015 at 09:18:14 UTC, Kagamin wrote:
> On Saturday, 18 July 2015 at 08:03:56 UTC, Jarl André Hübenthal wrote:
>> Its simple. In most cases you do an advanced aggregated search in mongo, and what you get is then a mongocursor. Lets say I am retrieving all projects for a given customer where the project is started.. I really am in no interest of lazily evaluating this result, because I want to return this data to the client (browser) immediately.
>
> How big is the slowdown you notice for lazy processing? Lazy processing is believed to be faster because it's less resource consuming.
>
>> And lets say I am in a prototype phase where i haven't yet implemented all those nasty mongo queries, I want to be able to filter, map and reduce the result and work with arrays not some sort of non evaluated lazy MapResult.
>
> I believe those algorithms were written to work on lazy ranges. What makes you think they can't do that?

I don't understand where you are going with this. I have solved my problem. Laziness is good for lets say take 5 out of infinite results. When you ask for a complete list and want the complete list, you take all. In clojure you actually say that, "doall". In D .array does the same thing. Converts lazy to non lazy.

On Saturday, 18 July 2015 at 09:33:37 UTC, Jarl André Hübenthal wrote:
> I don't understand where you are going with this. I have solved my problem. Laziness is good for lets say take 5 out of infinite results.

It's also good for saving resources, you don't spend time managing those resources and save that time to complete the processing earlier.