View mode: basic / threaded / horizontal-split · Log in · Help
July 04, 2013
Rust switches to external iteration
Evidence we've done the right thing by emphasizing ranges instead of 
opApply.

http://www.reddit.com/r/programming/comments/1hl2qr/rust_07_released/
https://mail.mozilla.org/pipermail/rust-dev/2013-June/004599.html

Andrei
July 04, 2013
Re: Rust switches to external iteration
On Wed, 2013-07-03 at 22:04 -0700, Andrei Alexandrescu wrote:
> Evidence we've done the right thing by emphasizing ranges instead of 
> opApply.
> 
> http://www.reddit.com/r/programming/comments/1hl2qr/rust_07_released/
> https://mail.mozilla.org/pipermail/rust-dev/2013-June/004599.html

I am not sure you can make that deduction. The question of internal vs.
external iterators is not black and white. Both are needed. The question
is what is the balance of usage, and it depends on which level of the
programming "stack " you are at. People working on implementing
frameworks and APIs often need explicit iteration to realize their
abstractions, but can also use some internal iteration as that
abstraction works for them at that point. For applications programmers,
I would expect most of them to benefit from using internal iteration as
much as possible since they are users of frameworks and APIs, but the
availability 

The difficulty is that we have some serious inertia in the system.

Many programmers are still using 1970s approaches to programming, even
though they may be 20-something, because that is what they have been
taught to do. As long as they use sequence, selection and iteration that
is all they need.

At the other extreme there are the Lambda Calculus purists who try to
instil that you have to undertake the Category Theory analysis to
construction the point-free function compositions prior to writing any
code.

And neither of these groups worry about test coverage that much.

The goal has to be to write code that is appropriate to the task,
readable, comprehensible, testable and tested. It is all about Domain
Specific Languages (DSL). Any program that uses functions and procedures
is creating a DSL. Abstraction and DSL go hand in hand. We are giving a
concept a label and a meaning.

By trying to make internal vs external a black and white type argument,
we lose something from our toolkit. By trying to work on where is
internal appropriate, where is external appropriate, we move software
development on further. The whole object-oriented vs. generic vs.
functional vs. logic models, especially the OO vs. functional just now
is an important move of programming into the future.

FOOPLOG may have failed but Scala has brought programming to functional
AND object-oriented. Groovy, Kotlin, Ceylon, etc. are pushing at this as
well. It is all about the balance of use of internal and external
iteration not treating it as warfare. Python had its own consideration
of these issues in the Python 2 → Python 3 change.

Native still seems to be in the 1990s OO vs. functional war: internal OR
external. cf. C++, Go, D, Rust, Haskell, OCaml, F#, etc.

My current hypothesis is that the more abstract the DSL the more
internal iteration. Moving from implementation focus to goal focus
naturally introduces more implicit as the DSL becomes more abstract. 

But I get too serious and philosophical. It is perhaps because I have
just finished a two day workshop guiding people to consider these
questions in a very practical way so as to improve their software
development.
-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
July 04, 2013
Re: Rust switches to external iteration
> In the future, Rust can have generators using a `yield` 
> statement like C#,
>  compiling down to a fast state machine without requiring 
> context switches,
>  virtual functions or even closures. This would eliminate the 
> difficulty of
>  coding recursive traversals by-hand with external iterators.

It would be nice to beat them to the punch on this. Yield is a 
great tool. You can implement not so bad yield-like behaviour 
yourself in D, though. I have an old example that I baked up a 
while ago.

http://pastebin.com/EGqLHk0U

I'd prioritise important matters like improving the garbage 
collector and other non-language changing, but definitely 
beneficial improvements, over a big language addition like yield, 
though.
July 04, 2013
Re: Rust switches to external iteration
On 07/04/2013 07:04 AM, Andrei Alexandrescu wrote:
> Evidence we've done the right thing by emphasizing ranges instead of
> opApply.
>
> http://www.reddit.com/r/programming/comments/1hl2qr/rust_07_released/
> https://mail.mozilla.org/pipermail/rust-dev/2013-June/004599.html
>
> Andrei

Well, probably, but implementing ranges is still too tedious.
It is very likely that rust is going to fix that.

It should be as simple as:

auto map(alias a,R)(R r){
    foreach(x;r)
        yield a(x);
}

auto filter(alias a,R)(R r){
    foreach(x;r)
        if(a(x))
            yield x;
}

auto take(R)(R r,size_t i){
    while(i--&&!r.empty){
        yield r.front;
        r.popFront();
    }
}

auto joiner(R,S)(R r,S s){
    if(r.empty) break;
    foreach(x;r.front)
        yield x;
    r.popFront();
    foreach(rr;r){
        yield s;
        foreach(x;rr)
            yield x;
    }
}

auto cprod(R,S)(R r,S s){
    foreach(x;r)
        foreach(y;s)
            yield tuple(x,y);
}

One can get painfully close with basically no effort (DMD regressed on 
it though):
http://dpaste.dzfl.pl/baa538af


Ideally the compiler would automatically turn definitions similar to 
those above into auto return function templates that create instances of 
range types.

The issue is how to design it to make it useful for generic wrapper 
ranges that should forward the source range capabilities.
July 04, 2013
Re: Rust switches to external iteration
On Thursday, 4 July 2013 at 07:20:18 UTC, Russel Winder wrote:
> [..]
> Many programmers are still using 1970s approaches to 
> programming, even
> though they may be 20-something, because that is what they have 
> been
> taught to do. As long as they use sequence, selection and 
> iteration that
> is all they need.

Even on the work environment.

Every time I see the usual shell and vi/emacs combo it brings me
back memories of when my home computer was a Timex 2068.

Or as someone puts it,

http://andrewbrookins.com/tech/one-year-later-an-epic-review-of-pycharm-2-7-from-a-vim-users-perspective/


> And neither of these groups worry about test coverage that much.

To be honest in the fortune 500 companies very few do. We always
try to push for it in our consulting projects, but not all 
customers
buy into it, specially if the project requires interaction with 
the
in-house developers.

>
> FOOPLOG may have failed but Scala has brought programming to 
> functional
> AND object-oriented. Groovy, Kotlin, Ceylon, etc. are pushing 
> at this as
> well. It is all about the balance of use of internal and 
> external
> iteration not treating it as warfare. Python had its own 
> consideration
> of these issues in the Python 2 → Python 3 change.

I think Microsoft plays a big role here. By sponsoring OCaml,
Haskel and F# development, as well as, introducing FP to the
enterprise via LINQ.


>
> Native still seems to be in the 1990s OO vs. functional war: 
> internal OR
> external. cf. C++, Go, D, Rust, Haskell, OCaml, F#, etc.
>

The main issue is that this is very human related.

Sometimes you really need a few generations to make people adopt 
new ideas.

--
Paulo
July 04, 2013
Re: Rust switches to external iteration
On Thursday, 4 July 2013 at 08:18:22 UTC, Timon Gehr wrote:
> It should be as simple as:
>
> auto map(alias a,R)(R r){
>     foreach(x;r)
>         yield a(x);
> }

... if all you care about is input ranges ...

I don't think there's any way to automatically infer and 
implement random access, bidirectionality etc. on ranges from an 
internal iteration definition.
July 04, 2013
Re: Rust switches to external iteration
On Thu, 2013-07-04 at 11:35 +0200, Paulo Pinto wrote:
[…]
> Every time I see the usual shell and vi/emacs combo it brings me
> back memories of when my home computer was a Timex 2068.
> 
> Or as someone puts it,
> 
> http://andrewbrookins.com/tech/one-year-later-an-epic-review-of-pycharm-2-7-from-a-vim-users-perspective/

I think the USP for IDEs is when you have to debug, otherwise it is just
a matter of personal taste.  I tend to use Emacs because it can reformat
just comment blocks in a file, none of the IDE editors can do this, and
for me this is an important thing. 

> > And neither of these groups worry about test coverage that much.
> 
> To be honest in the fortune 500 companies very few do. We always
> try to push for it in our consulting projects, but not all 
> customers
> buy into it, specially if the project requires interaction with 
> the
> in-house developers.

Perhaps stock prices should be linked to code coverage statistics for
that company. It is about as good a measure as the gamblers, sorry
traders, use – which seems to rely most on whether the CEO sneezed at
the time the traders specified they should.

> I think Microsoft plays a big role here. By sponsoring OCaml,
> Haskel and F# development, as well as, introducing FP to the
> enterprise via LINQ.

Agreed, and I hate to give Microsoft credit for anything positive, but
here they definitely have. Sad they dropped IronPython and IronRuby
though.

> The main issue is that this is very human related.
> 
> Sometimes you really need a few generations to make people adopt 
> new ideas.

Not if ideas are reified in the programming language. Java brought
shared memory multi-threading front and centre (or should that be
center?) and it remains the biggest barrier to quality software we have.
Programmers see the feature and use it despite the fact that it had been
known for 20 years prior to 1995 that it was the wrong way forward for
applications development. Actors, dataflow, CSP, data parallelism,
masses of great high level models for concurrency and parallelism all
ignore. And now C++11 nearly did almost the same thing. At least thanks
to the UK vote C++ includes asynchronous function call and futures. All
the rest of C++ threads stuff, like Java, is already 40 years behind
well tried and trusted approaches.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
July 04, 2013
Re: Rust switches to external iteration
w0rp:

> It would be nice to beat them to the punch on this. Yield is a 
> great tool. You can implement not so bad yield-like behaviour 
> yourself in D, though. I have an old example that I baked up a 
> while ago.
>
> http://pastebin.com/EGqLHk0U
>
> I'd prioritise important matters like improving the garbage 
> collector and other non-language changing, but definitely 
> beneficial improvements, over a big language addition like 
> yield, though.

+1

Bye,
bearophile
July 04, 2013
Re: Rust switches to external iteration
On 07/04/2013 11:49 AM, Peter Alexander wrote:
> On Thursday, 4 July 2013 at 08:18:22 UTC, Timon Gehr wrote:
>> It should be as simple as:
>>
>> auto map(alias a,R)(R r){
>>     foreach(x;r)
>>         yield a(x);
>> }
>
> ... if all you care about is input ranges ...
>
> I don't think there's any way to automatically infer and implement
> random access, bidirectionality etc. on ranges from an internal
> iteration definition.

There is no way that works in the general case, but it should be easy 
enough to infer from eg. the above implementation.
July 04, 2013
Re: Rust switches to external iteration
On Thursday, 4 July 2013 at 05:04:59 UTC, Andrei Alexandrescu 
wrote:
> Evidence we've done the right thing by emphasizing ranges 
> instead of opApply.

One downside is with opApply, you have a little more control over 
escaping references. Suppose you take a range to view a 
container, then .save it somewhere and keep that copy. Then when 
the container dies, your range points at invalid memory.

The Rust people must have solved that though. Probably not so 
hard there since their compiler has a borrow check, whereas in D 
I was trying to do it with library hacks...
« First   ‹ Prev
1 2 3
Top | Discussion index | About this forum | D home