View mode: basic / threaded / horizontal-split · Log in · Help
March 01, 2012
Tuples citizenship
I think std.typecons.Tuples merit to be a little more citizens in D and Phobos.
I think reducing the usage of "out" argument, and replacing them with a tuple result, will avoid mistakes and make the code look better. In std.math there are functions that maybe are better to use std.typecons.Tuple:

pure nothrow @trusted real frexp(real value, out int exp);
==>
pure nothrow @trusted Tuple!(real, int) frexp(real value);


nothrow @trusted real remquo(real x, real y, out int n); 
==>
nothrow @trusted Tuple!(real, int) remquo(real x, real y); 

------------------------

It's good for tuples to become more common in D code. Some time ago I have asked the built-in associative arrays to grow a method to iterate on key-value pairs, named "byPair":
http://d.puremagic.com/issues/show_bug.cgi?id=5466

Andrei answered:
>byPair is tricky because std.tuple is not visible from object.

How do you solve this problem?


Now I think about a "pairs" method too:

int[string] aa;
Tuple!(string, int) ps = aa.pairs;
foreach (p; aa.byPair)
	assert(is( typeof(p) == Tuple!(string, int) ));



This example shows an use case of byItem. Given a string, the task here is to show the char frequencies, putting higher frequencies on top, and sorting alphabetically the chars that share the same frequency.

A Python 2.6 program that solves the problem:


from collections import defaultdict
text = "the d programming language is an object oriented " + \
      "imperative multi paradigm system programming " + \
      "language created by walter bright of digital mars"
frequences = defaultdict(int)
for c in text:
   frequences[c] += 1
pairs = sorted(frequences.iteritems(), key=lambda (c,f): (-f,c))
for (c, f) in pairs:
   print f, c



The output of the Python program:

20  
14 a
12 e
11 g
11 i
11 r
10 t
9 m
6 n
5 d
5 l
5 o
4 p
4 s
3 b
3 u
2 c
2 h
2 y
1 f
1 j
1 v
1 w



Three different solutions in D (probably there are other solutions, maybe even better ones) using Phobos:


import std.stdio, std.typecons, std.algorithm, std.array;

void main() {
   auto text = "the d programming language is an object oriented " ~
               "imperative multi paradigm system programming " ~
               "language created by walter bright of digital mars";

   int[char] frequences;
   foreach (char c; text)
       frequences[c]++;

   Tuple!(int,char)[] pairs1 = array(map!(c => tuple(frequences[c], c))(frequences.byKey));
   schwartzSort!(p => tuple(-p[0], p[1]))(pairs1);
   foreach (pair; pairs1)
       writeln(pair[0], " ", pair[1]);
   writeln();

   import std.conv;
   dchar[] keys = to!(dchar[])(frequences.keys);
   schwartzSort!(c => tuple(-frequences[cast(char)c], c))(keys);
   foreach (dchar c; keys)
       writeln(frequences[cast(char)c], " ", c);
   writeln();

   Tuple!(int,char)[] pairs1b = array(map!(c => tuple(-frequences[c], c))(frequences.byKey));
   sort(pairs1b);
   foreach (pair; pairs1b)
       writeln(-pair[0], " ", pair[1]);
   writeln();
}



A version using AA.pairs (or array(AA.byPair)), I have not used 'auto' for type clarity:

Tuple!(char,int)[] pairs2 = frequences.pairs;
schwartzSort!(c_f => tuple(-c_f[1], c_f[0]))(pairs2);
foreach (c_f; pairs2)
   writeln(c_f[1], " ", c_f[0]);



I am now writing a good amount of D2 code, some of it is functional style, or it's just translated from Python, and one annoying thing that comes out quite frequently is the lack of syntax to unpack a tuple into some variables.

If you want a short self-contained example, this is a small program (and it's not much functional. Tuples aren't just for functional-style code):
http://rosettacode.org/wiki/Sokoban#D

It contains code like:

immutable dirs = [tuple( 0, -1, 'u', 'U'),
                 tuple( 1,  0, 'r', 'R'),
                 tuple( 0,  1, 'd', 'D'),
                 tuple(-1,  0, 'l', 'L')];
// ...
foreach (di; dirs) {
   CTable temp = cur;
   immutable int dx = di[0];
   immutable int dy = di[1];


With tuple unpacking becomes:

foreach (di; dirs) {
   CTable temp = cur;
   immutable (dx, dy) = di.slice!(0, 2);


I don't know if this is going to work, because di[0..2] creates a 2-typetuple!

foreach (di; dirs) {
   CTable temp = cur;
   immutable (dx, dy) = di[0 .. 2];


Maybe this is not efficient:

foreach (di; dirs) {
   CTable temp = cur;
   immutable (dx, dy, _1, _2) = di;


Another example from that little program:


alias Tuple!(CTable, string, int, int) Four;
GrowableCircularQueue!Four open;
// ...
while (open.length) {
   immutable item = open.pop();
   immutable CTable cur = item[0];
   immutable string cSol = item[1];
   immutable int x = item[2];
   immutable int y = item[3];


With a tuple unpacking syntax becomes:

while (open.length) {
   immutable (cur, cSol, x, y) = open.pop();


While I use tuples I hit similar situations often.

For people interested in trying this idea, there is a patch by Kenji Hara (one or two parts are missing, like tuple unpacking in a foreach(...) and in function signatures, but I think most meat is already present):
https://github.com/D-Programming-Language/dmd/pull/341

Bye,
bearophile
March 01, 2012
Re: Tuples citizenship
On Thursday, March 01, 2012 17:08:44 bearophile wrote:
> I think std.typecons.Tuples merit to be a little more citizens in D and
> Phobos. I think reducing the usage of "out" argument, and replacing them
> with a tuple result, will avoid mistakes and make the code look better. In
> std.math there are functions that maybe are better to use
> std.typecons.Tuple:
> 
> pure nothrow @trusted real frexp(real value, out int exp);
> ==>
> pure nothrow @trusted Tuple!(real, int) frexp(real value);
[snip]

Having good tuple support is great, but out parameters are great too. I'm sure 
that there are plenty of cases where using out parameters is actually far 
cleaner than using tuples, since you don't have multiple return values to deal 
with. So, better enabling tuples is great, but I don't think that we're 
necessarily moving in a good direction if we're trying to eliminate out 
parameters in favor of tuples.

- Jonathan M Davis
March 01, 2012
Re: Tuples citizenship
bearophile <bearophileHUGS@lycos.com> wrote:
(snip) 
> It's good for tuples to become more common in D code. Some time ago I
> have asked the built-in associative arrays to grow a method to iterate on
> key-value pairs, named "byPair":
> http://d.puremagic.com/issues/show_bug.cgi?id=5466
> 
(snip)
> Bye,
> bearophile

Associative arrays (should) have UFCS, so one could just define

@property auto byPair(AA)(AA aa) {
   return zip(aa.byKey, aa.byValue);
}

in std.array. Or object.di could just define a Pair!(K,V) structure which a
Tuple!(K,V) has an opAssign defined for it.
March 01, 2012
Re: Tuples citizenship
Le 01/03/2012 23:33, Jonathan M Davis a écrit :
> On Thursday, March 01, 2012 17:08:44 bearophile wrote:
>> I think std.typecons.Tuples merit to be a little more citizens in D and
>> Phobos. I think reducing the usage of "out" argument, and replacing them
>> with a tuple result, will avoid mistakes and make the code look better. In
>> std.math there are functions that maybe are better to use
>> std.typecons.Tuple:
>>
>> pure nothrow @trusted real frexp(real value, out int exp);
>> ==>
>> pure nothrow @trusted Tuple!(real, int) frexp(real value);
> [snip]
>
> Having good tuple support is great, but out parameters are great too. I'm sure
> that there are plenty of cases where using out parameters is actually far
> cleaner than using tuples, since you don't have multiple return values to deal
> with. So, better enabling tuples is great, but I don't think that we're
> necessarily moving in a good direction if we're trying to eliminate out
> parameters in favor of tuples.
>
> - Jonathan M Davis

I don't think out parameter is a great idea. This is rather confusing.

I tend to think as function's parameter as input of the function and 
return value as an output. Books usualy agree, so I guess it is a valid 
point.

OOP give us another parameter to play with : this. It act as a state 
that can be modified. UFCS is nice to extends that outside OOP. This is 
a way better alternative than out parameter.

Tuples are nice too. Since we have auto, this isn't a big deal.

Both should be preferred to out parameters IMO, because the later cannot 
be differentiated at the calling point and force the programer to refers 
to the function declaration all the time or use its - limited and 
sometime inaccurate - memory. This is something we want to avoid.
March 01, 2012
Re: Tuples citizenship
Jonathan M Davis:

> I'm sure 
> that there are plenty of cases where using out parameters is actually far 
> cleaner than using tuples, since you don't have multiple return values to deal 
> with.

Are you able to show me one or more examples where using one or more out arguments is in your opinion more clear (and safer!) than using a tuple with the proposed unpacking syntax?

With a tuple you have to deal with multiple return values, but the semantics is cleaner (function => resulting tuple). The number of variables doesn't change, because with "out" you need to define them any way, before the call.

If you think with tuples you lose the names of the out arguments there is a way to avoid this problem (but I don't know if in some situations out arguments are more efficient than tuples):

pure nothrow @trusted real frexp(real value, out int exp);
==>
pure nothrow @trusted Tuple!(real, int,"exp") frexp(real value);


Also, with a tuple result there is no risk of confusion if an argument is "out" (do you remember the discussion where people have asked a ref/out annotation at the calling point too, as in C#? With tuples this problem doesn't exists).

Thank you for your answer,
bye,
bearophile
March 01, 2012
Re: Tuples citizenship
On Friday, March 02, 2012 00:05:22 deadalnix wrote:
> Le 01/03/2012 23:33, Jonathan M Davis a écrit :
> > On Thursday, March 01, 2012 17:08:44 bearophile wrote:
> >> I think std.typecons.Tuples merit to be a little more citizens in D and
> >> Phobos. I think reducing the usage of "out" argument, and replacing them
> >> with a tuple result, will avoid mistakes and make the code look better.
> >> In
> >> std.math there are functions that maybe are better to use
> >> std.typecons.Tuple:
> >> 
> >> pure nothrow @trusted real frexp(real value, out int exp);
> >> ==>
> >> pure nothrow @trusted Tuple!(real, int) frexp(real value);
> > 
> > [snip]
> > 
> > Having good tuple support is great, but out parameters are great too. I'm
> > sure that there are plenty of cases where using out parameters is
> > actually far cleaner than using tuples, since you don't have multiple
> > return values to deal with. So, better enabling tuples is great, but I
> > don't think that we're necessarily moving in a good direction if we're
> > trying to eliminate out parameters in favor of tuples.
> > 
> > - Jonathan M Davis
> 
> I don't think out parameter is a great idea. This is rather confusing.
> 
> I tend to think as function's parameter as input of the function and
> return value as an output. Books usualy agree, so I guess it is a valid
> point.
> 
> OOP give us another parameter to play with : this. It act as a state
> that can be modified. UFCS is nice to extends that outside OOP. This is
> a way better alternative than out parameter.
> 
> Tuples are nice too. Since we have auto, this isn't a big deal.
> 
> Both should be preferred to out parameters IMO, because the later cannot
> be differentiated at the calling point and force the programer to refers
> to the function declaration all the time or use its - limited and
> sometime inaccurate - memory. This is something we want to avoid.

When you're looking to mutate existing variables in the caller, using out 
parameters results in cleaner code. Tuples are inherently messier, because you 
have to deal with multiple return values. They also often do poorly when you 
need to use the functional programming, because often you want all of the 
return values for later use but only want to pass _one_ of them to the 
function that you're passing the result to.

At other times, tuples are nicer - like when you actually _want_ the return 
value packed together (though often, if that's what you really want, a struct 
might be better). And if you're not looking to assign the parts of a tuple to 
existing variables, then they're not quite as bad as when you are, since you 
don't necessarily have to then assign the pieces to other variables.

Both have value, though if you need a lot of either, you should probably 
consider whether a struct or class would suit what you're doing better.

- Jonathan M Davis
March 01, 2012
Re: Tuples citizenship
"Jonathan M Davis" <jmdavisProg@gmx.com> wrote:

> When you're looking to mutate existing variables in the caller, using out 
> parameters results in cleaner code. Tuples are inherently messier, because you 
> have to deal with multiple return values. They also often do poorly when you 
> need to use the functional programming, because often you want all of the 
> return values for later use but only want to pass _one_ of them to the 
> function that you're passing the result to.
> 

You use 'ref' to mutate existing variable, not 'out'. Multiple return value
doesn't replace the use case of ref. 

Not sure how messy it is to extract one value out of the tuple. It is much
more messier to ignore an 'out' parameter than a field in a tuple in the
call site. 

> At other times, tuples are nicer - like when you actually _want_ the return 
> value packed together (though often, if that's what you really want, a struct 
> might be better). And if you're not looking to assign the parts of a tuple to 
> existing variables, then they're not quite as bad as when you are, since you 
> don't necessarily have to then assign the pieces to other variables.
> 
> Both have value, though if you need a lot of either, you should probably 
> consider whether a struct or class would suit what you're doing better.
> 
> - Jonathan M Davis
March 01, 2012
Re: Tuples citizenship
Jonathan M Davis:

> When you're looking to mutate existing variables in the caller, using out 
> parameters results in cleaner code. Tuples are inherently messier, because you 
> have to deal with multiple return values.

out arguments have two risks:
- If you assign a value to a variable and then use it to call a function, the precedent value is ignored and overwritten.
- If in a function you forget to assign an out argument, the D compiler produces no errors.
Both source of bugs are not present with tuples.


> They also often do poorly when you 
> need to use the functional programming,

As you know functional languages use tuples all the time.


> At other times, tuples are nicer - like when you actually _want_ the return 
> value packed together (though often, if that's what you really want, a struct 
> might be better).

Defining a struct makes your code messier. D tuples support named fields too, so the advantage of using a struct is limited.


> And if you're not looking to assign the parts of a tuple to 
> existing variables, then they're not quite as bad as when you are, since you 
> don't necessarily have to then assign the pieces to other variables.

There are parts of your post that I don't fully understand.


> Both have value, though if you need a lot of either, you should probably 
> consider whether a struct or class would suit what you're doing better.

For most usages of a tuple a class instance means useless heap activity and more work for the GC.
Using a struct to return the results of a function is sometimes acceptable, but most times I don't use tuples for that purpose. Consider this code in my original post, defining two static structs doesn't do much good to such kind of code:

Tuple!(char,int)[] pairs2 = frequences.pairs;
schwartzSort!(c_f => tuple(-c_f[1], c_f[0]))(pairs2);
foreach (c_f; pairs2)
   writeln(c_f[1], " ", c_f[0]);


Tuples are often defined and used on the fly, in-place.

Bye,
bearophile
March 02, 2012
Re: Tuples citizenship
On Thursday, March 01, 2012 18:57:15 bearophile wrote:
> Jonathan M Davis:
> > They also often do poorly when you
> > need to use the functional programming,
> 
> As you know functional languages use tuples all the time.

Yes, but chaining functions is the issue. It doesn't work well with tuples 
unless the function you're passing the result to wants the tuple. If all it 
wants is one piece of the tuple, then that doesn't work well at all. You're 
forced to assign the tuple to something else and then call then function 
rather than chain calls. That's one of the reasons that you constantly end up 
using stuff like let expressions and pattern matching in functional languages. 
You don't _want_ a tuple. Dealing with a tuple is annoying. It's just that 
it's often the best tool that you have to pass disparate stuff around, so 
that's what you use.

> > And if you're not looking to assign the parts of a tuple to
> > existing variables, then they're not quite as bad as when you are, since
> > you don't necessarily have to then assign the pieces to other variables.
> There are parts of your post that I don't fully understand.

It is often really annoying to have to deal with tuple return values, because 
you have to worry about unpacking the result. I don't want to use a tuple in 
the caller. Tuples are generally for grouping unrelated data that you don't 
necessarily want to keep togother (since if you did, you'd generally use a 
struct). I want the result to actually be assigned to variables. That is 
definitely cleaner with out than with tuples.

int exp;
auto result = frexp(value, exp);

vs

auto tup = frexp(value);
result = tup[0];
exp = tup[1];

Getting tuple return values is annoying. Yes, it can be useful, but most stuff 
doesn't operate on tuples. It operates on the pieces of tuples. So, you have 
to constantly break them up. So, using out results in much nicer code.

It always feels like I'm fighting the code when I have to deal with tuple 
return values.

- Jonathan M Davis
March 02, 2012
Re: Tuples citizenship
Jonathan M Davis:

> Yes, but chaining functions is the issue. It doesn't work well with tuples 
> unless the function you're passing the result to wants the tuple. If all it 
> wants is one piece of the tuple, then that doesn't work well at all. You're 
> forced to assign the tuple to something else and then call then function 
> rather than chain calls.

In the years I have used a mountain of tuples in Python, but I barely perceive that problem, so I think it's not so bad.


> int exp;
> auto result = frexp(value, exp);
> 
> vs
> 
> auto tup = frexp(value);
> result = tup[0];
> exp = tup[1];

I have assumed to use a sane tuple unpacking syntax. So the second part of your comparison is:

immutable (result, exp) = frexp(value);

That is better than your version with the out argument, safer, looks better, and you are even able to make both results constant.


> Getting tuple return values is annoying. Yes, it can be useful, but most stuff 
> doesn't operate on tuples. It operates on the pieces of tuples. So, you have 
> to constantly break them up. So, using out results in much nicer code.

I think that the tuple unpacking syntax is able to avoid part of your problems. The point of my original post, that maybe was lost in the bulk of the text, was that an unpacking syntax sugar is very useful if you want to use tuples for real in D.

Bye and thank you,
bearophile
« First   ‹ Prev
1 2 3
Top | Discussion index | About this forum | D home