On 30/05/14 22:45, monarch_dodra via Digitalmars-d-learn wrote:
> Didn't you make changes to how and when the global PRNG is popped and accessed
> in randomShuffle? I figured it *could* be an explanation.

No, it was partialShuffle that I tweaked, and that shouldn't affect the results here.  There was a bug where partialShuffle would _always_ shuffle the whole of the input; but that wouldn't affect the results of randomShuffle itself, which calls partialShuffle with the explicit request to shuffle the entire array.

Just to be sure I checked with random/partialShuffle pre- and post- my patches, and the result is identical: [1, 8, 4, 2, 0, 7, 5, 6, 9, 3] just as the OP gets for dmd.

I think it's more likely that the culprit is either your set of patches to the Mersenne Twister, or the patches made to uniform() (which is called by partialShuffle).  I'll look more deeply into this.

On 31/05/14 22:37, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
> On 30/05/14 22:45, monarch_dodra via Digitalmars-d-learn wrote:
>> Didn't you make changes to how and when the global PRNG is popped and accessed
>> in randomShuffle? I figured it *could* be an explanation.
>
> I think it's more likely that the culprit is either your set of patches to the
> Mersenne Twister, or the patches made to uniform() (which is called by
> partialShuffle).  I'll look more deeply into this.

It's due to the the updated uniform() provided in this pull request:
https://github.com/D-Programming-Language/phobos/commit/fc48d56284f19bf171780554b63b4ae83808b894

On 31/05/14 23:22, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
> It's due to the the updated uniform() provided in this pull request:
> https://github.com/D-Programming-Language/phobos/commit/fc48d56284f19bf171780554b63b4ae83808b894

You can see the effects in action by running e.g. the following D code:

//--------------------------------------------------------------
import std.algorithm, std.random, std.range, std.stdio;

void main()
{
    rndGen.seed(12);
    iota(0, 10).map!(a => uniform(0, 10 - a, rndGen)).writeln;
}
//--------------------------------------------------------------

... which corresponds to the underlying calls to uniform() found in the shuffle process.

On Saturday, 31 May 2014 at 21:22:48 UTC, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
> On 31/05/14 22:37, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
>> On 30/05/14 22:45, monarch_dodra via Digitalmars-d-learn wrote:
>>> Didn't you make changes to how and when the global PRNG is popped and accessed
>>> in randomShuffle? I figured it *could* be an explanation.
>>
>> I think it's more likely that the culprit is either your set of patches to the
>> Mersenne Twister, or the patches made to uniform() (which is called by
>> partialShuffle).  I'll look more deeply into this.
>
> It's due to the the updated uniform() provided in this pull request:
> https://github.com/D-Programming-Language/phobos/commit/fc48d56284f19bf171780554b63b4ae83808b894

I second the thought that reproducibility across different versions is an important feature of any random generation library.  Sadly, I didn't use a language yet which supported such a flavor of reproducibility for a significant period of time in its default random library, so I have to use my own randomness routines when it matters.  I've reported my concern [1] at the moment of breakage, but apparently it didn't convince people.  Perhaps I should make a more significant effort next time (like a pull request) for the things that matter to me.  Well, now I know it does matter for others, at least.

In short, if uniform() has to be tweaked, the sooner it happens, the better.

Alternatively, the library design could allow different uniform() implementations to be plugged in, and provide legacy implementations along with the current (default) one.  In that case, all one has to do to reproduce the old behavior is to plug the appropriate one in.

[1] http://forum.dlang.org/thread/vgmdoyyqhcqurpmobyuy@forum.dlang.org#post-gjuprkxzmcbdixtbucea:40forum.dlang.org

Thank you for hunting down the difference, in my case it's not a deal breaking problem. I can just specify the compiler and language version, then the results become reproducible. And I'm sure I'll appreciate the performance boost!

On Sunday, 1 June 2014 at 12:11:22 UTC, Ivan Kazmenko wrote:
> On Saturday, 31 May 2014 at 21:22:48 UTC, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
>> On 31/05/14 22:37, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
>>> On 30/05/14 22:45, monarch_dodra via Digitalmars-d-learn wrote:
>>>> Didn't you make changes to how and when the global PRNG is popped and accessed
>>>> in randomShuffle? I figured it *could* be an explanation.
>>>
>>> I think it's more likely that the culprit is either your set of patches to the
>>> Mersenne Twister, or the patches made to uniform() (which is called by
>>> partialShuffle).  I'll look more deeply into this.
>>
>> It's due to the the updated uniform() provided in this pull request:
>> https://github.com/D-Programming-Language/phobos/commit/fc48d56284f19bf171780554b63b4ae83808b894
>
> I second the thought that reproducibility across different versions is an important feature of any random generation library.  Sadly, I didn't use a language yet which supported such a flavor of reproducibility for a significant period of time in its default random library, so I have to use my own randomness routines when it matters.  I've reported my concern [1] at the moment of breakage, but apparently it didn't convince people.  Perhaps I should make a more significant effort next time (like a pull request) for the things that matter to me.  Well, now I know it does matter for others, at least.
>
> In short, if uniform() has to be tweaked, the sooner it happens, the better.
>
> Alternatively, the library design could allow different uniform() implementations to be plugged in, and provide legacy implementations along with the current (default) one.  In that case, all one has to do to reproduce the old behavior is to plug the appropriate one in.
>
> [1] http://forum.dlang.org/thread/vgmdoyyqhcqurpmobyuy@forum.dlang.org#post-gjuprkxzmcbdixtbucea:40forum.dlang.org

On 01/06/14 14:11, Ivan Kazmenko via Digitalmars-d-learn wrote:
> I second the thought that reproducibility across different versions is an
> important feature of any random generation library.  Sadly, I didn't use a
> language yet which supported such a flavor of reproducibility for a significant
> period of time in its default random library, so I have to use my own randomness
> routines when it matters.  I've reported my concern [1] at the moment of
> breakage, but apparently it didn't convince people. Perhaps I should make a more
> significant effort next time (like a pull request) for the things that matter to
> me.  Well, now I know it does matter for others, at least.

Yes, there probably should be a high bar for changes that break reproducibility in this way (although there certainly shouldn't be a ban: we shouldn't artificially constrain ourselves to avoid significant improvements to the module).

I missed the debate at the time, but actually, I'm slightly more concerned over the remark in that discussion that the new uniform was ported from java.util.Random.  Isn't OpenJDK GPL-licensed ... ?

On Sunday, 1 June 2014 at 14:22:31 UTC, Joseph Rushton Wakeling via Digitalmars-d-learn wrote:
> I missed the debate at the time, but actually, I'm slightly more concerned over the remark in that discussion that the new uniform was ported from java.util.Random.  Isn't OpenJDK GPL-licensed ... ?

To clarify, no, the uniform was not ported from Java's Random.

The comment in question:
https://github.com/D-Programming-Language/phobos/pull/1717#issuecomment-29712117

It was found after the code was made. It's more-or-less the same approach (so, it gives credit to the mathematical soundness) but it was discovered independently.

Having read more of the debate, I think coverage is more important than reproducibility. From my point of view, I'm not sure if there's much point in giving reproducible wrong answers.

On Sunday, 1 June 2014 at 12:11:22 UTC, Ivan Kazmenko wrote:
> I second the thought that reproducibility across different versions is an important feature of any random generation library.  Sadly, I didn't use a language yet which supported such a flavor of reproducibility for a significant period of time in its default random library, so I have to use my own randomness routines when it matters.

It can be done by using standardized algorithms like one in FIPS 186-2. Because it's standardized, it can't be improved.

On 02/06/14 08:57, Chris Cain via Digitalmars-d-learn wrote:
> On Sunday, 1 June 2014 at 14:22:31 UTC, Joseph Rushton Wakeling via
> Digitalmars-d-learn wrote:
>> I missed the debate at the time, but actually, I'm slightly more concerned
>> over the remark in that discussion that the new uniform was ported from
>> java.util.Random.  Isn't OpenJDK GPL-licensed ... ?
>
> To clarify, no, the uniform was not ported from Java's Random.

I'm really sorry, Chris, I was obviously mixing things up: on rereading, the person in the earlier forum discussion (not PR thread) who talks about porting from Java wasn't you.  I'm very glad to be corrected on that; your PR was a nice submission and I'm glad we have it in Phobos.