On 6/11/2014 2:41 AM, Joseph Rushton Wakeling wrote:
>> 5. Another possible improvement would be something akin to a "remix"
>> function. It should work identically to reseeding, but instead of
>> setting the internal state to match the seed (as I see in
>> https://github.com/WebDrake/hap/blob/master/source/hap/random/generator.d#L485),
>> remixing should probably be XOR'd into the current state. That way if
>> you have a state based on some real entropy, you can slowly, over
>> time, drip in more entropy into the state.
>
> Also a very interesting suggestion.  Is there a standard name for this
> kind of procedure?
>

NIST's crypto-RNG papers just refer to it as "reseeding", so there might not be a standard name for it. FWIW, I've taken to calling it "accumulating entropy".

On Wednesday, 11 June 2014 at 06:41:34 UTC, Joseph Rushton Wakeling wrote:
> That would be very cool.  Can you point me at your code examples?

It's written in Nimrod (in a way that someone who learned Nimrod the day before would write them, because I learned Nimrod the day before and worked on it for something like 17 hours straight to produce everything):

https://github.com/Zshazz/Ramrod/blob/master/util.nim

I'd like to make this concept a range in D. Not sure what exactly to call it but it's an "adaptor." Honestly, I wouldn't be surprised if something like this didn't already exist in D in some form, but it didn't seem like Nimrod had anything like it.

> The paranoiac in me feels that anything that involves getting random data via HTTPS is probably insecure crypto-wise :-)

Paranoia is good in this case. I appreciate the caution.

> However, I think sourcing random.org is a perfect case for an entry in hap.random.device.  I think the best thing to do would probably be to offer a RandomOrgClient (which offers a very thin API around the random.org HTTP API) and then to wrap that in a range type that uses the client internally to generate random numbers with particular properties.

This sounds like it would be beautiful. As a note, if we expose this via a part of the standard library, we would have to make certain that we follow the guidelines outlined on random.org (in particular, I'm concerned about having an internal locking mechanism to prevent multiple threads from asking for bits at the same time because that will cause clients to be banned ... global state, impurity, and all the nasty things will likely have to be a natural part of such a thing).

> Also a very interesting suggestion.  Is there a standard name for this kind of procedure?

I'm not really aware if there is. I remember hearing about the concept when talking with my cryptography professor awhile back (it may have even been in a lecture). IIRC, the description used was "mixing" in entropy, so my first thought is to call it a mix/remix function.

> Just for clarity, here's how I see things rolling out for the future:
>
>   * First goal is to ensure the existing codebase "plays nice" with
>     people's programs and that it works OK with dub, rdmd, etc. and
>     doesn't have any serious architectural or other bugs.  The 1.0.0
>     release will not have any new functionality compared to what is
>     in place now.
>
>   * Once it seems to be reasonably stable then work can begin on a
>     1.x release series that brings in successive pieces of new
>     functionality.

I like this procedure. Definitely confidence inspiring.

On Wednesday, 11 June 2014 at 06:48:24 UTC, Joseph Rushton Wakeling wrote:
> Incidentally, would it be a good idea to post a link to the blog post on r/programming?  Haven't done so yet, as generally I prefer to leave decisions about D publicity to others, but can do so if people would like.

I almost always like all the D posts I see on r/programming, but in general if any language highlighted the efforts in the RNG part of the standard library, I would like it. Too many languages get it wrong or don't care enough about it. (My most basic litmus test is to check a language's shuffle function... Too many fail, even if they claim to do the Knuth shuffle, often they make the small mistakes that matter)

It definitely gets my vote for publicizing.

Have you any plan to implement CMWC?

http://en.wikipedia.org/wiki/Multiply-with-carry#Complementary-multiply-with-carry_generators

On Monday, 9 June 2014 at 18:09:21 UTC, Joseph Rushton Wakeling wrote:
> Hello all,
>
> Some of you may remember my earlier draft of a class-based std.random successor:
> http://forum.dlang.org/thread/cyytvhixkqlbwkmiugex@forum.dlang.org
>
> Following revisions made in response to feedback, and some further development, I decided that it would be best to release the results as a dub package with a new library name:
> http://code.dlang.org/packages/hap
>
> Source code and documentation is available here:
> https://github.com/WebDrake/hap
> http://code.braingam.es/hap/random/
>
> I've also written a blog post describing new features and the motivations behind this library:
> http://braingam.es/2014/06/hap-random-a-new-random-number-library-for-d/
>
> I think that hap.random fixes certain fundamental design issues with std.random.  However, this needs to be put to the test "in the wild", so I'd really appreciate it if as many people as possible could try it out with their code, and report on the experience:
>
>    * Does it run faster, slower, etc?
>
>    * Do any undesirable memory allocation issues arise?
>
>    * Is the API (broadly similar but not identical to std.random)
>      pleasant to use?
>
> If it proves to be effective for everyone, then I will begin the process of submission as a new Phobos module.
>
> Thanks in advance for all testing and feedback.
>
> Best wishes,
>
>     -- Joe

On Wednesday, 11 June 2014 at 07:42:10 UTC, Andrea Fontana wrote:
> Have you any plan to implement CMWC?
>
> http://en.wikipedia.org/wiki/Multiply-with-carry#Complementary-multiply-with-carry_generators

I hadn't made any concrete plans about that particular family of generators (my impression was that Xorshift and its successors are superior), but I'll happily take patches or a feature request :-)

On Tuesday, 10 June 2014 at 10:57:32 UTC, bearophile wrote:
> Kagamin:
>
>> Pass it by reference, I see no reason why MT can't be pure.
>
> I meant strongly pure :-)

I'm afraid, this pure rng pattern precludes all pure optimizations, so it's effectively weakly pure.

On Tuesday, 10 June 2014 at 23:08:33 UTC, Chris Cain wrote:
> 4. I'd just like to say the idea of using ranges for seeds gets me giddy because I could totally see a range that queries https://random.org for true random bits to seed with, wrapped by a range that zeroes out the memory on popFront. Convenient and safe (possibly? Needs review before I get excited, obviously) for crypto purposes!

In some scenarios impredictability is not enough. For example, when you generate a session id, an attacker doesn't have to predict it ahead of time, he can guess it at any time later. And if they listen to radio waves - that's an "open protocol", an attacker can setup antenna near their antenna and get the same readings. Cryptographic PRNG and quantum TRNG are better isolated, so it's harder to read them.

On 6/11/2014 12:35 PM, Kagamin wrote:
>
> In some scenarios impredictability is not enough. For example, when you
> generate a session id, an attacker doesn't have to predict it ahead of
> time, he can guess it at any time later. And if they listen to radio
> waves - that's an "open protocol", an attacker can setup antenna near
> their antenna and get the same readings.

An interesting point.

> Cryptographic PRNG and quantum
> TRNG are better isolated, so it's harder to read them.

FWIW, a cryptographic PRNG isn't necessarily well-isolated. Being a PRNG, the isolation of a cryptographic PRNG is primarily limited to two main things:

- The isolation of its entropy source(s) (which are not normally part of a crypto-PRNG's specification - it's just left as "choose a good one"), and

- The patterns of how data is drawn from the PRNG.

If the entropy source is poorly isolated (via poor choice of entropy source, or a failure within the entropy source), and the requests being made to the PRNG are relatively predictable or even guessable (quite likely given the nature of software), then a cryptographic PRNG won't be any better isolated than, say, the digits of PI.

TL;DR: The isolation of a cryptographic PRNG is that of its external entropy source, not the cryptographic PRNG algorithm itself.

On Wednesday, 11 June 2014 at 07:24:11 UTC, Chris Cain wrote:
> I almost always like all the D posts I see on r/programming, but in general if any language highlighted the efforts in the RNG part of the standard library, I would like it. Too many languages get it wrong or don't care enough about it. (My most basic litmus test is to check a language's shuffle function... Too many fail, even if they claim to do the Knuth shuffle, often they make the small mistakes that matter)
>
> It definitely gets my vote for publicizing.

http://www.reddit.com/r/programming/comments/27wohj/haprandom_a_new_random_number_library_for_the_d/

:-)

On Wednesday, 11 June 2014 at 16:35:31 UTC, Kagamin wrote:
> In some scenarios impredictability is not enough. For example, when you generate a session id, an attacker doesn't have to predict it ahead of time, he can guess it at any time later. And if they listen to radio waves - that's an "open protocol", an attacker can setup antenna near their antenna and get the same readings. Cryptographic PRNG and quantum TRNG are better isolated, so it's harder to read them.

That's an interesting thought on a potential attack. I wouldn't say "same readings" but similar readings is possible and might make attacks easier.

It might not be a bad idea as part of a solution though, since it can be used to supplement other sources of local-machine crypto-grade entropy (since often such sources are exhaustible). But yes, just straight up using it alone appears to have a few critical problems.