On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
> Hi,
>
> I found many times that people use unpredictableSeed in combination with normal PRNG for cryptographic purpose. Some even go as far as reseeding at each call to try making it more secure.
>
> It is a dangerous practice, most PRNG are not designed with security (and unpredictability) in mind, and unpredictableSeed was definitely not designed with security in mind (or it failed heavily at it). It's a good tool when one needs randomness, not security.
>
> I wrote a blog post to present exactly why this is a bad idea and how it could be exploited [1].
>
> The best would be to add a standard CSPRNG interface to Phobos but we aren't there yet.
>
> [1]: https://cym13.github.io/article/unpredictableSeed.html

When I see the code for unpredictableSeed I went face palm really hard.

I did some digging, and it was way way worse:

https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529

https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e

On 3/2/17 4:50 PM, Yuxuan Shui wrote:
> On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
>> Hi,
>>
>> I found many times that people use unpredictableSeed in combination
>> with normal PRNG for cryptographic purpose. Some even go as far as
>> reseeding at each call to try making it more secure.
>>
>> It is a dangerous practice, most PRNG are not designed with security
>> (and unpredictability) in mind, and unpredictableSeed was definitely
>> not designed with security in mind (or it failed heavily at it). It's
>> a good tool when one needs randomness, not security.
>>
>> I wrote a blog post to present exactly why this is a bad idea and how
>> it could be exploited [1].
>>
>> The best would be to add a standard CSPRNG interface to Phobos but we
>> aren't there yet.
>>
>> [1]: https://cym13.github.io/article/unpredictableSeed.html
>
> When I see the code for unpredictableSeed I went face palm really hard.
>
> I did some digging, and it was way way worse:
>
> https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529
>
>
> https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e

Could you please submit a PR that makes is better? Thanks! -- Andrei

On Thursday, 2 March 2017 at 21:50:36 UTC, Yuxuan Shui wrote:
> On Sunday, 26 February 2017 at 18:23:27 UTC, cym13 wrote:
>> Hi,
>>
>> I found many times that people use unpredictableSeed in combination with normal PRNG for cryptographic purpose. Some even go as far as reseeding at each call to try making it more secure.
>>
>> It is a dangerous practice, most PRNG are not designed with security (and unpredictability) in mind, and unpredictableSeed was definitely not designed with security in mind (or it failed heavily at it). It's a good tool when one needs randomness, not security.
>>
>> I wrote a blog post to present exactly why this is a bad idea and how it could be exploited [1].
>>
>> The best would be to add a standard CSPRNG interface to Phobos but we aren't there yet.
>>
>> [1]: https://cym13.github.io/article/unpredictableSeed.html
>
> When I see the code for unpredictableSeed I went face palm really hard.
>
> I did some digging, and it was way way worse:
>
> https://github.com/dlang/phobos/commit/ff54d867e41abc8261075f0dce1261d68ee09180#diff-713ce153554afc99a07767cc8ba940aeL529
>
> https://github.com/dlang/phobos/commit/c433c36658df45677bf90b00e93cba051883294e

This is a misunderstanding: unpredictableSeed is perfectly fine as it is. What's wrong is 1) using it for cryptographic purpose and 2) systematic reseeding.

1) There is no way to make a cryptographically secure pseudo-random number generator that is seedable. If a PRNG is seedable then his number of states is finite which makes it cycle one way or an other once you've expended all possible states. So no cryptographic application should use such PRNG, and therefore any seed. For non-cryptographic purpose unpredictableSeed is, honnestly, random enough. It isn't you're actual PRNG (or shouldn't be, see point 2 but is only used to reseed it from time to time.

2) The big mistake is systematic reseeding which is far more common than it should be. Using unpredictableSeed as a seed is fine, the actual PRNG that is seeded will add a lot of entropy to the output. However by systematically reseeding it makes unpredictableSeed the PRNG that is actually used (ie, it doesn't leave it any time to add entropy). And that is something that should never happen because the PRNG in unpredictableSeed is the weakest possible. It is not meant to be the actual PRNG.

So this article wasn't meant to be "Haha, Phobos is broken, *facepalm*". It was about using tools for what they're meant and nothing else (especially when dealing with cryptographic problems). The problem, if anything, is in the documentation that doesn't enforce that point.

On 03/05/2017 05:48 AM, cym13 wrote:
> The problem, if anything, is in the documentation that doesn't enforce
> that point.

Thanks. Could we have you, Yishan, or other security expert submit a PR for the documentation? I'm not an expert and my coding of unpredictableSeed has been at the level of "I don't like those games that always do the same `random` things when you start playing."

Another good thing pointed by the article would be to use the Mersenne twister for unpredictableSeed, which would make it difficult to infer the sequence from a few samples. Please share if that would be a good thing to do.


Thanks! -- Andrei

On Sunday, 5 March 2017 at 15:30:29 UTC, Andrei Alexandrescu wrote:
> Another good thing pointed by the article would be to use the Mersenne twister for unpredictableSeed, which would make it difficult to infer the sequence from a few samples. Please share if that would be a good thing to do.

"From just a few samples" yes, but in general the internal state of Mersenne twister is easy to reverse – e.g. check out https://github.com/fx5/not_random for a nice little demonstration.

IMHO there is no problem with unpredictableSeed not being cryptographically secure, as long as the docs point out it is not to be used anywhere near where a CSPRNG should be. But I'm hardly an expert either.

 — David

On Sun, Mar 05, 2017 at 10:30:29AM -0500, Andrei Alexandrescu via Digitalmars-d wrote: [...]
> Another good thing pointed by the article would be to use the Mersenne twister for unpredictableSeed, which would make it difficult to infer the sequence from a few samples. Please share if that would be a good thing to do.
[...]

Wait, isn't that missing the point?

I thought the whole point of the article was that you shouldn't be using unpredictableSeed as your PRNG.  It's only supposed to give a random-enough value to get your chosen PRNG into a (hopefully) unpredictable initial state. But you should be using values from the PRNG, not from unpredictableSeed!  Otherwise that's totally missing the point.

It's possible to use unpredictableSeed for occasionally reseeding your PRNG, but that should be quite infrequent.  If you find yourself reseeding your PRNG every other minute, or worse, every time you call your PRNG, then you're doing something very, very wrong.

Using the Mersenne twister to generate unpredictableSeed seems to me to be completely backwards.  It should be the other way round: the value of unpredictableSeed should be random enough to be suitable for seeding a Mersenne twister algorithm, so that it will start off the algorithm in a random initial state (and you should be getting values from the algorithm thereafter, not from unpredictableSeed).


T

-- 
Unix is my IDE. -- Justin Whear

On 03/05/2017 07:25 PM, H. S. Teoh via Digitalmars-d wrote:
> On Sun, Mar 05, 2017 at 10:30:29AM -0500, Andrei Alexandrescu via Digitalmars-d wrote:
> [...]
>> Another good thing pointed by the article would be to use the Mersenne
>> twister for unpredictableSeed, which would make it difficult to infer
>> the sequence from a few samples. Please share if that would be a good
>> thing to do.
> [...]
>
> Wait, isn't that missing the point?
>
> I thought the whole point of the article was that you shouldn't be using
> unpredictableSeed as your PRNG.  It's only supposed to give a
> random-enough value to get your chosen PRNG into a (hopefully)
> unpredictable initial state. But you should be using values from the
> PRNG, not from unpredictableSeed!  Otherwise that's totally missing the
> point.
>
> It's possible to use unpredictableSeed for occasionally reseeding your
> PRNG, but that should be quite infrequent.  If you find yourself
> reseeding your PRNG every other minute, or worse, every time you call
> your PRNG, then you're doing something very, very wrong.
>
> Using the Mersenne twister to generate unpredictableSeed seems to me to
> be completely backwards.  It should be the other way round: the value of
> unpredictableSeed should be random enough to be suitable for seeding a
> Mersenne twister algorithm, so that it will start off the algorithm in a
> random initial state (and you should be getting values from the
> algorithm thereafter, not from unpredictableSeed).

Well, the big point is that nothing involving seeds should come near anything security-releated. Then secondly, (like you say) don't use any initial-seed-getter *as* an RNG.

But that aside, *if*[1] we do want to increase the entropy in unpredictableSeed, we should use /dev/(u)random and _RtlGenRandom. Like you say, anything less is kinda missing the point (unless someone can argue the current one is insufficient for non-security randomization).

[1] And I'm not sure we necessarily do want to. Current unpredictableSeed seems good enough as-is for non-security purposes, and we don't want to give people more reason to erroneously think it's ok to use unpredictableSeed for salts, tokens and the like.

On Sunday, 5 March 2017 at 15:30:29 UTC, Andrei Alexandrescu wrote:
> Another good thing pointed by the article would be to use the Mersenne twister for unpredictableSeed, which would make it difficult to infer the sequence from a few samples. Please share if that would be a good thing to do.

Seeding a PRNG from the same PRNG often gives bad results.
Also maybe rename it to arbitrarySeed, unpredictability is not the right claim here.

On 26/02/17 20:23, cym13 wrote:
> Hi,
>
> I found many times that people use unpredictableSeed in combination with
> normal PRNG for cryptographic purpose. Some even go as far as reseeding
> at each call to try making it more secure.
>
> It is a dangerous practice, most PRNG are not designed with security
> (and unpredictability) in mind, and unpredictableSeed was definitely not
> designed with security in mind (or it failed heavily at it). It's a good
> tool when one needs randomness, not security.
>
> I wrote a blog post to present exactly why this is a bad idea and how it
> could be exploited [1].
>
> The best would be to add a standard CSPRNG interface to Phobos but we
> aren't there yet.
>
> [1]: https://cym13.github.io/article/unpredictableSeed.html

Excuse me if I'm asking a trivial question. Why not just seed it from /dev/urandom? (or equivalent on non-Linux platforms. I know at least Windows has an equivalent).

Shachar

On Monday, 6 March 2017 at 10:12:09 UTC, Shachar Shemesh wrote:
> Excuse me if I'm asking a trivial question. Why not just seed it from /dev/urandom? (or equivalent on non-Linux platforms. I know at least Windows has an equivalent).
>
> Shachar

One reason is that /dev/urandom isn't always available, e.g., in a chroot.  Sure, these are corner cases, but it's annoying when stuff like this doesn't "just work".