Jump to page: 1 24  
Page
Thread overview
std.random suggestions
5 days ago
H. S. Teoh
5 days ago
Serg Gini
5 days ago
H. S. Teoh
5 days ago
Monkyyy
5 days ago
Monkyyy
5 days ago
monkyyy
5 days ago
monkyyy
3 days ago
IchorDev
1 day ago
IchorDev
1 day ago
monkyyy
1 day ago
IchorDev
1 day ago
Dennis
1 day ago
IchorDev
21 hours ago
Dennis
9 hours ago
IchorDev
8 hours ago
monkyyy
7 hours ago
claptrap
5 hours ago
Mike Parker
11 hours ago
0xEAB
3 minutes ago
Denis Feklushkin
1 hour ago
GrimMaple
5 days ago
I don't like the way the module std.random is designed

From a software side we have three types of ~random numbers sources:

1. Hardware (or environmental) "true" noise.
The most reliable source, which may not be that fast, i.e. it may be emptied in some cases, so it can be blocking and non-blocking in block devices terms.
This is exacatly what provided by /dev/random in Linux. It should also be noted that this random source type does not exist on all platforms.

2. Based on hardware noise (described in 1) seeded pseudo-random sequence.
Less (but still) reliable because it extrapolates a true random number using a deterministic algorithm (like described in 3 below). Suitable for generating large volumes of numbers.
This is exacatly what provided by /dev/urandom in Linux. Again, not all platforms providing it.

3. Pre-determined pseudorandom sequences based on some predictable algorithm.
Also good for getting large amounts of numbers and fast, but it strictly can't be used for cryptography etc. Its advantage is that it is always available in all systems since (at worst) it is just a mathematical function.

I think if we save users from deepening into details this will only go to the benefit of security. So, my suggestion:

1. Do not provide any access about entropy sources (I am about std.internal.entropy.EntropySource.tryAll and forceEntropySource). At the application programming level we usually have one source of true entropy. (If this is not so please correct me.) No need to make ambitious interfaces describing the theoretical diversity of RNGs. The "entropy" word can be excluded from the API description completely.

2. Completely exclude "seeding" concept: this is a source of potential issues (https://github.com/dlang/phobos/pull/10865/commits/3c2f87ef745ca6de6e392007421af81e661aecbe). Seeding can be encapsulated inside of of urandom generator (see 2 above) if needed.

In fact, you know exactly what amount and quality of random bytes you want to get at some point of your code. And, for example, if system does not provides true RNG needed by you, then let the corresponding function be totally unavailable for compilation and leads to compile time error. Then you can't accidentally build your neat designed software with weak predictable RNG.

It follows from this that it is necessary to provide only four points for obtaining random numbers, all without the need for any combining of them by users. (My suggestion is place each of it in dedicated std.random.* module)

1. std.random.truerandom: implemented as OS/hardware call if system provides hardware (or environmental) random number generator. Suitable for encryption key generation, etc. Maybe three functions will be provided, something like:

ubyte[Size] trueRandom(size_t Size)(); // blocks and waits if no enough entropy
bool trueRandom(ref ubyte[] result); // returns false if no enough entropy
void trueRandomEx(ref ubyte[] result); // throws if no enough entropy

If there is no random number generator in the system, then these functions will not be available and the compilation may end with the error!

2. std.random.seededrandom: function(s) that is implemented either by OS call (for Linux/Windows/Mac) or by some another TRNG call + PseudoRandom (on baremetal platforms). Does not blocks and not throws. Does not exist if there is no TRNG avalable because internally uses seed value. So, again: if there is no random number generator in the system, then these functions will not be available and the compilation may fail.
These are pretty good random numbers for general purpose like UUID generation.

3. std.random.pseudorandom.pseudoRandom: not for cryptography at all. Name was specifically chosen so that the user would clearly see the "pseudo" prefix.
Suitable for drawing a starry sky in a retro games or so one. Internally calls std.random.seededrandom if exists or uses std.random.predetermined (4) with seed value if seededrandom doesn't exist. Guaranteed to exist on all platforms.

4. std.random.predetermined: functions implementing pseudorandom number generators (PRNG). Mostly for internal use, but sometimes users may want to get guaranteed repeatability of pseudorandom sequences.

That's all, and nothing superfluous! I.e., if you do not use something, it does not creates any global variables, etc. From the point of view of the user, seems, everything is also simple and clear. It will be difficult to make a issue with RNG in this case.

Perhaps this is suitable for Phobos 3?

5 days ago
On 17/09/2025 12:04 AM, Denis Feklushkin wrote:
> Perhaps this is suitable for Phobos 3?

Not for V2.

https://github.com/dlang/PhobosV3-Design

5 days ago
On Tuesday, 16 September 2025 at 12:04:20 UTC, Denis Feklushkin wrote:
> I don't like the way the module std.random is designed
> Perhaps this is suitable for Phobos 3?

I think we also should take into account

- http://mir-random.libmir.org/
- https://dlangscience.github.io/dstats/api/dstats/random.html
5 days ago
On Tuesday, 16 September 2025 at 12:36:41 UTC, Serg Gini wrote:
> On Tuesday, 16 September 2025 at 12:04:20 UTC, Denis Feklushkin wrote:
>> I don't like the way the module std.random is designed
>> Perhaps this is suitable for Phobos 3?
>
> I think we also should take into account
>
> - http://mir-random.libmir.org/
> - https://dlangscience.github.io/dstats/api/dstats/random.html

This is more high-level functions

mir-random have ~roughly same design as current std.random

5 days ago
On Tuesday, 16 September 2025 at 12:08:38 UTC, Richard (Rikki) Andrew Cattermole wrote:

> Not for V2.
>
> https://github.com/dlang/PhobosV3-Design

https://github.com/dlang/PhobosV3-Design/discussions/42
5 days ago
On Tue, Sep 16, 2025 at 12:04:20PM +0000, Denis Feklushkin via Digitalmars-d wrote:
> I don't like the way the module std.random is designed
[...]
> I think if we save users from deepening into details this will only go to the benefit of security. So, my suggestion:
> 
> 1. Do not provide any access about entropy sources (I am about std.internal.entropy.EntropySource.tryAll and forceEntropySource).
[...]

Users are not supposed to use anything from std.internal.  It's named "internal" for a reason.


> 2. Completely exclude "seeding" concept: this is a source of potential
> issues
> (https://github.com/dlang/phobos/pull/10865/commits/3c2f87ef745ca6de6e392007421af81e661aecbe).
> Seeding can be encapsulated inside of of urandom generator (see 2
> above) if needed.

Seeding is useful for Monte Carlo simulations where you need pseudorandom numbers in large quantity, but completely reproducible from a seed value (e.g. for verification of certain results).  Being unable to use seeding means users have to roll their own generators, which usually means it's done poorly because generating random numbers is not as simple as it looks.


[...]
> It follows from this that it is necessary to provide only four points for obtaining random numbers, all without the need for any combining of them by users. (My suggestion is place each of it in dedicated std.random.* module)

std.random is not designed for cryptographically-safe random generation. We neither have the resources nor the expertise to do this.  It should NOT be used for anything related to authentication, security, cryptography, or anything similar.  If you need random numbers for any of these areas, please use a battle-tested library like openssl or one of the similar variants.  It's unsafe to use anything that wasn't explicitly designed for security.


> 1. std.random.truerandom: implemented as OS/hardware call if system provides hardware (or environmental) random number generator. Suitable for encryption key generation, etc. Maybe three functions will be provided, something like:

There's no need for this.  You could just open /dev/random or /dev/urandom as a file yourself, and read whatever you need from it. It's OS-dependent anyway, so there's no need to add another layer of abstraction to pretend that you're portable.

The problem with offering a crypto-level RNG in the standard library is that you need an active maintainer who's keeping on top of the latest security weaknesses and updates, and who has the expertise to actually understand the issues involved, because when it comes to crypto, rolling your own is extremely risky and almost certain to fall into unexpected pitfalls that will greatly weaken the security of your applications.  We have no such person and no such expertise currently.  Unless a crypto researcher shows up who has the time and energy to spend in such a project, I don't think we should go this route.

Or if what you're really asking for is an API to /dev/random or /dev/urandom, then why not just open them as files yourself?  That's why these devices are in /dev/ in the first place.


[...]
> 3. std.random.pseudorandom.pseudoRandom: not for cryptography at all. Name was specifically chosen so that the user would clearly see the "pseudo" prefix.
[...]

What's currently in std.random falls in this category. Wouldn't be a bad idea to rename it this way.  Good idea.


T

-- 
Real men don't take backups. They put their source on a public FTP-server and let the world mirror it. -- Linus Torvalds
5 days ago

On Tuesday, 16 September 2025 at 15:06:39 UTC, H. S. Teoh wrote:

>

On Tue, Sep 16, 2025 at 12:04:20PM +0000, Denis Feklushkin via Digitalmars-d wrote:

>

I don't like the way the module std.random is designed
[...]
I think if we save users from deepening into details this will only go to the benefit of security. So, my suggestion:

  1. Do not provide any access about entropy sources (I am about std.internal.entropy.EntropySource.tryAll and forceEntropySource).
    [...]

Users are not supposed to use anything from std.internal. It's named "internal" for a reason.

Yes. But I saw that this function is public out and the fact that it even exists is strange.

> >
  1. Completely exclude "seeding" concept: this is a source of potential
    issues
    (https://github.com/dlang/phobos/pull/10865/commits/3c2f87ef745ca6de6e392007421af81e661aecbe).
    Seeding can be encapsulated inside of of urandom generator (see 2
    above) if needed.

Seeding is useful for Monte Carlo simulations where you need pseudorandom numbers in large quantity, but completely reproducible from a seed value (e.g. for verification of certain results).

Sugggested std.random.predetermined exactly for this purpose

At the moment I see that unpredictableSeed used for seeding returns 32 bits by default and called everywhere. 128 or 256 bits is more preferable. I am convinced that this occurs because we did not provide a simple function that returns just an array of random bytes. More precisely, we hid its name behind this "Seed" name

>

std.random is not designed for cryptographically-safe random generation.

It is in vain!

>

We neither have the resources nor the expertise to do this.

We need just a copy bytes from TRNG to our arrays or ranges. This is not looks unsafe.

> >
  1. std.random.truerandom: implemented as OS/hardware call if system provides hardware (or environmental) random number generator. Suitable for encryption key generation, etc. Maybe three functions will be provided, something like:

There's no need for this. You could just open /dev/random or /dev/urandom as a file yourself, and read whatever you need from it. It's OS-dependent anyway, so there's no need to add another layer of abstraction to pretend that you're portable.

But in fact, this has already been implemented in unpredictableSeed

>

The problem with offering a crypto-level RNG in the standard library is that you need an active maintainer who's keeping on top of the latest security weaknesses and updates,

All of this is already implemented in the operating systems we use - we just need to use the appropriate APIs.

>

Or if what you're really asking for is an API to /dev/random or /dev/urandom, then why not just open them as files yourself?

Because I do not want to research this for MacOS during developing on Linux. Usula all I want is 8-16 random bytes obtained in the way recommended by the target system.

>

That's why these devices are in /dev/ in the first place.

Also /dev/* can be unavailable at load stages, getrandom is recommended way on Linux

>

[...]

>
  1. std.random.pseudorandom.pseudoRandom: not for cryptography at all. Name was specifically chosen so that the user would clearly see the "pseudo" prefix.
    [...]

What's currently in std.random falls in this category. Wouldn't be a bad idea to rename it this way. Good idea.

Thanks

5 days ago
On Tuesday, 16 September 2025 at 15:06:39 UTC, H. S. Teoh wrote:
> to roll their own generators, which usually means it's done poorly because generating random numbers is not as simple as it looks.

> std.random is not designed for cryptographically-safe random

Insecure random numbers are trivial, very trivial. The cope of meta programming a whole theory o randomness that no one uses and writing it off as "not designed to be advanced randomness" is a worse of both worlds take.

I don't know what goes into crypto level randomness but that airnt what I want or and what I want wasn't offered
5 days ago
On Tuesday, 16 September 2025 at 12:04:20 UTC, Denis Feklushkin wrote:
> 2. Completely exclude "seeding" concept: this is a source of

You can make it implict called, and leave the data public for whoever wants it. Private whatever meme is a bad take in general, but especially in such a well understood and you unable to predict all usecases, magic value.
5 days ago
On Tuesday, 16 September 2025 at 18:04:20 UTC, Monkyyy wrote:
> On Tuesday, 16 September 2025 at 12:04:20 UTC, Denis Feklushkin wrote:
>> 2. Completely exclude "seeding" concept: this is a source of
>
> You can make it implict called, and leave the data public for whoever wants it.

It is difficult to achieve this because in most cases it will be just a libc getrandom() call (system-wide seeded PRNG)

« First   ‹ Prev
1 2 3 4