Hi there,

As always when I make an appearance it's that something has gone wrong. Either in a popular library or D itself... This time it's a little bit of both.

There are many projects out there that use UUIDs to generate secrets such as session tokens and password recovery tokens. Other projects use it to protect against things like symlink attacks by generating temporary file names, or reduce the risk of IDOR by generating user IDs etc.

The issue is, they shouldn't use Phobos' randomUUID() for that. The Phobos implementation of UUID4 doesn't use cryptographic randomness which leads to predictable UUIDs, putting all these projects at risk.

Now, the question is: who's at fault? Is it Phobos? The RFC doesn't enforce the use of cryptographic randomness and the library doesn't claim that these UUIDs are suitable for secret generation. Is it the developers then? Every other language that I can think of uses cryptographic randomness for UUID4 because this kind of use is very common, so I think it is normal for developers to expect it to work similarly in D.

I personally think that Phobos must be changed. We can nag projects all day long, but any new developer will come and fall into the same trap. Furthermore I have thought for a long time that Phobos needs to provide a standard interface to the system's CSPRNG (CryptGenRandom on Windows, getrandom() or /dev/urandom on Linux, /dev/random on Unix). This is a component that does not involve the difficulty of cryptographic code (since you're not recoding anything) and is absolutely critical to so many operations that are very common at the age of secure communications. People use the path of least resistance and at the moment it is incredibly harder to use secure randomness compared to reaching for std.random.uniform(). Clearly randomUUID itself fell into this case.

Then again some might argue that developers should read the RFC of anything they use and, well, as much as I disagree, the RFC is the RFC, wrong as it may be.

I obviously can't solve that conundrum on my own and there are far too many projects impacted for me to take the time and reach them all, so I've written an article showing how to predict Phobos UUIDs in practice. I hope this shows that this is a very practical and important issue that must be dealt with, and the best way for that would be to generate cryptographically secure UUIDs in Phobos.

https://breakpoint.purrfect.fr/article/cracking_phobos_uuid.html

tl;dr: At the moment predicting a future UUID takes only a few thousand requests. It's nothing when it comes to things like trying session cookies or password recovery tokens. If you are a project manager, check whether you use randomUUID to generate secrets and replace it by cryptographically secure UUID.

Even if you don't care about the details of the attack, there's a section at the end of the article that details how to fix the issue and (maybe more importantly) how *not* to fix the issue.

Enjoy your day!

On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
> Hi there,
>
> As always when I make an appearance it's that something has gone wrong. Either in a popular library or D itself... This time it's a little bit of both.
>
> [...]

Thanks for the post! Read the crypto review before and surely enough this time again it was really fun to read through the whole post. I also love the random pictures in your posts :p

I'm not too sure if I ever use randomUUID now, but if it was used in vibe.d applications by default that's terrifying to me.

On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
> [..]

Very well written article, thank you for your efforts!

On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
> [...]

Very insightful article, thank you.

I don't think the right approach to solve any problem is to silently support misuse.

A better way, IMHO, would be to
1) Add a generateSecret() function to Phobos, and
2) Add a Warning to the docs and the use of the function that says something like: "A Universally Unique Identifier (UUID) is *not* a secret and shouldn't be used as such. This includes session cookies, password reset tokens, etc.. For such purposes use generateSecret() instead. This message can be disabled by providing version=IUnderstandThatUUIDsAreNotSecrets", and
3) Silently implement randomUUID() to use generateSecret() because people aren't listening.

I believe a big part of any solution is to destroy assumptions and to make people aware of their misconceptions/mistakes.

On Monday, 31 August 2020 at 14:02:00 UTC, wjoe wrote:
> On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
>> [...]
>
> Very insightful article, thank you.
>
> I don't think the right approach to solve any problem is to silently support misuse.
>
> A better way, IMHO, would be to
> 1) Add a generateSecret() function to Phobos, and
> 2) Add a Warning to the docs and the use of the function that says something like: "A Universally Unique Identifier (UUID) is *not* a secret and shouldn't be used as such. This includes session cookies, password reset tokens, etc.. For such purposes use generateSecret() instead. This message can be disabled by providing version=IUnderstandThatUUIDsAreNotSecrets", and
> 3) Silently implement randomUUID() to use generateSecret() because people aren't listening.
>
> I believe a big part of any solution is to destroy assumptions and to make people aware of their misconceptions/mistakes.

It's not a bad battleplan even though a warning at function use might be a bit cumbersome :) One of my favourite initiatives for security, all languages considered, is Python's standard module "secrets" (see [1]). It exposes very few things: 1) the system's CSPRNG, 2) various token generators (for example a binary one and a url-base64 encoded one to reduced encoding mistakes) and 3) a constant-time string comparison function. That's it. Most of these things are actually defined elsewhere. But it worked well and now whenever a python project requires something that has to do with manipulating secrets people are redirected toward that module where they can use stuff with minimal chance to get it wrong.

[1] https://docs.python.org/3/library/secrets.html

On Monday, 31 August 2020 at 14:14:12 UTC, Cym13 wrote:
> On Monday, 31 August 2020 at 14:02:00 UTC, wjoe wrote:
>> On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
>> 2) Add a Warning to the docs and the use of the function that says something like: "A Universally Unique Identifier (UUID)
>
> It's not a bad battleplan even though a warning at function use might be a bit cumbersome :)

This is true and on purpose. Originally I had
2) Add a Warning to the docs *and/or* the use of the function ...

But people may not read the docs daily/retroactively and how to make people aware of problems in existing and forgotten code otherwise ?

> [1] https://docs.python.org/3/library/secrets.html

This looks really good.

On 8/31/20 9:17 AM, WebFreak001 wrote:
> On Monday, 31 August 2020 at 07:49:24 UTC, Cym13 wrote:
>> Hi there,
>>
>> As always when I make an appearance it's that something has gone wrong. Either in a popular library or D itself... This time it's a little bit of both.
>>
>> [...]
> 
> Thanks for the post! Read the crypto review before and surely enough this time again it was really fun to read through the whole post. I also love the random pictures in your posts :p

I share this sentiment, great article!

> 
> I'm not too sure if I ever use randomUUID now, but if it was used in vibe.d applications by default that's terrifying to me.

I had to look it up to make sure. The session id producer uses what is recommended in the article:

https://github.com/vibe-d/vibe.d/blob/master/crypto/vibe/crypto/cryptorand.d#L125

whew!

-Steve

On Monday, 31 August 2020 at 16:10:33 UTC, Steven Schveighoffer wrote:
> On 8/31/20 9:17 AM, WebFreak001 wrote:
>> 
>> Thanks for the post! Read the crypto review before and surely enough this time again it was really fun to read through the whole post. I also love the random pictures in your posts :p
>
> I share this sentiment, great article!

Thank you both :)


> I had to look it up to make sure. The session id producer uses what is recommended in the article:
>
> https://github.com/vibe-d/vibe.d/blob/master/crypto/vibe/crypto/cryptorand.d#L125
>
> whew!

Yes, cryptorand.d is good. Also, I know I'm trying to detonate a bomb here, but I made sure it wasn't *that* big of a bomb first. If it got you looking for randomUUID in your projects or others half of my goal is accomplished already. There cannot be too many eyes on the matter IMHO.

On 8/31/20 3:49 AM, Cym13 wrote:
...

Outstanding! I am certainly scouring my codebases :-)

On Monday, 31 August 2020 at 14:14:12 UTC, Cym13 wrote:
> [1] https://docs.python.org/3/library/secrets.html

or https://ruby-doc.org/stdlib-2.7.1/libdoc/securerandom/rdoc/SecureRandom.html
But then how do you know that session ids are secrets and not just ids?