On Thursday, December 13, 2018 5:22:35 AM MST Sebastiaan Koppe via Digitalmars-d wrote:
> On Tuesday, 11 December 2018 at 22:32:45 UTC, Jonathan M Davis
>
> wrote:
> > the property isNull became confusing to some folks, because they thought that the null value of a pointer and whether the Nullable itself was null were related, when they're not (and if they were, it would cause subtle bugs - especially in generic code).
>
> Could you elaborate on some of those subtle bug? And are they worse than the following?
>
> ---
> import std.typecons;
> class C {
> }
> void main() {
>      Nullable!C c;
>      assert(c.isNull);
>      c = null;
>      assert(!c.isNull);
> }
> ---

I don't see anything wrong with that code. It's exactly how it's supposed to work. It's more confusing than would desirable, because Nullable was not originally intended to contain pointers, but having isNull be true would be a serious problem - especially for generic code.

> What about having Nullable!T* decay into a wrapped pointer. Why does it need to keep its own bool when that information can be captured in the pointer?

Take this code for example:

Nullable!(int*) n = getValue();
assert(!n.isNull);

Right now, you can rely on the fact that the Nullable contains a value after it's assigned a value and that therefore isNull is false, and calling get won't assert. However, if Nullable were to look at whether the internal pointer was null in order to determine whether the Nullable was "null," then suddenly, the fact that you assigned the Nullable a value would not guarantee that isNull was false, and calling get could therefore fail its assertion when it's called. That's bad enough when the code in question is specifically written with the idea that it's operating on a Nullable that contains a pointer. But it's absolutely horrible once you're dealing with generic code.

Right now, you can write a templated function that operates on a Nullable without caring one whit about what type it contains, and it will behave the same regardless of the type. Meaning that something like this

auto foo(T)(T t)
{
    Nullable!T n = t;
    ...
    auto bar = t.get;
    ...
}

is guaranteed to work without asserting in get. However, if Nullable treated pointers differently from other types and didn't used a separate bool, then that code would have to worry about checking the type that the Nullable contains to see whether it was a pointer or class reference and in that case do extra checks rather than assuming that a Nullable that was assigned a value actually had a value and that calling get wouldn't assert. As such, treating types that can be given the value of null differently from types that can't be given a value of null is just begging for bugs. In addition, it defeats the entire purpose of Nullable allowing pointers.

Originally, Nullable did not allow pointers or class references, because they could already be null. The main purpose of Nullable was to make it possible to have value types be nullable without allocating on the heap so that you could have a pointer. However, that meant that Nullable could not be used very well in generic code, because it could not contain any types that were naturally nullable. So, Nullable was changed to allow for such types, and it became possible to use Nullable in generic code without caring what type it contained. And as such, special-casing pointers in Nullable's implementation would completely defeat that, once again requiring that generic code using Nullable special-case pointers in order to avoid bugs.

Now, a side benefit of allowing Nullable to contain pointers and class references while using a separate bool to denote whether the Nullable contains a value or not is that you can have a Nullable treat null as a valid value and differentiate between having a pointer whose value is null and one which hasn't been given a value yet - e.g. if you have a case where you want to know whether a variable has been initialized yet, and it's allowed to be given the value of null, the fact that pointers are null by default makes it so that you can't tell whether the pointer has been given a value just by looking at the pointer. In that case, you basically need a separate bool that keeps track of whether the pointer has been initialized yet. And while that wasn't really the point of allowing Nullable to contain pointers, some folks have been using it for that purpose. And it does fit in nicely with one of the main use cases of using Nullable with value types. Nullable is often used simply to denote whether the variable has been given a value yet, because the default value is considered valid in a particular piece of code and therefore can't be used to distinguish between a default-initialized variable and one that's been given a real value. Having Nullable treat pointers differently would destroy that, and it would break existing code.

Honestly, there's no point in having Nullable use the null value of pointers to indicate whether isNull is true, because if that's what you wanted, you could just use the pointer directly without using Nullable at all. And not being able to rely on isNull being false after you assign a value to a Nullable would seriously hamper Nullable in generic code, because it couldn't actually be used generically. So, while the name Nullable has become somewhat confusing now that Nullable can contain pointers, from the point of view of how Nullable is used, it makes no sense for it to treat pointers differently. If it were going to do that, it might as well have never been changed to allow pointers.

> Or is there some difference between a Nullable!T* with a null value and one with no value? I thought no value and null to be the same thing. But if it is, why is it called Nullable?

It's called Nullable, because it was originally intended just for types which can't naturally be null. However, as I explained above, that doesn't play well with generic code, and it's sometimes useful to differentiate between a variable that's been default-initialized and one that's been given an actual value, and in the case of a pointer, that actual value could be null.

The fact that Nullable now allows pointers and class references does unfortunately make it more confusing than it was originally. In light of the changes that have been made to it, it probably should be called something else, and isNull should probably be isEmpty instead, but it's against current policy to rename things in Phobos. We replace things when their design turns out to be bad, but we don't simply rename them anymore. Walter and Andrei do not consider renaming things to be worth the code breakage.

And IMHO, if the documentation is sufficiently clear, this is really a non-issue. isNull indicates whether the Nullable is null, not whether the value it contains is null. The fact that there is any confusion here does indicate that the documentation needs to be improved, but as long as you understand that isNull refers to the state of the Nullable itself and not the value that it contains, it works perfectly fine as-is.

- Jonathan M Davis

On Tuesday, 11 December 2018 at 22:32:45 UTC, Jonathan M Davis wrote:
> On Tuesday, December 11, 2018 2:48:36 PM MST aliak via Digitalmars-d wrote:
>> [...]
>
> Ultimately, allocation is the main difference here. Nullable provides a way to emulate the behavior of a pointer with regards to nullability without having to allocate on the heap. An Optional or Maybe type is ultimately the same thing, just with a different name. If it weren't for the issue of heap allocation, it could easily argued that pointers negate the need for any kind of Nullable/Optional/Maybe type, because they provide that functionality. And they don't even cause memory safety issues if you're not doing pointer arithmetic.

There's another fundamental difference with regards to pointers: there's no equivalent to `null`, which is this weird special value that is compatible with all pointer types.

On Thursday, December 13, 2018 4:41:01 AM MST aliak via Digitalmars-d wrote:
> On Tuesday, 11 December 2018 at 22:32:45 UTC, Jonathan M Davis
>
> wrote:
> > Ultimately, allocation is the main difference here. Nullable provides a way to emulate the behavior of a pointer with regards to nullability without having to allocate on the heap. An Optional or Maybe type is ultimately the same thing, just with a different name. If it weren't for the issue of heap allocation, it could easily argued that pointers negate the need for any kind of Nullable/Optional/Maybe type, because they provide that functionality. And they don't even cause memory safety issues if you're not doing pointer arithmetic. So, really, I think that the need for heap allocation is _exactly_ the issue here that these types are designed to solve and that without that, a Nullable/Optional/Maybe type isn't adding much.
>
> This is not the point of optional types. They're just there to denote if a value exists or not.

Which you can do with a pointer. There is no need to have an "optional" type if you don't care about heap allocations. If you don't care about heap allocations, than you can just use T* and not bother with creating a library type. It even works if you already have a pointer, because you can have a pointer to a pointer: T**. Having an optional/maybe type allows you to indicate whether a value is present _without_ that extra heap allocation. It gives you pointer semantics with regards to whether a value is present without needing a pointer. So, ultimately, avoiding a heap allocation is the reason that such optional/maybe types exist, and it's why Nullable exists.

> > At most, it's making it clear that it's expected that the value can be null/empty/missing, because not all functions that involve pointers consider null to be an acceptable value.
>
> That makes a big difference. With everything. Writing the code, maintaining code, reviewing code, coming back to code, getting in to a language. Principle of least surprises. Intuitiveness. These all matter:
>
> class C {}
> struct S {}
>
> * What is Nullable!(int*)? An int * that can be null? But wait...
> * What is Nullable!C? A reference type that can be null? Isn't
> that just "C" ? What is this adding? How does this make sense?
> * What does this do: "Nullable!C a = null; a.isNull;"? if it's
> false does that mean C exists? So C exists?
> * What is Nullable!S? A struct that can be null or a struct that
> may exist? Or was it just to avoid heap allocation?
> * What's the difference between null and any of those other
> types' isNull?
> * Nullable!C a = null; a.get; // should that throw/assert?
>
> These are all non-obvious.

The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.

And if you're not writing generic code, then the only reason to even put a pointer or reference in a Nullable in the first place is so that you can differentiate between a pointer or reference that has been given a value rather than being simply default-initialized. If you didn't care about that, then you would just use the pointer or reference directly, because the Nullable would do _nothing_ for you. It would be a pointless wrapper.

And if you're writing generic code, then having isNull act differently for pointers than it does for value types would not work at all, because you'd have to special-case the code to handle the fact that assigning the Nullable a value could still result in isNull being true in the case where the Nullable contained a pointer or reference. So, the code would no longer be generic. And if it wasn't going to be generic, then you'd just use the naked pointer or reference rather than using a Nullable. The entire reason that Nullable was changed to allow types that are naturally nullable was so that generic code could use Nullable without having to be special-cased for pointers or references. That does come with the downside that isNull might be confusing if you don't understand what Nullable is, and the documentation clearly needs to be improved on that front, but there's no problem there with how Nullable works. It's just now a worse name, because it was changed to allow types that are naturally nullable and that can therefore have the value of null.

> > IMHO, the only real naming issue that we have with Nullable is that once it was changed to allow actual pointers (whereas originally, it just contained types that could not themselves be null), the property isNull became confusing to some folks, because they thought that the null value of a pointer and whether the Nullable itself was null were related, when they're not (and if they were, it would cause subtle bugs - especially in generic code).
>
> Yes, and this is just bad. Bad bad bad. How can any API that causes subtle bugs be something anyone would want to keep around??

How on earth does Nullable cause subtle bugs? The only issue with it that I see is that the fact that it's aliased to its get member risks bugs when you change code that used a T to use Nullable!T. Without that alias, you'd be forced to update the code and make sure that it could handle when isNull was true, whereas right now, it can be trivial to just change a variable's type from T to Nullable!T and assume that it works, and even if you're careful and try to make the code use get and isNull properly, you can easily miss a place where the alias is used and end up with a bug. But if the alias is removed like the OP wants, then that entire problem goes away, and then I don't see why Nullable would cause subtle bugs.

I'll grant you that now that Nullable allows types which are naturally nullable, the name is worse, because it does sometimes cause confusion around isNull. But it's against current policy to rename stuff in Phobos simply to give it a better name, and if the documentation is improved, that problem should largely go away. At that point, the only folks who would be confused are those that don't read the docs, and folks who don't read the documentation for the code they're reading or maintaining are going to be confused in general. If we were to start over, then yeah, it should probably be named Optional or Maybe or whatever rather than Nullable given that it accepts naturally nullable types, but I fail to see how its design causes bugs aside from the problems caused by the alias. And we should be able to fix that via deprecation, only breaking code that arguably should be broken in the process, whereas renaming stuff breaks _all_ code using Nullable, the vast majority of which is perfectly fine - which is why Walter and Andrei are against allowing renaming stuff. It breaks tons of code, and in general, the benefit is highly subjective if not outright negligible. It's probably worth more in this case than in many other cases, but it would still involve breaking tons of perfectly valid code.

- Jonathan M Davis

On Thursday, 13 December 2018 at 13:47:34 UTC, Jonathan M Davis wrote:
> Which you can do with a pointer. There is no need to have an "optional" type if you don't care about heap allocations. If you don't care about heap allocations, than you can just use T* and not bother with creating a library type. It even works if you already have a pointer, because you can have a pointer to a pointer: T**. Having an optional/maybe type allows you to indicate whether a value is present _without_ that extra heap allocation. It gives you pointer semantics with regards to whether a value is present without needing a pointer. So, ultimately, avoiding a heap allocation is the reason that such optional/maybe types exist, and it's why Nullable exists.

Maybe it's why Nullable in D exists for *some* types, but it is certainly not why optional types exist -> https://en.wikipedia.org/wiki/Option_type.

Why would you use Nullable!Class, or Nullable!(T*)? It's not to avoid allocation.

Now whether or not you think a pointer is good enough is a different argument. If you are in the camp where a pointer is good enough then that's fine but I completely disagree with a pointer being good enough to represent the absence of a value. That's a hack with pointers. Just like you can use functions to model object oriented programming, it's a hack. In the case of pointers it only works if you consider null as not being part of a pointer's value domain. If you think null is a valid value of a pointer, then you're stuck.

Also, "if (!null) then do something" is becoming more recognized as an anti pattern these days, and having pointers represent existence just helps you litter your code with null checks. Checking for null becomes an unenforcible requirement for stability. Instead, with a properly designed optional type, this requirement becomes checked and/or forced.

> The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.

The behavior of many things becomes completely obvious once you understand a set of things, generally speaking. The brain power needed to understand that is unnecessarily more than what would be required with a better/different API.

>
> And if you're not writing generic code, then the only reason to even put a pointer or reference in a Nullable in the first place is so that you can differentiate between a pointer or reference that has been given a value rather than being simply default-initialized. If you didn't care about that, then you would just use the pointer or reference directly, because the Nullable would do _nothing_ for you. It would be a pointless wrapper.

The Nullable implementation does indeed make it a pointless wrapper. Which is the problem. It actually make it worse. Because with a pointer or reference, if you check is null, then you know you can use it, but with Nullable, checking isNull doesn't give you any guaranteers.

>
> And if you're writing generic code, then having isNull act differently for pointers than it does for value types would not work at all, because you'd have to special-case the code to handle the fact that assigning the Nullable a value could still result in isNull being true in the case where the Nullable contained a pointer or reference. So, the code would no longer be generic. And if it wasn't going to be generic, then you'd

Why would you want to use a pointer or reference that is null?

Right now you have to do this:

void f(T)(Nullable!T a) {
  if (!a.isNull) {
    static if (is(T == class) || isPointer!T || is(T == interface)) {
      if (a.get !is null) {
        // can use a
      }
    } else {
      // can you a
    }
  }
}


> just use the naked pointer or reference rather than using a Nullable. The entire reason that Nullable was changed to allow types that are naturally nullable was so that generic code could use Nullable without having to be special-cased for pointers or references. That does come with the downside that isNull might be confusing if you don't understand what Nullable is, and the documentation clearly needs to be improved on that front, but there's no problem there with how Nullable works. It's just now a worse name, because it was changed to allow types that are naturally nullable and that can therefore have the value of null.
>
>> > IMHO, the only real naming issue that we have with Nullable is that once it was changed to allow actual pointers (whereas originally, it just contained types that could not themselves be null), the property isNull became confusing to some folks, because they thought that the null value of a pointer and whether the Nullable itself was null were related, when they're not (and if they were, it would cause subtle bugs - especially in generic code).
>>
>> Yes, and this is just bad. Bad bad bad. How can any API that causes subtle bugs be something anyone would want to keep around??
>
> How on earth does Nullable cause subtle bugs?

Au. I misread your sentence as saying Nullable causes subtle bugs :o Sorry.

Cheers,
- Ali

On Thursday, 13 December 2018 at 13:47:34 UTC, Jonathan M Davis wrote:
> The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.

Isn't the behavior of everything obvious if you know how it works? The reasons things should be named the way they are is to help identify what they do. What you are describing is more easily understood as a class named "Optional".

You mention that the template didn't work with types that were already nullable. I want to know what the individual was thinking when they expanded the functionality to include nullable types. They could have easily added extra code that does an additional check to see if the types are null. That's the logical thing to do for a class named "Nullable". The way it is currently implemented is the logical way to implement "Optional". You wouldn't store a float point number in a type called int, would you ?

On Thursday, 13 December 2018 at 13:47:34 UTC, Jonathan M Davis wrote:
> On Thursday, December 13, 2018 4:41:01 AM MST aliak via Digitalmars-d wrote:
>> On Tuesday, 11 December 2018 at 22:32:45 UTC, Jonathan M Davis
>>
>> wrote:
>> > Ultimately, allocation is the main difference here. Nullable provides a way to emulate the behavior of a pointer with regards to nullability without having to allocate on the heap. An Optional or Maybe type is ultimately the same thing, just with a different name. If it weren't for the issue of heap allocation, it could easily argued that pointers negate the need for any kind of Nullable/Optional/Maybe type, because they provide that functionality. And they don't even cause memory safety issues if you're not doing pointer arithmetic. So, really, I think that the need for heap allocation is _exactly_ the issue here that these types are designed to solve and that without that, a Nullable/Optional/Maybe type isn't adding much.
>>
>> This is not the point of optional types. They're just there to denote if a value exists or not.
>
> Which you can do with a pointer. There is no need to have an "optional" type if you don't care about heap allocations. If you don't care about heap allocations, than you can just use T* and not bother with creating a library type. It even works if you already have a pointer, because you can have a pointer to a pointer: T**. Having an optional/maybe type allows you to indicate whether a value is present _without_ that extra heap allocation. It gives you pointer semantics with regards to whether a value is present without needing a pointer. So, ultimately, avoiding a heap allocation is the reason that such optional/maybe types exist, and it's why Nullable exists.
>
>> > At most, it's making it clear that it's expected that the value can be null/empty/missing, because not all functions that involve pointers consider null to be an acceptable value.
>>
>> That makes a big difference. With everything. Writing the code, maintaining code, reviewing code, coming back to code, getting in to a language. Principle of least surprises. Intuitiveness. These all matter:
>>
>> class C {}
>> struct S {}
>>
>> * What is Nullable!(int*)? An int * that can be null? But wait...
>> * What is Nullable!C? A reference type that can be null? Isn't
>> that just "C" ? What is this adding? How does this make sense?
>> * What does this do: "Nullable!C a = null; a.isNull;"? if it's
>> false does that mean C exists? So C exists?
>> * What is Nullable!S? A struct that can be null or a struct that
>> may exist? Or was it just to avoid heap allocation?
>> * What's the difference between null and any of those other
>> types' isNull?
>> * Nullable!C a = null; a.get; // should that throw/assert?
>>
>> These are all non-obvious.
>
> The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.
>
> And if you're not writing generic code, then the only reason to even put a pointer or reference in a Nullable in the first place is so that you can differentiate between a pointer or reference that has been given a value rather than being simply default-initialized. If you didn't care about that, then you would just use the pointer or reference directly, because the Nullable would do _nothing_ for you. It would be a pointless wrapper.
>
> And if you're writing generic code, then having isNull act differently for pointers than it does for value types would not work at all, because you'd have to special-case the code to handle the fact that assigning the Nullable a value could still result in isNull being true in the case where the Nullable contained a pointer or reference. So, the code would no longer be generic. And if it wasn't going to be generic, then you'd just use the naked pointer or reference rather than using a Nullable. The entire reason that Nullable was changed to allow types that are naturally nullable was so that generic code could use Nullable without having to be special-cased for pointers or references. That does come with the downside that isNull might be confusing if you don't understand what Nullable is, and the documentation clearly needs to be improved on that front, but there's no problem there with how Nullable works. It's just now a worse name, because it was changed to allow types that are naturally nullable and that can therefore have the value of null.
>
>> > IMHO, the only real naming issue that we have with Nullable is that once it was changed to allow actual pointers (whereas originally, it just contained types that could not themselves be null), the property isNull became confusing to some folks, because they thought that the null value of a pointer and whether the Nullable itself was null were related, when they're not (and if they were, it would cause subtle bugs - especially in generic code).
>>
>> Yes, and this is just bad. Bad bad bad. How can any API that causes subtle bugs be something anyone would want to keep around??
>
> How on earth does Nullable cause subtle bugs? The only issue with it that I see is that the fact that it's aliased to its get member risks bugs when you change code that used a T to use Nullable!T. Without that alias, you'd be forced to update the code and make sure that it could handle when isNull was true, whereas right now, it can be trivial to just change a variable's type from T to Nullable!T and assume that it works, and even if you're careful and try to make the code use get and isNull properly, you can easily miss a place where the alias is used and end up with a bug. But if the alias is removed like the OP wants, then that entire problem goes away, and then I don't see why Nullable would cause subtle bugs.
>
> I'll grant you that now that Nullable allows types which are naturally nullable, the name is worse, because it does sometimes cause confusion around isNull. But it's against current policy to rename stuff in Phobos simply to give it a better name, and if the documentation is improved, that problem should largely go away. At that point, the only folks who would be confused are those that don't read the docs, and folks who don't read the documentation for the code they're reading or maintaining are going to be confused in general. If we were to start over, then yeah, it should probably be named Optional or Maybe or whatever rather than Nullable given that it accepts naturally nullable types, but I fail to see how its design causes bugs aside from the problems caused by the alias. And we should be able to fix that via deprecation, only breaking code that arguably should be broken in the process, whereas renaming stuff breaks _all_ code using Nullable, the vast majority of which is perfectly fine - which is why Walter and Andrei are against allowing renaming stuff. It breaks tons of code, and in general, the benefit is highly subjective if not outright negligible. It's probably worth more in this case than in many other cases, but it would still involve breaking tons of perfectly valid code.
>
> - Jonathan M Davis

https://github.com/dlang/phobos/commit/bfa6cd72ae83a977f272bf3520f3ef5da99eb7cf#diff-4e008aedb3026d4a84f58323e53bf017R1862

Had a good laugh when I found this. Looks like someone just didn't know what they were doing and someone was picking up their pieces.

On Friday, December 14, 2018 3:23:57 PM MST Rubn via Digitalmars-d wrote:
> On Thursday, 13 December 2018 at 13:47:34 UTC, Jonathan M Davis
>
> wrote:
> > The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.
>
> Isn't the behavior of everything obvious if you know how it works? The reasons things should be named the way they are is to help identify what they do. What you are describing is more easily understood as a class named "Optional".
>
> You mention that the template didn't work with types that were already nullable. I want to know what the individual was thinking when they expanded the functionality to include nullable types. They could have easily added extra code that does an additional check to see if the types are null. That's the logical thing to do for a class named "Nullable". The way it is currently implemented is the logical way to implement "Optional". You wouldn't store a float point number in a type called int, would you ?

As I already explained in multiple posts here, the entire reason that Nullable was expanded to work with pointers and classes was so that it would work in generic code. So, it's now possible to have a templated function or type use Nullable without caring about what type it actually contains. It's just whatever type the template was instantiated with, whereas previously it would have had to do stuff like

static if(is(Nullable!T))
    n.nullify()
else
    n = null;

and

static if(is(Nullable!T))
    return n.get;
else
    return *n;

in order to work with both naturally nullable types and those that need Nullable to be "null" in a single piece of templated code. Now, you can just do

n.nullify();

and

return n.get;

regardless of whether the Nullable contains a naturally nullable type or not. If Nullable were to treat pointers differently so that isNull were true if the value were null instead of using a separate bool, then you'd have to special case Nullable in generic code again, because you could no longer rely on assigning a value to Nullable meaning that it had a value.

n = value;
return n.get;

could then actually fail the assertion in get if the code was instantiated with a type that was naturally nullable. The current approach - regardless of the name of Nullable - prevents that problem.

Having a Nullable treat pointers differently from value types would completely defeat the purpose of having Nullable work with pointers in the first place, and if you're not writing generic code, a Nullable that doesn't have a separate bool indicating whether it has a value or not is pointless for pointers, because it would just be creating a wrapper that did nothing that pointers don't do naturally.

As for the name, Nullable was a perfectly fine name when it couldn't contain pointers. Is it a great name now that it can contain pointers? No, and the documentation should probably be improved to make it even clearer that Nullable has nothing to do with the value of the type it contains, just about whether the Nullable itself is "null," but the way that Nullable works is the way that it should work - regardless of its name. If we were adding Nullable now, we would almost certainly give it a different name, but it's against Phobos policy to simply rename stuff, because it breaks everything that uses the symbol when you do that. If the current design caused bugs, then we might replace it, because we'd be fixing and preventing bugs in the process, but it doesn't. If anything, its current design prevents bugs by ensuring that Nullable functions the same regardless of what type it contains, making it work well in generic code, whereas having Nullable treat pointers differently would create bugs in generic code.

The problem with Nullable's design that does cause bugs is the fact that it's aliased to its get member - which is what this thread was originally about - and the way to fix that is to deprecate the alias, thereby only breaking code that uses the alias. There's no need to break _all_ code that uses Nullable, which is exactly what would happen if we were to rename it - which is why we don't rename things in Phobos anymore just because it's thought that the name isn't as good as it could be.

If we decide that we need to replace something, because its design is subtoptimal, and we can't fix it in place, then we can end up with a new name, because we're actually adding something new to replace something broken, not simply renaming it. If this were 2010, then yeah, we'd probably rename it (e.g. a number of years ago, I fixed a bunch of the names in std.string, because they didn't follow Phobos' naming conventions), but Walter and Andrei no longer consider renaming a construct to give it a better name worth the breakage; there was already tons of complaining about that when we did it when we had a much smaller user base. It would be much worse now.

So, if there's a fix that we're going to do at this point, it's to improve Nullable's documentation, not to rename it.

- Jonathan M Davis

On Thursday, 13 December 2018 at 13:14:13 UTC, Jonathan M Davis wrote:
>> import std.typecons;
>> class C {
>> }
>> void main() {
>>      Nullable!C c;
>>      assert(c.isNull);
>>      c = null;
>>      assert(!c.isNull);
>> }
>> ---
>
> I don't see anything wrong with that code. It's exactly how it's supposed to work.

It is confusing, it doesn't follow the principle of least astonishment.

> Right now, you can write a templated function that operates on a Nullable without caring one whit about what type it contains, and it will behave the same regardless of the type. Meaning that something like this
>
> auto foo(T)(T t)
> {
>     Nullable!T n = t;
>     ...
>     auto bar = t.get;
>     ...
> }

In languages where nullable/optional/maybe is more central, you are advised to avoid calling get directly, and you are expected to always map or foreach over it.

Essentially it is a range with 0 or 1 element.

> Nullable is often used simply to denote whether the variable
> has been given a value yet

Do you consider null a value? Because Nullable does.

---
auto p = Nullable!(int*)(someFunctionThatReturnsNull());
while (p.isNull) {
  // never happens
}
---

---
int* p = someFunctionThatReturnsNull();
while (p is null) {
  // does
}
---

On Friday, 14 December 2018 at 23:08:30 UTC, Jonathan M Davis wrote:
> On Friday, December 14, 2018 3:23:57 PM MST Rubn via Digitalmars-d wrote:
>> On Thursday, 13 December 2018 at 13:47:34 UTC, Jonathan M Davis
>>
>> wrote:
>> > The behavior is completely obvious if you understand that whether a Nullable is null refers to the Nullable itself and not the value that it contains. Looking at the documentation, it does need to be improved so that that is clearer. But if you understand that, then there is no confusion.
>>
>> Isn't the behavior of everything obvious if you know how it works? The reasons things should be named the way they are is to help identify what they do. What you are describing is more easily understood as a class named "Optional".
>>
>> You mention that the template didn't work with types that were already nullable. I want to know what the individual was thinking when they expanded the functionality to include nullable types. They could have easily added extra code that does an additional check to see if the types are null. That's the logical thing to do for a class named "Nullable". The way it is currently implemented is the logical way to implement "Optional". You wouldn't store a float point number in a type called int, would you ?
>
> As I already explained in multiple posts here, the entire reason that Nullable was expanded to work with pointers and classes was so that it would work in generic code. So, it's now possible to have a templated function or type use Nullable without caring about what type it actually contains. It's just whatever type the template was instantiated with, whereas previously it would have had to do stuff like
>
> static if(is(Nullable!T))
>     n.nullify()
> else
>     n = null;
>
> and
>
> static if(is(Nullable!T))
>     return n.get;
> else
>     return *n;
>
> in order to work with both naturally nullable types and those that need Nullable to be "null" in a single piece of templated code. Now, you can just do
>
> n.nullify();
>
> and
>
> return n.get;
>
> regardless of whether the Nullable contains a naturally nullable type or not. If Nullable were to treat pointers differently so that isNull were true if the value were null instead of using a separate bool, then you'd have to special case Nullable in generic code again, because you could no longer rely on assigning a value to Nullable meaning that it had a value.

So for this naturally nullable type, would you say it has a value?

int* ptr = null;

> n = value;
> return n.get;
>
> could then actually fail the assertion in get if the code was instantiated with a type that was naturally nullable. The current approach - regardless of the name of Nullable - prevents that problem.

When would you ever want to use get() if you know isNull is true (including for if isNull() returned true for pointers that have a null value) ?

> Having a Nullable treat pointers differently from value types would completely defeat the purpose of having Nullable work with pointers in the first place, and if you're not writing generic code, a Nullable that doesn't have a separate bool indicating whether it has a value or not is pointless for pointers, because it would just be creating a wrapper that did nothing that pointers don't do naturally.

Are you really saying it's pointless after the you spent a paragraph writing up about how it's useful to have 1 type provide the same interface ?

struct SomeType {
   void foo();
}

test( Nullable!SomeType() );
test( Nullable!(SomeType*) );

void test(T)(Nullable!T value) {

    if( !value.isNull() ) {
        value.get().foo()
    }
}

> As for the name, Nullable was a perfectly fine name when it couldn't contain pointers. Is it a great name now that it can contain pointers? No, and the documentation should probably be improved to make it even clearer that Nullable has nothing to do with the value of the type it contains, just about whether the Nullable itself is "null," but the way that Nullable works is the way that it should work - regardless of its name. If we were adding Nullable now, we would almost certainly give it a different name, but it's against Phobos policy to simply rename stuff, because it breaks everything that uses the symbol when you do that. If the current design caused bugs, then we might replace it, because we'd be fixing and preventing bugs in the process, but it doesn't. If anything, its current design prevents bugs by ensuring that Nullable functions the same regardless of what type it contains, making it work well in generic code, whereas having Nullable treat pointers differently would create bugs in generic code.

You keep saying this, but the very first version of Nullable could in fact contain pointers.

https://github.com/dlang/phobos/commit/0c142994d9b5cb9f379eca28f3a625c749370e4a#diff-4e008aedb3026d4a84f58323e53bf017R873

https://run.dlang.io/is/bu0hqt

Maybe there was a Nullable type in D1, but I can't find one at least.

> The problem with Nullable's design that does cause bugs is the fact that it's aliased to its get member - which is what this thread was originally about - and the way to fix that is to deprecate the alias, thereby only breaking code that uses the alias. There's no need to break _all_ code that uses Nullable, which is exactly what would happen if we were to rename it - which is why we don't rename things in Phobos anymore just because it's thought that the name isn't as good as it could be.

I don't think you would break that much code if you made isNull() return true if the value is nullable and is actually null. Any code that is checking isNull() would also have to be checking if the pointer is null. Then only use the pointer if it's optional value is not null.

> If we decide that we need to replace something, because its design is subtoptimal, and we can't fix it in place, then we can end up with a new name, because we're actually adding something new to replace something broken, not simply renaming it. If this were 2010, then yeah, we'd probably rename it (e.g. a number of years ago, I fixed a bunch of the names in std.string, because they didn't follow Phobos' naming conventions), but Walter and Andrei no longer consider renaming a construct to give it a better name worth the breakage; there was already tons of complaining about that when we did it when we had a much smaller user base. It would be much worse now.
>
> So, if there's a fix that we're going to do at this point, it's to improve Nullable's documentation, not to rename it.
>
> - Jonathan M Davis

Don't have to rename it, can just make it do what it actually is named after.

On Friday, December 14, 2018 4:39:16 PM MST Sebastiaan Koppe via Digitalmars-d wrote:
> On Thursday, 13 December 2018 at 13:14:13 UTC, Jonathan M Davis
>
> wrote:
> >> import std.typecons;
> >> class C {
> >> }
> >> void main() {
> >>
> >>      Nullable!C c;
> >>      assert(c.isNull);
> >>      c = null;
> >>      assert(!c.isNull);
> >>
> >> }
> >> ---
> >
> > I don't see anything wrong with that code. It's exactly how it's supposed to work.
>
> It is confusing, it doesn't follow the principle of least astonishment.

Honestly, I have never found it counfusing, but I don't disagree that having a different name would be less confusing for some folks. Regardless of the naming though, Nullable has exactly the semantics that it needs to have to make any sense to use it with naturally nullable types. If isNull cared about the value of what the Nullable contained, pointers would not work the same as other types inside a Nullable, making Nullable treacherous to use in generic code, and if the code is not generic, and isNull just tells you whether the pointer is null, then you might as well use the pointer directly.

Using Nullable!(T*) can make sense in non-generic when you want to indicate whether the variable has been assigned a value rather than being default-initialized, and it makes sense in generic code where you don't care what type you're operating on. But Nullable!(T*) makes no sense whatsoever if isNull indicates anything about the pointer's value.

> In languages where nullable/optional/maybe is more central, you are advised to avoid calling get directly, and you are expected to always map or foreach over it.
>
> Essentially it is a range with 0 or 1 element.

I have never seen an Optional type that worked that way or worked with code that did anything like that. That obviously does not mean that such code doesn't exist, but personally, it strikes me as a bizarre way to operate, and I've never written code that way. Having a type that contains zero or more elements (such as a range) makes sense to me, but mapping that API onto a type that can contain one or zero just seems bizarre to me. That being said, it's pretty trivial for someone to write a function that operates on a pointer or Nullable that way if that's what they want.

Either way, it would totally screw with code that does consider null to be distinct from not having a value for Nullable to treat isNull as having anything to do with the value of that's contained in the Nullable. e.g. IIRC, when someone previously suggested making such a change to Nullable, it was pointed out that it would break vibe.d, because it has code that uses Nullable!(T*) with null having a distinct meaning separate from the Nullable having no value.

> > Nullable is often used simply to denote whether the variable has been given a value yet
>
> Do you consider null a value? Because Nullable does.

Whether null is a valid value depends entirely on how the code works. Putting a pointer inside a Nullable is essentially the same as having a pointer to a pointer. T** is nullable, but whether the outer pointer is null says nothing about whether the T* it points to is null. Some code would consider having the T* pointed to by the T** as null to be a perfectly valid value, whereas other code would not.

And if you want to indicate whether a variable has been given a value or not when the init value for that type is a valid value, then you pretty much have to use either a pointer to that value or something like Nullable which uses a separate bool to indicate whether variable has been given a value.

- Jonathan M Davis