I think pluggable type systems will become more common in the following years (see also the optional annotations of Python3 that are designed for that too). This is more or less related:

http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

(but nonnullability is so basic that it's better inside the language, and not left out to a plug-in type system).

Bye,
bearophile

On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS@lycos.com> said:

> I think pluggable type systems will become more common in the following years (see also the optional annotations of Python3 that are designed for that too). This is more or less related:
> 
> http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

Nice post.

> (but nonnullability is so basic that it's better inside the language, and not left out to a plug-in type system).

I agree for non-nullability.

In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

On Mon, 19 Jan 2009 16:26:47 +0300, Michel Fortin <michel.fortin@michelf.com> wrote:

> On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS@lycos.com> said:
>
>> I think pluggable type systems will become more common in the following years (see also the optional annotations of Python3 that are designed for that too). This is more or less related:
>>  http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/
>
> Nice post.
>
>> (but nonnullability is so basic that it's better inside the language, and not left out to a plug-in type system).
>
> I agree for non-nullability.
>
> In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier.
>

Agree.

"bearophile" <bearophileHUGS@lycos.com> wrote in message news:gl0ohe$1gda$1@digitalmars.com...
>I think pluggable type systems will become more common in the following years (see also the optional annotations of Python3 that are designed for that too). This is more or less related:
>
> http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/
>
> (but nonnullability is so basic that it's better inside the language, and not left out to a plug-in type system).
>
> Bye,
> bearophile

Interesting article. Sounds a lot like D2 but with that flow-analysis nice-ity, and @PolyNull (which has been proposed here...often). Although I wonder...maybe @PolyNull should be the default (wherever applicable) instead of @Nullable or @NonNull? (Or maybe I'm overlooking some obvious problem with that.)

I see pluggable type sytems as potentially being a specific form of a more general thing I've been observing an increasing need for/trend towards: customizable languages.

Ex: A language that doesn't say "Semicolons mark end-of-statement" or "Newline marks end-of-statement and lines can be split with a backslash", but provides both as mutually-exclusive programmer-selectable options. Or a choice of C-style curly-braces vs Python-style indentation. Etc. (Side note: This stuff could really cut down on religious language debates. Not eliminate, of course, but reduce.)

Also, a tool could then be used to translate code between the various options, maybe even integrated into the IDE, so that people working on multi-person projects could each set up their own environments and use their own "customized language" even while working on the same line of code. The only project-wide standard would be what settings are used for checked-in code, just like tab/space settings and EOL markers (but ideally performed automatically (and reliably) by the check-in/check-out process). Eventually this might even be generalizable to the point where most languages can be reduced to nothing more than a collection of language settings. Which I think is the direction we appear to be headed anyway, since it seems like everyone and their grandmother is designing their own language (or two) these days, and many of these languages are more-or-less "just like language X, but with A, B and C changed to be more like language Y."

Obviously this is all much easier said than done, but it's something I've been thinking a lot about lately, and I think may be worth exploring.

I also believe that non-null should be the default. Most references in a typical program are assumed to be non-null.

Like for every default, there should be an escape mechanism from it. However, adding "nullable" to the type system might be controversial, especially because it requires special treatment of constructors and polymorphism (hence additional type modifiers, raw and poly-null, and I haven't even mentioned wildcard annotations).

So some time ago I came up with a more modest proposal. The compiler should catch nullable references at their source. It should enforce what I call "declaration is initialization" (DII). For instance, the following is correct:

Foo foo = new Foo;

and the following is not:

Foo foo; // error
...
foo = new Foo;

Obviously this rule is too strict, so there must be a way to override it. Here's the simple syntax:

Foo foo = null; // not an error!

Notice that the programmer explicitly states that one should be careful when using foo, because it might be null.

Since this solution doesn't involve the type system, the nullability doesn't propagate across function calls. But it might be a tradeoff worth taking, if only for the simplicity.

Constructors still need some special treatment. Every class member variable must be initialized in the constructor. Again, the escape is to _explicitly_ initialize it to null. This might seem redundant, since the variable is already default initialized to null, but the compiler can easily eliminate such redundancies.

I also found out that Java's editor, Eclipse, enforces DII--a very useful feature that paid off for me in the first 100 lines of code I wrote after a long period of inactivity in Java ;-)

> Michel Fortin Wrote:

> In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier.

Bartosz Milewski wrote:
> I also found out that Java's editor, Eclipse, enforces DII--a very useful feature that paid off for me in the first 100 lines of code I wrote after a long period of inactivity in Java ;-) 

The Java spec does, actually. Eclipse JDT just reports compiler errors as you type.

Shows you what a beginner in Java I am. Again and again I keep reinventing the wheel. That settles it--we have to have it in D!


Robert Fraser Wrote:

> Bartosz Milewski wrote:
> > I also found out that Java's editor, Eclipse, enforces DII--a very useful feature that paid off for me in the first 100 lines of code I wrote after a long period of inactivity in Java ;-)
> 
> The Java spec does, actually. Eclipse JDT just reports compiler errors as you type.

Bartosz Milewski:
> I also believe that non-null should be the default.

That' probably right.
And you can add ? to the things that can be null. Better to let the type system deal with this.

Bye,
bearophile

On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz@relisoft.com> wrote:

> I also believe that non-null should be the default. Most references in a typical program are assumed to be non-null.
>

Oh, that's true!

> Like for every default, there should be an escape mechanism from it. However, adding "nullable" to the type system might be controversial, especially because it requires special treatment of constructors and polymorphism (hence additional type modifiers, raw and poly-null, and I haven't even mentioned wildcard annotations).
>

Common opinion is that this is worth the trouble these days. More and more people discuss, write articles on non-nullable types and incorporate them into languages. Main C# architects has stated that absence of non-nullable types by default is his greatest mistake that is too late to be fixed.

> So some time ago I came up with a more modest proposal. The compiler should catch nullable references at their source. It should enforce what I call "declaration is initialization" (DII). For instance, the following is correct:
>
> Foo foo = new Foo;
>
> and the following is not:
>
> Foo foo; // error
> ...
> foo = new Foo;
>
> Obviously this rule is too strict, so there must be a way to override it. Here's the simple syntax:
>
> Foo foo = null; // not an error!
>

And what about value types? Why special treatment for reference types only? I don't like rules like that in a language:

Foo foo; // error
int i; // okay?

If not, and this is an error, too, then this defeats the whole purpose of having T.init in D, because it will never be used per your proposal:

int i = 0; // initialized explicitly
char c = char.init; // would you use that?

> Notice that the programmer explicitly states that one should be careful when using foo, because it might be null.
>
> Since this solution doesn't involve the type system, the nullability doesn't propagate across function calls. But it might be a tradeoff worth taking, if only for the simplicity.
>

Naah.. The whole purpose of having non-nullable types is that given a pointer you are sure it is not null:

string toString(Object o)
{
  // is it null or not? Should I check for null or not? What if it is null?
  // Should I throw? What kind of exception?
  // But was going to make it nothrow! So what, no checks?
  // Just fail with access violation? Is this the best D can do?

  return o.toString();
}

As an end user, you neglect and avoid null checks in your code ("I'm sure I won't pass null here"). And then you run your program half an hour, it segfaults with access violation and you don't have any clue where did null dereference take place. I have came across this situation so many times that I hate it.

As a library designer, you end up having null checks everywhere in your code which is some kind of a paranoia because in 99% of cases the pointer will be not null. But why put so many special case and checks if it is *disallowed* to run code with null pointer in first place? So people decided - let's enforce that at compile time! That's the whole point of the feature - disallow invalid use cases at compile time. You get cleaner code (no unnecessary null-checks and throws) and small performance improvement as a bonus.

I would like to write code as this as be sure that it will never fail:

string toString(Object o) // non-nullability is enforced at compile time
{
   return o.toString();
}

This is D style - do as much as possible at compile time - and I believe non-nullable types fit D type system very well.

> Constructors still need some special treatment. Every class member variable must be initialized in the constructor. Again, the escape is to _explicitly_ initialize it to null. This might seem redundant, since the variable is already default initialized to null, but the compiler can easily eliminate such redundancies.
>
> I also found out that Java's editor, Eclipse, enforces DII--a very useful feature that paid off for me in the first 100 lines of code I wrote after a long period of inactivity in Java ;-)
>
>> Michel Fortin Wrote:
>
>> In fact, I'd even argue that non-nullability should be the default for
>> pointers and class references, because it is either safer (if the
>> compiler doesn't do the null check for you) or less troublesome to use
>> (if the compiler forces you to check for null everytime). Another
>> reason being consistency: value-types can't be null by default and so
>> should be class references and pointers. And making pointers
>> non-nullable by default can only be done at the language level, so
>> obviously it's better in the language than as a user-defined type
>> modifier.
>

I just assume that any extension of the already complex D type system will be met with a lot of resistance. I remember the all-out wars about const and immutable (a.k.a invariant). Even those extensions are still half-assed: the construction of immutable objects and const polymorphism issues remain.

My impression is that theoreticians and very advanced programmers love elaborate type systems. Nine-to-five programmers, which are in the majority, prefer simplicity even if the cost is reliability. Just look at the comments to my blog post.

I know we have to do something about null references in D, but I'm still on the fence about how we should accomplish that.

Denis Koroskin Wrote:

> On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz@relisoft.com> wrote:
> 
> > I also believe that non-null should be the default. Most references in a typical program are assumed to be non-null.
> >
> 
> Oh, that's true!
> 
> > Like for every default, there should be an escape mechanism from it. However, adding "nullable" to the type system might be controversial, especially because it requires special treatment of constructors and polymorphism (hence additional type modifiers, raw and poly-null, and I haven't even mentioned wildcard annotations).
> >
> 
> Common opinion is that this is worth the trouble these days. More and more people discuss, write articles on non-nullable types and incorporate them into languages. Main C# architects has stated that absence of non-nullable types by default is his greatest mistake that is too late to be fixed.
> 
> > So some time ago I came up with a more modest proposal. The compiler should catch nullable references at their source. It should enforce what I call "declaration is initialization" (DII). For instance, the following is correct:
> >
> > Foo foo = new Foo;
> >
> > and the following is not:
> >
> > Foo foo; // error
> > ...
> > foo = new Foo;
> >
> > Obviously this rule is too strict, so there must be a way to override it. Here's the simple syntax:
> >
> > Foo foo = null; // not an error!
> >
> 
> And what about value types? Why special treatment for reference types only? I don't like rules like that in a language:
> 
> Foo foo; // error
> int i; // okay?
> 
> If not, and this is an error, too, then this defeats the whole purpose of having T.init in D, because it will never be used per your proposal:
> 
> int i = 0; // initialized explicitly
> char c = char.init; // would you use that?
> 
> > Notice that the programmer explicitly states that one should be careful when using foo, because it might be null.
> >
> > Since this solution doesn't involve the type system, the nullability doesn't propagate across function calls. But it might be a tradeoff worth taking, if only for the simplicity.
> >
> 
> Naah.. The whole purpose of having non-nullable types is that given a pointer you are sure it is not null:
> 
> string toString(Object o)
> {
>    // is it null or not? Should I check for null or not? What if it is null?
>    // Should I throw? What kind of exception?
>    // But was going to make it nothrow! So what, no checks?
>    // Just fail with access violation? Is this the best D can do?
> 
>    return o.toString();
> }
> 
> As an end user, you neglect and avoid null checks in your code ("I'm sure I won't pass null here"). And then you run your program half an hour, it segfaults with access violation and you don't have any clue where did null dereference take place. I have came across this situation so many times that I hate it.
> 
> As a library designer, you end up having null checks everywhere in your code which is some kind of a paranoia because in 99% of cases the pointer will be not null. But why put so many special case and checks if it is *disallowed* to run code with null pointer in first place? So people decided - let's enforce that at compile time! That's the whole point of the feature - disallow invalid use cases at compile time. You get cleaner code (no unnecessary null-checks and throws) and small performance improvement as a bonus.
> 
> I would like to write code as this as be sure that it will never fail:
> 
> string toString(Object o) // non-nullability is enforced at compile time
> {
>     return o.toString();
> }
> 
> This is D style - do as much as possible at compile time - and I believe non-nullable types fit D type system very well.
> 
> > Constructors still need some special treatment. Every class member variable must be initialized in the constructor. Again, the escape is to _explicitly_ initialize it to null. This might seem redundant, since the variable is already default initialized to null, but the compiler can easily eliminate such redundancies.
> >
> > I also found out that Java's editor, Eclipse, enforces DII--a very useful feature that paid off for me in the first 100 lines of code I wrote after a long period of inactivity in Java ;-)
> >
> >> Michel Fortin Wrote:
> >
> >> In fact, I'd even argue that non-nullability should be the default for pointers and class references, because it is either safer (if the compiler doesn't do the null check for you) or less troublesome to use (if the compiler forces you to check for null everytime). Another reason being consistency: value-types can't be null by default and so should be class references and pointers. And making pointers non-nullable by default can only be done at the language level, so obviously it's better in the language than as a user-defined type modifier.
> >
> 
>