View mode: basic / threaded / horizontal-split · Log in · Help
November 05, 2008
Re: null and type safety
Robert Fraser wrote:
> Walter Bright wrote:
>> My focus is on eliminating bugs that cannot be reliably detected even 
>> at run time. This will be a big win for D.
> FWIW, I've _never_ run into a bug const could have prevented.

That isn't really the point of const. The point of const is to be able 
to write functions that can accommodate both mutable and invariant 
arguments. The point of invariantness is to be able to prove that code 
has certain properties. This is much better than relying on your 
programming team never making a mistake.

For example, you can do functional programming in C++. It's just that 
the compiler cannot know you're doing that, and so cannot take advantage 
of it. Furthermore, the compiler cannot detect when code is not 
functional, and so if someone hands you a million lines of code you have 
no freakin' way of determining if it adheres to functional principles or 
not.

This really matters when one starts doing concurrent programming.
November 05, 2008
What is the plan for when Tango turns 1.0 ?
Can someone explain what is the plan for when Tango turns 1.0

Will the code be frozen ?

Will the version of D it runs with be frozen ?


cheers
Nick B
November 05, 2008
Re: null and type safety
On 2008-11-05 03:18:50 -0500, Walter Bright <newshound1@digitalmars.com> said:

> I don't see what you've gained here. The compiler certainly can do flow 
> analysis in some cases to know that a pointer isn't null, but that 
> isn't generalizable. If a function takes a pointer parameter, no flow 
> analysis will tell you if it is null or not.

I'm not sure how you're reading things, but to me having two kinds of 
pointers (nullable and non-nullable) is exactly what you need to enable 
nullness flow analysis across function boundaries.

Basically, if you declare some pointer to be restricted to not-null in 
a function signature, and then try to call the function by passing it a 
possibly null pointer, the compiler can tell you that you need to check 
for null at the call site before calling the function.

It then ensue that when given a non-nullable pointer you can call a 
function requiring a non-nullable pointer directly without any check 
for null, because you know the pointer you recieved can't be null.

Currently, you can acheive this with proper documentation of functions 
saying whether arguments accept null and if return values can return 
null, and write your code with those assumptions in mind. Most often 
than not however there is no such documentation and you find yourself 
checking for null a lot more than necessary. If this property about 
pointers in function parameters and return values were known to the 
compiler, the compiler could check for you that you're doing things 
correctly, warn you whenever you're forgetting a null check, and 
optimise away checks for null on these pointers.

I know the null-dereferencing problem can generally be caught easily at 
runtime, but sometime your null comes from far away in the program 
(someone set a global to null for instance) and you're left to wonder 
who put a null value there in the first place. Non-nullable pointers 
would help a lot in those cases because you no longer have to test 
every code path and the error of giving a null value would be caught at 
the source (with the compiler telling you to check against null), not 
only where it's being dereferenced.

- - -

That said, I think this could be done using an template. Consider this:

	struct NotNullPtr(Type) {
		private Type* ptr;
		this(Type* ptr) {
			opAssign(ptr);
		}
		void opAssign(Type* ptr) {
			// if this gets inlined and you have already checked for null, then
			// hopefully the optimizer will remove this redundant check.
			if (ptr)
				this.ptr = ptr;
			else
				throw new Exception("Unacceptable null value.");
		}
		void opAssign(NotNullPtr other) {
			this.ptr = other.ptr;
		}
		Type* opCast() {
			return ptr;
		}
		ref Type opDeref() {
			return &ptr;
		}
		alias opDeref opStar;
		// ... implement the rest yourself
	}

(not tested)

You could use this template everywhere you want to be sure a pointer 
isn't null. It guarenties that its value will never be null, and will 
throw an exception at the source where you attempt to put a null value 
in it, not when you attempt to dereference it later, when it's too late 
and your program has already been put in an incorrect state.

	NotNullPtr!(int) globalThatShouldNotBeNull;

	int* foo();
	globalThatShouldBeNull = foo(); // will throw if you attempt to set it 
to null.

	void bar(NotNullPtr!(int) arg);
	bar(globalThatShouldNotBeNull); // no need to check for null.

The greatest downside to this template is that since it isn't part of 
the language, almost no one will use it in their function prototypes 
and return types. That's not counting that its syntax is verbose and 
not very appealing (although it's not much worse than boost::shared_ptr 
or std::auto_ptr).

But still, if you have a global or member variable that must not be 
null, it can be of use; and if you have a function where you want to 
put the burden of checking for null on the caller, it can be of use.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
November 05, 2008
Re: null and type safety
Michel Fortin:
> Basically, if you declare some pointer to be restricted to not-null in 
> a function signature, and then try to call the function by passing it a 
> possibly null pointer, the compiler can tell you that you need to check 
> for null at the call site before calling the function.
> 
> It then ensue that when given a non-nullable pointer you can call a 
> function requiring a non-nullable pointer directly without any check 
> for null, because you know the pointer you recieved can't be null.
> 
> Currently, you can acheive this with proper documentation of functions 
> saying whether arguments accept null and if return values can return 
> null, and write your code with those assumptions in mind. Most often 
> than not however there is no such documentation and you find yourself 
> checking for null a lot more than necessary. If this property about 
> pointers in function parameters and return values were known to the 
> compiler, the compiler could check for you that you're doing things 
> correctly, warn you whenever you're forgetting a null check, and 
> optimise away checks for null on these pointers.

The same is true making integral values become range values. If I want to write a function that takes an iterable of results of throwing a dice, I can use an enum, or control every item of the iterable for being in range 1 - 6. If range values are available I can just:

StatsResults stats(Dice[] throwing_results) { ...

Where Dice is:
typedef int:1..7 Dice;

I then don't need to remember to control items for being in 1-6 inside stats(), and the control is pushed up, toward the place where that throwing_results was created (or where it comes from disk, user input, etc). This avoids some bugs and reduces some code.

Bye,
bearophile
November 05, 2008
Re: null and type safety
On Wed, Nov 5, 2008 at 3:18 AM, Walter Bright
<newshound1@digitalmars.com> wrote:
> Jarrett Billingsley wrote:
>>
>> The implication of non-nullable types isn't that nullable types
>> disappear; quite the opposite, in fact.  Nullable types have obvious
>> use for exactly the reason you explain.  The problem arises when
>> nullable types are used in situations where it makes _no sense_ for
>> null to appear.  This is where bugs show up.  In a system that has
>> both nullable and non-null types, nullable types act as a sort of
>> container, preventing you from accessing anything through them as it
>> cannot be statically proven that the access will be legal at runtime.
>> In order to access something from a nullable type, you have to convert
>> it to a non-null type.  Delight uses D's "declare a variable in the
>> condition of an if or while" to great effect here:
>>
>> if(auto f = someFuncThatReturnsNullableFoo()) // f is declared as non-null
>> {
>>    // f is known not to be null.
>> }
>> else
>> {
>>    // something else happened.  Handle it.
>> }
>
> I don't see what you've gained here. The compiler certainly can do flow
> analysis in some cases to know that a pointer isn't null, but that isn't
> generalizable. If a function takes a pointer parameter, no flow analysis
> will tell you if it is null or not.
>

What?  Is your response in response to my post at all?  I am not
talking about flow analysis on "normal" pointer types.  I am talking
about the typing system actually being modified to allow a programmer
to express the idea, with a _type_, and not with static checking, that
a reference/pointer value _may not be null_.

In a type system with non-null types, if a function takes a non-null
parameter and you pass it a nullable pointer, _you get an error at
compile time_.

// foo takes a non-null int*.
void foo(int* x) { writefln("%s", *x); }

// bar returns a nullable int* - an int*?.
int*? bar(int x) { if(x < 10) return new int(x); else return null; }

foo(bar(3)); // compiler error, you can't pass a potentially null type
into a parameter that can't be null, moron

if(auto p = bar(3))
   foo(p); // ok
else
   throw new Exception("Wah wah wah bar returned null");

With nullable types, flow analysis doesn't have to be done.  It is
implicit in the types.  It is mangled into function names.  foo
_cannot_ take a pointer that may be null.  End of story.
November 05, 2008
Re: null and type safety
On Wed, Nov 5, 2008 at 10:43 PM, Jarrett Billingsley
<jarrett.billingsley@gmail.com> wrote:
> On Wed, Nov 5, 2008 at 3:18 AM, Walter Bright
> <newshound1@digitalmars.com> wrote:
>> Jarrett Billingsley wrote:
>>> cannot be statically proven that the access will be legal at runtime.
>>> In order to access something from a nullable type, you have to convert
>>> it to a non-null type.  Delight uses D's "declare a variable in the
>>> condition of an if or while" to great effect here:
>>>
>>> if(auto f = someFuncThatReturnsNullableFoo()) // f is declared as non-null
>>> {
>>>    // f is known not to be null.
>>> }
>>> else
>>> {
>>>    // something else happened.  Handle it.
>>> }
>>
>> I don't see what you've gained here. The compiler certainly can do flow
>> analysis in some cases to know that a pointer isn't null, but that isn't
>> generalizable. If a function takes a pointer parameter, no flow analysis
>> will tell you if it is null or not.
>>
>
> What?  Is your response in response to my post at all?  I am not
> talking about flow analysis on "normal" pointer types.  I am talking
> about the typing system actually being modified to allow a programmer
> to express the idea, with a _type_, and not with static checking, that
> a reference/pointer value _may not be null_.
>
> In a type system with non-null types, if a function takes a non-null
> parameter and you pass it a nullable pointer, _you get an error at
> compile time_.
>
> // foo takes a non-null int*.
> void foo(int* x) { writefln("%s", *x); }
>
> // bar returns a nullable int* - an int*?.
> int*? bar(int x) { if(x < 10) return new int(x); else return null; }
>
> foo(bar(3)); // compiler error, you can't pass a potentially null type
> into a parameter that can't be null, moron
>
> if(auto p = bar(3))
>    foo(p); // ok
> else
>    throw new Exception("Wah wah wah bar returned null");
>
> With nullable types, flow analysis doesn't have to be done.  It is
> implicit in the types.  It is mangled into function names.  foo
> _cannot_ take a pointer that may be null.  End of story.

I didn't really get what you meant the first time either.  The thing
about Delight's use of auto "to great effect" wasn't clear.   I
assumed it was basically the same as D's auto inside an if, but I see
now that it's not.  Looks like a run-time type deduction, even though
its not really.  Kinda neat.

--bb
November 05, 2008
Re: null and type safety
"Walter Bright" wrote
> Jarrett Billingsley wrote:
>> Don't you think that eliminating something that's
>> always a bug at compile time is a worthwhile investment?
>
> Not always. There's a commensurate increase in complexity that may not 
> make it worth while.
>
> My focus is on eliminating bugs that cannot be reliably detected even at 
> run time. This will be a big win for D.

I was working in C# today, and I realized one very excellent design 
advantage for D -- the array.  In C#, if null, an array is subject to null 
dereference errors.  In D, it simply doesn't happen, because the array has a 
guard that is stored with the reference -- the length.  I think these 
similar to the kinds of things that Jarrett is referring to.  Something 
that's like a pointer, but can't ever be null, so you never have to check it 
for null before using it.  Except Jarrett's idea eliminates it at compile 
time vs. run time.

Couldn't one design a struct wrapper that implements this behavior? 
Something like:

NonNullable!(T)
{
  opAssign(T t) {/* make sure t is not null */}
  opAssign(NonNullable!(T) t) {/* no null check */}
  ...
}

-Steve
November 05, 2008
Re: null and type safety
On Wed, Nov 5, 2008 at 9:33 AM, Bill Baxter <wbaxter@gmail.com> wrote:
> I didn't really get what you meant the first time either.  The thing
> about Delight's use of auto "to great effect" wasn't clear.   I
> assumed it was basically the same as D's auto inside an if, but I see
> now that it's not.  Looks like a run-time type deduction, even though
> its not really.  Kinda neat.

It's almost the same as D's variable-inside-an-if, with the addition
that you can use it to convert a nullable type to a non-null type.
Hence, in:

if(auto f = someFunctionThatCanReturnNull())

if someFunctionThatCanReturnNull returns an int*? (nullable pointer to
int), typeof(f) will just be int* (non-null pointer to int), since in
the scope of the if statement, f is provably non-null.
November 06, 2008
Re: null and type safety
Jarrett Billingsley wrote:
> With nullable types, flow analysis doesn't have to be done.  It is
> implicit in the types.  It is mangled into function names.  foo
> _cannot_ take a pointer that may be null.  End of story.

Sure, which is why I was puzzled at the example given, which is about 
something else entirely.

What you're talking about is a type constructor to create another kind 
of pointer. It's a significant increase in complexity. That's why I was 
talking about complexity being a downside of this - there is a tradeoff.
November 06, 2008
Re: null and type safety
Steven Schveighoffer wrote:
> Couldn't one design a struct wrapper that implements this behavior? 

If that cannot be done in D, then D needs some design improvements. 
Essentially, any type should be "wrappable" in a struct which can alter 
the behavior of the wrapped type.

For example, you should also be able to create a ranged int that can 
only contain values from n to m:

RangedInt!(N, M) i;

Preserving this property of structs has driven many design choices in D, 
particularly with regards to how const fits into the type system.
1 2 3 4
Top | Discussion index | About this forum | D home