View mode: basic / threaded / horizontal-split · Log in · Help
July 29, 2005
Walter - Should we use arrays as Null?
Hi Walter,

This is something that's confused me quite a bit and I think you are the only
one that can settle it for good. The question is whether we should be using null
as a special array value. Maybe it can be broken down to pieces:

1) Why can objects be null but arrays can't (given that _both_ are by-ref)?

# // Example:
# Object obj  = null; // This is a "proper" null.
# int[] array = null; // This is not. Only via array.ptr.

IMHO this is inconsistent. The former makes sense, the latter is weird. Another
way of looking at it is: why dumb-down arrays but not objects?

2) Is it a technical limitation (for now)?
3) Is support for "proper" null arrays planned?

I, for one, would _like_ to see support for both null arrays and continued
support for null objects. As Regan has argued (and now I'm a believer), the null
special value is very useful, and we should keep this distinction (vs. empty).
Perhaps you can clarify whether this is going to happen properly or not.

In my view, proper array nulls do _not_ exist. What we have right now is very
confusing because sometimes we can use the null value and sometimes we can't. It
is also fickle because the null value is tied to the pointer. Regan thinks that
you are planning on merging emptiness and existence into one (a bad thing).

Some of the problems (not technically "bugs"):
- array.length = 0 sets the pointer to null.
- static int[0] is not null, but new int[0] is.
- .dup of an empty string (static or not) also sets the pointer to null.
- static arrays can't have null pointers.

4) What exactly does [if (array)] mean (or theoretically should mean)?
- if (array.ptr)
- if (array.length)
- if (array == null)
- if (array is null)
- or some combination thereof?

==============

In short, I think it would be dangerous to use this feature if you are planning
on subtly phasing it out. Could you please shed some light on the situation?

Thanks!
--AJG.
July 29, 2005
Re: Walter - Should we use arrays as Null?
On Fri, 29 Jul 2005 05:30:52 +0000 (UTC), AJG wrote:

> Hi Walter,

I know I'm not the big W. but here's my take this anyhow ;-)

> This is something that's confused me quite a bit and I think you are the only
> one that can settle it for good. The question is whether we should be using null
> as a special array value. Maybe it can be broken down to pieces:
> 
> 1) Why can objects be null but arrays can't (given that _both_ are by-ref)?
> 
> # // Example:
> # Object obj  = null; // This is a "proper" null.
> # int[] array = null; // This is not. Only via array.ptr.

I don't understand your concern. Both these are allowed and both work as
expected; that is to say in both cases, after the assignment, the variable
does not reference anything. What more than this are you expecting?

> IMHO this is inconsistent. The former makes sense, the latter is weird. Another
> way of looking at it is: why dumb-down arrays but not objects?

Huh? I still can't see what you are worried about.

 Object obj = null;  // Makes sense.
 int[]array = null;  // Also makes sense (to me anyhow).

> 2) Is it a technical limitation (for now)?

Is *what* a limitation?

> 3) Is support for "proper" null arrays planned?
> 
> I, for one, would _like_ to see support for both null arrays and continued
> support for null objects. 

Who says that this support is going away?

>As Regan has argued (and now I'm a believer), the null
> special value is very useful, and we should keep this distinction (vs. empty).

Absolutely! Both are good and distinct concepts.

> Perhaps you can clarify whether this is going to happen properly or not.
> 
> In my view, proper array nulls do _not_ exist. 

But they do. If array.ptr is null, then array is a null array. 

>What we have right now is very
> confusing because sometimes we can use the null value and sometimes we can't.

Huh? When can't you use it?

> It is also fickle because the null value is tied to the pointer. 

Huh? Of course it is. What else could it be?

> Regan thinks that
> you are planning on merging emptiness and existence into one (a bad thing).

I don't think that Walter is planning on this.

> Some of the problems (not technically "bugs"):
> - array.length = 0 sets the pointer to null.

This is a bug and Walter has said so. He will fix this.

> - static int[0] is not null, but new int[0] is.

static arrays cannot be null (not reference anything) by their very nature.
A static array must always reference some RAM somewhere.

What do you think that 'new int[0]' should return? 

> - .dup of an empty string (static or not) also sets the pointer to null.

This is because of the bug.

> - static arrays can't have null pointers.

Of course not. The 'static' attribute means that they occupy RAM that is
allocated at compile time.

> 4) What exactly does [if (array)] mean (or theoretically should mean)?
> - if (array.ptr)
> - if (array.length)
> - if (array == null)
> - if (array is null)
> - or some combination thereof?

It actually means ...

 if (array.ptr !is null || array.length != 0)

which is a bit redundant because we can never have the situation where the
ptr is null and the length is > 0.

> ==============
> 
> In short, I think it would be dangerous to use this feature if you are planning
> on subtly phasing it out. Could you please shed some light on the situation?

Once Walter fixes the bug in which setting the length to zero also clears
the ptr, I think we will have what you want. Hope I've helped.

-- 
Derek
Melbourne, Australia
29/07/2005 4:17:38 PM
July 29, 2005
Re: Walter - Should we use arrays as Null?
AJG wrote:
> 1) Why can objects be null but arrays can't (given that _both_ are by-ref)?
> 
> # // Example:
> # Object obj  = null; // This is a "proper" null.
> # int[] array = null; // This is not. Only via array.ptr.

Do you want all operations on a null array, such as:

# int[] array = null;
# int len = array.length; // <-- segfault

to segfault (to throw a NullPointerException in managed environments 
parlance), like they do on a null object reference?

-- 
Niko Korhonen
SW Developer
July 29, 2005
Re: Walter - Should we use arrays as Null?
On Fri, 29 Jul 2005 16:39:54 +1000, Derek Parnell <derek@psych.ward> wrote:
>> Regan thinks that
>> you are planning on merging emptiness and existence into one (a bad  
>> thing).
>
> I don't think that Walter is planning on this.

I hope not. Someone once mentioned it as a goal Walter had. I've not heard  
from Big W himself.

Regan
July 29, 2005
Re: Walter - Should we use arrays as Null?
On Fri, 29 Jul 2005 16:39:54 +1000, Derek Parnell <derek@psych.ward> wrote:
>> - static int[0] is not null, but new int[0] is.
>
> static arrays cannot be null (not reference anything) by their very  
> nature.
> A static array must always reference some RAM somewhere.

Why? I mean, I know what you mean: What's the point in having a  
non-existant static array. A static array always exists, therefore cannot  
be null. But doesn't that then make:

int[0] a;

illegal?

I thought about this for a sec and decided that no, to make it illegal  
would likely annoy the heck out of a template programmer some time in the  
future.

But, it can be null, can't it? I mean the data pointer, not the array  
'reference'. I'm not sure an 'array reference' even exists for static  
arrays? My impression is that a static array is simply implemented as a  
pointer, the length property which is static is 'macro replaced' at  
compile time.

In which case, the data pointer could be null, right? Statements like  
a.length would be fine, it's marco replaced after all. Statements like  
a[0] = 'a'; would crash, or give array bounds errors, just like any other  
array would.

Maybe I'm missing some secret of their implementation.


> What do you think that 'new int[0]' should return?

Well, at first glance 'null'. You're asking for (0 * int.sizeof) memory  
which is 0 bytes. But, have you tried it in C/C++?

The MSDN documentation states:
  "If size is 0, malloc allocates a zero-length item in the heap and  
returns a valid pointer to that item"

There is nothing in the docs for "new" but a quick experiment showed the  
same behaviour as malloc for the statement "new int[0]".

Ilya wondered immediately how it was possible to have a "zero-length item  
in the heap" so he tried DMC and Cygwin-GCC and found: "both returned at  
least 8 bytes."


If you step back and just look it at a conceptual level you'd expect the  
statements:

int[0] a;
int[] a = new int[0];

to result in the same thing, surely? i mean 'a' is an instance of an  
'int[0]' in both cases (whatever that is decided to be).

Currently they don't and there appears to be 3 choices:
 - leave it as it, nothing is wrong.
 - make "int[0] a" null.
 - make "new int[0]" non-null.

Regan
July 29, 2005
Re: Walter - Should we use arrays as Null?
>> Some of the problems (not technically "bugs"):
>> - array.length = 0 sets the pointer to null.
>
> This is a bug and Walter has said so. He will fix this.

I don't remember what Walter said but I hope he thinks about the options. 
There are three factors involved (that I can see):
1) setting length to/from 0
2) slicing to a 0 length array and appending to a 0 length array
3) the +1 that gets added to every array allocation which makes powers-of-2 
allocations the most inefficient (takes 2x the memory of what you asked for)

Currently item 3 is added because one can slice off the end of an array and 
then ask to grow that. Should 2 behave like 1 or should 1 behave like 2? I 
could imagine a solution where appending to a zero-length array reallocs 
like setting the length from 0 reallocs. In any case there isn't a pain-free 
solution.
July 29, 2005
Re: Walter - Should we use arrays as Null?
Hi Derek,

>I know I'm not the big W. but here's my take this anyhow ;-)
>
>> This is something that's confused me quite a bit and I think you are the only
>> one that can settle it for good. The question is whether we should be using null
>> as a special array value. Maybe it can be broken down to pieces:
>> 
>> 1) Why can objects be null but arrays can't (given that _both_ are by-ref)?
>> 
>> # // Example:
>> # Object obj  = null; // This is a "proper" null.
>> # int[] array = null; // This is not. Only via array.ptr.
>
>I don't understand your concern. Both these are allowed and both work as
>expected; that is to say in both cases, after the assignment, the variable
>does not reference anything. What more than this are you expecting?

Sure, they both "work" to a certain extent. But if you try to access a property
on a null object -that's illegal. On a null array, it's not. That's not a null
array. That's a pseudo-null reference that never goes away.

>> IMHO this is inconsistent. The former makes sense, the latter is weird. Another
>> way of looking at it is: why dumb-down arrays but not objects?
>
>Huh? I still can't see what you are worried about.

Arrays are dumbed-down so that you can do things like:

foreach (char c; null) { // do something }

NULL, in my opinion, is _not_ the same as empty. BUT, the above operation makes
it so. Instead, it should throw an exception at the very least, or even better,
it could be detected at compile-time.

>> 2) Is it a technical limitation (for now)?
>Is *what* a limitation?

The distinction I made before between the nullness of objects (which is
complete), and that of arrays (which is incomplete). I asked this because
perhaps it was due to the way D properties worked (more like functions), or
somesuch, meaning, it was not _intended_ to be that way.

>> 3) Is support for "proper" null arrays planned?
>> 
>> I, for one, would _like_ to see support for both null arrays and continued
>> support for null objects. 
>
>Who says that this support is going away?

If Walter decides emptiness and existence should be one. This already happens in
the language. Maybe it's due to bugs, maybe not. That's why I asked Walter for
his "vision," if you will, regarding arrays and nulls. If array null disappears,
it's likely object null will also disappear. Both of these worry me. But if
indeed they are just bugs, then why doesn't Walter say so?

>>As Regan has argued (and now I'm a believer), the null
>> special value is very useful, and we should keep this distinction (vs. empty).
>
>Absolutely! Both are good and distinct concepts.

In theory, yes. In D, not entirely. Please see below:

>> Perhaps you can clarify whether this is going to happen properly or not.
>> 
>> In my view, proper array nulls do _not_ exist. 
>
>But they do. If array.ptr is null, then array is a null array. 

Why should be have to resort to array.ptr for nullness? Why can't the _array
itself_ be null? An object _can_ be null by itself, no need to check an
"object.ptr". In fact, on a null object .ptr would throw. You have to
acknowledge this is a significant difference.

>>What we have right now is very
>> confusing because sometimes we can use the null value and sometimes we can't.
>
>Huh? When can't you use it?

I can't use it when the "bugs" get in my way. And they just so happen to get in
the way a lot. I'm working with databases right not, and essentially there's no
way to have a string represent a DBNULL, because when I dup an empty string, it
_too_ becomes NULL.

>> It is also fickle because the null value is tied to the pointer. 
>
>Huh? Of course it is. What else could it be?

It could be, say, a simple boolean. Or, it could be, say, like objects. Objects
don't rely on object.ptr, why should arrays?

>> Regan thinks that
>> you are planning on merging emptiness and existence into one (a bad thing).
>
>I don't think that Walter is planning on this.

I certainly hope not, but how can we be sure? This is why I asked.

>> Some of the problems (not technically "bugs"):
>> - array.length = 0 sets the pointer to null.
>
>This is a bug and Walter has said so. He will fix this.

Just out of curiosity, is there a post that I could read regarding this? I'd
really like to see what he said.

>> - static int[0] is not null, but new int[0] is.
>
>static arrays cannot be null (not reference anything) by their very nature.

Their very nature says nothing of nullness. It just means allocate in a
different area of memory.

>A static array must always reference some RAM somewhere.

Why? Why can't it reference null? Conceptually, I don't see a problem. But maybe
this is one of the "technical limitations" I was talking about.

>What do you think that 'new int[0]' should return? 

It should return a NON-null empty array. In current terminology:

in[] arr = new int[0];
if (arr)        // this should be TRUE.
if (arr.length) // this should be FALSE.

>> - .dup of an empty string (static or not) also sets the pointer to null.
>
>This is because of the bug.

Well this subtle bugs renders DB-null impossible because as it happens .dups are
fairly common. This is what I said about it being "fickle."

char[] s = ""; // here you have it.
char[] p = s.dup; // now you don't. very fickle.

>> - static arrays can't have null pointers.
>
>Of course not. The 'static' attribute means that they occupy RAM that is
>allocated at compile time.

This is a technical limitation. Once again, conceptually, it should be able to
point to static null just as well. Perhaps allocate 0-bytes? In other words,
this is a problem with the implementation. The language itself shouldn't be
limited because of this.

> 
>> 4) What exactly does [if (array)] mean (or theoretically should mean)?
>> - if (array.ptr)
>> - if (array.length)
>> - if (array == null)
>> - if (array is null)
>> - or some combination thereof?
>
>It actually means ...
>
>  if (array.ptr !is null || array.length != 0)

>which is a bit redundant because we can never have the situation where the
>ptr is null and the length is > 0.

IIRC this was deduced from a single dissasembly, wasn't it?
Is it _always_ the same thing? (static/dynamic/associative)?

>> ==============
>> 
>> In short, I think it would be dangerous to use this feature if you are planning
>> on subtly phasing it out. Could you please shed some light on the situation?
>
>Once Walter fixes the bug in which setting the length to zero also clears
>the ptr, I think we will have what you want. Hope I've helped.

And the very important duping "bug" (for DBs). And the inconsistency with static
arrays. And I'm sure I could find some more problems. But first I need to know
whether they are problems in the first place. Only Walter knows...

Cheers,
--AJG.
July 29, 2005
Re: Walter - Should we use arrays as Null?
Hi,

In article <dcclvb$v0i$1@digitaldaemon.com>, Niko Korhonen says...
>
>AJG wrote:
>> 1) Why can objects be null but arrays can't (given that _both_ are by-ref)?
>> 
>> # // Example:
>> # Object obj  = null; // This is a "proper" null.
>> # int[] array = null; // This is not. Only via array.ptr.
>
>Do you want all operations on a null array, such as:
>
># int[] array = null;
># int len = array.length; // <-- segfault
>
>to segfault (to throw a NullPointerException in managed environments 
>parlance), like they do on a null object reference?

For starters, yes. Why should objects be different than arrays when they are
both reference types? This is inconsistent IMHO.

Thanks,
--AJG.
July 29, 2005
Re: Walter - Should we use arrays as Null?
Hi,

>If you step back and just look it at a conceptual level you'd expect the  
>statements:
>
>int[0] a;
>int[] a = new int[0];
>
>to result in the same thing, surely? i mean 'a' is an instance of an  
>'int[0]' in both cases (whatever that is decided to be).

Exactly.

>Currently they don't and there appears to be 3 choices:
>  - leave it as it, nothing is wrong.

Something _is_ wrong, IMHO. 

>  - make "int[0] a" null.

Two wrongs don't a right make. ;)

>  - make "new int[0]" non-null.

Bingo!

>
>Regan

--AJG.
July 29, 2005
Re: Walter - Should we use arrays as Null?
"AJG" <AJG_member@pathlink.com> wrote in message 
news:dcde0p$1kop$1@digitaldaemon.com...
> Hi,
>
> In article <dcclvb$v0i$1@digitaldaemon.com>, Niko Korhonen says...
>>
>>AJG wrote:
>>> 1) Why can objects be null but arrays can't (given that _both_ are 
>>> by-ref)?
>>>
>>> # // Example:
>>> # Object obj  = null; // This is a "proper" null.
>>> # int[] array = null; // This is not. Only via array.ptr.
>>
>>Do you want all operations on a null array, such as:
>>
>># int[] array = null;
>># int len = array.length; // <-- segfault
>>
>>to segfault (to throw a NullPointerException in managed environments
>>parlance), like they do on a null object reference?
>
> For starters, yes. Why should objects be different than arrays when they 
> are
> both reference types? This is inconsistent IMHO.

I think you'll have a hard time getting lots of support for that. I much 
prefer the current behavior and I bet there is lots of existing D code that 
assumes one can test the length of an array at any time. Since an array is 
not an object I see no problem with the "inconistency" - an array is an 
array.
« First   ‹ Prev
1 2 3 4 5
Top | Discussion index | About this forum | D home