July 24, 2005
On Sun, 24 Jul 2005 20:47:43 +1200, Regan Heath <regan@netwin.co.nz> wrote:
>> Thus a good solution should be built into the language.
>
> IMO a good solution _is_ built into the language. Arrays, are a good solution to the problem posed by types which can represent non-existance, that problem being that the added expressiveness comes with greater risk of accidental use. Arrays cannot be null, yet can represent non-existance, they're a great solution.

Re-reading this I don't think I was clear enough. What I meant here is that arrays themselves are types which can represent non-existance, they're done in a clever way which enables the expressiveness without the cost. I didn't mean to imply that you could store a value type in an array and represent non-existance in some way (except of the whole array).

Regan
July 24, 2005
In article <dbvajm$1p2i$1@digitaldaemon.com>, AJG says...
>
>Hi,
>
>In article <dbv8fr$1nnm$1@digitaldaemon.com>, Dave says...
>>
>>In article <dbv45d$1ju8$1@digitaldaemon.com>, AJG says...
>>>
>>>C# 2.0 will "solve" this problem with the concept of nullable types. Even ints will be nullable. I'm not sure how this is going to work (haven't tried it),
>>
>>Interesting stuff.. I looked into this a bit and apparently the underlying implementation is done through System.Nullable<T>.
>>
>>System.Nullable<int> j;
>>int? k;
>>
>>The 'T?' form is shorthand.
>
>Interesting. I didn't know that. I was actually kinda hoping they found a magic "native" way, but I guess not.
>
>>As you can imagine, there appears to be quite a bit of overhead involved as the nullable types aren't native.
>
>Yes, I agree. Though I shouldn't speculate without having even tested for performance.
>

I did run a quick test w/ a simple loop using a few assignment operators and the penalty was on the order of 4-5x.

Even if that is actually indicative of what you could expect with 'real world' code, I'm not implying that C# nullable types aren't useful just because of a performance penalty. Who knows, if built into D 'natively', maybe all/most of that penalty could be (practically) optimized away or maybe there's an internal implementation possible that wouldn't cause any 'penalty'?

>>But there's nothing stopping you from retrieving,
>>say, a DB value into a nullable type, checking if it's null and then assigning
>>it to a native variable if it's not. But assigning it requires accessing a
>>property (int k = j.Value;) or a cast (int k = (int)j;).
>
>This looks a little cumbersome. It will remain cumbersome without language support, IMHO.
>
>>I like the idea, but given that you will still always have to check if a
>>nullable variable is not null before using it or even assigning it to another
>>(non-nullable) variable, I'm having trouble imagining how much more >productive /
>>readable it's going to make coding for most chores where "nullable native >types" would be useful.
>
>I disagree here. I think the non-existent concept is a good one. It's useful in arrays (and possibly classes), and I think the usefulness extends across primitives as well.
>
>>For example, for database applications, I can still see a need
>>to write a library of wrapper functions to assign a column to native data >types,
>>or if the table was represented by a class, to check for null each time a >column was retrieved in order to assign the value to a native type.
>
>For instance:
>
># void someDataFunc(nullableInt dbValue) {
>#     // Handle the special case:
>#     if (!dbValue) throw new Exception("Value must be specified.");
>#
>#     // These can now all be valid values:
>#          if (dbValue  < 0) { /* Case 1 */ }
>#     else if (dbValue == 0) { /* Case 2 */ }
>#     else if (dbValue  > 0) { /* Case 3 */ }
>#     else assert(false);
># }
>
>>Either way it seems like it will require about the same amount of code to write most applications, but with added complexity to the language.
>
>Some (most?) of the complexity is already there. Arrays and Classes both are already capable of existing vs. being empty. This would merely extend the feature for orthogonality. I think it would be fairly useful.
>
>Just my 2 cents.
>--AJG.
>
>


July 24, 2005
Holger schrieb:
> Hi Regan, you're of course spot-on.
> It's not the first time that someone expressed misguided perceptions of "not
> existant" vs "empty" in the C language here. I really wonder how often we'll
> need to discuss such basic C-isms on the D NG? People should better learn their
> stuff before making such bold statements.

Misconceptions? That was a typo and Regan could, if he cared to read through the beginnings of the preceding newsgroup, know me, know that, although he pretty much popped up shortly before i disapperared. And, i really wonder why anyone would have to listen to someone who neither leaves his complete name nor a real e-mail adress, who can just drop a bomb a disappear.

You are free to google for my name, it is not that common, and make yourself a picture.

-i.
July 24, 2005
Regan Heath schrieb:
> Sure, memory management makes things complicated. But, uniqueness has  nothing to do with non-existance. The fact that non-existance is typically  represented by null is the same regardless of memory management model.

And the emptyness?

>> There is a problem with "exists but empty". What does malloc do when you  request 0 bytes?
> 
> Allocates a zero length item on the heap. (I checked this recently).

How? If this was so, it would break the promise that malloc never returns the same adress unless this memory was returned by free. So, my bet would be that it returns a byte or a word of memory, which basically doesn't matter since you may not dereference it anyway. Does it segfault or throw when you access the first byte?

Which implementation did you check? It is still possible that other implementations do it differently. I just checked DMC and Cygwin-GCC, and both returned at least 8 bytes.

>> So, in C there is no other way than to embed the information on the  non-existance into your data structure.

ARRGH. I obviously meant the emptyness, probably was very tired when i wrote the message.

And you, you should know that i have posts in the digitalmars newsgroups dating as early as two and a half years back, and if you ever looked at them you would know that you don't have to explain such stuff to me, but that i just mixed up the words.

The current newsgroup is just too much of a bunch of people not able nor willing to take time for each other and the subject, which was why i basically left. I think i was quite right about it and i don't think i'd like to show here up ever again.

> No, you simply use null. A non-existant string in C is a null pointer. An  empty string in C is a non-null pointer which contains a \0 as the first  character. The same applies to any other object. A null pointer indicates  non-existance, and emptiness is represented in whatever fashion makes  sense for the object i.e. a length property set to 0.

Blah. True.

>> In the case of strings, this is a string having '\0' character at the  very beginning.
> 
> No, that is an "empty" string, not a "non-existant" one.

I know dammit.

> None of this is necessary.

Now, if you like to explain *how* exactly you want to distinguish the empty something by pointer? You (a) embed the information into the target, or (b) you make an empty singleton and distinguish it by pointer comparison.

Please note that we're not talking about the D arrays here, it's about your statement: "a pointer is a good example of a type which can represent  non-existance, exists but empty, exists and not empty" and i'm waiting for you to either see your mistake or show me exactly how.

>> What do you mean by can't be null?
> 
> char[] p = null;
> if (p.length == 0) { //does not crash, p itself is never 'null' }

Ok, if you like to see it so.

I think i would call p "null" if i cannot dereference any element out of it. Just like a null pointer is something you cannot dereference, and where you can distinguish that by looking at the representation of the pointer, lust like you can by looking at the fields of an array slice. Whether you get a specialized exception or a general memory protection fault if you try to dereference it nontheless, is implementation detail. BTW, to add more to this similarity, you can catch the exception resulting from dereferencing a null pointer, just as you can catch the one resulting from dereferencing an element from the null array.

>> Ok, given we still have the ability to manipulate the pointer and the  length separately, how should array conversion to boolean condition be  defined then?
> 
> The same way it works for every other type in D, the statement "if(x)"  means "compare x to null or 0". In the case of a reference it compares the  reference to null.

> The confusion arises in this case because arrays in D cannot be null, and  because arrays are in fact implemented as stack based structs in the  background. This makes arrays appear to be a struct and not a reference,  however currently in all (I believe) situations they behave as references.  I believe this was done on purpose.

I still cannot quite grasp the statement "arrays cannot be null". :)

Yes, your writing looks very confused to me. :) It is quite correct to think of array (or, perhaps more correctly slice) as a value struct. However, i don't see how it would "behave as a reference" as opposed to the pointer... An array references a bunch of objects, which you don't access directly, but you "dereference" a certain element of an array by using operator[], similarly like operator* is used to dereference a single pointer.

Though i'm not that sure whether the distinction between pointers and references needs to be kept upright. It is only of syntactical nature, but so many sorts of pointers in D do some sort of syntactical forwarding to their target - the function pointer in the same way as in C, but also a struct pointer makes forwarding of the dot '.' operator.

> As I've noted in all cases where an array reference would be null, i.e.
> 
> char[] p = null;
> 
> it isn't, but instead the data pointer p.ptr is null.

...

> So, in order for them to behave as references it's logically consistent  for "if(p)" to check the data ptr vs null. Change that and you need to  code special cases for arrays vs other reference types, eg.
> 
> template doWrite(Type) { void doWrite(Type p) {
>   if (p) writefln(p);
> }
}
> 
> class C {
>   char[] toString() { return "C"; }
> }
> 
> char[] p = "test";
> C c = new C();
> 
> doWrite!(char[])(p);
doWrite!(C)(c);

This just works. I don't see any *specific* problem with templates if array is defined to always be null when it is empty, only the *general* tiny loss of freedom we just discussed.

There is another thing that comes to my mind: when you do a lot of sclicing and the slices get nulled-out as soon as you cannot reference any element through them, it can make the garbage collector reclaim the memory sooner.

-eye
July 24, 2005
On Sun, 24 Jul 2005 20:21:30 +0200, Ilya Minkov <minkov@cs.tum.edu> wrote:
> Holger schrieb:
>> Hi Regan, you're of course spot-on.
>> It's not the first time that someone expressed misguided perceptions of "not
>> existant" vs "empty" in the C language here. I really wonder how often we'll
>> need to discuss such basic C-isms on the D NG? People should better learn their
>> stuff before making such bold statements.
>
> Misconceptions? That was a typo and Regan could, if he cared to read through the beginnings of the preceding newsgroup, know me

I do, in fact, "know you" well enough to have thought as I typed my reply that you must have simply made a mistake. Nor, did I make the above statements, so, I really have no idea why you'd react in such a way towards *me*?

> , know that, although he pretty much popped up shortly before i disapperared. And, i really wonder why anyone would have to listen to someone who neither leaves his complete name nor a real e-mail adress, who can just drop a bomb a disappear.

Are you referring to me? I'm using my complete name and my real email address.

> You are free to google for my name, it is not that common, and make yourself a picture.

You can google mine as well 7 of the top 10 are me.

Regan
July 24, 2005
On Sun, 24 Jul 2005 22:09:26 +0200, Ilya Minkov <minkov@cs.tum.edu> wrote:
> Regan Heath schrieb:
>> Sure, memory management makes things complicated. But, uniqueness has  nothing to do with non-existance. The fact that non-existance is typically  represented by null is the same regardless of memory management model.
>
> And the emptyness?
>
>>> There is a problem with "exists but empty". What does malloc do when you  request 0 bytes?
>>  Allocates a zero length item on the heap. (I checked this recently).
>
> How?

I have no idea, I am just repeating what the MSDN documentation says ;) I wrote some C to test it, and malloc(0) and new char[0] both return non-null. That is all I tried.

> If this was so, it would break the promise that malloc never returns the same adress unless this memory was returned by free. So, my bet would be that it returns a byte or a word of memory, which basically doesn't matter since you may not dereference it anyway. Does it segfault or throw when you access the first byte?

Not sure. I didn't check it. I suspect your guesses are correct.

> Which implementation did you check? It is still possible that other implementations do it differently. I just checked DMC and Cygwin-GCC, and both returned at least 8 bytes.

I tested only the M$ compiler that comes with Visual Studio 6.0. I read only the MSDN documentation. It appears all implementations tested so far (mine and yours above) return something on a malloc of 0, rather than nothing.

>>> So, in C there is no other way than to embed the information on the  non-existance into your data structure.
>
> ARRGH. I obviously meant the emptyness, probably was very tired when i wrote the message.

No problem. I half suspected as much.

> And you, you should know that i have posts in the digitalmars newsgroups dating as early as two and a half years back, and if you ever looked at them you would know that you don't have to explain such stuff to me, but that i just mixed up the words.

I could not be certain it was a mistake. I replied not to correct *you* but to correct the *statements* so another would not read them and take them as truth. My reply was as much to you as to the newsgroup as a whole, so while you may not need the explainations, others might. My comments were never intended as personal criticism.

> The current newsgroup is just too much of a bunch of people not able nor willing to take time for each other and the subject, which was why i basically left. I think i was quite right about it and i don't think i'd like to show here up ever again.

I'm sorry you feel that way. I'm here because I like to discuss these sorts of things in my spare time. (can anyone say 'geek'). Regardless of the quality(*) of the replies I get they all influence and refine _my_ opinion, giving me a clearer idea of what I'm talking about and all the issues involved. In short, it's fun and it's self improvement.

(*) I'll leave that to the reader to define.

>> None of this is necessary.
>
> Now, if you like to explain *how* exactly you want to distinguish the empty something by pointer?

I wasn't suggesting you could. I thought you were confusing "empty" and "non-existant" at this stage and framed my reply with that in mind, essentially I thought you were suggesting you needed a singleton to represent non-existant, not empty.

> You (a) embed the information into the target, or (b) you make an empty singleton and distinguish it by pointer comparison.

You're correct.

> Please note that we're not talking about the D arrays here, it's about your statement: "a pointer is a good example of a type which can represent  non-existance, exists but empty, exists and not empty" and i'm waiting for you to either see your mistake or show me exactly how.

I see the point you're making, and you are correct.

char *a = null;
char *b = "";
char *c = "test";

a is non-existant.
b exists but is empty
c exists and is not empty.

It is not technically the pointer representing all 3, it only really represents exists, or not. Whether it is empty or not is represented by the data.

All I meant by my statement is that with a pointer you can do all 3. Compare that to 'int' which can only do non existance by using one of it's _values_ for non existance (this is limiting, and sort of illogical IMO - using a value to represent non-existance).

>>> What do you mean by can't be null?
>>  char[] p = null;
>> if (p.length == 0) { //does not crash, p itself is never 'null' }
>
> Ok, if you like to see it so.
>
> I think i would call p "null" if i cannot dereference any element out of it.

I would call it "null" if it was equal to "null". eg.
  if (p is null) {}

> Just like a null pointer is something you cannot dereference, and where you can distinguish that by looking at the representation of the pointer, lust like you can by looking at the fields of an array slice.

There is never a problem de-referencing an array reference itself, eg.

char[] p = null;
if (p.length) { //never crashes, so by your definition 'p' is never null, right? }

There _is_ a problem referencing data outside an array, but I don't think that's the same thing, eg.

char[] p = "one";
p[4] = 'a';  //crash (array bounds error)

this is dereferencing the data pointer (which doesn't crash) by an offset larger than the array (which is what crashes).

> Whether you get a specialized exception or a general memory protection fault if you try to dereference it nontheless, is implementation detail.

True.

> BTW, to add more to this similarity, you can catch the exception resulting from dereferencing a null pointer, just as you can catch the one resulting from dereferencing an element from the null array.

True.

>>> Ok, given we still have the ability to manipulate the pointer and the  length separately, how should array conversion to boolean condition be  defined then?
>>  The same way it works for every other type in D, the statement "if(x)"  means "compare x to null or 0". In the case of a reference it compares the  reference to null.
>
>> The confusion arises in this case because arrays in D cannot be null, and  because arrays are in fact implemented as stack based structs in the  background. This makes arrays appear to be a struct and not a reference,  however currently in all (I believe) situations they behave as references.  I believe this was done on purpose.
>
> I still cannot quite grasp the statement "arrays cannot be null". :)

See above for my meaning. The array reference itself is never "null" (and can always be dereferenced).

> Yes, your writing looks very confused to me. :) It is quite correct to think of array (or, perhaps more correctly slice) as a value struct.

I disagree. I believe the intention is that we think of them as references. The documentation calls them references, their behaviour is orthogonal with other references. I believe Walter made them behave as references on purpose.

They are however implemented as a stack based struct. Derek has shown that.

> However, i don't see how it would "behave as a reference" as opposed to the pointer... An array references a bunch of objects

I'd say an array 'contains' a bunch of objects, but that's beside the point. I'm referring to the array variable itself, eg.
  char[] p;

'p' behaves like a reference, just as:

class C {}
C c;

'c' behaves like a reference.

> , which you don't access directly, but you "dereference" a certain element of an array by using operator[], similarly like operator* is used to dereference a single pointer.

You access it directly when you use [] or any other property or member of it. You de-reference the array reference in order to use these properties. In the case of [] you dereference the data pointer in the array reference by an offset.

> Though i'm not that sure whether the distinction between pointers and references needs to be kept upright. It is only of syntactical nature, but so many sorts of pointers in D do some sort of syntactical forwarding to their target - the function pointer in the same way as in C, but also a struct pointer makes forwarding of the dot '.' operator.

Indeed.. when I think about pointers and references I find myself thinking of them as the "same thing". A reference is perhaps just a specific type of pointer. I mean, a reference is a pointer to an object, but a pointer is more general, you can have a pointer to a pointer to a ... to an object. You can create pointers to any other type.

>> As I've noted in all cases where an array reference would be null, i.e.
>>  char[] p = null;
>>  it isn't, but instead the data pointer p.ptr is null.
>
> ...
>
>> So, in order for them to behave as references it's logically consistent  for "if(p)" to check the data ptr vs null. Change that and you need to  code special cases for arrays vs other reference types, eg.
>>  template doWrite(Type) { void doWrite(Type p) {
>>   if (p) writefln(p);
>> }
> }
>>  class C {
>>   char[] toString() { return "C"; }
>> }
>>  char[] p = "test";
>> C c = new C();
>>  doWrite!(char[])(p);
> doWrite!(C)(c);
>
> This just works. I don't see any *specific* problem with templates if array is defined to always be null when it is empty, only the *general* tiny loss of freedom we just discussed.

"general tiny loss of freedom"? you mean the ability to express non-existance?

> There is another thing that comes to my mind: when you do a lot of sclicing and the slices get nulled-out as soon as you cannot reference any element through them, it can make the garbage collector reclaim the memory sooner.

I don't see how this is relevant, sorry.

Regan
July 24, 2005
On Sun, 24 Jul 2005 14:38:05 +0000 (UTC), Dave <Dave_member@pathlink.com> wrote:
>> In article <dbv8fr$1nnm$1@digitaldaemon.com>, Dave says...
>>>
>>> In article <dbv45d$1ju8$1@digitaldaemon.com>, AJG says...
>>>>
>>>> C# 2.0 will "solve" this problem with the concept of nullable types. Even ints
>>>> will be nullable. I'm not sure how this is going to work (haven't tried it),
>>>
>>> Interesting stuff.. I looked into this a bit and apparently the underlying
>>> implementation is done through System.Nullable<T>.
>>>
>>> System.Nullable<int> j;
>>> int? k;
>>>
>>> The 'T?' form is shorthand.
>>
>> Interesting. I didn't know that. I was actually kinda hoping they found a magic
>> "native" way, but I guess not.
>>
>>> As you can imagine, there appears to be quite a bit of overhead involved as the
>>> nullable types aren't native.
>>
>> Yes, I agree. Though I shouldn't speculate without having even tested for
>> performance.
>>
>
> I did run a quick test w/ a simple loop using a few assignment operators and the
> penalty was on the order of 4-5x.
>
> Even if that is actually indicative of what you could expect with 'real world'
> code, I'm not implying that C# nullable types aren't useful just because of a
> performance penalty. Who knows, if built into D 'natively', maybe all/most of
> that penalty could be (practically) optimized away or maybe there's an internal
> implementation possible that wouldn't cause any 'penalty'?

D's arrays are exactly this. They are a nullable type which is implemented as a stack based value type. They are fast and efficient. Walter could do the same thing with boxing i.e. build boxing into the language, implement it using a stack based value type.

Regan
July 24, 2005
In article <42E3DC2A.1040406@cs.tum.edu>, Ilya Minkov says...
>
>Holger schrieb:
>> Hi Regan, you're of course spot-on.
>> It's not the first time that someone expressed misguided perceptions of "not
>> existant" vs "empty" in the C language here. I really wonder how often we'll
>> need to discuss such basic C-isms on the D NG? People should better learn their
>> stuff before making such bold statements.
>
>Misconceptions? That was a typo and Regan could, if he cared to read through the beginnings of the preceding newsgroup, know me, know that, although he pretty much popped up shortly before i disapperared. And, i really wonder why anyone would have to listen to someone who neither leaves his complete name nor a real e-mail adress, who can just drop a bomb a disappear.
>
>You are free to google for my name, it is not that common, and make yourself a picture.
>
>-i.


Good answer Ilya, you hit the mark. However, my philippic wasn't specifically
addressed at you. It's just that this particular misconception has popped
up quite a few times in the past and I felt annoyed. Anyway, I apologize for
being caustically. I didn't mean to question you personal abilities. Sorry ...

Still,
Holger


July 24, 2005
In article <opsufqj2rn23k2f5@nrage.netwin.co.nz>, Regan Heath says...
>
>On Sun, 24 Jul 2005 20:21:30 +0200, Ilya Minkov <minkov@cs.tum.edu> wrote:
>> Holger schrieb:
>>> Hi Regan, you're of course spot-on.
>>> It's not the first time that someone expressed misguided perceptions of
>>> "not
>>> existant" vs "empty" in the C language here. I really wonder how often
>>> we'll
>>> need to discuss such basic C-isms on the D NG? People should better
>>> learn their
>>> stuff before making such bold statements.
>>
>> Misconceptions? That was a typo and Regan could, if he cared to read through the beginnings of the preceding newsgroup, know me
>
>I do, in fact, "know you" well enough to have thought as I typed my reply that you must have simply made a mistake. Nor, did I make the above statements, so, I really have no idea why you'd react in such a way towards *me*?
>
>> , know that, although he pretty much popped up shortly before i disapperared. And, i really wonder why anyone would have to listen to someone who neither leaves his complete name nor a real e-mail adress, who can just drop a bomb a disappear.
>
>Are you referring to me? I'm using my complete name and my real email address.
>
>> You are free to google for my name, it is not that common, and make yourself a picture.
>
>You can google mine as well 7 of the top 10 are me.
>
>Regan

Regan, calm down please. It's me, Holger, that is the hooligan here! Again, I apologize for my tone.

Cheers,
Holger


July 24, 2005
On Sun, 24 Jul 2005 21:37:09 +0000 (UTC), Holger <Holger_member@pathlink.com> wrote:
> In article <opsufqj2rn23k2f5@nrage.netwin.co.nz>, Regan Heath says...
>>
>> On Sun, 24 Jul 2005 20:21:30 +0200, Ilya Minkov <minkov@cs.tum.edu> wrote:
>>> Holger schrieb:
>>>> Hi Regan, you're of course spot-on.
>>>> It's not the first time that someone expressed misguided perceptions of
>>>> "not
>>>> existant" vs "empty" in the C language here. I really wonder how often
>>>> we'll
>>>> need to discuss such basic C-isms on the D NG? People should better
>>>> learn their
>>>> stuff before making such bold statements.
>>>
>>> Misconceptions? That was a typo and Regan could, if he cared to read
>>> through the beginnings of the preceding newsgroup, know me
>>
>> I do, in fact, "know you" well enough to have thought as I typed my reply
>> that you must have simply made a mistake. Nor, did I make the above
>> statements, so, I really have no idea why you'd react in such a way
>> towards *me*?
>>
>>> , know that, although he pretty much popped up shortly before i
>>> disapperared. And, i really wonder why anyone would have to listen to
>>> someone who neither leaves his complete name nor a real e-mail adress,
>>> who can just drop a bomb a disappear.
>>
>> Are you referring to me? I'm using my complete name and my real email
>> address.
>>
>>> You are free to google for my name, it is not that common, and make
>>> yourself a picture.
>>
>> You can google mine as well 7 of the top 10 are me.
>>
>> Regan
>
> Regan, calm down please. It's me, Holger, that is the hooligan here!

You are right. ;) I am calm, I did not intend for my comments above to sound angry.

> Again, I apologize for my tone.

We're all adults here, you reply shows as much (no condescention meant/implied).

Regan