View mode: basic / threaded / horizontal-split · Log in · Help
September 27, 2009
Re: Null references redux
downs wrote:
> Jeremie Pelletier wrote:
>> Andrei Alexandrescu wrote:
>>> downs wrote:
>>>> Walter Bright wrote:
>>>>> Nick Sabalausky wrote:
>>>>>
>>>>> I agree with you that if the compiler can detect null dereferences at
>>>>> compile time, it should.
>>>>>
>>>>>
>>>>>>> Also, by "safe" I presume you mean "memory safe" which means free of
>>>>>>> memory corruption. Null pointer exceptions are memory safe. A null
>>>>>>> pointer could be caused by memory corruption, but it cannot *cause*
>>>>>>> memory corruption.
>>>>>> No, he's using the real meaning of "safe", not the
>>>>>> misleadingly-limited "SafeD" version of "safe" (which I'm still
>>>>>> convinced is going to get some poor soul into serious trouble from
>>>>>> mistakingly thinking their SafeD program is much safer than it really
>>>>>> is). Out here in reality, "safe" also means a lack of ability to
>>>>>> crash, or at least some level of protection against it. 
>>>>> Memory safety is something that can be guaranteed (presuming the
>>>>> compiler is correctly implemented). There is no way to guarantee that a
>>>>> non-trivial program cannot crash. It's the old halting problem.
>>>>>
>>>> Okay, I'm gonna have to call you out on this one because it's simply
>>>> incorrect.
>>>>
>>>> The halting problem deals with a valid program state - halting.
>>>>
>>>> We cannot check if every program halts because halting is an
>>>> instruction that must be allowed at almost any point in the program.
>>>>
>>>> Why do crashes have to be allowed? They're not an allowed instruction!
>>>>
>>>> A compiler can be turing complete and still not allow crashes. There
>>>> is nothing wrong with this, and it has *nothing* to do with the
>>>> halting problem.
>>>>
>>>>>> You seem to be under the impression that nothing can be made
>>>>>> uncrashable without introducing the possibility of corrupted state.
>>>>>> That's hogwash.
>>>>> I read that statement several times and I still don't understand
>>>>> what it
>>>>> means.
>>>>>
>>>>> BTW, hardware null pointer checking is a safety feature, just like
>>>>> array
>>>>> bounds checking is.
>>>> PS: You can't convert segfaults into exceptions under Linux, as far
>>>> as I know.
>>> How did Jeremie do that?
>>>
>>> Andrei
>> A signal handler with the undocumented kernel parameters attaches the
>> signal context to the exception object, repairs the stack frame forged
>> by the kernel to make us believe we called the handler ourselves, does a
>> backtrace right away and attaches it to the exception object, and then
>> throw it.
>>
>> The error handling code will unwind down to the runtime's main() where a
>> catch clause is waiting for any Throwables, sending them back into the
>> unhandled exception handler, and a crash window appears with the
>> backtrace, all finally blocks executed, and gracefully shutting down.
>>
>> All I need to do is an ELF/DWARF reader to extract symbolic debug info
>> under linux, its already working for PE/CodeView on windows.
>>
>> Jeremie
> 
> 
> Woah, nice. I stand corrected. Is this in druntime already?

Not yet, its part of a custom runtime I'm working on and wish to release 
under a public domain license when I get the time. The code is linked 
from a thread in D.announce.
September 27, 2009
Re: Null references redux
Jeremie Pelletier wrote:
>> Is this Linux specific? what about other *nix systems, like BSD and 
>> solaris?
> 
> Signal handler are standard to most *nix platforms since they're part of 
> the posix C standard libraries, maybe some platforms will require a 
> special handling but nothing impossible to do.

Let me write a message on behalf of Sean Kelly. He wrote that to Walter 
and myself this morning, then I suggested him to post it but probably he 
is off email for a short while. Hopefully the community will find a 
solution to the issue he's raising. Let me post this:

===================
Sean Kelly wrote:

There's one minor problem with his code.  It's not safe to throw an 
exception from a signal handler.  Here's a quote from the POSIX spec at 
opengroup.org:

"In order to prevent errors arising from interrupting non-reentrant 
function calls, applications should protect calls to these functions 
either by blocking the appropriate signals or through the use of some 
programmatic semaphore (see semget() , sem_init() , sem_open() , and so 
on). Note in particular that even the "safe" functions may modify errno; 
the signal-catching function, if not executing as an independent thread, 
may want to save and restore its value. Naturally, the same principles 
apply to the reentrancy of application routines and asynchronous data 
access. Note thatlongjmp() and siglongjmp() are not in the list of 
reentrant functions. This is because the code executing after longjmp() 
and siglongjmp() can call any unsafe functions with the same danger as 
calling those unsafe functions directly from the signal handler. 
Applications that use longjmp() andsiglongjmp() from within signal 
handlers require rigorous protection in order to be portable."

If this were an acceptable approach it would have been in druntime ages 
ago :-)
===================



Andrei
September 27, 2009
Re: Null references redux
Jeremie Pelletier wrote:
> downs wrote:
>> Jeremie Pelletier wrote:
>>> Andrei Alexandrescu wrote:
>>>> downs wrote:
>>>>> Walter Bright wrote:
>>>>>> Nick Sabalausky wrote:
>>>>>>
>>>>>> I agree with you that if the compiler can detect null dereferences at
>>>>>> compile time, it should.
>>>>>>
>>>>>>
>>>>>>>> Also, by "safe" I presume you mean "memory safe" which means 
>>>>>>>> free of
>>>>>>>> memory corruption. Null pointer exceptions are memory safe. A null
>>>>>>>> pointer could be caused by memory corruption, but it cannot *cause*
>>>>>>>> memory corruption.
>>>>>>> No, he's using the real meaning of "safe", not the
>>>>>>> misleadingly-limited "SafeD" version of "safe" (which I'm still
>>>>>>> convinced is going to get some poor soul into serious trouble from
>>>>>>> mistakingly thinking their SafeD program is much safer than it 
>>>>>>> really
>>>>>>> is). Out here in reality, "safe" also means a lack of ability to
>>>>>>> crash, or at least some level of protection against it. 
>>>>>> Memory safety is something that can be guaranteed (presuming the
>>>>>> compiler is correctly implemented). There is no way to guarantee 
>>>>>> that a
>>>>>> non-trivial program cannot crash. It's the old halting problem.
>>>>>>
>>>>> Okay, I'm gonna have to call you out on this one because it's simply
>>>>> incorrect.
>>>>>
>>>>> The halting problem deals with a valid program state - halting.
>>>>>
>>>>> We cannot check if every program halts because halting is an
>>>>> instruction that must be allowed at almost any point in the program.
>>>>>
>>>>> Why do crashes have to be allowed? They're not an allowed instruction!
>>>>>
>>>>> A compiler can be turing complete and still not allow crashes. There
>>>>> is nothing wrong with this, and it has *nothing* to do with the
>>>>> halting problem.
>>>>>
>>>>>>> You seem to be under the impression that nothing can be made
>>>>>>> uncrashable without introducing the possibility of corrupted state.
>>>>>>> That's hogwash.
>>>>>> I read that statement several times and I still don't understand
>>>>>> what it
>>>>>> means.
>>>>>>
>>>>>> BTW, hardware null pointer checking is a safety feature, just like
>>>>>> array
>>>>>> bounds checking is.
>>>>> PS: You can't convert segfaults into exceptions under Linux, as far
>>>>> as I know.
>>>> How did Jeremie do that?
>>>>
>>>> Andrei
>>> A signal handler with the undocumented kernel parameters attaches the
>>> signal context to the exception object, repairs the stack frame forged
>>> by the kernel to make us believe we called the handler ourselves, does a
>>> backtrace right away and attaches it to the exception object, and then
>>> throw it.
>>>
>>> The error handling code will unwind down to the runtime's main() where a
>>> catch clause is waiting for any Throwables, sending them back into the
>>> unhandled exception handler, and a crash window appears with the
>>> backtrace, all finally blocks executed, and gracefully shutting down.
>>>
>>> All I need to do is an ELF/DWARF reader to extract symbolic debug info
>>> under linux, its already working for PE/CodeView on windows.
>>>
>>> Jeremie
>>
>>
>> Woah, nice. I stand corrected. Is this in druntime already?
> 
> Not yet, its part of a custom runtime I'm working on and wish to release 
> under a public domain license when I get the time. The code is linked 
> from a thread in D.announce.

Some of this functionality is also in Tango (SVN version). Signals are 
catched only to print a backtrace.
September 27, 2009
Re: Null references redux
BCS wrote:

> Hello Lutger,
> 
>> The answer may
>> depend on [...]
>> the habits of the 'programmers' in question, I don't know.
>> 
> 
> If you can't trust the programmer to write good code, replace them with
> someone you can trust. There will never be a usable language that can take
> in garbage and spit out correct programs.

Hi. I don't think this argument will work, for several reasons:

First, there is a huge demand for programmers, so much that even I got hired 
in this time of crisis ;) Good programmers don't suddenly fall from the 
skies apparently. 
Second, there are lot's of tasks doable by programmers with less skill than 
others using tools that trade safety for performance / expressiveness / 
whatever. 
Finally, programmers are humans, humans make mistakes, have quirks and bad 
days. All of them. What it comes down to is that languages are made in order 
to service and adapt to the programmers, not the other way around.

Do you maintain that a programmer who can't deal with non-nullable 
references without hacking them away is unusually incompetent? I don't know 
about this. Actually I suspect non-nullable references by default are in the 
end safer (whatever that means), but only if they don't complicate the use 
of nullable references.
September 27, 2009
Re: Null references redux
Hello Lutger,

> BCS wrote:
> 
>> Hello Lutger,
>> 
>>> The answer may
>>> depend on [...]
>>> the habits of the 'programmers' in question, I don't know.
>> If you can't trust the programmer to write good code, replace them
>> with someone you can trust. There will never be a usable language
>> that can take in garbage and spit out correct programs.
>> 
> Hi. I don't think this argument will work, for several reasons:
> 
[...]
> 
> Do you maintain that a programmer who can't deal with non-nullable
> references without hacking them away is unusually incompetent?

Incompetent? No. But I wouldn't want to hire a programer that *habitually* 
(and unnecessarily) hacks past a feature designed to prevent bugs. The best 
race car driver in the world is clearly not incompetent but would still get 
a ticket on public roads for speeding or following to close.

> I don't
> know about this. Actually I suspect non-nullable references by default
> are in the end safer (whatever that means), but only if they don't
> complicate the use of nullable references.

I'll second that.
September 27, 2009
Re: Null references redux
On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail@erdani.org> said:

> Michel Fortin wrote:
>> On 2009-09-26 23:28:30 -0400, Michel Fortin <michel.fortin@michelf.com> said:
>> 
>>> On 2009-09-26 22:07:00 -0400, Walter Bright <newshound1@digitalmars.com> said:
>>> 
>>>> [...] The facilities in D enable one to construct a non-nullable type, 
>>>> and they are appropriate for many designs. I just don't see them as a 
>>>> replacement for *all* reference types.
>>> 
>>> As far as I understand this thread, no one here is arguing that 
>>> non-nullable references/pointers should replace *all* reference/pointer 
>>> types. The argument made is that non-nullable should be the default and 
>>> nullable can be specified explicitly any time you need it.
>>> 
>>> So if you need a reference you use "Object" as the type, and if you 
>>> want that reference to be nullable you write "Object?". The static 
>>> analysis can then assert that your code properly check for null prior 
>>> dereferencing a nullable type and issues a compilation error if not.
>> 
>> I just want to add: some people here are suggesting the compiler adds 
>> code to check for null and throw exceptions... I believe like you that 
>> this is the wrong approach because, like you said, it makes people add 
>> dummy try/catch statements to ignore the error. What you want a 
>> prorammer to do is check for null and properly handle the situation 
>> before the error occurs, and this is exactly what the static analysis 
>> approach I suggest forces.
>> 
>> Take this example where "a" is non-nullable and "b" is nullable:
>> 
>> string test(Object a, Object? b)
>> {
>>     auto x = a.toString();
>>     auto y = b.toString();
>>         return x ~ y;
>> }
>> 
>> This should result in a compiler error on line 4 with a message telling 
>> you that "b" needs to be checked for null prior use. The programmer 
>> must then fix his error with an if (or some other control structure), 
>> like this:
>> 
>> string test(Object a, Object? b)
>> {
>>     audo result = a.toString();
>>     if (b)
>>         result ~= b.toString();
>> 
>>     return result;
>> }
>> 
>> And now the compiler will let it pass. This is what I'd like to see. 
>> What do you think?
>> 
>> I'm not totally against throwing exceptions in some cases, but the 
>> above approach would be much more useful. Unfortunatly, throwing 
>> exceptions it the best you can do with a library type approach.
> 
> I don't think this would fly.

You want me to add wings? Please explain.

> One good thing about nullable references is that they are dynamically 
> checked for validity at virtually zero cost.

When you say they are dynamically checked, do you mean it throws an 
exception when you assign null? I'm not totally against this idea, but 
I find the above a supperior solution because it forces the programmer 
to handle the problem where it occurs and it doesn't require any 
runtime check.

> Non-nullable references, therefore, would not add value in that 
> respect, but would add value by reducing the cases when programmers 
> forgot to initialize references properly.

To me it looks like you're supporting an inferior concept for 
non-nullable references because the better one will not "fly" (whatever 
that means). Well, I support both concepts of non-nullable because they 
both can be very useful, but I believe static checking is a better way 
than throwing exceptions.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
September 27, 2009
Re: Null references redux
Jarrett Billingsley:

> And if you have a nullable reference that you know is not null for the
> rest of the function? Just put "assert(x !is null)" and everything
> that follows will assume it's not null.

Asserts tend to vanish in release mode, so it may be better to use something different. A possibility is to use the enforce() some people have shown here.

Another possibility is the very strange assume() of Visual C++, that I may appreciate for other purposes too:
http://msdn.microsoft.com/en-us/library/1b3fsfxw(loband).aspx

Bye,
bearophile
September 27, 2009
Re: Null references redux
Andrei Alexandrescu wrote:
> Jeremie Pelletier wrote:
>>> Is this Linux specific? what about other *nix systems, like BSD and 
>>> solaris?
>>
>> Signal handler are standard to most *nix platforms since they're part 
>> of the posix C standard libraries, maybe some platforms will require a 
>> special handling but nothing impossible to do.
> 
> Let me write a message on behalf of Sean Kelly. He wrote that to Walter 
> and myself this morning, then I suggested him to post it but probably he 
> is off email for a short while. Hopefully the community will find a 
> solution to the issue he's raising. Let me post this:
> 
> ===================
> Sean Kelly wrote:
> 
> There's one minor problem with his code.  It's not safe to throw an 
> exception from a signal handler.  Here's a quote from the POSIX spec at 
> opengroup.org:
> 
> "In order to prevent errors arising from interrupting non-reentrant 
> function calls, applications should protect calls to these functions 
> either by blocking the appropriate signals or through the use of some 
> programmatic semaphore (see semget() , sem_init() , sem_open() , and so 
> on). Note in particular that even the "safe" functions may modify errno; 
> the signal-catching function, if not executing as an independent thread, 
> may want to save and restore its value. Naturally, the same principles 
> apply to the reentrancy of application routines and asynchronous data 
> access. Note thatlongjmp() and siglongjmp() are not in the list of 
> reentrant functions. This is because the code executing after longjmp() 
> and siglongjmp() can call any unsafe functions with the same danger as 
> calling those unsafe functions directly from the signal handler. 
> Applications that use longjmp() andsiglongjmp() from within signal 
> handlers require rigorous protection in order to be portable."
> 
> If this were an acceptable approach it would have been in druntime ages 
> ago :-)
> ===================
> 
> 
> 
> Andrei

Yes but the segfault signal handler is not made to design code that can 
live with these exceptions, its just a feature to allow segfaults to be 
sent to the crash handler to get a backtrace dump. Even on windows while 
you can recover from access violations, its generally a bad idea to 
allow for bugs to be turned into features.

Jeremie
September 27, 2009
Re: Null references redux
Michel Fortin wrote:
> On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu 
> <SeeWebsiteForEmail@erdani.org> said:
> 
>> Michel Fortin wrote:
>>> On 2009-09-26 23:28:30 -0400, Michel Fortin 
>>> <michel.fortin@michelf.com> said:
>>>
>>>> On 2009-09-26 22:07:00 -0400, Walter Bright 
>>>> <newshound1@digitalmars.com> said:
>>>>
>>>>> [...] The facilities in D enable one to construct a non-nullable 
>>>>> type, and they are appropriate for many designs. I just don't see 
>>>>> them as a replacement for *all* reference types.
>>>>
>>>> As far as I understand this thread, no one here is arguing that 
>>>> non-nullable references/pointers should replace *all* 
>>>> reference/pointer types. The argument made is that non-nullable 
>>>> should be the default and nullable can be specified explicitly any 
>>>> time you need it.
>>>>
>>>> So if you need a reference you use "Object" as the type, and if you 
>>>> want that reference to be nullable you write "Object?". The static 
>>>> analysis can then assert that your code properly check for null 
>>>> prior dereferencing a nullable type and issues a compilation error 
>>>> if not.
>>>
>>> I just want to add: some people here are suggesting the compiler adds 
>>> code to check for null and throw exceptions... I believe like you 
>>> that this is the wrong approach because, like you said, it makes 
>>> people add dummy try/catch statements to ignore the error. What you 
>>> want a prorammer to do is check for null and properly handle the 
>>> situation before the error occurs, and this is exactly what the 
>>> static analysis approach I suggest forces.
>>>
>>> Take this example where "a" is non-nullable and "b" is nullable:
>>>
>>> string test(Object a, Object? b)
>>> {
>>>     auto x = a.toString();
>>>     auto y = b.toString();
>>>         return x ~ y;
>>> }
>>>
>>> This should result in a compiler error on line 4 with a message 
>>> telling you that "b" needs to be checked for null prior use. The 
>>> programmer must then fix his error with an if (or some other control 
>>> structure), like this:
>>>
>>> string test(Object a, Object? b)
>>> {
>>>     audo result = a.toString();
>>>     if (b)
>>>         result ~= b.toString();
>>>
>>>     return result;
>>> }
>>>
>>> And now the compiler will let it pass. This is what I'd like to see. 
>>> What do you think?
>>>
>>> I'm not totally against throwing exceptions in some cases, but the 
>>> above approach would be much more useful. Unfortunatly, throwing 
>>> exceptions it the best you can do with a library type approach.
>>
>> I don't think this would fly.
> 
> You want me to add wings? Please explain.

I did explain. You suggest that we replace an automated, no-cost 
checking with a manual, compulsory, conservative, and costly scheme. 
That pretty much summarizes its disadvantages too :o).

Andrei
September 27, 2009
Re: Null references redux
grauzone, el 27 de septiembre a las 22:31 me escribiste:
> >>Woah, nice. I stand corrected. Is this in druntime already?
> >
> >Not yet, its part of a custom runtime I'm working on and wish to
> >release under a public domain license when I get the time. The
> >code is linked from a thread in D.announce.
> 
> Some of this functionality is also in Tango (SVN version). Signals
> are catched only to print a backtrace.

I think this is a very bad idea. When the program receive a segfault
I want my lovely core dumped. A core dump is way more useful than any
possible backtrace.

I really don't see any use for it except if an uncaught exception could
generate a core dump (just as GCC do for C++ code). But I *really*
*really* want my core dump, so I can open my debugger and inspect the dead
program exactly in the point where it failed.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
The average person laughs 13 times a day
12 13 14 15 16 17 18 19 20
Top | Discussion index | About this forum | D home