September 27, 2009
downs wrote:
> Jeremie Pelletier wrote:
>> Andrei Alexandrescu wrote:
>>> downs wrote:
>>>> Walter Bright wrote:
>>>>> Nick Sabalausky wrote:
>>>>>
>>>>> I agree with you that if the compiler can detect null dereferences at
>>>>> compile time, it should.
>>>>>
>>>>>
>>>>>>> Also, by "safe" I presume you mean "memory safe" which means free of
>>>>>>> memory corruption. Null pointer exceptions are memory safe. A null
>>>>>>> pointer could be caused by memory corruption, but it cannot *cause*
>>>>>>> memory corruption.
>>>>>> No, he's using the real meaning of "safe", not the
>>>>>> misleadingly-limited "SafeD" version of "safe" (which I'm still
>>>>>> convinced is going to get some poor soul into serious trouble from
>>>>>> mistakingly thinking their SafeD program is much safer than it really
>>>>>> is). Out here in reality, "safe" also means a lack of ability to
>>>>>> crash, or at least some level of protection against it. 
>>>>> Memory safety is something that can be guaranteed (presuming the
>>>>> compiler is correctly implemented). There is no way to guarantee that a
>>>>> non-trivial program cannot crash. It's the old halting problem.
>>>>>
>>>> Okay, I'm gonna have to call you out on this one because it's simply
>>>> incorrect.
>>>>
>>>> The halting problem deals with a valid program state - halting.
>>>>
>>>> We cannot check if every program halts because halting is an
>>>> instruction that must be allowed at almost any point in the program.
>>>>
>>>> Why do crashes have to be allowed? They're not an allowed instruction!
>>>>
>>>> A compiler can be turing complete and still not allow crashes. There
>>>> is nothing wrong with this, and it has *nothing* to do with the
>>>> halting problem.
>>>>
>>>>>> You seem to be under the impression that nothing can be made
>>>>>> uncrashable without introducing the possibility of corrupted state.
>>>>>> That's hogwash.
>>>>> I read that statement several times and I still don't understand
>>>>> what it
>>>>> means.
>>>>>
>>>>> BTW, hardware null pointer checking is a safety feature, just like
>>>>> array
>>>>> bounds checking is.
>>>> PS: You can't convert segfaults into exceptions under Linux, as far
>>>> as I know.
>>> How did Jeremie do that?
>>>
>>> Andrei
>> A signal handler with the undocumented kernel parameters attaches the
>> signal context to the exception object, repairs the stack frame forged
>> by the kernel to make us believe we called the handler ourselves, does a
>> backtrace right away and attaches it to the exception object, and then
>> throw it.
>>
>> The error handling code will unwind down to the runtime's main() where a
>> catch clause is waiting for any Throwables, sending them back into the
>> unhandled exception handler, and a crash window appears with the
>> backtrace, all finally blocks executed, and gracefully shutting down.
>>
>> All I need to do is an ELF/DWARF reader to extract symbolic debug info
>> under linux, its already working for PE/CodeView on windows.
>>
>> Jeremie
> 
> 
> Woah, nice. I stand corrected. Is this in druntime already?

Not yet, its part of a custom runtime I'm working on and wish to release under a public domain license when I get the time. The code is linked from a thread in D.announce.
September 27, 2009
Jeremie Pelletier wrote:
>> Is this Linux specific? what about other *nix systems, like BSD and solaris?
> 
> Signal handler are standard to most *nix platforms since they're part of the posix C standard libraries, maybe some platforms will require a special handling but nothing impossible to do.

Let me write a message on behalf of Sean Kelly. He wrote that to Walter and myself this morning, then I suggested him to post it but probably he is off email for a short while. Hopefully the community will find a solution to the issue he's raising. Let me post this:

===================
Sean Kelly wrote:

There's one minor problem with his code.  It's not safe to throw an exception from a signal handler.  Here's a quote from the POSIX spec at opengroup.org:

"In order to prevent errors arising from interrupting non-reentrant function calls, applications should protect calls to these functions either by blocking the appropriate signals or through the use of some programmatic semaphore (see semget() , sem_init() , sem_open() , and so on). Note in particular that even the "safe" functions may modify errno; the signal-catching function, if not executing as an independent thread, may want to save and restore its value. Naturally, the same principles apply to the reentrancy of application routines and asynchronous data access. Note thatlongjmp() and siglongjmp() are not in the list of reentrant functions. This is because the code executing after longjmp() and siglongjmp() can call any unsafe functions with the same danger as calling those unsafe functions directly from the signal handler. Applications that use longjmp() andsiglongjmp() from within signal handlers require rigorous protection in order to be portable."

If this were an acceptable approach it would have been in druntime ages ago :-)
===================



Andrei
September 27, 2009
Jeremie Pelletier wrote:
> downs wrote:
>> Jeremie Pelletier wrote:
>>> Andrei Alexandrescu wrote:
>>>> downs wrote:
>>>>> Walter Bright wrote:
>>>>>> Nick Sabalausky wrote:
>>>>>>
>>>>>> I agree with you that if the compiler can detect null dereferences at
>>>>>> compile time, it should.
>>>>>>
>>>>>>
>>>>>>>> Also, by "safe" I presume you mean "memory safe" which means free of
>>>>>>>> memory corruption. Null pointer exceptions are memory safe. A null
>>>>>>>> pointer could be caused by memory corruption, but it cannot *cause*
>>>>>>>> memory corruption.
>>>>>>> No, he's using the real meaning of "safe", not the
>>>>>>> misleadingly-limited "SafeD" version of "safe" (which I'm still
>>>>>>> convinced is going to get some poor soul into serious trouble from
>>>>>>> mistakingly thinking their SafeD program is much safer than it really
>>>>>>> is). Out here in reality, "safe" also means a lack of ability to
>>>>>>> crash, or at least some level of protection against it. 
>>>>>> Memory safety is something that can be guaranteed (presuming the
>>>>>> compiler is correctly implemented). There is no way to guarantee that a
>>>>>> non-trivial program cannot crash. It's the old halting problem.
>>>>>>
>>>>> Okay, I'm gonna have to call you out on this one because it's simply
>>>>> incorrect.
>>>>>
>>>>> The halting problem deals with a valid program state - halting.
>>>>>
>>>>> We cannot check if every program halts because halting is an
>>>>> instruction that must be allowed at almost any point in the program.
>>>>>
>>>>> Why do crashes have to be allowed? They're not an allowed instruction!
>>>>>
>>>>> A compiler can be turing complete and still not allow crashes. There
>>>>> is nothing wrong with this, and it has *nothing* to do with the
>>>>> halting problem.
>>>>>
>>>>>>> You seem to be under the impression that nothing can be made
>>>>>>> uncrashable without introducing the possibility of corrupted state.
>>>>>>> That's hogwash.
>>>>>> I read that statement several times and I still don't understand
>>>>>> what it
>>>>>> means.
>>>>>>
>>>>>> BTW, hardware null pointer checking is a safety feature, just like
>>>>>> array
>>>>>> bounds checking is.
>>>>> PS: You can't convert segfaults into exceptions under Linux, as far
>>>>> as I know.
>>>> How did Jeremie do that?
>>>>
>>>> Andrei
>>> A signal handler with the undocumented kernel parameters attaches the
>>> signal context to the exception object, repairs the stack frame forged
>>> by the kernel to make us believe we called the handler ourselves, does a
>>> backtrace right away and attaches it to the exception object, and then
>>> throw it.
>>>
>>> The error handling code will unwind down to the runtime's main() where a
>>> catch clause is waiting for any Throwables, sending them back into the
>>> unhandled exception handler, and a crash window appears with the
>>> backtrace, all finally blocks executed, and gracefully shutting down.
>>>
>>> All I need to do is an ELF/DWARF reader to extract symbolic debug info
>>> under linux, its already working for PE/CodeView on windows.
>>>
>>> Jeremie
>>
>>
>> Woah, nice. I stand corrected. Is this in druntime already?
> 
> Not yet, its part of a custom runtime I'm working on and wish to release under a public domain license when I get the time. The code is linked from a thread in D.announce.

Some of this functionality is also in Tango (SVN version). Signals are catched only to print a backtrace.
September 27, 2009
BCS wrote:

> Hello Lutger,
> 
>> The answer may
>> depend on [...]
>> the habits of the 'programmers' in question, I don't know.
>> 
> 
> If you can't trust the programmer to write good code, replace them with someone you can trust. There will never be a usable language that can take in garbage and spit out correct programs.

Hi. I don't think this argument will work, for several reasons:

First, there is a huge demand for programmers, so much that even I got hired in this time of crisis ;) Good programmers don't suddenly fall from the skies apparently. Second, there are lot's of tasks doable by programmers with less skill than others using tools that trade safety for performance / expressiveness / whatever. Finally, programmers are humans, humans make mistakes, have quirks and bad days. All of them. What it comes down to is that languages are made in order to service and adapt to the programmers, not the other way around.

Do you maintain that a programmer who can't deal with non-nullable references without hacking them away is unusually incompetent? I don't know about this. Actually I suspect non-nullable references by default are in the end safer (whatever that means), but only if they don't complicate the use of nullable references.








September 27, 2009
Hello Lutger,

> BCS wrote:
> 
>> Hello Lutger,
>> 
>>> The answer may
>>> depend on [...]
>>> the habits of the 'programmers' in question, I don't know.
>> If you can't trust the programmer to write good code, replace them
>> with someone you can trust. There will never be a usable language
>> that can take in garbage and spit out correct programs.
>> 
> Hi. I don't think this argument will work, for several reasons:
> 
[...]
> 
> Do you maintain that a programmer who can't deal with non-nullable
> references without hacking them away is unusually incompetent?

Incompetent? No. But I wouldn't want to hire a programer that *habitually* (and unnecessarily) hacks past a feature designed to prevent bugs. The best race car driver in the world is clearly not incompetent but would still get a ticket on public roads for speeding or following to close.

> I don't
> know about this. Actually I suspect non-nullable references by default
> are in the end safer (whatever that means), but only if they don't
> complicate the use of nullable references.

I'll second that.


September 27, 2009
On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> Michel Fortin wrote:
>> On 2009-09-26 23:28:30 -0400, Michel Fortin <michel.fortin@michelf.com> said:
>> 
>>> On 2009-09-26 22:07:00 -0400, Walter Bright <newshound1@digitalmars.com> said:
>>> 
>>>> [...] The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. I just don't see them as a replacement for *all* reference types.
>>> 
>>> As far as I understand this thread, no one here is arguing that non-nullable references/pointers should replace *all* reference/pointer types. The argument made is that non-nullable should be the default and nullable can be specified explicitly any time you need it.
>>> 
>>> So if you need a reference you use "Object" as the type, and if you want that reference to be nullable you write "Object?". The static analysis can then assert that your code properly check for null prior dereferencing a nullable type and issues a compilation error if not.
>> 
>> I just want to add: some people here are suggesting the compiler adds code to check for null and throw exceptions... I believe like you that this is the wrong approach because, like you said, it makes people add dummy try/catch statements to ignore the error. What you want a prorammer to do is check for null and properly handle the situation before the error occurs, and this is exactly what the static analysis approach I suggest forces.
>> 
>> Take this example where "a" is non-nullable and "b" is nullable:
>> 
>> string test(Object a, Object? b)
>> {
>>     auto x = a.toString();
>>     auto y = b.toString();
>>         return x ~ y;
>> }
>> 
>> This should result in a compiler error on line 4 with a message telling you that "b" needs to be checked for null prior use. The programmer must then fix his error with an if (or some other control structure), like this:
>> 
>> string test(Object a, Object? b)
>> {
>>     audo result = a.toString();
>>     if (b)
>>         result ~= b.toString();
>> 
>>     return result;
>> }
>> 
>> And now the compiler will let it pass. This is what I'd like to see. What do you think?
>> 
>> I'm not totally against throwing exceptions in some cases, but the above approach would be much more useful. Unfortunatly, throwing exceptions it the best you can do with a library type approach.
> 
> I don't think this would fly.

You want me to add wings? Please explain.

> One good thing about nullable references is that they are dynamically checked for validity at virtually zero cost.

When you say they are dynamically checked, do you mean it throws an exception when you assign null? I'm not totally against this idea, but I find the above a supperior solution because it forces the programmer to handle the problem where it occurs and it doesn't require any runtime check.

> Non-nullable references, therefore, would not add value in that respect, but would add value by reducing the cases when programmers forgot to initialize references properly.

To me it looks like you're supporting an inferior concept for non-nullable references because the better one will not "fly" (whatever that means). Well, I support both concepts of non-nullable because they both can be very useful, but I believe static checking is a better way than throwing exceptions.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

September 27, 2009
Jarrett Billingsley:

> And if you have a nullable reference that you know is not null for the rest of the function? Just put "assert(x !is null)" and everything that follows will assume it's not null.

Asserts tend to vanish in release mode, so it may be better to use something different. A possibility is to use the enforce() some people have shown here.

Another possibility is the very strange assume() of Visual C++, that I may appreciate for other purposes too:
http://msdn.microsoft.com/en-us/library/1b3fsfxw(loband).aspx

Bye,
bearophile
September 27, 2009
Andrei Alexandrescu wrote:
> Jeremie Pelletier wrote:
>>> Is this Linux specific? what about other *nix systems, like BSD and solaris?
>>
>> Signal handler are standard to most *nix platforms since they're part of the posix C standard libraries, maybe some platforms will require a special handling but nothing impossible to do.
> 
> Let me write a message on behalf of Sean Kelly. He wrote that to Walter and myself this morning, then I suggested him to post it but probably he is off email for a short while. Hopefully the community will find a solution to the issue he's raising. Let me post this:
> 
> ===================
> Sean Kelly wrote:
> 
> There's one minor problem with his code.  It's not safe to throw an exception from a signal handler.  Here's a quote from the POSIX spec at opengroup.org:
> 
> "In order to prevent errors arising from interrupting non-reentrant function calls, applications should protect calls to these functions either by blocking the appropriate signals or through the use of some programmatic semaphore (see semget() , sem_init() , sem_open() , and so on). Note in particular that even the "safe" functions may modify errno; the signal-catching function, if not executing as an independent thread, may want to save and restore its value. Naturally, the same principles apply to the reentrancy of application routines and asynchronous data access. Note thatlongjmp() and siglongjmp() are not in the list of reentrant functions. This is because the code executing after longjmp() and siglongjmp() can call any unsafe functions with the same danger as calling those unsafe functions directly from the signal handler. Applications that use longjmp() andsiglongjmp() from within signal handlers require rigorous protection in order to be portable."
> 
> If this were an acceptable approach it would have been in druntime ages ago :-)
> ===================
> 
> 
> 
> Andrei

Yes but the segfault signal handler is not made to design code that can live with these exceptions, its just a feature to allow segfaults to be sent to the crash handler to get a backtrace dump. Even on windows while you can recover from access violations, its generally a bad idea to allow for bugs to be turned into features.

Jeremie
September 27, 2009
Michel Fortin wrote:
> On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:
> 
>> Michel Fortin wrote:
>>> On 2009-09-26 23:28:30 -0400, Michel Fortin <michel.fortin@michelf.com> said:
>>>
>>>> On 2009-09-26 22:07:00 -0400, Walter Bright <newshound1@digitalmars.com> said:
>>>>
>>>>> [...] The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. I just don't see them as a replacement for *all* reference types.
>>>>
>>>> As far as I understand this thread, no one here is arguing that non-nullable references/pointers should replace *all* reference/pointer types. The argument made is that non-nullable should be the default and nullable can be specified explicitly any time you need it.
>>>>
>>>> So if you need a reference you use "Object" as the type, and if you want that reference to be nullable you write "Object?". The static analysis can then assert that your code properly check for null prior dereferencing a nullable type and issues a compilation error if not.
>>>
>>> I just want to add: some people here are suggesting the compiler adds code to check for null and throw exceptions... I believe like you that this is the wrong approach because, like you said, it makes people add dummy try/catch statements to ignore the error. What you want a prorammer to do is check for null and properly handle the situation before the error occurs, and this is exactly what the static analysis approach I suggest forces.
>>>
>>> Take this example where "a" is non-nullable and "b" is nullable:
>>>
>>> string test(Object a, Object? b)
>>> {
>>>     auto x = a.toString();
>>>     auto y = b.toString();
>>>         return x ~ y;
>>> }
>>>
>>> This should result in a compiler error on line 4 with a message telling you that "b" needs to be checked for null prior use. The programmer must then fix his error with an if (or some other control structure), like this:
>>>
>>> string test(Object a, Object? b)
>>> {
>>>     audo result = a.toString();
>>>     if (b)
>>>         result ~= b.toString();
>>>
>>>     return result;
>>> }
>>>
>>> And now the compiler will let it pass. This is what I'd like to see. What do you think?
>>>
>>> I'm not totally against throwing exceptions in some cases, but the above approach would be much more useful. Unfortunatly, throwing exceptions it the best you can do with a library type approach.
>>
>> I don't think this would fly.
> 
> You want me to add wings? Please explain.

I did explain. You suggest that we replace an automated, no-cost checking with a manual, compulsory, conservative, and costly scheme. That pretty much summarizes its disadvantages too :o).

Andrei
September 27, 2009
grauzone, el 27 de septiembre a las 22:31 me escribiste:
> >>Woah, nice. I stand corrected. Is this in druntime already?
> >
> >Not yet, its part of a custom runtime I'm working on and wish to release under a public domain license when I get the time. The code is linked from a thread in D.announce.
> 
> Some of this functionality is also in Tango (SVN version). Signals are catched only to print a backtrace.

I think this is a very bad idea. When the program receive a segfault I want my lovely core dumped. A core dump is way more useful than any possible backtrace.

I really don't see any use for it except if an uncaught exception could generate a core dump (just as GCC do for C++ code). But I *really* *really* want my core dump, so I can open my debugger and inspect the dead program exactly in the point where it failed.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
The average person laughs 13 times a day