On 12/2/2020 9:52 AM, H. S. Teoh wrote:
> It seems like a minor detail, but it makes a huge difference when the
> length of the array is always kept together with the pointer to the
> array contents, and is supported by the language.  I work with C code
> daily, and I cannot tell you how many times I've seen absolutely
> terrifying code that simply passes a bare pointer around willy-nilly,
> making implicit assumptions about array size that, almost inevitably,
> some user code somewhere violates.  Or the number of times I've fixed
> bugs involving checking the wrong size against the wrong pointer,
> because when you have to manually pass them around, it's easy to make
> mistakes.

I wrote C every day for 15 years before I was able to reliably write complex code that didn't have buffer overflows and other pointer bugs. The conversion of DMD from C to D did not uncover a single pointer bug, which I'm rather proud of. But with D, there's no longer a need to train 15 years to write reliable code.


> The worst is C strings.  The number of bugs I've caught involving
> (potentially) unterminated strings is absolutely scary.

I've talked many times about whenever I review C code, I'll look at the use of string functions first and will nearly always find a bug.


> D made a bunch of seemingly-minor, but actually game-changing decisions
> that eliminate 95% of the above-mentioned problems.  The single biggest
> one is probably the D array aka fat pointer, as far as memory bugs are
> concerned.  There are a bunch of others, which others have mentioned.
> The general design in D is to make the simplest, most naïve code
> memory-safe, and you have to work at it if you want to bypass that
> safety net for systems programming reasons.  Which means you'll be
> thinking harder about your code, and hopefully more aware of potential
> issues and catch yourself before making slip-ups.  That's the way the
> incentives should be, not the other way round as it is in C.

I couldn't have said it better!

On Friday, 4 December 2020 at 07:10:51 UTC, Walter Bright wrote:
> On 12/2/2020 9:52 AM, H. S. Teoh wrote:
>> [...]
>
> I wrote C every day for 15 years before I was able to reliably write complex code that didn't have buffer overflows and other pointer bugs. The conversion of DMD from C to D did not uncover a single pointer bug,

Yes *but*. There's still strange code in DMD. For example in dmd.root.aav or dmd.doc there strange lazy initializations with double pointers ( ** ;) )to structures.

On 04.12.20 08:03, Walter Bright wrote:
> On 12/3/2020 8:13 PM, Adam D. Ruppe wrote:
>> On Friday, 4 December 2020 at 04:08:31 UTC, Walter Bright wrote:
>>> One of them, the simplest, is just execute a halt instruction.
>>
>> which wouldn't help kernel code at all fyi
> 
> Infinitely better than a buffer overflow.

In ring 0 where the kernel runs, `HLT` does not prevent the buffer overflow, it's just delayed until the next external interrupt.

Essentially, it would behave in a way similar to this:

if(i > a.length){
    Thread.sleep();
}
a.ptr[i]=x;

The only reason why `HLT` terminates execution of userspace code is that such code does not have sufficient permissions to execute the instruction; in the kernel, it would not do much.

On Friday, 4 December 2020 at 09:24:43 UTC, Timon Gehr wrote:
>
> In ring 0 where the kernel runs, `HLT` does not prevent the buffer overflow, it's just delayed until the next external interrupt.
>
> Essentially, it would behave in a way similar to this:
>
> if(i > a.length){
>     Thread.sleep();
> }
> a.ptr[i]=x;
>
> The only reason why `HLT` terminates execution of userspace code is that such code does not have sufficient permissions to execute the instruction; in the kernel, it would not do much.

Correct so if this was a kernel, then if you get a interrupt like pressing a key or if there are any pending interrupts, the HLT instruction would just continue. For kernels, the best choice would be some kind of function that is supposed to be called or a panic function that the programmer can fill in. This would be the most versatile option for those scenarios.

On 12/4/2020 2:12 AM, IGotD- wrote:
> Correct so if this was a kernel, then if you get a interrupt like pressing a key or if there are any pending interrupts, the HLT instruction would just continue. For kernels, the best choice would be some kind of function that is supposed to be called or a panic function that the programmer can fill in. This would be the most versatile option for those scenarios.

And DMD has that option as well:

    -checkaction=C

which calls the C Standard library assert fail function. Note that this function doesn't actually have to be in the C Standard library, it just has to have the same name and arguments.

On 12/4/2020 12:41 AM, user1234 wrote:
> Yes *but*. There's still strange code in DMD. For example in dmd.root.aav or dmd.doc there strange lazy initializations with double pointers ( ** ;) )to structures.

Oh, I'm not going to argue that the code is a paragon of virtue. Just that it didn't have pointer bugs in it.

(aav.d was optimized for speed, not clarity.)

On Friday, 4 December 2020 at 09:24:43 UTC, Timon Gehr wrote:
> The only reason why `HLT` terminates execution of userspace code is that such code does not have sufficient permissions to execute the instruction; in the kernel, it would not do much.

I think every time dmd uses hlt it would be better off with int 3; the debug trap instruction. It is also one byte - 0xcc - and is actually defined to do something more appropriate.

Or maybe not cuz of side effects... idk really, just the misuse of hlt has always bugged me.

On 12/4/20 4:24 AM, Timon Gehr wrote:
> On 04.12.20 08:03, Walter Bright wrote:
>> On 12/3/2020 8:13 PM, Adam D. Ruppe wrote:
>>> On Friday, 4 December 2020 at 04:08:31 UTC, Walter Bright wrote:
>>>> One of them, the simplest, is just execute a halt instruction.
>>>
>>> which wouldn't help kernel code at all fyi
>>
>> Infinitely better than a buffer overflow.
> 
> In ring 0 where the kernel runs, `HLT` does not prevent the buffer overflow, it's just delayed until the next external interrupt.
> 
> Essentially, it would behave in a way similar to this:
> 
> if(i > a.length){
>      Thread.sleep();
> }
> a.ptr[i]=x;
> 
> The only reason why `HLT` terminates execution of userspace code is that such code does not have sufficient permissions to execute the instruction; in the kernel, it would not do much.

Had no idea. Thanks!

On 2020-12-04 10:24, Timon Gehr wrote:

> In ring 0 where the kernel runs, `HLT` does not prevent the buffer overflow, it's just delayed until the next external interrupt.
> 
> Essentially, it would behave in a way similar to this:
> 
> if(i > a.length){
>      Thread.sleep();
> }
> a.ptr[i]=x;
> 
> The only reason why `HLT` terminates execution of userspace code is that such code does not have sufficient permissions to execute the instruction; in the kernel, it would not do much.

The just use another instructions that the kernel doesn't have access to. There's always a more privileged mode.

-- 
/Jacob Carlborg

On 2020-12-02 18:52, H. S. Teoh wrote:

> D made a bunch of seemingly-minor, but actually game-changing decisions
> that eliminate 95% of the above-mentioned problems.  The single biggest
> one is probably the D array aka fat pointer, as far as memory bugs are
> concerned.  There are a bunch of others, which others have mentioned.
> The general design in D is to make the simplest, most naïve code
> memory-safe, and you have to work at it if you want to bypass that
> safety net for systems programming reasons.  Which means you'll be
> thinking harder about your code, and hopefully more aware of potential
> issues and catch yourself before making slip-ups.  That's the way the
> incentives should be, not the other way round as it is in C.

Unfortunately it's still very easy to bypass most safety features in D. Especially since everything is @system by default. All the features of C are still available, one have to pick the D specific features to be safe. I've seen many many times on the forums that people are asking questions with examples containing C style code with raw pointers and calling functions in the C standard library instead of using the D equivalent.

-- 
/Jacob Carlborg