June 17, 2021
On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:
> What they all have in common is they're programming
> language crack cocaine.
>
> By that I mean that these features make programming faster
> and easier. It's like that first hit of crack enables
> a superpower where prodigious quantities of code can be
> quickly churned out. Wow! What could be wrong with that?

No, Walter, the actual crack cocaine here is function-level `@trusted`. You want to call this function from `@safe` code? Just slap `@trusted` on it! Problem solved, no thinking required.
June 17, 2021
On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:
>
>
> [...]

So let's say we keep it as it is with this @safe, @trusted, @system on function level. How is this going to work with FFI? With ImportC if you are going to be able to call C functions from safe code, then you need to designate the either @safe or @trusted. Now many C function actually want a pointer, like strings for example which might be potentially harmful (probably a cast required). Either you must create @trusted trampolines that works with D primitives or C functions can only be called from @system code.

In this particular case @system blocks could be useful if you don't want to create trampoline code for many C functions.

How @safe, @trusted, @system work with FFI is not documented here.

https://dlang.org/spec/memory-safe-d.html

and I think it needs to be defined.
June 17, 2021
On Wednesday, 16 June 2021 at 17:59:19 UTC, IGotD- wrote:
>
> I have a better idea, throw it all out. What is @safe? It's a limitation of operations you can do in D that might cause memory corruption, like pointer casts and such. Wouldn't be enough that the programmer self know about this and do not use those potentially harmful operations? That would be enough according to me but let's say that the programmer doesn't remember what is unsafe/safe. Then a compiler switch that gives a warning would be enough, at least for me.
>
> I couldn't care less about this safe/unsafe and it just gets in the way. It is also clear that despite you want to automate safe code verification, you are unable to do so and the responsibility falls to the programmer anyway. That you are unable to solve how FFI should act (remember the famous DIP 1028) is also a reminder of that.

That is a no go. Why should I leave verification of code to a human that is known to fail from times to times?

C has no verification, and what is the result of this? Lot's and lots of bugs due to human errors.

One more thing for verification to be present, is that it saves me time. I don't have to be extra careful while writing code, and certainly won't need to spend more time debugging a bug that could be prevented by automatic code verification.

June 17, 2021
On 6/17/2021 2:06 AM, Max Samukha wrote:
> No, the lambda approach will not lead to those. Just like every first D project out there abuses "static if" to hack around the limitations of "version", people will use something like "int a = () @trusted{ ... terrible hacks... }();" everywhere.

I actually agree with you. The lambdas should be replaced with a static nested functions, so the arguments come in through the front door.
June 17, 2021
On 6/17/2021 2:13 AM, Dennis wrote:
> Nice post! Just one thing I don't understand:
> 
> On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:
>> Consider the following controversial language features:
>> ...
>> 2. non-void default initialization
>> ...
>> 6. transitive const
>> ...
>> Ok, I have successfully sold (1) and (2). But
>> the others are all uphill struggles for D.
> 
> Are you trying to "sell" the idea that this list of features is bad? In that case, are you saying D's `const` and default initialization of `T x = T.init;` instead of `T x = void;` are bad? I get the other items on the list, but (2) and (6) confuse me.

Yeah, I could have phrased that better. For 2, I'm referring to the C feature of no initializer means initialize to random garbage. For 6, I meant that people want to have "logical const", and so want a way out of transitive const.
June 17, 2021
On 6/17/2021 2:56 AM, IGotD- wrote:
> How @safe, @trusted, @system work with FFI is not documented here.
> 
> https://dlang.org/spec/memory-safe-d.html
> 
> and I think it needs to be defined.

For FFI, the programmer of the import file for it gets to decide. You can see that at work in the core.stdc.* files.

For ImportC, it's going to be @system. But that may get relaxed in the future by having the compiler infer the safety by analyzing the function bodies, if they are provided.
June 17, 2021

On Thursday, 17 June 2021 at 09:56:52 UTC, Alexandru Ermicioi wrote:

>

C has no verification, and what is the result of this? Lot's and lots of bugs due to human errors.

True, although there are dialects of C that have more advanced verification than D, both research projects and industrial projects.

>

One more thing for verification to be present, is that it saves me time. I don't have to be extra careful while writing code, and certainly won't need to spend more time debugging a bug that could be prevented by automatic code verification.

Indeed. But if you think about C functions that require arrays of zero terminated strings… Ok, you can create a simple @trusted wrapper, but then that wrapper has to check that all the strings are zero terminated, which adds unacceptable overhead. So even in this trivial example the @trusted code has to assume that the provided data structure is correct, and thus it enables @safe code to make correct @trusted code unsafe.

It gets even more complicated in real system level programming where you might make a function @trusted because you know that when this function is called no other threads are running. That is an assumption about an invariant bound to time.

Proving things about timelines and concurrency is difficult/impossible. So, in practice, the correctness of @trusted is ad hoc, cannot be assumed to be local and requires audits as the code base changes.

But it could be helpful to list the invariants unsafe code depends on, e.g.:

@unsafe(assumes_singlethreaded){
   …fast update of shared datastructure…
}

@unsafe(pointer_packing, pointer_arithmetics){
 …
}

@unsafe(innocent_compiler_workaround){
 …
}

Now you have something to scan for. Like, in testing you could inject a check before the code that assumes no threads to be running. If you build with GC then you can scan all used libraries that does tricks with pointers and so on.

For true system level programming something like this (or more advanced) is needed for people to use it. Otherwise just slapping @system on all the code is the easier option. There has to be some significant benefits if you want programmers to add visual noise to their codebase.

You could also add a tag that says when the unsafe code was last audited (or at all):

@unsafe(pointer_arithmetics, 2021-06-17){
…
}
June 17, 2021
On Thursday, 17 June 2021 at 02:02:58 UTC, Walter Bright wrote:
>
> That's for context. The rest will be about the @trusted proposal.
>
> The question is: Why is @trusted at the function level rather than
> at the statement level? It certainly seems more convenient to apply
> it with statement granularity, and it will save 4 characters of typing
> over the lambda approach. What could be wrong with that?
> And indeed, that so far appears to be the general reaction.
>
> The idea of putting it at the function level is to force (I know,
> that sounds bad, but indulge me for a moment) the programmer
> to think about the decomposition of programs into safe and unsafe
> code. Ideally, untrusted code should be encapsulated and segregated
> into separate sections of code, with clean, well-defined, well-considered,
> and well thought through interfaces.
Not always possible. Sometimes you have objects, that 90% are safe, and only 10% not. Having dedicated functions or interfaces for those 10% is just plain and unneeded clutter. How would I even name those methods/interfaces?
>
> At statement level, one just schlepps @trusted in front and gives it
> no more consideration. It is thoughtlessly applied, the compiler error
> goes away, Mission Accomplished! It might as well be renamed
> the @shaddup attribute. Zero thought is given to carefully crafting
> a safe interface, because a safe interface to it is not required.
> Of course that's tempting.
Truth to be told, I gave in to this temptation, though they were one liners. But still I fear that this temptation is quite great, as not every software engineer is keen at keeping highest degree of safety and code quality. That is my concern why current use of @trusted, and trusted lambda might not be sufficient, to make it quite convenient for ordinary engineer to use them properly.

Note: There is a better proposal flying around, which making trusted code be verified as safe, but allowing system blocks insid., That or it's derived version might be the best approach here imho.



June 17, 2021
On 17.06.21 12:28, Ola Fosheim Grøstad wrote:
> Indeed. But if you think about C functions that require arrays of zero terminated strings… Ok, you can create a simple @trusted wrapper, but then that wrapper has to check that all the strings are zero terminated, which adds unacceptable overhead. So even in this trivial example the @trusted code has to assume that the provided data structure is correct, and thus it enables @safe code to make correct @trusted code unsafe.

The function you describe simply can't be @trusted. If you need to call a function with a zero-terminated string, and you cannot afford to check that the string is indeed zero-terminated, then you just can't guarantee safety. A function that is not guaranteed to be safe is @system, not @trusted.

> It gets even more complicated in real system level programming where you might make a function @trusted because you know that when this function is called no other threads are running. That is an assumption about an invariant bound to time.

That's also not a valid @trusted function. "It's safe as long as [some condition that's not guaranteed by the language]" describes an @system function, not an @trusted one.

If you want to be extra clever and exploit conditions that are not guaranteed by the language, then you either have to make sure inside the @trusted function that the conditions are actually met, or you settle for @system.

[...]
> For true system level programming something like this (or more advanced) is needed for people to use it. Otherwise just slapping @system on all the code is the easier option. There has to be some significant benefits if you want programmers to add visual noise to their codebase.

True system level programming is going to be @system in D. I don't think that's much of a surprise.
June 17, 2021
On Thursday, 17 June 2021 at 10:57:01 UTC, ag0aep6g wrote:
> The function you describe simply can't be @trusted. If you need to call a function with a zero-terminated string, and you cannot afford to check that the string is indeed zero-terminated, then you just can't guarantee safety. A function that is not guaranteed to be safe is @system, not @trusted.

That basically means that all interesting system level code is @system, including all the code that calls it. That also means that you prevent system level programmers from benefiting from language safety checks!?

Here is the problem with that viewpoint, there is no way for the function to prove that the memory it receives has not been freed. So there is in fact no way for the function to ensure that it is @trusted. That applies to @safe functions too. There has to be a contract between caller and callee, those are the invariants that the unsafe code (and safe code) depends on.

So I strongly disagree with the viewpoint that @trusted cannot assume invariants to hold for the data it receives. That is mandatory for all correct code of some complexity.

For instance, in order to make the dmd lexer @trusted you would then require the lexer to do the allocation itself. If it accepts a filebuffer allocated outside the lexer then there is no way for the lexer to ensure that the sentinels (zeros at the end) are not overwritten by other code.

That is an unreasonable restriction that makes @trusted and @safe useless. The lexer should be allowed to assume that the invariants of the filebuffer holds when it takes ownership of it. It is difficult to prove without language level unique ownership, but it is unreasonable to make the lexer and everything that calls it @system, just because it accepts a filebuffer object.

> That's also not a valid @trusted function. "It's safe as long as [some condition that's not guaranteed by the language]" describes an @system function, not an @trusted one.

What are the invariants that are guaranteed by the language in a multi-threaded program that calls C code?

What can you depend on?

Is it at all possible to write a performant 3D game that isn't @system?


> If you want to be extra clever and exploit conditions that are not guaranteed by the language, then you either have to make sure inside the @trusted function that the conditions are actually met, or you settle for @system.

But that is the signature of a very high level language, not of a system level language. In system level programming you cannot have dynamic checks all over the place, except in debug builds.

> True system level programming is going to be @system in D. I don't think that's much of a surprise.

That makes Rust a much better option for people who cares about safety. That is a problem.