November 04, 2009
Andrei Alexandrescu wrote:
> SafeD is, unfortunately, not finished at the moment. I want to leave in place a stub that won't lock our options. Here's what we currently have:
> 
> module(system) calvin;
> 
> This means calvin can do unsafe things.
> 
> module(safe) susie;
> 
> This means susie commits to extra checks and therefore only a subset of D.
>
> module hobbes;
>
> This means hobbes abides to whatever the default safety setting is.

Where did susie come from? Only module(system) has been discussed before. Why the need for THREE types of modules? Distinguishing hobbes and susie seems pointless -- either hobbes is safe, or else it will not compile with the -safe switch (and it won't compile at all, on a compiler which makes safe the default!!). It seems that module(safe) is simply a comment, "yes, I've tested it with the -safe switch, and it does compile". Doesn't add any value that I can see.

As I understood it, the primary purpose of 'SafeD' was to confine the usage of dangerous constructs to a small number of modules. IMHO, the overwhelming majority of modules should not require any marking.

> \item Cross-module function calls must only go to other @safe@ modules

> So these are my thoughts so far. There is one problem though related to the last \item - there's no way for a module to specify "trusted", meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me no problem". Many modules in std fit that mold.
> 
> How can we address that? Again, I'm looking for a simple, robust, extensible design that doesn't lock our options.

This actually seems pretty similar to public/private.
I see three types of modules:

module  : the default, should compile in -safe mode.
module(system) : Modules which need to do nasty stuff inside, but for which all the public functions are safe.
module(sysinternal/restricted/...): Modules which exist only to support system modules. This will include most APIs to C libraries.

Modules in the outer ring need to be prevented from calling ones in the inner ring.
November 04, 2009
On 2009-11-03 17:33:39 -0500, Andrei Alexandrescu <SeeWebsiteForEmail@erdani.org> said:

> So these are my thoughts so far. There is one problem though related to the last \item - there's no way for a module to specify "trusted", meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me no problem". Many modules in std fit that mold.
> 
> How can we address that? Again, I'm looking for a simple, robust, extensible design that doesn't lock our options.

What you want is to define the safety of the implementation separately from the safety of the interface. A safe module interface means that you can use the module in safe code, while a system interface forbid using the module in safe code. You could do this with two values in the parenthesis:

	module (<interface-safety>, <implementation-safety>) <name>;

	module (system, system) name; // interface: unsafe   impl.: unsafe
	module (safe, safe) name;     // interface: safe     impl.: safe
	module (safe, system) name;   // interface: safe     impl.: unsafe
	module (system, safe) name;   // interface: unsafe   impl.: safe

(The last one is silly, I know.)

Then define a shortcut so you don't have to repeat yourself when the safety of the two is the same:

	module (<interface-and-implementation-safety>) <name>;

	module (system) name;         // interface: unsafe   impl.: unsafe
	module (safe) name;           // interface: safe     impl.: safe

Of note, this also leaves the door open to a more fine-grained security policy in the future. We could add an 'extra-safe' or 'half-safe' mode if we wanted.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

November 04, 2009
Andrei Alexandrescu Wrote:

> Jason House wrote:
> > Walter Bright Wrote:
> > 
> >> Andrei Alexandrescu wrote:
> >>> Sketch of the safe rules:
> >>> 
> >>> \begin{itemize*} \item No @cast@ from a pointer type to an integral type and vice versa
> >> replace integral type with non-pointer type.
> >> 
> >>> \item No @cast@ between unrelated pointer types \item Bounds checks on all array accesses \item  No  unions  that  include  a reference  type  (array,  @class@, pointer, or @struct@ including such a type)
> >> pointers are not a reference type. Replace "reference type" with "pointers or reference types".
> >> 
> >>> \item No pointer arithmetic \item No escape of a pointer  or reference to a local variable outside its scope
> >> revise: cannot take the address of a local or a reference.
> >> 
> >>> \item Cross-module function calls must only go to other @safe@ modules \end{itemize*}
> >> add: . no inline assembler . no casting away of const, immutable, or shared
> > 
> > How does casting away const, immutable, or shared cause memory corruption?
> 
> If you have an immutable string, the compiler may cache or enregister the length and do anything (such as hoisting checks out of loops) in confidence the length will never change. If you do change it -> memory error.

These arguments are pretty reversible to show casting to XXX is as unsafe as casting away XXX. Consider code that creates thread-local mutable data, leaks it (e.g. assign to a global), and then casts it to immutable or shared and makes another call? To me, this is indistinguushable from the unsafe case.
November 04, 2009
Aelxx Wrote:

> 
> "Andrei Alexandrescu" <SeeWebsiteForEmail@erdani.org> ÓÏÏÂÝÉÌ/ÓÏÏÂÝÉÌÁ × ÎÏ×ÏÓÔÑÈ ÓÌÅÄÕÀÝÅÅ: news:hcr2hb$dvm$1@digitalmars.com...
> > Jesse Phillips wrote:
> >> On Tue, 03 Nov 2009 17:55:15 -0600, Andrei Alexandrescu wrote:
> >>
> >>> There's a lot more, but there are a few useful subspaces. One is, if an entire application only uses module(safe) that means there is no memory error in that application, ever.
> >>>
> >>> Andrei
> >>
> >> Does that mean that a module that uses a "trusted" module must also be marked as "trusted?" I would see this as pointless since system modules are likely to be used in safe code a lot.
> >
> > Same here.
> >
> >> I think the only real option is to have the importer decide if it is trusted.
> >
> > That can't work. I can't say that stdc.stdlib is trusted no matter how hard I try. I mean free is there!
> >
> >> I don't see a reasonable way to have third party certification. It is between the library writer and application developer. Since the library writer's goal should be to have a system module that is safe, he would likely want to mark it as trusted. This would leave "system" unused because everyone wants to be safe.
> >
> > Certain modules definitely can't aspire to be trusted. But for example std.stdio can claim to be trusted because, in spite of using untrusted stuff like FILE* and fclose, they are encapsulated in a way that makes it impossible for a safe client to engender memory errors.
> >
> >> In conclusion, here is a chunk of possible import options. I vote for the top two.
> >>
> >> import(system) std.stdio;
> >> system import std.stdio;
> >> trusted import std.stdio;
> >> import(trusted) std.stdio;
> >> import("This is a system module and I know that it is potentially unsafe,
> >> but I still want to use it in my safe code") std.stdio;
> >
> > Specifying a clause with import crossed my mind too, it's definitely something to keep in mind.
> >
> >
> > Andrei
> >
> How about this:
> system module foo ;
> ... (code)
> trusted module foo2 ;
> ... (code)
> safe module bar ;
> ... (code)
> 
> import foo, foo2, bar ; // status defined automatically from module
> declaration.
> //  error: used system module 'foo' in safe application.
> 
> 


What stops an irritated programmer from marking every one of his modules as trusted?

An even worse scenario would be if they create a safe facade module and importing all his pseudo-trusted code. As described so far, trust isn't transitive/viral.
November 04, 2009
Hello Michel,

> module (system) name;         // interface: unsafe   impl.: unsafe
> module (safe) name;           // interface: safe     impl.: safe

I thought that first (unsafe-unsafe) case is currently available just by:

module name; // interface: unsafe   impl.: unsafe

separating modules to unsafe-unsafe and safe-safe  has no usefulness - as those modules could not interact, specifically you need modules that are implemented by unsafe means, but provides only safe interface, so I see it as:

module name;                  // interface: unsafe   impl.: unsafe
module (system) name;         // interface: safe     impl.: unsafe
module (safe) name;           // interface: safe     impl.: safe

so you can call system modules (io, network...) from safe code.


November 04, 2009
Don wrote:
> Andrei Alexandrescu wrote:
>> SafeD is, unfortunately, not finished at the moment. I want to leave in place a stub that won't lock our options. Here's what we currently have:
>>
>> module(system) calvin;
>>
>> This means calvin can do unsafe things.
>>
>> module(safe) susie;
>>
>> This means susie commits to extra checks and therefore only a subset of D.
>  >
>  > module hobbes;
>  >
>  > This means hobbes abides to whatever the default safety setting is.
> 
> Where did susie come from?  Only module(system) has been discussed
> before.

Well actually it's always been at least in the early discussions, I was actually surprised that dmd doesn't yet accept it lexically.

> Why the need for THREE types of modules? Distinguishing hobbes and susie seems pointless -- either hobbes is safe, or else it will not compile with the -safe switch (and it won't compile at all, on a compiler which makes safe the default!!). It seems that module(safe) is simply a comment, "yes, I've tested it with the -safe switch, and it does compile". Doesn't add any value that I can see.

Agreed, module(safe) would be unnecessary if everything was safe unless overrode with "system". This would be a hard sell to Walter, however. It would be a hard sell to you, too - don't forget that safe + no bounds checking = having the cake and eating it.

module(safe) is not a comment. We need three types of modules because of the interaction between what the module declares and what the command line wants.

Let's assume the default, no-flag build allows unsafe code, like right now. Then, module(safe) means that the module volunteers itself for tighter checking, and module(system) is same as module unadorned.

But then if the user compiles with -safe, module(safe) is the same as module unadorned, and module(system) allows for unchecked operations in that particular module. I was uncomfortable with this, but Walter convinced me that D's charter is not to allow sandbox compilation and execution of malicious code. If you have the sources, you may as well take a look at their module declarations if you have some worry.

Regardless on the result of the debate regarding the default compilation mode, if the change of that default mode is allowed in the command line, then we need both module(safe) and module(system).

On a loosely-related vein, I am starting to think it would be a good idea to refine the module declaration some more. It's a great way to have fine-grained compilation options without heavy command-line options.

module(safe, contracts, debug) mymodule;

This means the module forces safety, contract checks, and debug mode within itself, regardless on the command line.

> As I understood it, the primary purpose of 'SafeD' was to confine the usage of dangerous constructs to a small number of modules. IMHO, the overwhelming majority of modules should not require any marking.

Indeed. I hope so :o).

>> \item Cross-module function calls must only go to other @safe@ modules
> 
>> So these are my thoughts so far. There is one problem though related to the last \item - there's no way for a module to specify "trusted", meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me no problem". Many modules in std fit that mold.
>>
>> How can we address that? Again, I'm looking for a simple, robust, extensible design that doesn't lock our options.
> 
> This actually seems pretty similar to public/private.
> I see three types of modules:
> 
> module  : the default, should compile in -safe mode.
> module(system) : Modules which need to do nasty stuff inside, but for which all the public functions are safe.
> module(sysinternal/restricted/...): Modules which exist only to support system modules. This will include most APIs to C libraries.
> 
> Modules in the outer ring need to be prevented from calling ones in the inner ring.

Well I wouldn't want to go any dirtier than "system", so my "system" would be your "sysinternal". I'd like to milden "system" a bit like in e.g. "trusted", which would be your "system".


Andrei
November 04, 2009
Michal Minich wrote:
> Hello Michel,
> 
>> module (system) name;         // interface: unsafe   impl.: unsafe
>> module (safe) name;           // interface: safe     impl.: safe
> 
> I thought that first (unsafe-unsafe) case is currently available just by:
> 
> module name; // interface: unsafe   impl.: unsafe
> 
> separating modules to unsafe-unsafe and safe-safe  has no usefulness - as those modules could not interact, specifically you need modules that are implemented by unsafe means, but provides only safe interface, so I see it as:
> 
> module name;                  // interface: unsafe   impl.: unsafe
> module (system) name;         // interface: safe     impl.: unsafe
> module (safe) name;           // interface: safe     impl.: safe
> 
> so you can call system modules (io, network...) from safe code.

That's a pretty clean design. How would it interact with a -safe command-line flag?

Andrei
November 04, 2009
Hello Andrei,

> Michal Minich wrote:
> 
>> Hello Michel,
>> 
>>> module (system) name;         // interface: unsafe   impl.: unsafe
>>> module (safe) name;           // interface: safe     impl.: safe
>>> 
>> I thought that first (unsafe-unsafe) case is currently available just
>> by:
>> 
>> module name; // interface: unsafe   impl.: unsafe
>> 
>> separating modules to unsafe-unsafe and safe-safe  has no usefulness
>> - as those modules could not interact, specifically you need modules
>> that are implemented by unsafe means, but provides only safe
>> interface, so I see it as:
>> 
>> module name;                  // interface: unsafe   impl.: unsafe
>> module (system) name;         // interface: safe     impl.: unsafe
>> module (safe) name;           // interface: safe     impl.: safe
>> 
>> so you can call system modules (io, network...) from safe code.
>> 
> That's a pretty clean design. How would it interact with a -safe
> command-line flag?
> 
> Andrei
> 

When compiling with -safe flag, you are doing it because you need your entire application to be safe*.

Safe flag would just affect modules with no safety flag specified - making them (safe):

module name; --> module (safe) name;

and then compile.

It would not affect system modules, because you already *belive* that the modules are *safe to use* (by using or not using -safe compiler flag).

*note: you can also partially compile only some modules/package.


November 04, 2009
On 2009-11-04 09:29:21 -0500, Michal Minich <michal@minich.sk> said:

> Hello Andrei,
> 
>> Michal Minich wrote:
>> 
>>> Hello Michel,
>>> 
>>>> module (system) name;         // interface: unsafe   impl.: unsafe
>>>> module (safe) name;           // interface: safe     impl.: safe
>>>> 
>>> I thought that first (unsafe-unsafe) case is currently available just
>>> by:
>>> 
>>> module name; // interface: unsafe   impl.: unsafe
>>> 
>>> separating modules to unsafe-unsafe and safe-safe  has no usefulness
>>> - as those modules could not interact, specifically you need modules
>>> that are implemented by unsafe means, but provides only safe
>>> interface, so I see it as:
>>> 
>>> module name;                  // interface: unsafe   impl.: unsafe
>>> module (system) name;         // interface: safe     impl.: unsafe
>>> module (safe) name;           // interface: safe     impl.: safe
>>> 
>>> so you can call system modules (io, network...) from safe code.
>>> 
>> That's a pretty clean design. How would it interact with a -safe
>> command-line flag?
>> 
>> Andrei
>> 
> 
> When compiling with -safe flag, you are doing it because you need your entire application to be safe*.
> 
> Safe flag would just affect modules with no safety flag specified - making them (safe):
> 
> module name; --> module (safe) name;
> 
> and then compile.

I'm not sure this works so well. Look at this:

	module memory;   // unsafe interface - unsafe impl.
	extern (C) void* malloc(int);
	extern (C) void free(void*);

	module (system) my.system;   // safe interface - unsafe impl.
	import memory;
	void test() { auto i = malloc(10); free(i); }   // ok: unsafe impl. allowed

	module (safe) my.safe;   // safe interface - safe impl.
	import memory;
	void test() { auto i = malloc(10); free(i); }   // error: malloc, free are unsafe

How is this supposed to work correctly with and without the "-safe" compiler flag? The way you define things "-safe" would make module memory safe for use while it is not.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/

November 04, 2009
Andrei Alexandrescu wrote:
> Don wrote:
>> Andrei Alexandrescu wrote:

> module(safe) is not a comment. We need three types of modules because of the interaction between what the module declares and what the command line wants.
> 
> Let's assume the default, no-flag build allows unsafe code, like right now. Then, module(safe) means that the module volunteers itself for tighter checking, and module(system) is same as module unadorned.
> 
> But then if the user compiles with -safe, module(safe) is the same as module unadorned, and module(system) allows for unchecked operations in that particular module. I was uncomfortable with this, but Walter convinced me that D's charter is not to allow sandbox compilation and execution of malicious code. If you have the sources, you may as well take a look at their module declarations if you have some worry.
> 
> Regardless on the result of the debate regarding the default compilation mode, if the change of that default mode is allowed in the command line, then we need both module(safe) and module(system).

When would it be MANDATORY for a module to be compiled in safe mode?
If module(safe) implies bound-checking *cannot* be turned off for that module, would any standard library modules be module(safe)?

>> This actually seems pretty similar to public/private.
>> I see three types of modules:
>>
>> module  : the default, should compile in -safe mode.
>> module(system) : Modules which need to do nasty stuff inside, but for which all the public functions are safe.
>> module(sysinternal/restricted/...): Modules which exist only to support system modules. This will include most APIs to C libraries.
>>
>> Modules in the outer ring need to be prevented from calling ones in the inner ring.
> 
> Well I wouldn't want to go any dirtier than "system", so my "system" would be your "sysinternal". I'd like to milden "system" a bit like in e.g. "trusted", which would be your "system".

Yeah, the names don't matter. The thing is, modules in the inner ring are extremely rare. I'd hope there'd be just a few in druntime, and no public ones at all in Phobos.