View mode: basic / threaded / horizontal-split · Log in · Help
November 04, 2009
Re: safety model in D
Andrei Alexandrescu wrote:
> SafeD is, unfortunately, not finished at the moment. I want to leave in 
> place a stub that won't lock our options. Here's what we currently have:
> 
> module(system) calvin;
> 
> This means calvin can do unsafe things.
> 
> module(safe) susie;
> 
> This means susie commits to extra checks and therefore only a subset of D.
>
> module hobbes;
>
> This means hobbes abides to whatever the default safety setting is.

Where did susie come from? Only module(system) has been discussed 
before. Why the need for THREE types of modules? Distinguishing hobbes 
and susie seems pointless -- either hobbes is safe, or else it will not 
compile with the -safe switch (and it won't compile at all, on a 
compiler which makes safe the default!!). It seems that module(safe) is 
simply a comment, "yes, I've tested it with the -safe switch, and it 
does compile". Doesn't add any value that I can see.

As I understood it, the primary purpose of 'SafeD' was to confine the 
usage of dangerous constructs to a small number of modules. IMHO, the 
overwhelming majority of modules should not require any marking.

> \item Cross-module function calls must only go to other @safe@ modules

> So these are my thoughts so far. There is one problem though related to 
> the last \item - there's no way for a module to specify "trusted", 
> meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me 
> no problem". Many modules in std fit that mold.
> 
> How can we address that? Again, I'm looking for a simple, robust, 
> extensible design that doesn't lock our options.

This actually seems pretty similar to public/private.
I see three types of modules:

module  : the default, should compile in -safe mode.
module(system) : Modules which need to do nasty stuff inside, but for 
which all the public functions are safe.
module(sysinternal/restricted/...): Modules which exist only to support 
system modules. This will include most APIs to C libraries.

Modules in the outer ring need to be prevented from calling ones in the 
inner ring.
November 04, 2009
Re: safety model in D
On 2009-11-03 17:33:39 -0500, Andrei Alexandrescu 
<SeeWebsiteForEmail@erdani.org> said:

> So these are my thoughts so far. There is one problem though related to 
> the last \item - there's no way for a module to specify "trusted", 
> meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me 
> no problem". Many modules in std fit that mold.
> 
> How can we address that? Again, I'm looking for a simple, robust, 
> extensible design that doesn't lock our options.

What you want is to define the safety of the implementation separately 
from the safety of the interface. A safe module interface means that 
you can use the module in safe code, while a system interface forbid 
using the module in safe code. You could do this with two values in the 
parenthesis:

	module (<interface-safety>, <implementation-safety>) <name>;

	module (system, system) name; // interface: unsafe   impl.: unsafe
	module (safe, safe) name;     // interface: safe     impl.: safe
	module (safe, system) name;   // interface: safe     impl.: unsafe
	module (system, safe) name;   // interface: unsafe   impl.: safe

(The last one is silly, I know.)

Then define a shortcut so you don't have to repeat yourself when the 
safety of the two is the same:

	module (<interface-and-implementation-safety>) <name>;

	module (system) name;         // interface: unsafe   impl.: unsafe
	module (safe) name;           // interface: safe     impl.: safe

Of note, this also leaves the door open to a more fine-grained security 
policy in the future. We could add an 'extra-safe' or 'half-safe' mode 
if we wanted.

-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
November 04, 2009
Re: safety model in D
Andrei Alexandrescu Wrote:

> Jason House wrote:
> > Walter Bright Wrote:
> > 
> >> Andrei Alexandrescu wrote:
> >>> Sketch of the safe rules:
> >>> 
> >>> \begin{itemize*} \item No @cast@ from a pointer type to an
> >>> integral type and vice versa
> >> replace integral type with non-pointer type.
> >> 
> >>> \item No @cast@ between unrelated pointer types \item Bounds
> >>> checks on all array accesses \item  No  unions  that  include  a
> >>> reference  type  (array,  @class@, pointer, or @struct@ including
> >>> such a type)
> >> pointers are not a reference type. Replace "reference type" with 
> >> "pointers or reference types".
> >> 
> >>> \item No pointer arithmetic \item No escape of a pointer  or
> >>> reference to a local variable outside its scope
> >> revise: cannot take the address of a local or a reference.
> >> 
> >>> \item Cross-module function calls must only go to other @safe@
> >>> modules \end{itemize*}
> >> add: . no inline assembler . no casting away of const, immutable,
> >> or shared
> > 
> > How does casting away const, immutable, or shared cause memory
> > corruption?
> 
> If you have an immutable string, the compiler may cache or enregister
> the length and do anything (such as hoisting checks out of loops) in 
> confidence the length will never change. If you do change it -> memory 
> error.

These arguments are pretty reversible to show casting to XXX is as unsafe as casting away XXX. Consider code that creates thread-local mutable data, leaks it (e.g. assign to a global), and then casts it to immutable or shared and makes another call? To me, this is indistinguushable from the unsafe case.
November 04, 2009
Re: safety model in D
Aelxx Wrote:

> 
> "Andrei Alexandrescu" <SeeWebsiteForEmail@erdani.org> ÓÏÏÂÝÉÌ/ÓÏÏÂÝÉÌÁ × 
> ÎÏ×ÏÓÔÑÈ ÓÌÅÄÕÀÝÅÅ: news:hcr2hb$dvm$1@digitalmars.com...
> > Jesse Phillips wrote:
> >> On Tue, 03 Nov 2009 17:55:15 -0600, Andrei Alexandrescu wrote:
> >>
> >>> There's a lot more, but there are a few useful subspaces. One is, if an
> >>> entire application only uses module(safe) that means there is no memory
> >>> error in that application, ever.
> >>>
> >>> Andrei
> >>
> >> Does that mean that a module that uses a "trusted" module must also be 
> >> marked as "trusted?" I would see this as pointless since system modules 
> >> are likely to be used in safe code a lot.
> >
> > Same here.
> >
> >> I think the only real option is to have the importer decide if it is 
> >> trusted.
> >
> > That can't work. I can't say that stdc.stdlib is trusted no matter how 
> > hard I try. I mean free is there!
> >
> >> I don't see a reasonable way to have third party certification. It is 
> >> between the library writer and application developer. Since the library 
> >> writer's goal should be to have a system module that is safe, he would 
> >> likely want to mark it as trusted. This would leave "system" unused 
> >> because everyone wants to be safe.
> >
> > Certain modules definitely can't aspire to be trusted. But for example 
> > std.stdio can claim to be trusted because, in spite of using untrusted 
> > stuff like FILE* and fclose, they are encapsulated in a way that makes it 
> > impossible for a safe client to engender memory errors.
> >
> >> In conclusion, here is a chunk of possible import options. I vote for the 
> >> top two.
> >>
> >> import(system) std.stdio;
> >> system import std.stdio;
> >> trusted import std.stdio;
> >> import(trusted) std.stdio;
> >> import("This is a system module and I know that it is potentially unsafe, 
> >> but I still want to use it in my safe code") std.stdio;
> >
> > Specifying a clause with import crossed my mind too, it's definitely 
> > something to keep in mind.
> >
> >
> > Andrei
> >
> How about this:
> system module foo ;
> ... (code)
> trusted module foo2 ;
> ... (code)
> safe module bar ;
> ... (code)
> 
> import foo, foo2, bar ; // status defined automatically from module 
> declaration.
> //  error: used system module 'foo' in safe application.
> 
> 


What stops an irritated programmer from marking every one of his modules as trusted? 

An even worse scenario would be if they create a safe facade module and importing all his pseudo-trusted code. As described so far, trust isn't transitive/viral.
November 04, 2009
Re: safety model in D
Hello Michel,

> module (system) name;         // interface: unsafe   impl.: unsafe
> module (safe) name;           // interface: safe     impl.: safe

I thought that first (unsafe-unsafe) case is currently available just by:

module name; // interface: unsafe   impl.: unsafe

separating modules to unsafe-unsafe and safe-safe  has no usefulness - as 
those modules could not interact, specifically you need modules that are 
implemented by unsafe means, but provides only safe interface, so I see it 
as:

module name;                  // interface: unsafe   impl.: unsafe
module (system) name;         // interface: safe     impl.: unsafe
module (safe) name;           // interface: safe     impl.: safe

so you can call system modules (io, network...) from safe code.
November 04, 2009
Re: safety model in D
Don wrote:
> Andrei Alexandrescu wrote:
>> SafeD is, unfortunately, not finished at the moment. I want to leave 
>> in place a stub that won't lock our options. Here's what we currently 
>> have:
>>
>> module(system) calvin;
>>
>> This means calvin can do unsafe things.
>>
>> module(safe) susie;
>>
>> This means susie commits to extra checks and therefore only a subset 
>> of D.
>  >
>  > module hobbes;
>  >
>  > This means hobbes abides to whatever the default safety setting is.
> 
> Where did susie come from?  Only module(system) has been discussed
> before.

Well actually it's always been at least in the early discussions, I was 
actually surprised that dmd doesn't yet accept it lexically.

> Why the need for THREE types of modules? Distinguishing hobbes 
> and susie seems pointless -- either hobbes is safe, or else it will not 
> compile with the -safe switch (and it won't compile at all, on a 
> compiler which makes safe the default!!). It seems that module(safe) is 
> simply a comment, "yes, I've tested it with the -safe switch, and it 
> does compile". Doesn't add any value that I can see.

Agreed, module(safe) would be unnecessary if everything was safe unless 
overrode with "system". This would be a hard sell to Walter, however. It 
would be a hard sell to you, too - don't forget that safe + no bounds 
checking = having the cake and eating it.

module(safe) is not a comment. We need three types of modules because of 
the interaction between what the module declares and what the command 
line wants.

Let's assume the default, no-flag build allows unsafe code, like right 
now. Then, module(safe) means that the module volunteers itself for 
tighter checking, and module(system) is same as module unadorned.

But then if the user compiles with -safe, module(safe) is the same as 
module unadorned, and module(system) allows for unchecked operations in 
that particular module. I was uncomfortable with this, but Walter 
convinced me that D's charter is not to allow sandbox compilation and 
execution of malicious code. If you have the sources, you may as well 
take a look at their module declarations if you have some worry.

Regardless on the result of the debate regarding the default compilation 
mode, if the change of that default mode is allowed in the command line, 
then we need both module(safe) and module(system).

On a loosely-related vein, I am starting to think it would be a good 
idea to refine the module declaration some more. It's a great way to 
have fine-grained compilation options without heavy command-line options.

module(safe, contracts, debug) mymodule;

This means the module forces safety, contract checks, and debug mode 
within itself, regardless on the command line.

> As I understood it, the primary purpose of 'SafeD' was to confine the 
> usage of dangerous constructs to a small number of modules. IMHO, the 
> overwhelming majority of modules should not require any marking.

Indeed. I hope so :o).

>> \item Cross-module function calls must only go to other @safe@ modules
> 
>> So these are my thoughts so far. There is one problem though related 
>> to the last \item - there's no way for a module to specify "trusted", 
>> meaning: "Yeah, I do unsafe stuff inside, but safe modules can call me 
>> no problem". Many modules in std fit that mold.
>>
>> How can we address that? Again, I'm looking for a simple, robust, 
>> extensible design that doesn't lock our options.
> 
> This actually seems pretty similar to public/private.
> I see three types of modules:
> 
> module  : the default, should compile in -safe mode.
> module(system) : Modules which need to do nasty stuff inside, but for 
> which all the public functions are safe.
> module(sysinternal/restricted/...): Modules which exist only to support 
> system modules. This will include most APIs to C libraries.
> 
> Modules in the outer ring need to be prevented from calling ones in the 
> inner ring.

Well I wouldn't want to go any dirtier than "system", so my "system" 
would be your "sysinternal". I'd like to milden "system" a bit like in 
e.g. "trusted", which would be your "system".


Andrei
November 04, 2009
Re: safety model in D
Michal Minich wrote:
> Hello Michel,
> 
>> module (system) name;         // interface: unsafe   impl.: unsafe
>> module (safe) name;           // interface: safe     impl.: safe
> 
> I thought that first (unsafe-unsafe) case is currently available just by:
> 
> module name; // interface: unsafe   impl.: unsafe
> 
> separating modules to unsafe-unsafe and safe-safe  has no usefulness - 
> as those modules could not interact, specifically you need modules that 
> are implemented by unsafe means, but provides only safe interface, so I 
> see it as:
> 
> module name;                  // interface: unsafe   impl.: unsafe
> module (system) name;         // interface: safe     impl.: unsafe
> module (safe) name;           // interface: safe     impl.: safe
> 
> so you can call system modules (io, network...) from safe code.

That's a pretty clean design. How would it interact with a -safe 
command-line flag?

Andrei
November 04, 2009
Re: safety model in D
Hello Andrei,

> Michal Minich wrote:
> 
>> Hello Michel,
>> 
>>> module (system) name;         // interface: unsafe   impl.: unsafe
>>> module (safe) name;           // interface: safe     impl.: safe
>>> 
>> I thought that first (unsafe-unsafe) case is currently available just
>> by:
>> 
>> module name; // interface: unsafe   impl.: unsafe
>> 
>> separating modules to unsafe-unsafe and safe-safe  has no usefulness
>> - as those modules could not interact, specifically you need modules
>> that are implemented by unsafe means, but provides only safe
>> interface, so I see it as:
>> 
>> module name;                  // interface: unsafe   impl.: unsafe
>> module (system) name;         // interface: safe     impl.: unsafe
>> module (safe) name;           // interface: safe     impl.: safe
>> 
>> so you can call system modules (io, network...) from safe code.
>> 
> That's a pretty clean design. How would it interact with a -safe
> command-line flag?
> 
> Andrei
> 

When compiling with -safe flag, you are doing it because you need your entire 
application to be safe*.

Safe flag would just affect modules with no safety flag specified - making 
them (safe):

module name; --> module (safe) name;

and then compile.

It would not affect system modules, because you already *belive* that the 
modules are *safe to use* (by using or not using -safe compiler flag).

*note: you can also partially compile only some modules/package.
November 04, 2009
Re: safety model in D
On 2009-11-04 09:29:21 -0500, Michal Minich <michal@minich.sk> said:

> Hello Andrei,
> 
>> Michal Minich wrote:
>> 
>>> Hello Michel,
>>> 
>>>> module (system) name;         // interface: unsafe   impl.: unsafe
>>>> module (safe) name;           // interface: safe     impl.: safe
>>>> 
>>> I thought that first (unsafe-unsafe) case is currently available just
>>> by:
>>> 
>>> module name; // interface: unsafe   impl.: unsafe
>>> 
>>> separating modules to unsafe-unsafe and safe-safe  has no usefulness
>>> - as those modules could not interact, specifically you need modules
>>> that are implemented by unsafe means, but provides only safe
>>> interface, so I see it as:
>>> 
>>> module name;                  // interface: unsafe   impl.: unsafe
>>> module (system) name;         // interface: safe     impl.: unsafe
>>> module (safe) name;           // interface: safe     impl.: safe
>>> 
>>> so you can call system modules (io, network...) from safe code.
>>> 
>> That's a pretty clean design. How would it interact with a -safe
>> command-line flag?
>> 
>> Andrei
>> 
> 
> When compiling with -safe flag, you are doing it because you need your 
> entire application to be safe*.
> 
> Safe flag would just affect modules with no safety flag specified - 
> making them (safe):
> 
> module name; --> module (safe) name;
> 
> and then compile.

I'm not sure this works so well. Look at this:

	module memory;   // unsafe interface - unsafe impl.
	extern (C) void* malloc(int);
	extern (C) void free(void*);

	module (system) my.system;   // safe interface - unsafe impl.
	import memory;
	void test() { auto i = malloc(10); free(i); }   // ok: unsafe impl. allowed

	module (safe) my.safe;   // safe interface - safe impl.
	import memory;
	void test() { auto i = malloc(10); free(i); }   // error: malloc, free 
are unsafe

How is this supposed to work correctly with and without the "-safe" 
compiler flag? The way you define things "-safe" would make module 
memory safe for use while it is not.


-- 
Michel Fortin
michel.fortin@michelf.com
http://michelf.com/
November 04, 2009
Re: safety model in D
Andrei Alexandrescu wrote:
> Don wrote:
>> Andrei Alexandrescu wrote:

> module(safe) is not a comment. We need three types of modules because of 
> the interaction between what the module declares and what the command 
> line wants.
> 
> Let's assume the default, no-flag build allows unsafe code, like right 
> now. Then, module(safe) means that the module volunteers itself for 
> tighter checking, and module(system) is same as module unadorned.
> 
> But then if the user compiles with -safe, module(safe) is the same as 
> module unadorned, and module(system) allows for unchecked operations in 
> that particular module. I was uncomfortable with this, but Walter 
> convinced me that D's charter is not to allow sandbox compilation and 
> execution of malicious code. If you have the sources, you may as well 
> take a look at their module declarations if you have some worry.
> 
> Regardless on the result of the debate regarding the default compilation 
> mode, if the change of that default mode is allowed in the command line, 
> then we need both module(safe) and module(system).

When would it be MANDATORY for a module to be compiled in safe mode?
If module(safe) implies bound-checking *cannot* be turned off for that 
module, would any standard library modules be module(safe)?

>> This actually seems pretty similar to public/private.
>> I see three types of modules:
>>
>> module  : the default, should compile in -safe mode.
>> module(system) : Modules which need to do nasty stuff inside, but for 
>> which all the public functions are safe.
>> module(sysinternal/restricted/...): Modules which exist only to 
>> support system modules. This will include most APIs to C libraries.
>>
>> Modules in the outer ring need to be prevented from calling ones in 
>> the inner ring.
> 
> Well I wouldn't want to go any dirtier than "system", so my "system" 
> would be your "sysinternal". I'd like to milden "system" a bit like in 
> e.g. "trusted", which would be your "system".

Yeah, the names don't matter. The thing is, modules in the inner ring 
are extremely rare. I'd hope there'd be just a few in druntime, and no 
public ones at all in Phobos.
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home