There seem to have been a few tickets thus far from people wanting to do relative file imports using the -J switch.  Walter has stated that this is explicitly disallowed as a defensive measure, which is fine.

I was going to post the following as an enhancement request, but I thought it might be better if it was discussed first.

TLDR: relative paths should be safe if we disallow symlinks, disallow -J to be used except for a small whitelist and require a specially-named file to be present in any imported directory.

Disclaimer: I'm not a security guru.

  -- Daniel

There have been at least two reports thus far (2103 and 2759) of people attempting to use relative imports with the -J compile-time file import feature.

It has been stated that this is explicitly disabled due to security concerns.

I suggest that adding relative paths (provided paths cannot escape the base directory) is not less secure; it should already be possible to read an arbitrary file using the feature.

Let us assume relative paths are allowed, and that the path is "collapsed" to remove all parent (..) references, and clipped to the base directory.  The only way to "escape" this directory is for one of three things to happen:

1. there is a symlink to a parent directory in the import path,
2. there is a device of interest mounted in the import path or
3. the user was tricked into specifying a higher directory in the first
place.

Clearly, #3 is already a problem: one could always specify -J/etc in a Makefile to read any system configuration file that the user has access to.  #2 is only a problem if the compiling user is an administrator and tremendously stupid, which is not so much an issue with DMD but with the user.  #1 is something which could also be done in a Makefile.

It should be noted that this is really no different to executing arbitrary code on a machine.  That said, compiling a program is not typically thought of as "executing" code, so some restrictions in this case would probably be prudent.

Therefore, I believe the following should be sufficient to ensure relative paths are safe to include:

1. Ensure that symlinks are NOT followed.  This also applies under Windows for junctions and symlinks (I believe it should be enough to use GetFileAttributesEx on each path component and reject it if it has the FILE_ATTRIBUTE_REPARSE_POINT attribute, although I cannot verify this with XP; see http://blogs.msdn.com/oldnewthing/archive/2004/12/27/332704.aspx).

2. Disallow -J to be the root of any drive or filesystem, and only allow -J to be used from the following roots: %USERPROFILE% for Windows and ~ for *nix.  Possibly, this should be configured or overridable in sc.ini; an extendible whitelist would be good.

3. Require that any directory you attempt to read from using file imports MUST have either a "_import" or ".import" file (or whatever name seems appropriate) present in it.

The first should prevent symlinks from being a potential attack vector, the second should prevent most casual malicious behaviour, and the third will ensure that the user not only have read access but WRITE access as well (or whoever owns the directory explicitly allowed for D file imports.)

This should hopefully be sufficient to allow for relative imports.

Daniel Keep schrieb:
> There have been at least two reports thus far (2103 and 2759) of people attempting to use relative imports with the -J compile-time file import feature.

Please note also my posting "Proposal for fixing import("file")" on 14th
March in this NG.

On Wed, Mar 25, 2009 at 8:28 AM, Daniel Keep <daniel.keep.lists@gmail.com> wrote:

> 2. Disallow -J to be the root of any drive or filesystem, and only allow -J to be used from the following roots: %USERPROFILE% for Windows and ~ for *nix.  Possibly, this should be configured or overridable in sc.ini; an extendible whitelist would be good.

%USERPROFILE% is probably a bad choice on Windows pre-Vista, mostly because OPTLINK will die with paths with spaces in them (go OPTLINK). That and, I have never used my %USERPROFILE% directory for anything :P (probably because it's such a long god-damned path!)

Jarrett Billingsley wrote:
> On Wed, Mar 25, 2009 at 8:28 AM, Daniel Keep <daniel.keep.lists@gmail.com> wrote:
> 
>> 2. Disallow -J to be the root of any drive or filesystem, and only allow -J to be used from the following roots: %USERPROFILE% for Windows and ~ for *nix.  Possibly, this should be configured or overridable in sc.ini; an extendible whitelist would be good.
> 
> %USERPROFILE% is probably a bad choice on Windows pre-Vista, mostly because OPTLINK will die with paths with spaces in them (go OPTLINK). That and, I have never used my %USERPROFILE% directory for anything :P (probably because it's such a long god-damned path!)

True, but what else is there?  That's where "My Documents" lives; there's no other "safe" directory I could think of for Windows.

If the error message including the language "You may need to add this directory, or a parent of it, to the whitelist in %PATHTODMD%\sc.ini" it should be OK.

The point here is to allow it safely.

  -- Daniel

Daniel Keep wrote:
> It should be noted that this is really no different to executing
> arbitrary code on a machine.  That said, compiling a program is not
> typically thought of as "executing" code, so some restrictions in this
> case would probably be prudent.

Here's the scenario I'm concerned about. Let's say you set up a website that instead of supporting javascript, supports D used as a scripting language. The site thus must run the D compiler on the source code. When it executes the resulting code, that execution presumably will run in a "sandbox" at a low privilege level.

But the compiler itself will be part of the server software, and may run at a higher privilege. The import feature could possible read any file in the system, inserting it into the executable being built. The running executable could then supply this information to the attacker, even though it is sandboxed.

This is why even using the import file feature must be explicitly enabled by a compiler switch, and which directories it can read must also be explicitly set with a compiler switch. Presumably, it's a lot easier for the server software to control the compiler switches than to parse the D code looking for obfuscated file imports.

Frank Benoit wrote:
> Daniel Keep schrieb:
>> There have been at least two reports thus far (2103 and 2759) of people
>> attempting to use relative imports with the -J compile-time file import
>> feature.
> 
> Please note also my posting "Proposal for fixing import("file")" on 14th
> March in this NG.

Should go into bugzilla with a link to that thread.

On Wed, Mar 25, 2009 at 2:18 PM, Walter Bright <newshound1@digitalmars.com> wrote:
>
> Here's the scenario I'm concerned about. Let's say you set up a website that instead of supporting javascript, supports D used as a scripting language. The site thus must run the D compiler on the source code. When it executes the resulting code, that execution presumably will run in a "sandbox" at a low privilege level.
>
> But the compiler itself will be part of the server software, and may run at a higher privilege. The import feature could possible read any file in the system, inserting it into the executable being built. The running executable could then supply this information to the attacker, even though it is sandboxed.
>
> This is why even using the import file feature must be explicitly enabled by a compiler switch, and which directories it can read must also be explicitly set with a compiler switch. Presumably, it's a lot easier for the server software to control the compiler switches than to parse the D code looking for obfuscated file imports.
>

One, you do not use Javascript as a server-side scripting language. That task is usually relegated to (shiver) PHP, Ruby, Python, Java or the like.  Javascript, as far as I know, only has a place in web programming in client-side (browser-side) scripting.  The only thing you can do from the client is maybe munge some POST data, and then that's a logic error on the part of your server script, not anything to do with the language.

Two, if you did replace your server-side scripting language with
something like D, there are no more or fewer security holes than with
any other server-side scripting language.  The entire point of
server-side scripts is that they are invisible to the clients, and the
clients never see, modify, or upload server-side scripts of their own.
 It's not possible to put a malicious file import in the D server page
unless you actually have access to the server, and if an attacker does
have that kind of access, D is no more or less secure than any other
server-side language.  They'd all be vulnerable.  The attacker could
just change the server's compiler flags to change the import directory
anyway.

I understand your desire to restrict the use of import expressions, but your example just does not add up.

Walter Bright wrote:
> Daniel Keep wrote:
>> It should be noted that this is really no different to executing arbitrary code on a machine.  That said, compiling a program is not typically thought of as "executing" code, so some restrictions in this case would probably be prudent.
> 
> Here's the scenario I'm concerned about. Let's say you set up a website that instead of supporting javascript, supports D used as a scripting language. The site thus must run the D compiler on the source code. When it executes the resulting code, that execution presumably will run in a "sandbox" at a low privilege level.
> 
> But the compiler itself will be part of the server software, and may run at a higher privilege.

If you're thinking of something like ASP.NET for D, whereby the server executes a D compiler, then I think this would be a serious design flaw of the software; the D compiler itself should be executing with the absolute minimum permissions necessary to function, and preferably in a VM or a jail.

> The import feature could possible read any file
> in the system, inserting it into the executable being built. The running
> executable could then supply this information to the attacker, even
> though it is sandboxed.
> 
> This is why even using the import file feature must be explicitly enabled by a compiler switch, and which directories it can read must also be explicitly set with a compiler switch. Presumably, it's a lot easier for the server software to control the compiler switches than to parse the D code looking for obfuscated file imports.

Well, my proposal should -improve- security because currently it's possible to read any file the compiler has access to simply by compiling via a Makefile or any other build process.

If the code cannot set command-line switches, then there's no difference, so let's ignore that case.  Let's assume the code CAN set switches.  There's nothing to stop it doing this:

> dmd -J/etc evil.d && evil

> module evil;
> const PASSWD = cast(char[]) import("passwd");
> import std.stdio;
> void main()
> {
>     writefln("200 OK\nContent-Type: text/plain\n");
>     writef("%s", PASSWD);
> }

Hence why I proposed a whitelist in sc.ini itself.  This way, you can't trick the compiler (either by controlling the command-line or by tricking the user) into giving you the sensitive file directly.

By disallowing symlinks, you also remove the ability to "escape" a directory tree.

Finally, the requirement for a ".dimport" or "_dimport" file or something ensures that folders have to explicitly cleared for file imports.  Combined with the whitelist and removal of symlinks, it should be more or less impossible to get access to a file you shouldn't.

Perhaps it might be worthwhile to do a mock compiler (that just attempts to read a provided filename using a -J switch) with these semantics and see if it's possible to exploit it in any way...

  -- Daniel

Daniel Keep wrote:
> If the code cannot set command-line switches, then there's no
> difference, so let's ignore that case.  Let's assume the code CAN set
> switches.  There's nothing to stop it doing this:

It's a lot easier to scrub command line switches than to try to scrub D source code. It's the server that runs dmd, not the client.

The rest of your proposal may be tight, I don't really know. I do wish to keep it, however, as simple as possible and the current scheme does that.

Walter Bright wrote:
> Daniel Keep wrote:
>> If the code cannot set command-line switches, then there's no difference, so let's ignore that case.  Let's assume the code CAN set switches.  There's nothing to stop it doing this:
> 
> It's a lot easier to scrub command line switches than to try to scrub D source code. It's the server that runs dmd, not the client.
> 
> The rest of your proposal may be tight, I don't really know. I do wish to keep it, however, as simple as possible and the current scheme does that.

Sorry, poor choice of words.  I should have said "Let's assume a malicious party who either provided the code or knows how to exploit the code can set switches."

I'm not proposing scrubbing the source in any way.

  -- Daniel