I have -J added to the command line like

-JC:\Temp

I then use import(r"C:\Temp\a.dat");

and I get an error about the file not existing in the

main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J

It seems dmd does internally compare the paths to see if they are identical.

Of course, removing the C:\Temp\ part of import works fine.

The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used.

essentially

version(X)
import(path)
else
load(path);

while I could do something like

import(baseName(path))

it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.

On 12/08/2017 3:34 AM, Mr. Pib wrote:
> I have -J added to the command line like
> 
> -JC:\Temp
> 
> I then use import(r"C:\Temp\a.dat");
> 
> and I get an error about the file not existing in the
> 
> main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J
> 
> It seems dmd does internally compare the paths to see if they are identical.
> 
> Of course, removing the C:\Temp\ part of import works fine.
> 
> The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used.
> 
> essentially
> 
> version(X)
> import(path)
> else
> load(path);
> 
> while I could do something like
> 
> import(baseName(path))
> 
> it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.

Most likely related or is the issue[0].

[0] https://issues.dlang.org/show_bug.cgi?id=3420

On Saturday, 12 August 2017 at 02:45:30 UTC, rikki cattermole wrote:
> On 12/08/2017 3:34 AM, Mr. Pib wrote:
>> I have -J added to the command line like
>> 
>> -JC:\Temp
>> 
>> I then use import(r"C:\Temp\a.dat");
>> 
>> and I get an error about the file not existing in the
>> 
>> main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J
>> 
>> It seems dmd does internally compare the paths to see if they are identical.

No, it just prefixes the given import() path will all those given by -J.

>> Of course, removing the C:\Temp\ part of import works fine.
>> 
>> The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used.
>> 
>> essentially
>> 
>> version(X)
>> import(path)
>> else
>> load(path);

I don't understand why you would want to do that. Using absolute paths makes assumptions about the system your software is being built on. Even if you never plan to build the software on any computer but your own, it still seems like a questionable design decision.

>> while I could do something like
>> 
>> import(baseName(path))
>> 
>> it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.

You could also use -J/ and import(path[1..$]) (or the Windows equivalent). Still a bad idea.

> Most likely related or is the issue[0].
>
> [0] https://issues.dlang.org/show_bug.cgi?id=3420

Nope.

On Saturday, 12 August 2017 at 03:02:31 UTC, Vladimir Panteleev wrote:
> On Saturday, 12 August 2017 at 02:45:30 UTC, rikki cattermole wrote:
>> On 12/08/2017 3:34 AM, Mr. Pib wrote:
>>> I have -J added to the command line like
>>> 
>>> -JC:\Temp
>>> 
>>> I then use import(r"C:\Temp\a.dat");
>>> 
>>> and I get an error about the file not existing in the
>>> 
>>> main.d: Error: file "C:\\Temp\\a.dat" cannot be found or not in a path specified with -J
>>> 
>>> It seems dmd does internally compare the paths to see if they are identical.
>
> No, it just prefixes the given import() path will all those given by -J.
>
>>> Of course, removing the C:\Temp\ part of import works fine.
>>> 
>>> The problem with that approach is it then is not consistent with other code. I need to specify the full path because sometimes it is used.
>>> 
>>> essentially
>>> 
>>> version(X)
>>> import(path)
>>> else
>>> load(path);
>
> I don't understand why you would want to do that. Using absolute paths makes assumptions about the system your software is being built on. Even if you never plan to build the software on any computer but your own, it still seems like a questionable design decision.
>

You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/

>>> while I could do something like
>>> 
>>> import(baseName(path))
>>> 
>>> it seems kinda clunky, in any case. It's pretty obvious that one excepts the same behavior so it could create bugs in code that except the behavior to work correctly.
>
> You could also use -J/ and import(path[1..$]) (or the Windows equivalent). Still a bad idea.
>
>> Most likely related or is the issue[0].
>>
>> [0] https://issues.dlang.org/show_bug.cgi?id=3420
>
> Nope.

If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location.

It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations.

If that is not the case, then a better error message should be given.

On Saturday, 12 August 2017 at 03:52:25 UTC, Mr. Pib wrote:
> You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/

Sorry, maybe I don't understand the question. Maybe you could explain in broader terms the higher-level goal or problem you're trying to solve, and maybe we can recommend a better way?

> If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location.
>
> It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations.

Fairly sure we forbid .. out of security considerations.

I don't think this applies to Windows, but on POSIX, depending on how .. is interpreted, \x\y x/y actually can mean a different file from x/../x/y.

It has taken a lot of consideration and research until we even allowed path separators in import paths. For a very long time, they were completely forbidden, and a long time after that, they were forbidden on Windows (because on Windows things can be more complicated due to the various kinds of reparse points and things such as short filenames).

> If that is not the case, then a better error message should be given.

Feel free to file a diagnostic enhancement request if you have specific suggestions.

On Saturday, 12 August 2017 at 03:58:30 UTC, Vladimir Panteleev wrote:
> On Saturday, 12 August 2017 at 03:52:25 UTC, Mr. Pib wrote:
>> You are making assumptions about me making assumptions... please don't make any more assumptions or we will be in an infinite regression ;/
>
> Sorry, maybe I don't understand the question. Maybe you could explain in broader terms the higher-level goal or problem you're trying to solve, and maybe we can recommend a better way?
>
>> If are aware, it has nothing to do with absolute paths as the bug should also be exhibited with relative paths. It is a comparison issue of the strings rather than checking to see if they represent the same physical location.
>>
>> It would be like saying that \x\y is different from \x\..\x\y. Dmd does a blind comparison, I bet, rather than what it should be doing. Obviously \x\y are different strings but they are not different file locations.
>
> Fairly sure we forbid .. out of security considerations.
>
> I don't think this applies to Windows, but on POSIX, depending on how .. is interpreted, \x\y x/y actually can mean a different file from x/../x/y.
>
> It has taken a lot of consideration and research until we even allowed path separators in import paths. For a very long time, they were completely forbidden, and a long time after that, they were forbidden on Windows (because on Windows things can be more complicated due to the various kinds of reparse points and things such as short filenames).
>
>> If that is not the case, then a better error message should be given.
>
> Feel free to file a diagnostic enhancement request if you have specific suggestions.

I'm pretty sure that on no OS does the same location mean different things?

I am not talking about strange stuff but simple stuff.

I have code that loads a file at runtime and requires the absolute path. This is only for debugging purposes. When built in release, everything is switched over to use imports and embed the files in the binary. The same path is used for other things like caching/uniqueID but are never actually read from. You see this sort of stuff a lot when you open an executable and see hard coded paths but obviously never used for file system purposes.

The files and paths are all the same but import doens't seem to think so. Adding baseName solves the problem immediately but that is a hack. import should know that the path is the same as the one specified by -J. The whole point of -J is to specify the path for security purposes, right? So why does it matter if I use path\filename or baseName(filename)? Both point to the same location and both are consistent with -J, import should understand that. It is an obvious oversight. But there is an obvious programmatic difference between the two versions. Luckily, using baseName does fix the problem so it is not a huge deal but it is still a bug/issue with import for being ignorant of what it is actually doing.

On Saturday, 12 August 2017 at 05:42:21 UTC, Mr. Pib wrote:
> I'm pretty sure that on no OS does the same location mean different things?

I understand what you mean, but just to clarify on the .. thing:

$ mkdir d d/x d/z
$ ln -s d/z x
$ echo foo > d/z/y
$ echo bar > d/x/y
$ cat x/y
foo
+ cat x/../x/y
bar

> I am not talking about strange stuff but simple stuff.
>
> I have code that loads a file at runtime and requires the absolute path. This is only for debugging purposes. When built in release, everything is switched over to use imports and embed the files in the binary. The same path is used for other things like caching/uniqueID but are never actually read from. You see this sort of stuff a lot when you open an executable and see hard coded paths but obviously never used for file system purposes.

Sounds like you can also work around it using e.g.

enum appRoot = `C:\Temp\`;
debug
    string data = readText(appRoot ~ "a.dat");
else
    enum data = import("a.dat");

(and build with -JC:\Temp).

That should now also work if a.dat is in a subdirectory relative to the -J path.

> The files and paths are all the same but import doens't seem to think so. Adding baseName solves the problem immediately but that is a hack. import should know that the path is the same as the one specified by -J. The whole point of -J is to specify the path for security purposes, right? So why does it matter if I use path\filename or baseName(filename)? Both point to the same location and both are consistent with -J, import should understand that. It is an obvious oversight. But there is an obvious programmatic difference between the two versions. Luckily, using baseName does fix the problem so it is not a huge deal but it is still a bug/issue with import for being ignorant of what it is actually doing.

Sounds reasonable, the compiler could check if paths start with a -J path.

On Saturday, 12 August 2017 at 06:02:57 UTC, Vladimir Panteleev wrote:
> Sounds reasonable, the compiler could check if paths start with a -J path.

There is a potential ambiguity here:

dmd -Jsomedir test.d

test.d: import("somedir/file.txt");

Does the user mean to import "somedir/file.txt" or "somedir/somedir/file.txt"? Currently the latter is understood. Simply checking for path prefix would break this case. I guess this could be done only with absolute paths, but that introduces an inconsistency with relative paths. I'm not sure it's worth it, considering it's easy to work around.

On Saturday, 12 August 2017 at 06:06:33 UTC, Vladimir Panteleev wrote:
> On Saturday, 12 August 2017 at 06:02:57 UTC, Vladimir Panteleev wrote:
>> Sounds reasonable, the compiler could check if paths start with a -J path.
>
> There is a potential ambiguity here:
>
> dmd -Jsomedir test.d
>
> test.d: import("somedir/file.txt");
>
> Does the user mean to import "somedir/file.txt" or "somedir/somedir/file.txt"? Currently the latter is understood. Simply checking for path prefix would break this case. I guess this could be done only with absolute paths, but that introduces an inconsistency with relative paths. I'm not sure it's worth it, considering it's easy to work around.

What is the inconsistency? Absolute paths are specific and well defined and so are relatives. Relatives are always relative to J, are they not?

so, import("somedir/file.txt"); obviously means somedir/somedir/file.txt.

If you wanted somedir/file.txt you would do import(`file.txt`).

I see no problem here. With absolute paths, one simply checks if the absolute base path matches the absolute base path specified by J. If it matches, then it passes and can be used directly or removed internally. If not then an error is given.

e.g.,

-JC:/basepath/somedir
C:/basepath/somedir/file.txt

import('C:/basepath/somedir/file.txt')
import('file.txt')
import('../somedir/file.txt')
import('C:/basepath/../basepath/somedir/file.txt')
import('C:/path/../basepath/file.txt')

should pass all pass.

import('../file.txt')
import('C:/basepath/../basepath/file.txt')
import('C:/path/file.txt')

should all fail.

(I'm only using .. for reference, not sure if they should actually be supported)

It seems quite simple to me

Any relative paths specified by import are prepended with the path specified by -J. Any absolute paths specified by an import are first matched to the path -J simply by

example:

if (J_path.length < basepath.length) assert("Import path must be located in the same path that is specified by -J");

if (basepath[0..J_path.length] != J_path) ditto; // makes sure absolute path is in the -J path. This is the check that is missing from dmd.

(this doesn't resolve ..'s, that could be done before and assumes basepath is an absolute path)