On 12/16/2012 5:10 PM, Jonathan M Davis wrote:
> On Sunday, December 16, 2012 16:57:31 Walter Bright wrote:
>> I've done such (precompiled headers for C++), I've done .di files, and I've
>> done Java bytecode. .di files are superior in nearly every way.
>
> Given that .di don't work with inlining or CTFE,

The other schemes outlined will not work any better, because they are fundamentally no different.

> in favor of a binary solution

Whether the file format is text or binary does not make any fundamental difference.

On Sunday, December 16, 2012 19:18:45 Walter Bright wrote:
> On 12/16/2012 5:10 PM, Jonathan M Davis wrote:
> > On Sunday, December 16, 2012 16:57:31 Walter Bright wrote:
> >> I've done such (precompiled headers for C++), I've done .di files, and
> >> I've
> >> done Java bytecode. .di files are superior in nearly every way.
> > 
> > Given that .di don't work with inlining or CTFE,
> 
> The other schemes outlined will not work any better, because they are fundamentally no different.
> 
>  > in favor of a binary solution
> 
> Whether the file format is text or binary does not make any fundamental difference.

If the entire .d file is there in binary form, then I don't see why it wouldn't work. .di files fail because they strip out the implementation. If a binary format were used, then we should be able to get away with keeping the implementation there, because then it's obfuscated rather than for sitting there for all to see, which is why corporations and the like insist on distributing only headers. Even with an object file, the best that you get is obfuscation, because it can always be reverse engineered, so it seems to me that what needs to be avoided is providing text. As long as we use text, we're forced to cut out the implementation and end up crippling any code that uses that module, since it can't inline it or use it in CTFE. In binary format, it's obfuscated, so the entire implementation can be there, allowing inlining and CTFE to work.

- Jonathan M Davis

On 12/16/2012 10:27 PM, Jonathan M Davis wrote:
> If the entire .d file is there in binary form, then I don't see why it wouldn't
> work. .di files fail because they strip out the implementation. If a binary
> format were used,

It's all about what is in the file, not whether it is text or binary.

> then we should be able to get away with keeping the
> implementation there, because then it's obfuscated rather than for sitting
> there for all to see, which is why corporations and the like insist on
> distributing only headers. Even with an object file, the best that you get is
> obfuscation, because it can always be reverse engineered, so it seems to me
> that what needs to be avoided is providing text. As long as we use text, we're
> forced to cut out the implementation and end up crippling any code that uses
> that module, since it can't inline it or use it in CTFE. In binary format,
> it's obfuscated, so the entire implementation can be there, allowing inlining
> and CTFE to work.

This method of obfuscation simply will not hide things from someone with even modest technical ability, because *all* the source information is *necessarily* there in the file.

Object files are resistant to reverse engineering because most of the information is gone.

On Sun, Dec 16, 2012 at 10:27:31PM -0800, Jonathan M Davis wrote:
> On Sunday, December 16, 2012 19:18:45 Walter Bright wrote:
[...]
> > Whether the file format is text or binary does not make any fundamental difference.
> 
> If the entire .d file is there in binary form, then I don't see why it wouldn't work. .di files fail because they strip out the implementation. If a binary format were used, then we should be able to get away with keeping the implementation there, because then it's obfuscated rather than for sitting there for all to see, which is why corporations and the like insist on distributing only headers. Even with an object file, the best that you get is obfuscation, because it can always be reverse engineered, so it seems to me that what needs to be avoided is providing text. As long as we use text, we're forced to cut out the implementation and end up crippling any code that uses that module, since it can't inline it or use it in CTFE. In binary format, it's obfuscated, so the entire implementation can be there, allowing inlining and CTFE to work.
[...]

>From a computer-theoretical point of view, whether you use a plaintext
.d file or a partially-compiled interface file makes no difference at all, because they are all representations of the same thing. But from a practical standpoint, some PTBs may feel more comfortable with a binary blob that's not obviously readable to the casual onlooker.

Of course, that provide zero theoretical security, because anyone who's desperate enough to reverse-engineer your library will do it even if they only have the final executable. But the casual onlooker factor is not to be dismissed when dealing with management types. So I contend that precompiling to some kind of IR is of some practical value, even if in theory it's not worth very much. (Yes decompilers and what-not will always be there. That didn't stop Java developers from preferentially shipping .class files vs. .java files. You can't stop the professionals, but you can at least stop the petty thieves. This can be important to some people.)


T

-- 
The fact that anyone still uses AOL shows that even the presence of options doesn't stop some people from picking the pessimal one. - Mike Ellis

On Sunday, December 16, 2012 22:58:26 Walter Bright wrote:
> On 12/16/2012 10:27 PM, Jonathan M Davis wrote:
> > If the entire .d file is there in binary form, then I don't see why it wouldn't work. .di files fail because they strip out the implementation. If a binary format were used,
> 
> It's all about what is in the file, not whether it is text or binary.

The concept of .di files and their ilk is fundamentally broken precisely because they're trying to strip what's in the file. That's what causes the problems.

> > then we should be able to get away with keeping the
> > implementation there, because then it's obfuscated rather than for sitting
> > there for all to see, which is why corporations and the like insist on
> > distributing only headers. Even with an object file, the best that you get
> > is obfuscation, because it can always be reverse engineered, so it seems
> > to me that what needs to be avoided is providing text. As long as we use
> > text, we're forced to cut out the implementation and end up crippling any
> > code that uses that module, since it can't inline it or use it in CTFE.
> > In binary format, it's obfuscated, so the entire implementation can be
> > there, allowing inlining and CTFE to work.
> 
> This method of obfuscation simply will not hide things from someone with even modest technical ability, because *all* the source information is *necessarily* there in the file.
> 
> Object files are resistant to reverse engineering because most of the information is gone.

True, but they can stll be reverse engineered, and if the problem is that companies don't want 3rd parties looking at their code, then a binary format has obfuscated it and possibly solved that problem. Object files do make it harder, but they can still be reverse engineered, so it really becomes a question of what it takes to satisfy the folks who think that not giving the whole information in a .d or .cpp file somehow protects their code (since it doesn't really). And if a binary format can do that (as it seems to in Java land), then that seems like a far better solution, because then we can leave all of the information in there that inlining and CTFE need in order to do their jobs. With .di files, we'll never get that.

- Jonathan M Davis

On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
> It doesn't hide the source in any effective way. There are enough Java byte code => source translators around to prove that. It only takes one such tool to exist (and it's especially easy to create such a tool given D being open source).
>

Oh, so that's why java is never used in any company !

On 2012-12-16 22:34, Walter Bright wrote:

> No "seem" about it!

Ok, I see. I'm just taking some precautions since I don't know what you're thinking.

-- 
/Jacob Carlborg

On Monday, December 17, 2012 08:24:27 deadalnix wrote:
> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
> > It doesn't hide the source in any effective way. There are enough Java byte code => source translators around to prove that. It only takes one such tool to exist (and it's especially easy to create such a tool given D being open source).
> 
> Oh, so that's why java is never used in any company !

Java is used quite heavily by companies.

- Jonathan M Davis

On 2012-12-17 02:10, Jonathan M Davis wrote:

> Given that .di don't work with inlining or CTFE, I'd consider them to be a
> very poor solution. You're seriously impairing yourself if you use them. It's
> pretty much BS that corporations insist on header files to hide implementation,
> since it really doesn't work, but if we're going to be forced to a have a
> solution which tries to hide implementation to make folks like that happy, we
> could at least have one that doesn't cripple the language like .di files do. It
> may not truly hide the implementation any better than .di files do, but at
> least it would allow us to still use the language properly.
>
> I'm not expecting this problem to be fixed any time soon (we have far higher
> priorites), but I really do think that in the long run .di files should be
> deprecated in favor of a binary solution which doesn't stop things like
> inlining or CTFE from working.

If a function needs to be template, inline or CTFE it can be manually put in the .di file. I assume that will work.

-- 
/Jacob Carlborg

On Monday, 17 December 2012 at 07:27:49 UTC, Jonathan M Davis wrote:
> On Monday, December 17, 2012 08:24:27 deadalnix wrote:
>> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
>> > It doesn't hide the source in any effective way. There are
>> > enough Java byte code => source translators around to prove
>> > that. It only takes one such tool to exist (and it's especially
>> > easy to create such a tool given D being open source).
>> 
>> Oh, so that's why java is never used in any company !
>
> Java is used quite heavily by companies.
>

No it isn't, because they are afraid of bytecode decoders like jad or JD.