On Monday, 17 December 2012 at 07:56:01 UTC, Paulo Pinto wrote:
>
> Similar solutions work for Ada, Modula-3, F#, Haskell, OCaml, just
> to cite a few languages with generic types and modules.
>
> So the issue is how .di files are implemented, not the general concept.
>
> --
> Paulo

If we want to allow D to fit into various niche markets overlooked by C++, for added security, encryption could be added, where the person compiling encrypted .di files would have to supply a key. That would work only for certain situations, not for mass distribution, but it may be useful to enough people.

--rt

On 12/16/2012 11:00 PM, H. S. Teoh wrote:
> Of course, that provide zero theoretical security, because anyone who's
> desperate enough to reverse-engineer your library will do it even if
> they only have the final executable. But the casual onlooker factor is
> not to be dismissed when dealing with management types. So I contend
> that precompiling to some kind of IR is of some practical value, even if
> in theory it's not worth very much. (Yes decompilers and what-not will
> always be there. That didn't stop Java developers from preferentially
> shipping .class files vs. .java files. You can't stop the professionals,
> but you can at least stop the petty thieves. This can be important to
> some people.)


I don't care if some people delude themselves about the "security" of a binary version of the IR, but there's no way I'll represent D as offering such security. It's fraudulent to even pretend that such would.

.class files offer ZERO security. You can get a free tool off the internet to decompile them at the push of a button.

There are no tools to do that with object files.

There are legitimate arguments for a binary file. Obscurity is NOT one of them, and cannot be.

On 12/16/2012 11:36 PM, Jacob Carlborg wrote:
> If a function needs to be template, inline or CTFE it can be manually put in the
> .di file. I assume that will work.

Of course it will. In fact, you can:

   copy foo.d foo.di

and it will always work. .di is really nothing more than a naming convention. There is no code whatsoever in the compiler that says:

   if (in a .di file)
       do this
   else
       do that

and that is on purpose.

On 12/16/2012 11:24 PM, deadalnix wrote:
> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
>> It doesn't hide the source in any effective way. There are enough Java byte
>> code => source translators around to prove that. It only takes one such tool
>> to exist (and it's especially easy to create such a tool given D being open
>> source).
>>
>
> Oh, so that's why java is never used in any company !

Yes, I know you're being sarcastic.

But consider that Amazon uses Java heavily. They don't care about source obfuscation, because they don't ship source code or .class libraries. They don't sell Java code at all. Source obfuscation is irrelevant to them.

On 12/16/2012 11:27 PM, Jacob Carlborg wrote:
> On 2012-12-16 22:34, Walter Bright wrote:
>
>> No "seem" about it!
>
> Ok, I see. I'm just taking some precautions since I don't know what you're
> thinking.

I know. I was just taking the opportunity to clarify.

On 12/17/2012 12:24 AM, Rob T wrote:
> If we want to allow D to fit into various niche markets overlooked by C++, for
> added security, encryption could be added, where the person compiling encrypted
> .di files would have to supply a key. That would work only for certain
> situations, not for mass distribution, but it may be useful to enough people.

That simply would never work any better than just emailing someone an encrypted file, a decryption key, and expecting the end user to decrypt the files, then run the compiler on it.

Consider that dmd is open source. You cannot embed secret information in it.

On Monday, 17 December 2012 at 08:34:26 UTC, Walter Bright wrote:
> On 12/16/2012 11:24 PM, deadalnix wrote:
>> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
>>> It doesn't hide the source in any effective way. There are enough Java byte
>>> code => source translators around to prove that. It only takes one such tool
>>> to exist (and it's especially easy to create such a tool given D being open
>>> source).
>>>
>>
>> Oh, so that's why java is never used in any company !
>
> Yes, I know you're being sarcastic.
>
> But consider that Amazon uses Java heavily. They don't care about source obfuscation, because they don't ship source code or .class libraries. They don't sell Java code at all. Source obfuscation is irrelevant to them.

More seriously, I understand that in some cases, di are interesting. Mostly if you want to provide a closed source library to be used by 3rd party devs.

But that is a use amongst many, and is irrelevant for an huge amount of companies. Many don't ship code because they are providing service (google, facebook, amazon, etc . . .) or because they deliver the final binary to the customer (most software you can buy are delivered that way).

On Monday, 17 December 2012 at 08:11:18 UTC, deadalnix wrote:
> On Monday, 17 December 2012 at 08:07:10 UTC, Paulo Pinto wrote:
>> As for not being used in companies, well my employer does consulting work for Fortune 500 companies. You only have JVM and .NET everywhere, with anything else considered legacy or stuff that kids play with.
>>
>
> That can't be true, you must be lying !
>
> How can such company be stupid enough to use a language with a bytecode that can be decoded ? Man, be realistic, no company will ever use such a technology. In what world are you living ? They have business to do, and you don't make business using a technology that everybody can reverse engineer !

Assembly is no different than reversing any other type of bytecode:

http://www.hopperapp.com/

On Monday, 17 December 2012 at 07:09:46 UTC, Jonathan M Davis wrote:
> On Sunday, December 16, 2012 22:58:26 Walter Bright wrote:
>> On 12/16/2012 10:27 PM, Jonathan M Davis wrote:
>> > If the entire .d file is there in binary form, then I don't see why it
>> > wouldn't work. .di files fail because they strip out the implementation.
>> > If a binary format were used,
>> 
>> It's all about what is in the file, not whether it is text or binary.
>
> The concept of .di files and their ilk is fundamentally broken precisely
> because they're trying to strip what's in the file. That's what causes the
> problems.

WRT to all opinions above (ie: binary vs text, what to put etc.)

I had some reflection on that some time ago: how about bundling a "header" file (that would be the .di file) and a binary file (the compiled .d file, that is the .obj file) into a single .zip (albeit with another extension), that will be recognized and processed by the D compiler (let's name that file a .dobj).

Idea may seem a bit crazy, but consider the following:

-the standard .zip format could be used by a user of that object/library to learn the interface of the functions provided by the object (just like a C header file)
-if he's a power user, he can simply extract the .zip/.dobj, modify the included header (adding comments, for example), then archive that back and present the compiler a "fresh" .dobj/library file

The responsability of maintaining the .obj and the header in sync will be of the compiler or of the power user, if the latter edit it manually. More, IDEs could simply extract relevant header information from the .zip archive and use it for code completion, documentation and so all.

Basically, this would be like bundling a .h file with the corresponding .obj file (if we speak C++), all that under a transparent format. The code is hidden and obfuscated, just like in a standard library (think -lstdc++ vs <iostream>). The use of a single file greatly facilitate synchronization, while the use of the standard .zip format allow a plethora of tools to manually tune the file (if desired).

This can be extended also to entire .dlib (that is, archive of .dobjs), which can become self-documenting, that way. I kinda of dreamt about that since programming in C++ and always needed to have the headers and the libs with me. Why do not include the headers in the lib, in a transparent and manually readable/editable format?

A checksum could guarantee also that the header information and the binary information are in sync inside the .zip archive.

What do you think?

On Monday, 17 December 2012 at 08:21:43 UTC, Walter Bright wrote:
> On 12/16/2012 11:08 PM, Jonathan M Davis wrote:
>> The concept of .di files and their ilk is fundamentally broken precisely
>> because they're trying to strip what's in the file. That's what causes the
>> problems.
>
> What is and what isn't in a .di is entirely under the user's control.
>
>
>>> Object files are resistant to reverse engineering because most of the
>>> information is gone.
>> True, but they can stll be reverse engineered,
>
> I know what I'm talking about with this. The only time they get reverse engineered is when somebody really really REALLY wants to do it, an expert is necessary to do the job, and it's completely impractical for larger sets of files. You cannot build a tool to do it, it must be done by hand, line by line. It's the proverbial turning of hamburger back into a cow.
>
> A binary format of the source code, however, is EASY to turn back into source code, and it's EASY to write a tool to do it automatically, and complete idiots can successfully run that tool and reverse engineer it by pushing a button.
>
> You cannot even begin to compare the difficulty levels between the two.
>
>> and if the problem is that
>> companies don't want 3rd parties looking at their code, then a binary format
>> has obfuscated it and possibly solved that problem.
>
> No, it has not obfuscated it at all.
>
>> Object files do make it
>> harder, but they can still be reverse engineered, so it really becomes a
>> question of what it takes to satisfy the folks who think that not giving the
>> whole information in a .d or .cpp file somehow protects their code (since it
>> doesn't really). And if a binary format can do that (as it seems to in Java
>> land), then that seems like a far better solution, because then we can leave
>> all of the information in there that inlining and CTFE need in order to do
>> their jobs. With .di files, we'll never get that.
>
> Java developers tried to "obfuscate" their code by distributing them as .class files, and someone promptly wrote a free tool to turn .class files back into source code.
>
> No such tool exists for object code, and never has. That should be a pretty strong indication of its possibilities.
>

Really?!

http://www.hopperapp.com/

I really like the way it generates pseudo-code and basic block graphs out of instruction sequences.

--
Paulo