On Monday, 17 December 2012 at 01:11:22 UTC, Jonathan M Davis wrote:
> On Sunday, December 16, 2012 16:57:31 Walter Bright wrote:
>> I've done such (precompiled headers for C++), I've done .di files, and I've
>> done Java bytecode. .di files are superior in nearly every way.
>
> Given that .di don't work with inlining or CTFE, I'd consider them to be a
> very poor solution. You're seriously impairing yourself if you use them. It's
> pretty much BS that corporations insist on header files to hide implementation,
> since it really doesn't work, but if we're going to be forced to a have a
> solution which tries to hide implementation to make folks like that happy, we
> could at least have one that doesn't cripple the language like .di files do. It
> may not truly hide the implementation any better than .di files do, but at
> least it would allow us to still use the language properly.
>
> I'm not expecting this problem to be fixed any time soon (we have far higher
> priorites), but I really do think that in the long run .di files should be
> deprecated in favor of a binary solution which doesn't stop things like
> inlining or CTFE from working.
>
> - Jonathan M Davis

Similar solutions work for Ada, Modula-3, F#, Haskell, OCaml, just
to cite a few languages with generic types and modules.

So the issue is how .di files are implemented, not the general concept.

--
Paulo

On Monday, December 17, 2012 08:39:56 deadalnix wrote:
> On Monday, 17 December 2012 at 07:27:49 UTC, Jonathan M Davis
> 
> wrote:
> > On Monday, December 17, 2012 08:24:27 deadalnix wrote:
> >> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright
> >> 
> >> wrote:
> >> > It doesn't hide the source in any effective way. There are
> >> > enough Java byte code => source translators around to prove
> >> > that. It only takes one such tool to exist (and it's
> >> > especially
> >> > easy to create such a tool given D being open source).
> >> 
> >> Oh, so that's why java is never used in any company !
> > 
> > Java is used quite heavily by companies.
> 
> No it isn't, because they are afraid of bytecode decoders like jad or JD.

There may very well be companies which avoid it because of that, but I don't see how you can possibly think that Java isn't used heavily in the corporate world, since it is.

- Jonathan M Davis

On Sunday, 16 December 2012 at 23:07:04 UTC, Jonathan M Davis wrote:
> On Sunday, December 16, 2012 23:32:38 Andrej Mitrovic wrote:
>> On 12/16/12, Paulo Pinto <pjmlp@progtools.org> wrote:
>> > If modules are used correctly, a .di should be created with the public
>> > interface and everything else is already in binary format, thus the
>> > compiler is not really parsing everything all the time.
>> 
>> A lot of D code tends to be templated code, .di files don't help you
>> in that case.
>
> And .di files don't work with CTFE or inlining. In general, .di files are a
> horrible idea.
>
> I tend to be of the opinion that they shouldn't even exist, but some corporate
> types require that sort of thing when distributing libraries to 3rd parties,
> so we need some sort of header solution. A better one probably would have been
> a binary format where the code is partially compiled with documentation
> providing a human-readable API, but that's something that we'll have to look
> into in the future. For now, we're stuck with .di files.
>
> - Jonathan M Davis

If you want the language to have a place in the enterprise, you have to support the enterprise use cases, as simple as that.

--
Paulo

On Sun, 16 Dec 2012 23:08:49 -0800, Jonathan M Davis <jmdavisProg@gmx.com> wrote:

> On Sunday, December 16, 2012 22:58:26 Walter Bright wrote:
>> On 12/16/2012 10:27 PM, Jonathan M Davis wrote:
>> > If the entire .d file is there in binary form, then I don't see why it
>> > wouldn't work. .di files fail because they strip out the  
>> implementation.
>> > If a binary format were used,
>>
>> It's all about what is in the file, not whether it is text or binary.
>
> The concept of .di files and their ilk is fundamentally broken precisely
> because they're trying to strip what's in the file. That's what causes the
> problems.
>
>> > then we should be able to get away with keeping the
>> > implementation there, because then it's obfuscated rather than for  
>> sitting
>> > there for all to see, which is why corporations and the like insist on
>> > distributing only headers. Even with an object file, the best that  
>> you get
>> > is obfuscation, because it can always be reverse engineered, so it  
>> seems
>> > to me that what needs to be avoided is providing text. As long as we  
>> use
>> > text, we're forced to cut out the implementation and end up crippling  
>> any
>> > code that uses that module, since it can't inline it or use it in  
>> CTFE.
>> > In binary format, it's obfuscated, so the entire implementation can be
>> > there, allowing inlining and CTFE to work.
>>
>> This method of obfuscation simply will not hide things from someone with
>> even modest technical ability, because *all* the source information is
>> *necessarily* there in the file.
>>
>> Object files are resistant to reverse engineering because most of the
>> information is gone.
>
> True, but they can stll be reverse engineered, and if the problem is that
> companies don't want 3rd parties looking at their code, then a binary format
> has obfuscated it and possibly solved that problem. Object files do make it
> harder, but they can still be reverse engineered, so it really becomes a
> question of what it takes to satisfy the folks who think that not giving the
> whole information in a .d or .cpp file somehow protects their code (since it
> doesn't really). And if a binary format can do that (as it seems to in Java
> land), then that seems like a far better solution, because then we can leave
> all of the information in there that inlining and CTFE need in order to do
> their jobs. With .di files, we'll never get that.
>
> - Jonathan M Davis

Putting it all in binary and calling it good isn't really a solution. Like it or not, the C/C++ header file provides what larger corporations with IP to protect want. Difficult to reverse (not impossible, but they know that and are willing to accept it) with easy to access plaintext headers so the legitimately licensed programmer can work with the product that is rightfully his without stealing it. Not to mention that if extracting binary was easy we'd all be doing it, and if we made a tool to do it, then we'd be back to it being no different than plaintext DI's which is I think what Walter was driving at.

With respect to those who hold one ideology above others, trying to impose those ideals on another is a great way to ensure animosity. What a business does with their code is entirely up to them, and I would guess that even Richard Stallman himself would take issue with trying to impose an ideology on another person. What does that mean for D practically? Using a close-to-home example, imagine if Remedy decided that shipping their ENTIRE codebase in .DI files with the product would cause them to give away some new rendering trick that they came up with that nobody else had. And they decided that this was unacceptable. What would they most likely do? Rewrite the project in C++ and tell the D community to kindly pound sand.

A license agreement is not enough to stop a thief. And once the new trick makes it into the wild, as long as a competitor can honestly say they had no idea how they got it (and they probably really don't, as they saw it on a legitimate game development website) the hands of the legal system are tied.

I know there are those here who think that D can thrive without the support of the corporates. But realistically, they are the ones who dictate the languages most coders use and therefore learn. THAT's what makes the Remedy thing so important. By showing other corps that D is safe, reliable, and understands THEIR needs, they will begin investing in training there people, and so on. This pushes more people to learn D. Is your goal the proliferation of D or the proliferation of ideology?

And this idea that somehow because we lose CTFE, Inlining and Auto support we should never allow DI's to be anything other then full source distro's is borderline ridiculous. C++ programmers are under ZERO illusions that inlining was ever going to work when using third party libraries, so the idea that some things won't work is beyond routine. They won't even notice, much less care. I'll put it to you this way, when I was considering D for our projects, I asked what features I would lose in redacted DI files. My entire response to the list was "That's it? Sweet! This thing still beats C++ six ways from Sunday."

Then I learned that the DI files weren't redacted. Now I have a pull for it.

-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/

On Monday, 17 December 2012 at 07:39:57 UTC, deadalnix wrote:
> On Monday, 17 December 2012 at 07:27:49 UTC, Jonathan M Davis wrote:
>> On Monday, December 17, 2012 08:24:27 deadalnix wrote:
>>> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright wrote:
>>> > It doesn't hide the source in any effective way. There are
>>> > enough Java byte code => source translators around to prove
>>> > that. It only takes one such tool to exist (and it's especially
>>> > easy to create such a tool given D being open source).
>>> 
>>> Oh, so that's why java is never used in any company !
>>
>> Java is used quite heavily by companies.
>>
>
> No it isn't, because they are afraid of bytecode decoders like jad or JD.

Bytecode is only one possible way to use Java.

People seem to forget on purpose that bytecodes are just a possible way to execute Java code and that actually there are quite a few Java native compilers to choose from.

Java is much more than the Oracle's JVM.

As for not being used in companies, well my employer does consulting work for Fortune 500 companies. You only have JVM and .NET everywhere, with anything else considered legacy or stuff that kids play with.

The last time I had the chance to do a C++ project at work was in 2006.

Now with the C++11 publicity going on, this might change, otherwise the enterprise is all about JVM and .NET based languages.

--
Paulo

On Monday, 17 December 2012 at 07:56:26 UTC, Jonathan M Davis wrote:
> On Monday, December 17, 2012 08:39:56 deadalnix wrote:
>> On Monday, 17 December 2012 at 07:27:49 UTC, Jonathan M Davis
>> 
>> wrote:
>> > On Monday, December 17, 2012 08:24:27 deadalnix wrote:
>> >> On Monday, 17 December 2012 at 00:57:30 UTC, Walter Bright
>> >> 
>> >> wrote:
>> >> > It doesn't hide the source in any effective way. There are
>> >> > enough Java byte code => source translators around to prove
>> >> > that. It only takes one such tool to exist (and it's
>> >> > especially
>> >> > easy to create such a tool given D being open source).
>> >> 
>> >> Oh, so that's why java is never used in any company !
>> > 
>> > Java is used quite heavily by companies.
>> 
>> No it isn't, because they are afraid of bytecode decoders like
>> jad or JD.
>
> There may very well be companies which avoid it because of that, but I don't
> see how you can possibly think that Java isn't used heavily in the corporate
> world, since it is.
>

OK, before I'll drive you crazy, I know that (and I professionally wrote quite a lot of java myself). Java is one of the most used language in companies.

But it seems that explaining it the forward classic way do not work, so I try another strategy : maintain the opposite point and let people notice how dumb it is.

I hope the point is made by now.

On Monday, 17 December 2012 at 08:07:10 UTC, Paulo Pinto wrote:
> As for not being used in companies, well my employer does consulting work for Fortune 500 companies. You only have JVM and .NET everywhere, with anything else considered legacy or stuff that kids play with.
>

That can't be true, you must be lying !

How can such company be stupid enough to use a language with a bytecode that can be decoded ? Man, be realistic, no company will ever use such a technology. In what world are you living ? They have business to do, and you don't make business using a technology that everybody can reverse engineer !

On Monday, 17 December 2012 at 08:02:12 UTC, Adam Wilson wrote:
> On Sun, 16 Dec 2012 23:08:49 -0800, Jonathan M Davis <jmdavisProg@gmx.com> wrote:
>
>> On Sunday, December 16, 2012 22:58:26 Walter Bright wrote:
>>> On 12/16/2012 10:27 PM, Jonathan M Davis wrote:
>>> > If the entire .d file is there in binary form, then I don't see why it
>>> > wouldn't work. .di files fail because they strip out the
>>> implementation.
>>> > If a binary format were used,
>>>
>>> It's all about what is in the file, not whether it is text or binary.
>>
>> The concept of .di files and their ilk is fundamentally broken precisely
>> because they're trying to strip what's in the file. That's what causes the
>> problems.
>>
>>> > then we should be able to get away with keeping the
>>> > implementation there, because then it's obfuscated rather than for
>>> sitting
>>> > there for all to see, which is why corporations and the like insist on
>>> > distributing only headers. Even with an object file, the best that
>>> you get
>>> > is obfuscation, because it can always be reverse engineered, so it
>>> seems
>>> > to me that what needs to be avoided is providing text. As long as we
>>> use
>>> > text, we're forced to cut out the implementation and end up crippling
>>> any
>>> > code that uses that module, since it can't inline it or use it in
>>> CTFE.
>>> > In binary format, it's obfuscated, so the entire implementation can be
>>> > there, allowing inlining and CTFE to work.
>>>
>>> This method of obfuscation simply will not hide things from someone with
>>> even modest technical ability, because *all* the source information is
>>> *necessarily* there in the file.
>>>
>>> Object files are resistant to reverse engineering because most of the
>>> information is gone.
>>
>> True, but they can stll be reverse engineered, and if the problem is that
>> companies don't want 3rd parties looking at their code, then a binary format
>> has obfuscated it and possibly solved that problem. Object files do make it
>> harder, but they can still be reverse engineered, so it really becomes a
>> question of what it takes to satisfy the folks who think that not giving the
>> whole information in a .d or .cpp file somehow protects their code (since it
>> doesn't really). And if a binary format can do that (as it seems to in Java
>> land), then that seems like a far better solution, because then we can leave
>> all of the information in there that inlining and CTFE need in order to do
>> their jobs. With .di files, we'll never get that.
>>
>> - Jonathan M Davis
>
> Putting it all in binary and calling it good isn't really a solution. Like it or not, the C/C++ header file provides what larger corporations with IP to protect want. Difficult to reverse (not impossible, but they know that and are willing to accept it) with easy to access plaintext headers so the legitimately licensed programmer can work with the product that is rightfully his without stealing it. Not to mention that if extracting binary was easy we'd all be doing it, and if we made a tool to do it, then we'd be back to it being no different than plaintext DI's which is I think what Walter was driving at.
>
> With respect to those who hold one ideology above others, trying to impose those ideals on another is a great way to ensure animosity. What a business does with their code is entirely up to them, and I would guess that even Richard Stallman himself would take issue with trying to impose an ideology on another person. What does that mean for D practically? Using a close-to-home example, imagine if Remedy decided that shipping their ENTIRE codebase in .DI files with the product would cause them to give away some new rendering trick that they came up with that nobody else had. And they decided that this was unacceptable. What would they most likely do? Rewrite the project in C++ and tell the D community to kindly pound sand.
>
> A license agreement is not enough to stop a thief. And once the new trick makes it into the wild, as long as a competitor can honestly say they had no idea how they got it (and they probably really don't, as they saw it on a legitimate game development website) the hands of the legal system are tied.
>
> I know there are those here who think that D can thrive without the support of the corporates. But realistically, they are the ones who dictate the languages most coders use and therefore learn. THAT's what makes the Remedy thing so important. By showing other corps that D is safe, reliable, and understands THEIR needs, they will begin investing in training there people, and so on. This pushes more people to learn D. Is your goal the proliferation of D or the proliferation of ideology?
>
> And this idea that somehow because we lose CTFE, Inlining and Auto support we should never allow DI's to be anything other then full source distro's is borderline ridiculous. C++ programmers are under ZERO illusions that inlining was ever going to work when using third party libraries, so the idea that some things won't work is beyond routine. They won't even notice, much less care. I'll put it to you this way, when I was considering D for our projects, I asked what features I would lose in redacted DI files. My entire response to the list was "That's it? Sweet! This thing still beats C++ six ways from Sunday."
>
> Then I learned that the DI files weren't redacted. Now I have a pull for it.


And as I referred in another thread, it is as not other languages don't support this, except maybe for CTFE. But even then it is mostly an implementation issue,
I think.

--
Paulo

On Monday, 17 December 2012 at 08:02:12 UTC, Adam Wilson wrote:
> With respect to those who hold one ideology above others, trying to impose those ideals on another is a great way to ensure animosity. What a business does with their code is entirely up to them, and I would guess that even Richard Stallman himself would take issue with trying to impose an ideology on another person. What does that mean for D practically? Using a close-to-home example, imagine if Remedy decided that shipping their ENTIRE codebase in .DI files with the product would cause them to give away some new rendering trick that they came up with that nobody else had. And they decided that this was unacceptable. What would they most likely do? Rewrite the project in C++ and tell the D community to kindly pound sand.
>
> A license agreement is not enough to stop a thief. And once the new trick makes it into the wild, as long as a competitor can honestly say they had no idea how they got it (and they probably really don't, as they saw it on a legitimate game development website) the hands of the legal system are tied.
>

But that what I say !

I can't stop myself laughing at people that may think any business can be based on java, PHP or C#. That is a mere dream ! Such technology will simply never get used in companies, because bytecode can be decoded !

On 12/16/2012 11:08 PM, Jonathan M Davis wrote:
> The concept of .di files and their ilk is fundamentally broken precisely
> because they're trying to strip what's in the file. That's what causes the
> problems.

What is and what isn't in a .di is entirely under the user's control.


>> Object files are resistant to reverse engineering because most of the
>> information is gone.
> True, but they can stll be reverse engineered,

I know what I'm talking about with this. The only time they get reverse engineered is when somebody really really REALLY wants to do it, an expert is necessary to do the job, and it's completely impractical for larger sets of files. You cannot build a tool to do it, it must be done by hand, line by line. It's the proverbial turning of hamburger back into a cow.

A binary format of the source code, however, is EASY to turn back into source code, and it's EASY to write a tool to do it automatically, and complete idiots can successfully run that tool and reverse engineer it by pushing a button.

You cannot even begin to compare the difficulty levels between the two.

> and if the problem is that
> companies don't want 3rd parties looking at their code, then a binary format
> has obfuscated it and possibly solved that problem.

No, it has not obfuscated it at all.

> Object files do make it
> harder, but they can still be reverse engineered, so it really becomes a
> question of what it takes to satisfy the folks who think that not giving the
> whole information in a .d or .cpp file somehow protects their code (since it
> doesn't really). And if a binary format can do that (as it seems to in Java
> land), then that seems like a far better solution, because then we can leave
> all of the information in there that inlining and CTFE need in order to do
> their jobs. With .di files, we'll never get that.

Java developers tried to "obfuscate" their code by distributing them as .class files, and someone promptly wrote a free tool to turn .class files back into source code.

No such tool exists for object code, and never has. That should be a pretty strong indication of its possibilities.

If you want to obfuscate your code, the way to do it is with the PIMPL idiom. That works, it works now, and has a long track record of working for statically compiled languages.