John Reimer wrote:
> Chris Miller wrote:
>> On Thu, 29 Dec 2005 15:40:24 -0500, Walter Bright <newshound@digitalmars.com> wrote:
>>
>>>
>>> "Derek Parnell" <derek@psych.ward> wrote in message
>>> news:op.s2kab6y06b8z09@ginger.vic.bigpond.net.au...
>>>> In other words, why does the generated header show implementation code and
>>>> why does it show private members?
>>>
>>> It shows implementation code for functions that are candidates for inlining.
>>> It shows private members because they might be used in const initializers
>>> and in inline functions.
>>>
>>> If a function cannot be inlined, its implementation will not be included.
>>>
>>
>> Perhaps it should only do this if -inline is present.
> 
> What happens, then, if you use "-inline" on the library and no "-inline" on the project that uses the library?  You get an interface file that's prepared for inline code per the library build (with all the extra private and function implementation details), but a project file that uses an interface file that won't be inlined in the project.  Kind of complicated.  I think it's just better to disable inlining on libraries that generate an interface file and on projects that uses that interface file.
> 
> -JJR

I should qualify the above.  "-inline" should be allowed on the project file as usual, but only for items that don't pertain to the interface file.

"John Reimer" <terminal.node@gmail.com> wrote in message news:dp1ja1$rh2$1@digitaldaemon.com...
> I realize the need to make private data visible for inlining purposes, but this appears to defeat the main purpose of the interface file -- hiding private members and implementation details.
>
> The need to accommodate inlining seems wrong too.  Don't you think it's better, at this point, to disable inlining for interface files and document it well so nobody is surprised by the absence of it.

This would be a perfectly valid solution, but Dave and I decided to come at it from the direction of having the results be identical whether importing a .d or a .di file.

Note that template bodies also wind up being totally reproduced in a .di file. Private data members are also needed in order to get the class member offsets right. In this way, it's like a C++ header file, or a Java .class file.

I could switch it to a binary format, and that would provide some level of obfuscation, but that shouldn't be confused with a secure format. Binary symbol formats are easilly hacked and reversed. Even if the file format was encrypted, the decryption keys would have to be supplied to the compiler, and once DVD Jon reverse engineers it, that's the end of any security.

The best way to achieve hiding is to use interface classes, not an interface file. The main advantage of a D interface file is compile performance.

(P.S. binary symbol table formats have been compared to object files as far as difficulty of reverse engineering. I believe this is a misguided and completely false analogy. Reverse engineering an object file is converting the proverbial hamburger back into a cow, because most semantic information is thrown away in the process of making an object file. Not so with a symbol file; all the semantic information *must* be present.

This thinking is the result of my discussions with C++ experts on having binary exported template symbol files, which many otherwise well informed people mistakenly imagine will make for a securely hidden implementation. )

(P.P.S. A D interface file can also be compared with a Java .class file, which can be imported by Java compilers rather than source code. But many tools exist which automatically turn .class files into reasonable Java source code. .class files don't really hide information at all.)

Regarding Derek's example: what exactly is the value in an "import file" like that, Walter? It bears little or no resemblance to what any related posts over the last two years have been requesting, and seems to break all understood rules vis-a-vis visability.

It certainly appears to have little bearing on the primary request point ~ that is, taking a non OSS component and exposing the minimal attributes to make it usable/linkable. The example below explicitly exposes the inner workings (private attributes also!), which is totally contrary to said requested purpose.

I really don't get the point of this at all, and it seems like others are in the same boat. Can you enlighten us, please? It's not April 1st yet, right? :)

- Kris


"Derek Parnell" <derek@psych.ward> wrote in message news:op.s2kab6y06b8z09@ginger.vic.bigpond.net.au...
> On Fri, 30 Dec 2005 06:11:54 +1100, Walter Bright <newshound@digitalmars.com> wrote:
>
>> This incorporates a new 'header' generator capability, written by Dave Fladebo, now working!
>
> Well it might be working, but it doesn't do what I was expecting.
>
> This source file ...
>
> //--------------------------------
> import std.stdio;
>
> private int qwerty;
> class A
> {
>     private int x;
>     void ths()
>     {
>       x = 1;
>       }
> }
>
> void xpub()
> {
>     A a = new A;
>     qwerty = 1;
> }
>
> private void xpriv()
> {
>     A b = new A;
>     qwerty = 2;
> }
> //--------------------------------
>
>
> was turned into this 'header' ...
>
> // D import file generated from 'test.d'
> import std.stdio;
> private
> {
>     int qwerty;
> }
> class A
> {
>     private
> {
>     int x;
> }
>     void ths()
> {
> x = 1;
> }
> }
> void xpub()
> {
> A a = new A;
> qwerty = 1;
> }
> private
> {
>     void xpriv()
> {
> A b = new A;
> qwerty = 2;
> }
> }
> //-------------------------
>
> but I was expecting something more like this ...
>
> // D import file generated by hand
> import std.stdio;
> class A
> {
>     void ths(){}
> }
> void xpub(){}
> //-------------------------
>
>
> In other words, why does the generated header show implementation code and why does it show private members? Currently, all it seems to do is reformat the original source by placing braces around private sections.
> -- 
> Derek Parnell
> Melbourne, Australia

On Thu, 29 Dec 2005 13:38:48 -0800, Walter Bright wrote:

> The main advantage of a D interface file is compile performance.

This is the *only* advantage and that's not such a problem anyhow. It seems then that I won't be needing the -H switch anytime soon.

Excuse me for sounding impertinent and selfish, but are you spending your time on D wisely? We get something of little value and still not receive things that are higher on the priority list.

How can I help? I know I can /donate/ library code and documentation, but the core language is still where the pre-1.0 issues lie. And even when I have given you library fixes, they are still to appear. I guess I don't really understand your organization's development processes.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 9:41:09 AM

"Kris" <fu@bar.com> wrote in message news:dp1on5$vsn$1@digitaldaemon.com...
> I really don't get the point of this at all, and it seems like others are in the same boat. Can you enlighten us, please? It's not April 1st yet, right? :)

Sure. See my other post in this thread.

On Fri, 30 Dec 2005 07:10:44 +1100, Derek Parnell wrote:

Ok, how about we try to get both camps happy about this DI file concept.
Here is a suggestion that does mean a change to the language though...

Given this source file ...

 //--------------------------------
 import std.stdio;

 private int  qwerty;
 private real asdfgh;

 pragma(generate): // Turn inlining off from here on.

 class A
 {
      private int x;
      void ths()
      {
        x = 1;
        }
 }

 void xpub()
 {
      A a = new A;
      qwerty = 1;
 }

 private void xpriv()
 {
      A b = new A;
      qwerty = 2;
 }
 //--------------------------------


It could be turned into this 'header' ...

 // D import file generated from 'test.d'
 import std.stdio;
 reserve:14; // fourteen bytes reserved
 class A
 {
  reserve:4; // four bytes reserved
  void ths() { }
 }
 void xpub() { }
 //-------------------------


In other words, let's enhance the language so we can support the needs of developers who want closed-source development *and* the needs of the compiler to generate correctly performing code. This would mean we need to resolve the issue of class member offsets, inlining, and const declarations. These are not impossible to resolve.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"A learning experience is one of those things that says,
 'You know that thing you just did? Don't do that.'" - D.N. Adams
30/12/2005 9:56:10 AM

"Derek Parnell" <derek@psych.ward> wrote in message news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg@40tude.net...
> In other words, let's enhance the language so we can support the needs of developers who want closed-source development *and* the needs of the compiler to generate correctly performing code. This would mean we need to resolve the issue of class member offsets, inlining, and const declarations. These are not impossible to resolve.

There's more than that, there are template bodies. At some point, we have to ask if we've just got the wrong approach to implementation hiding. I suggest that interface classes are the way to get true implementation hiding, not di files. di files' main advantage is improving build speeds for large projects.

They work a lot like "precompiled headers" do in C++.

"Derek Parnell" <derek@psych.ward> wrote in message news:1uxk73kt2jabq.1w7j11urctxbb.dlg@40tude.net...
> On Thu, 29 Dec 2005 13:38:48 -0800, Walter Bright wrote:
>> The main advantage of a D interface file is compile performance.
> This is the *only* advantage

No, it does wind up hiding the implementation details of most functions.

> and that's not such a problem anyhow.

Build speed isn't a problem now, but it will become one once D projects exceed a certain size.

> It seems then that I won't be needing the -H switch anytime soon.

It's certainly a forward looking capability.

> Excuse me for sounding impertinent and selfish, but are you spending your time on D wisely? We get something of little value and still not receive things that are higher on the priority list.

It is regularly asked for. The specific impetus in this case was that Dave Fladebo went ahead and implemented it.

> How can I help? I know I can /donate/ library code and documentation, but the core language is still where the pre-1.0 issues lie. And even when I have given you library fixes, they are still to appear. I guess I don't really understand your organization's development processes.

It's mainly driven by 1) what people feel like contributing and 2) things that are blocking use of D.

You suggest that using Interfaces is the right way to hide implementation detail. I'd agree, but how does one expose the means of accessing an instance? Does one provide an interface factory, and expose a header for that?

How is it supposed to work?


"Walter Bright" <newshound@digitalmars.com> wrote in message news:dp1pbv$10i7$2@digitaldaemon.com...
>
> "Kris" <fu@bar.com> wrote in message news:dp1on5$vsn$1@digitaldaemon.com...
>> I really don't get the point of this at all, and it seems like others are in the same boat. Can you enlighten us, please? It's not April 1st yet, right? :)
>
> Sure. See my other post in this thread.
> 

Walter Bright wrote:
> "Derek Parnell" <derek@psych.ward> wrote in message news:zqfcc85x3d7z$.1n4xv1fmu7njt$.dlg@40tude.net...
> 
>>In other words, let's enhance the language so we can support the needs of
>>developers who want closed-source development *and* the needs of the
>>compiler to generate correctly performing code. This would mean we need to
>>resolve the issue of class member offsets, inlining, and const
>>declarations. These are not impossible to resolve.
> 
> 
> There's more than that, there are template bodies. At some point, we have to ask if we've just got the wrong approach to implementation hiding. I suggest that interface classes are the way to get true implementation hiding, not di files. di files' main advantage is improving build speeds for large projects.
> 
> They work a lot like "precompiled headers" do in C++. 
> 
> 

Understood.

How about a separate interface class generation feature then?

If it were up to me I would rename the 'import file' feature to 'precompiled header', hoping that the term is not copyrighted/trademarked by Microsoft. =)  Then again, if they work 'a lot like' precompiled headers in C++ then what are the main differences if the two are not exactly alike?

For the closed source community, one could also invent a specially formatted D import source file which declares only the non-private methods of classes *and* at which offsets they can be found at in the vtable.  Of course include the public fields and at which offsets they are found at as well.