Kris wrote:
> Perhaps there's further misunderstanding here:

> I can point to one side-effect that actually does help: the compiler now looks for "di" files, which means that hand-coded "implementation bridging" files can (at last!) live side-by-side with the real implementation module, eliminating some prior grief for a developer. However, this conflicts with -H rather badly, and therefore is likely to be an unintended and somewhat precarious benefit. Was that aspect considered?
> 


Interesting!  I didn't really think of that aspect.  That could be useful, indeed. Though, as you said, it probably is precarious.

-JJR

Kris wrote:
> 
> And because of that, it really has no value for implementation hiding. I'd go as far as to say it simply complicates things instead. Do imports get exposed also? I'd try this myself, but the compiler GPFs. Further, exposing public attributes in this manner seems to defy logic, and appears to breaks every visibility rule in the book. Such declarations were made private because they were intended to be, /ahem/, private?

But, as Walter stated, private data might affect alignment and such so it must be exposed.  If implementation hiding is an issue and if interfaces are not appropriate, the pimpl idiom from C++ might be worth considering.

> Forgive me Dave, but I just can't see how that's relevant. What you describe is not what, for example, any Fortune-500 company would consider usable. Walter apparently agrees. In reference to software contract-exposure: ".di files aren't necessarilly going to help you here."
> 
> Then there's the question of what this feature thinks it addresses. Walter seems to feel it's about compiler performance: "di files' main advantage is improving build speeds for large projects", and this is borne out in the flurry of posts. You also intimated this in a recent post. Yet you're also saying it can be used for implementation-hiding?

I think the most relevant comparison is with C/C++ header files, as D headers are nearly identical in what they provide.  And as Walter said, the primary use of headers is to speed compilation.  That implementation can be concealed in some cases is secondary.  Another concern is template files.  C++ has 'export' in an attempt to address this, but it's a tremendously complex feature that provides questionable benefit.  I think Walter is right for not attempting anything like this for D quite yet.

> Pardon me, but I have to disagree on all points:
> 
> <rant>
> 
> 1) -H has little or no benefit for implementation hiding. It may look as though it could suffice, but I'll be happy to bet the current incarnation is a questionable toy in this regard. If I could actually get it to work, then I'd be happy to prove the point. Perhaps surprisingly, Walter appears to agree.

See my comment about the pimpl (pointer to implementation) idea above. This is commonly used both to obscure implementation details and to reduce coupling.
> 
> 2) -H certainly ought to cut down on import size. So, just how much does that affect the compiler performance? It's already blazingly fast. I'll be a monkey's uncle if -H makes true /real-world/ differences to more than 5% of users. On the other hand, a /binary/ symbolic representation (as discussed several times) could make a really notable dent for certain environments, if such speedups were to become a necessity. If -H is all about performance, then why isn't it based around such a representation?

Another issue is that header files are touched far less frequently than implementation files.  Thus, implementation changes can be made and so long as the header remains unchanged the compiler doesn't have to rebuild dependent modules.  In large projects, this can be of tremendous value.  To this end, it might be nice if DMD only touched header files for nonzero deltas (so timestamps only changed if the file contents changed).

> 3) You speculatively mentioned DWT and RAD IDEs. There's a few things to consider there: (a) if the API is stable, the headers can be produced by a standalone tool ~ it doesn't need to be part of the compiler; (b) it the headers in question are being revised, they need to be regenerated on the fly; and (c) a RAD tool would likely prefer to have a library made available that hides all the details of creating and managing such headers, and would likely prefer that happened at the binary-symbolic level, explicitly for reasons of performance. Especially in case (a). Partial source-code munging does little to help such a tool ~ it's just not going far enough.

True enough.  But since a front-end is really required to do the job, why not build the feature into the compiler?  Particularly since each compiler may have different metrics for inlining and such.

> I've no intent on causing discomfort to you or anyone here, but I really have to say this feature appears to have been given not /nearly/ sufficient thought. It fails to support industrial-level implementation-hiding (the one place it is needed) and it doesn't push the performance envelope very hard at all.

I see implementation hiding as a separate (though related) QOI issue. Typically, I want to see readable header files and let things get mangled in post-production before the CD is sent out to clients.

> You can call me an ass if you like, Dave, but this is surely feature-creep of the worst kind ~ there's apparently no meat. Can you, or anyone, please help me grasp where the real value lies? It's not that I'm somehow against new functionality. Part of what bugs me about this is the reference compiler sets an unwritten "standard" that others must somehow comply with. Whatever those standards are, they surely ought to be well considered.

I think it's important to separate QOI features from "standard" features.  Will it be necessary for all D compilers to have code coverage or profiling tools?  Of course not.  Header generation is in the same category.

> I can point to one side-effect that actually does help: the compiler now looks for "di" files, which means that hand-coded "implementation bridging" files can (at last!) live side-by-side with the real implementation module, eliminating some prior grief for a developer. However, this conflicts with -H rather badly, and therefore is likely to be an unintended and somewhat precarious benefit. Was that aspect considered?

I haven't installed this new version of DMD yet so I don't know much about how -H works yet.  What I said above is based on what I've read here today :-)


Sean

In article <dp4oq5$6qt$1@digitaldaemon.com>, Kris says...
>
>Perhaps there's further misunderstanding here:
>
>"Dave" <Dave_member@pathlink.com> wrote in
>>
>> "Kris" <fu@bar.com> wrote ...
>>> Another approach to implementation hiding is to simply strip out existing
>>> method bodies and leave the bare declarations, along with any enums
>>> and/or
>>> typedefs they may use. Many requests have been made for an automated tool
>>> to do exactly this: which would also work well for the Interface approach
>>> noted above. The end result would be the same in both cases ~ a secondary
>>
>> This is basically what -H does, except that inlinable function bodies are emitted. As pointed out earlier, private members often need to be there too, with or without function bodies (offsets, const initializers, templates), otherwise it gets pretty complicated, fast.
>
>
>And because of that, it really has no value for implementation hiding. I'd go as far as to say it simply complicates things instead. Do imports get exposed also? I'd try this myself, but the compiler GPFs. Further, exposing public attributes in this manner seems to defy logic, and appears to breaks every visibility rule in the book. Such declarations were made private because they were intended to be, /ahem/, private?
>
>
>> I experimented with most/all the implementation hiding stuff and got most of it working, I think, but in the end after testing it would have taken a whole heck of a lot more work (and time maintaining it) to make and keep the thing bulletproof (see: http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/29883).
>>
>> Why don't you give -H a try by using it against phobos, DWT and Mango (once the integration issues are taken care of)? Then see if it still really makes the implementation details all that apparent (or at least more apparent than C/++ library header files do now)?
>
>
>Forgive me Dave, but I just can't see how that's relevant. What you describe is not what, for example, any Fortune-500 company would consider usable. Walter apparently agrees. In reference to software contract-exposure: ".di files aren't necessarilly going to help you here."
>
>Then there's the question of what this feature thinks it addresses. Walter seems to feel it's about compiler performance: "di files' main advantage is improving build speeds for large projects", and this is borne out in the flurry of posts. You also intimated this in a recent post. Yet you're also saying it can be used for implementation-hiding? Pardon me, but I have to disagree on all points:
>
><rant>
>
>1) -H has little or no benefit for implementation hiding. It may look as though it could suffice, but I'll be happy to bet the current incarnation is a questionable toy in this regard. If I could actually get it to work, then I'd be happy to prove the point. Perhaps surprisingly, Walter appears to agree.
>
>2) -H certainly ought to cut down on import size. So, just how much does that affect the compiler performance? It's already blazingly fast. I'll be a monkey's uncle if -H makes true /real-world/ differences to more than 5% of users. On the other hand, a /binary/ symbolic representation (as discussed several times) could make a really notable dent for certain environments, if such speedups were to become a necessity. If -H is all about performance, then why isn't it based around such a representation?
>
>3) You speculatively mentioned DWT and RAD IDEs. There's a few things to consider there: (a) if the API is stable, the headers can be produced by a standalone tool ~ it doesn't need to be part of the compiler; (b) it the headers in question are being revised, they need to be regenerated on the fly; and (c) a RAD tool would likely prefer to have a library made available that hides all the details of creating and managing such headers, and would likely prefer that happened at the binary-symbolic level, explicitly for reasons of performance. Especially in case (a). Partial source-code munging does little to help such a tool ~ it's just not going far enough.
>
>I've no intent on causing discomfort to you or anyone here, but I really have to say this feature appears to have been given not /nearly/ sufficient thought. It fails to support industrial-level implementation-hiding (the one place it is needed) and it doesn't push the performance envelope very hard at all.
>
>You can call me an ass if you like, Dave, but this is surely feature-creep of the worst kind ~ there's apparently no meat. Can you, or anyone, please help me grasp where the real value lies? It's not that I'm somehow against new functionality. Part of what bugs me about this is the reference compiler sets an unwritten "standard" that others must somehow comply with. Whatever those standards are, they surely ought to be well considered.
>
></rant>
>
>I can point to one side-effect that actually does help: the compiler now looks for "di" files, which means that hand-coded "implementation bridging" files can (at last!) live side-by-side with the real implementation module, eliminating some prior grief for a developer. However, this conflicts with -H rather badly, and therefore is likely to be an unintended and somewhat precarious benefit. Was that aspect considered?
>
>Respectfully;
>
>- Kris
>
>

I think part of the confusion is lack of knowledge of how the DMD compiler works. When one compiles a DMD module - it parses and performs full analysis on all imported modules - and all modules they import etc. Theres a section of comments refering to importing only pre-compiled symbols for imported objects instead of just parsing the complete source. I believe this is some sort if middle ground between importing symbols and parsing the source. And I can very much see how it could be usefull in huge projects but I personally havent experienced compile times that are not lighting fast. Also lets not forget what walter mentioned about this feature fleshing out the code generation output of the compiler which will be usefull down the road. I just dont understand why people react with such negativity and disrespect towards walter if he implements a feature that they dont need. Its really immature. Some of you act like you are paying Walter or something. What ever.

"JT" <JT_member@pathlink.com> wrote
> I think part of the confusion is lack of knowledge of how the DMD compiler
> works. When one compiles a DMD module - it parses and performs full
> analysis on
> all imported modules - and all modules they import etc.

I think most folks understand this. Is it an issue?

> Theres a section of
> comments refering to importing only pre-compiled symbols for imported
> objects
> instead of just parsing the complete source. I believe this is some sort
> if
> middle ground between importing symbols and parsing the source.

The "middle ground" you speak of is exactly that. It's not needed at this point, so why burden the state of play with adding such a thing? It's called feature creep.


> And I can very
> much see how it could be usefull in huge projects but I personally havent
> experienced compile times that are not lighting fast.

There's no question that a binary-symbolic implementation would be great. But, as you point out, there's no pressing need for it right now. Each time I compile, it's with 200+ files on a mediocre machine. I just timed it, and it takes just over 2 seconds. That includes the time Build takes. The compiler is very, very, fast. I think that's indicitive of a good design and a lot of valuable experience. The same cannot be said for -H, since it doesn't even seem to be clear about what it addresses.


Also lets not forget what
> walter mentioned about this feature fleshing out the code generation
> output of
> the compiler which will be usefull down the road.

Sure. But that's not what's being addressed. What it being addressed is the latest claim that this is somehow for implementation-hiding. It is simply not. Performance wise? Different subject matter.


> I just dont understand why
> people react with such negativity and disrespect towards walter if he
> implements
> a feature that they dont need. Its really immature. Some of you act like
> you are
> paying Walter or something. What ever.

First, I don't see where there was /any/ negativity or disrespect towards Walter in that post. It was adressed entirely towards Dave. But then, so what if it were ardently disagreeing with Walter directly? Walter is a tough nut to crack ... there's no value in mincing words with him.

You are, of course, welcome to your opinion. As am I.

My opinion is this: The community has a rather large stake in the future of D. This is why Walter has this NG ~ it's a way to create the evangelizers, and get the message out. D is a grass-root effort. There are a few handfuls of people who have contributed an /extraordinary/ amount of their /own/ time, in many different ways. Whether it is through ideas and suggestions, through encouraging other to try D, through hosting the NG, through hosting dsource.org, writing libraries or building tools. Do you think the community has no claim whatsoever to the D language? Do you feel we should not be able to shape its direction? Or criticize it? Chastise it?

Thus, we're not paying Walter to write the compiler. Nor is he paying those who contribute so much. It goes both ways, and it's pretty immature of you to state otherwise. I'm sure you'd agree.

First, I must point out that this branch is about "implementation hiding" ~
it say so in the topic :)
Second, if there's a point in this reply, it's at the end.


"Sean Kelly" <sean@f4.ca> wrote
>
> But, as Walter stated, private data might affect alignment and such so it must be exposed.  If implementation hiding is an issue and if interfaces are not appropriate, the pimpl idiom from C++ might be worth considering.

Interfaces are in the same boat, Sean. D still needs something nice and easy to generate the "implementation bridge" module that folk have been asking after for a long time. You can always build it by hand, but there's always been problems of one kind or another. Thankfully, a side-effect of -H will help. I'm not familiar with pimpl.



> I think the most relevant comparison is with C/C++ header files, as D headers are nearly identical in what they provide.  And as Walter said, the primary use of headers is to speed compilation.  That implementation can be concealed in some cases is secondary.  Another concern is template files.  C++ has 'export' in an attempt to address this, but it's a tremendously complex feature that provides questionable benefit. I think Walter is right for not attempting anything like this for D quite yet.


Then let's call the darned thing a "performance enhancement" and stop all this continued pretense :)



> Another issue is that header files are touched far less frequently than implementation files.  Thus, implementation changes can be made and so long as the header remains unchanged the compiler doesn't have to rebuild dependent modules.  In large projects, this can be of tremendous value. To this end, it might be nice if DMD only touched header files for nonzero deltas (so timestamps only changed if the file contents changed).


In C, yes. D currently does not have any such issues.


> I see implementation hiding as a separate (though related) QOI issue. Typically, I want to see readable header files and let things get mangled in post-production before the CD is sent out to clients.

Ack ~ this is not about mangled headers!


>> You can call me an ass if you like, Dave, but this is surely feature-creep of the worst kind ~ there's apparently no meat. Can you, or anyone, please help me grasp where the real value lies? It's not that I'm somehow against new functionality. Part of what bugs me about this is the reference compiler sets an unwritten "standard" that others must somehow comply with. Whatever those standards are, they surely ought to be well considered.
>
> I think it's important to separate QOI features from "standard" features. Will it be necessary for all D compilers to have code coverage or profiling tools?  Of course not.  Header generation is in the same category.


Is it? If, just if, people start using -H in an attempt at implementation-hiding, then other compilers will have to follow suit. After all, code will be written to take advantage of those headers (the whole point of implementation-hiding), and thus will be dependent on the way the information is exposed. Thus, once we start down that road others will have to comply.

In article <dp4oq5$6qt$1@digitaldaemon.com>, Kris says...
>
>Perhaps there's further misunderstanding here:
>
>"Dave" <Dave_member@pathlink.com> wrote in
>>
>> "Kris" <fu@bar.com> wrote ...
>>> Another approach to implementation hiding is to simply strip out existing
>>> method bodies and leave the bare declarations, along with any enums
>>> and/or
>>> typedefs they may use. Many requests have been made for an automated tool
>>> to do exactly this: which would also work well for the Interface approach
>>> noted above. The end result would be the same in both cases ~ a secondary
>>
>
>I've no intent on causing discomfort to you or anyone here, but I really have to say this feature appears to have been given not /nearly/ sufficient thought. It fails to support industrial-level implementation-hiding (the one place it is needed) and it doesn't push the performance envelope very hard at all.
>

No discomfort at all - Everyone has their opinions. This obviously doesn't have to be the be-all and end-all to the 'header', symbol file, what-have-you functionality, and I don't think it was ever intended as such, certainly not by me.

At the very least it brought out the discussion -- included in my earlier post was a link to an older request for functionality and no-one replied, so I implemented what seemed to make the most sense and what I had time for, at least as a *first step*.

I would applaud anyone (Kris?, Derek?) who would take what's there and provide a better solution -- plain-and-simple fact is I've spent all the time I can on it.

- Dave

"JT" <JT_member@pathlink.com> wrote in message news:dp2ins$1gn3$1@digitaldaemon.com...
> And I agree - the pretty printing is usefull to have in the front end. I
> noticed
> while playing around in the front end that you have stubbed out the
> posibility
> of using 'precompiled symbol files' as imports - do you still have plans
> for
> this and do you expect it to be in version 1?

.di files replace precompiled symbol files.

In article <dp50o3$bt6$1@digitaldaemon.com>, Walter Bright says...
>
>
>"JT" <JT_member@pathlink.com> wrote in message news:dp2ins$1gn3$1@digitaldaemon.com...
>> And I agree - the pretty printing is usefull to have in the front end. I
>> noticed
>> while playing around in the front end that you have stubbed out the
>> posibility
>> of using 'precompiled symbol files' as imports - do you still have plans
>> for
>> this and do you expect it to be in version 1?
>
>.di files replace precompiled symbol files.
>
>

very cool. .di files make much more sense now, in fact I think its a very good solution for this part. Thanks for everything you do Walter.

"Dave" <Dave_member@pathlink.com> wrote
> No discomfort at all - Everyone has their opinions.

I'm very glad you feel that way.


> This obviously doesn't have
> to be the be-all and end-all to the 'header', symbol file, what-have-you
> functionality, and I don't think it was ever intended as such, certainly
> not by
> me.

Hey; it's tough to please anyone these days :)


> At the very least it brought out the discussion -- included in my earlier
> post
> was a link to an older request for functionality and no-one replied, so I
> implemented what seemed to make the most sense and what I had time for, at
> least
> as a *first step*.
>
> I would applaud anyone (Kris?, Derek?) who would take what's there and
> provide a
> better solution -- plain-and-simple fact is I've spent all the time I can
> on it.


It's certainly refreshing to hear such down-to-earth statements. Unfortunately, it's proven nigh impossible to get fixes into the darned library (phobos). It'll be harder still to get fixes/changes into the front-end. Perhaps you'll share your secret :)

I have to say, though, that's there's a fundamental flaw exposed by trying to please two masters: you can't do implementation-hiding correctly whilst exposing partial-implementation and private attributes. And you can't provide for faster-compilation without doing so. While I personally admire you tremendously for taking the initiative (no mean feat), I have to take serious issue with the imp-hiding aspect, since it has long-term ramifications.

Regards;

Kris wrote:
> First, I must point out that this branch is about "implementation hiding" ~ it say so in the topic :)

Sorry :-)  I suppose I read too much into some of the text I omitted.

>> Another issue is that header files are touched far less frequently than implementation files.  Thus, implementation changes can be made and so long as the header remains unchanged the compiler doesn't have to rebuild dependent modules.  In large projects, this can be of tremendous value. To this end, it might be nice if DMD only touched header files for nonzero deltas (so timestamps only changed if the file contents changed).
> 
> In C, yes. D currently does not have any such issues.

How so?  I'll admit I haven't looked into how the need for recompilation is detected.

>> I see implementation hiding as a separate (though related) QOI issue. Typically, I want to see readable header files and let things get mangled in post-production before the CD is sent out to clients.
> 
> Ack ~ this is not about mangled headers!

I had a few things going through my head and unfortunately only one of them made it out :-)  I was thinking of implementation hiding in .NET where mangling is pretty much necessary.  So what would be fitting then?  The ability to generate pure headers with no inlined function calls and no private data?

>> I think it's important to separate QOI features from "standard" features. Will it be necessary for all D compilers to have code coverage or profiling tools?  Of course not.  Header generation is in the same category.
> 
> Is it? If, just if, people start using -H in an attempt at implementation-hiding, then other compilers will have to follow suit. After all, code will be written to take advantage of those headers (the whole point of implementation-hiding), and thus will be dependent on the way the information is exposed. Thus, once we start down that road others will have to comply.

I'm not sure I understand.  The -H option must necessarily generate headers that contain much of the same symbolic information as the original module or the header will not be usable.  Hrm... perhaps instead of containing private data for alignment purposes, a pragma could be used to indicate data size?  ie.

class C {
private:
    int x;
    int y;
}

// translates to

class C {
pragma( pad, 8 );
}