November 01, 2018
On Thu, Nov 01, 2018 at 10:37:59PM +0000, unprotected-entity via Digitalmars-d-announce wrote:
> On Thursday, 1 November 2018 at 03:10:22 UTC, H. S. Teoh wrote:
> > 
> > Actually, code within a module *should* be tightly coupled and cohesive -- that's the whole reason to put that code inside a single module in the first place.  If two pieces of code inside a module are only weakly coupled or completely decoupled, that's a sign that they should not be in the same module at all.  Or at the very least, they should belong in separate submodules that are isolated from each other.
> 
> How does one determine, whether a 10,000 line module, is tightly coupled and cohesive?
> 
> Only the author can make that statement - which they naturally will, even if it's not true.
> 
> An outsider, seeking to verify that statement, has a hell of a job on their hands...(I for one, think code smell immediately).

The code reviewer can reject a 10,000-line module off the bat as being too large.  It's up to the project to enforce such conventions.


> As soon as you see a class interface in a module, in D, you have to assume their is other code in the module, perhaps down around line 9,900, that is bypassing its interface, and doing who knows what to it....
> 
> Sure, the author might be happy, but an auditor/code reviewer, will likely have a different view, when a 10,000 line module is shoved in front of them, and they know, that 'anything goes', 'all bets are off', inside the D module....

Yes, and this is when you, or the project manager, puts down the foot and say this is unacceptable, break this module up into smaller submodules or else it doesn't go into the master repository.  Simple.


[...]
> I don't use a particular language. I'm more interested in design and architecture.
> 
> In the age of 'lock-down-everything', increased modularity is becoming more important. A monolithic module approach, is already outdated, and risky, in terms of developing secure, maintainable software....

I don't understand what D's approach to modules has to do with being monolithic.  Why can't you just write smaller modules if private being module-wide bothers you so much?  D has package.d that can make a set of smaller submodules appear as a larger module to outside code.  Use it.


> I think providing an additional tool, to those who seek to use D, such as 'strict private' (syntax can be argued about), would aid better design - it can't make it any worse, that's for sure).
> 
> Although. I don't mean strict private like freepascal, but strict private, as in it inherits everything that private already does, but additionally, become private within the module too.
> 
> Is that really such a bad idea? Are there no programmers out there in the D world that might think this could be a good, additional tool, to give programmers, so they can better architect their solution?

It's not a bad idea. In fact, if I were in charge of designing D, I'd probably do the same.  But D chose to do it differently -- and part of the rationale, if I understand it correctly, is to eliminate the need for `friend` declarations like in C++.  But maybe Andrei or whoever it was that made this decision can step up and explain why it was designed this way.

All we're saying is that it's not the end of the world if private is module-wide rather than aggregate-wide.  You can still have your encapsulation by splitting up your code into smaller files, which, given what you said above, you want to be doing anyway since the idea of a 10,000-line source file is so abhorrent to you.

The fundamental issue here is that there needs to be some kind of unit of encapsulation.  Java and C# chose to make the class that unit; D chose the module.  If the Java/C# unit of encapsulation is what you want, all you have to do is to make the module equivalent to the class, problem solved.  It's not as though there is no encapsulation at all and you're left out in the wild west of C's global namespace, or that the workaround is too onerous to be practical.


> The amount of push back in the D community on this idea, is really odd to me. I'm still trying to understand why that is. Are D programmers just hackers, insterested in getting their code to work, no matter what? Are their not enough Java/C# programmers coming to D - and bringing their design skills with them?

It's not so much push back, as being worn out from every other newcomer clamoring for the same thing year after year without ever having tried to do it the D way.  Not saying that you're doing that, but it just becomes a sort of knee-jerk reaction after X number of newcomers barge in repeating the same old complaints that has the same old answers that everyone is tired of repeating.

Having said that, though, there are some here who *do* want something like what you describe... IIRC Manu has voiced this before, and there may be others. (I myself don't consider it a big enough issue to be worth agonizing over.)  So far, there has yet to be a strong enough argument for per-aggregate private to convince Walter & Andrei.  It's them you have to convince, not the rest of us.  Even if we were to all agree with you, it doesn't mean squat if Walter and Andrei won't budge on the issue.

And they are unlikely to budge as long as (1) there is a perfectly workable solution to the problem -- in this case, split up your code into submodules, and use package.d to group them into a logical parent module if you wish; and (2) the proposed language addition/change brings strong enough benefits to offset the technical debt of yet another complication added to the language.

(Personally, I think they're a bit too conservative on changes that would ostensibly improve the language. But again, what I think means squat -- they are the decision-makers, not me.)


T

-- 
Which is worse: ignorance or apathy? Who knows? Who cares? -- Erich Schubert
November 02, 2018
On Wednesday, 31 October 2018 at 10:48:24 UTC, Sebastien Alaiwan wrote:
>
> "Encapsulation" is about implementation hiding and access control ("public/private"), and requires programming language support (e.g most dynamic languages don't have it).

"Encapsulation is sometimes referred to as the first pillar or principle of object-oriented programming. According to the principle of encapsulation, a class or struct can specify how accessible each of its members is to code outside of the class or struct. Methods and variables that are not intended to be used from outside of the class .. can be hidden to limit the potential for coding errors or malicious exploits."

https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/

D simply has *zero* support for encapsulation of classes or structs, within the module (only encapsulation from code that is outside the module).

Any programmers interested in enforcing encapsulation in D (the first pillar of OOP), are *forced* (because the language doesn't provide the tool to do anything else) to use the one class, one struct, per file solution. Which seems really silly to me. D forces you into Java like programming - just to encapsulate a little class or struct.

Speaking of 'structs', I don't see anyone in the D community, telling others to use 'one struct per module'.


November 01, 2018
On Fri, Nov 02, 2018 at 12:25:21AM +0000, unprotected-entity via Digitalmars-d-announce wrote: [...]
> "Encapsulation is sometimes referred to as the first pillar or principle of object-oriented programming. According to the principle of encapsulation, a class or struct can specify how accessible each of its members is to code outside of the class or struct. Methods and variables that are not intended to be used from outside of the class .. can be hidden to limit the potential for coding errors or malicious exploits."
>
> https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/

That is a narrow definition of encapsulation that is very OO-centric, and IMO, it sorta misses the larger point behind encapsulation, focusing instead on the mechanics of it, and that the specific implementation of it in the OO paradigm.  The concept of encapsulation goes beyond OO and classes; it has to do with modularity and well-defined interfaces between modules (or units of encapsulation -- this is not necessary the same thing as a D module).

It's fine to explain how OO's concept of classes and methods implement encapsulation, but to *define* encapsulation in terms of classes and methods is IMO missing the forest for the trees.  If *that's* your basis for understanding encapsulation, it would it explain a lot of your reactions to this thread.  But there is more to the world than the narrow realm of the OO paradigm, and in a multi-paradigm language like D, it doesn't seem to make sense to be handicapped by a definition of encapsulation that really only makes sense in the context of an OO language.

What encapsulation is really about, is the ability to modularize your code into self-contained units which have well-defined interfaces, with private code and data that cannot be accessed outside of that unit.  In OO languages, this unit would be the class.  But in D, it's the module. That doesn't make D non-encapsulated; it just means we're using a different unit than an OO language.  It's just a difference between pounds and dollars, not some fundamental discrepancy.  You just have to convert one currency to another, that's all.


> D simply has *zero* support for encapsulation of classes or structs, within the module (only encapsulation from code that is outside the module).
> 
> Any programmers interested in enforcing encapsulation in D (the first pillar of OOP), are *forced* (because the language doesn't provide the tool to do anything else) to use the one class, one struct, per file solution. Which seems really silly to me. D forces you into Java like programming - just to encapsulate a little class or struct.
> 
> Speaking of 'structs', I don't see anyone in the D community, telling others to use 'one struct per module'.

Because we love UFCS, and structs just lend themselves very well to that sort of usage.

And along that line, recent wisdom is that it's better to move things *out* of classes (and structs) if they don't need access to private members. (Sorry, I wanted to include a link for this, but I couldn't find the article -- the title eludes me and google isn't turning up anything useful.)  Class and struct APIs should be as minimal as possible -- just enough to do what needs to be done and no more, and the rest of the syntactic sugar (that makes it more palatable to your users) belongs outside as optional convenience functions.


T

-- 
There is no gravity. The earth sucks.
November 02, 2018
On Thursday, 1 November 2018 at 23:58:15 UTC, H. S. Teoh wrote:
>
> Having said that, though, there are some here who *do* want something like what you describe... IIRC Manu has voiced this before, and there may be others. (I myself don't consider it a big enough issue to be worth agonizing over.)  So far, there has yet to be a strong enough argument for per-aggregate private to convince Walter & Andrei.  It's them you have to convince, not the rest of us.  Even if we were to all agree with you, it doesn't mean squat if Walter and Andrei won't budge on the issue.

Again, I feel it's Walter and Andrei that should be doing the convincing.. they are always silent on this matter, which is a real shame,as I think their insights might actually be useful ;-)

If it's simply not practical, from a language design, to consider this further, then I'd like to know that - I work at a higher leve of abstraction and have no idea about this. So not hearing from Walter or Andrei, is really unhelpful, as to whether it's an idea worth pursuing, or an idea that Walter and Andrei would never accept - which is it?

Anyway, the real 'problem' (in my opinion), is not the idea. The idea is sound (or at least, it's not unsound), and it's represented in all major langauges, already.

The real problem, is that whenever this idea pops up (as it has, and will continue to do so), far too many people on this forum attempt to divert the discussion from 'what is the usefulness of this new idea' to 'what's the difficulty of finding ways to avoid any change.'

The discussion can never move forward under those circumstances.

I would like the discussion to move forwards, and people start to think about how such a change could be adopted, how difficult it would be to implement, how will it play with other features in the langauge..etc....

It's a real shame we can't get beyond "No. We don't like your idea, and even if we did, we don't want to do it".


November 02, 2018
On Thu, 01 Nov 2018 22:37:59 +0000, unprotected-entity wrote:
> On Thursday, 1 November 2018 at 03:10:22 UTC, H. S. Teoh wrote:
>>
>> Actually, code within a module *should* be tightly coupled and cohesive -- that's the whole reason to put that code inside a single module in the first place.  If two pieces of code inside a module are only weakly coupled or completely decoupled, that's a sign that they should not be in the same module at all.  Or at the very least, they should belong in separate submodules that are isolated from each other.
> 
> How does one determine, whether a 10,000 line module, is tightly coupled and cohesive?

You can get a pretty accurate result by just saying "no". 10KLOC in one module is almost always a problem.

Failing that, you can do a code review. Which will take a very long time. The solution to that is to submit less code at a time, review as you go, and keep modules smaller.

For instance, Phobos keeps most modules below 3000 lines of code, including unittests. The largest module, std.datetime.systime, has about 6000 lines of code -- but if you exclude unittests and test-only code, it's more like 1000.

> Only the author can make that statement - which they naturally will, even if it's not true.

I think a lot of programmers are well aware of the failings of their code.

> As soon as you see a class interface in a module, in D, you have to assume their is other code in the module, perhaps down around line 9,900, that is bypassing its interface, and doing who knows what to it....

And so you have a rule against casting from an interface to a concrete type, if that's a thing that worries you. It's something you can check rather easily in a reasonably sized code review.

> In the age of 'lock-down-everything', increased modularity is becoming more important. A monolithic module approach, is already outdated, and risky, in terms of developing secure, maintainable software....

That statement could be taken as being against an approach that recommends structuring a project as a monolithic module, or against an approach that treats modules as a monolith in terms of protection.

> I think providing an additional tool, to those who seek to use D,
> such as 'strict private' (syntax can be argued about), would aid better
> design - it can't make it any worse, that's for sure).

It would be more language complexity in order to make it easier to have large modules. But you already said that large modules are a problem.

> Is that really such a bad idea? Are there no programmers out there in the D world that might think this could be a good, additional tool, to give programmers, so they can better architect their solution?

The use case is when you don't want to break up a module and you don't trust yourself not to modify private members from the wrong parts of the module.

That's not useless.

It's also not obviously so useful as to merit inclusion. A lot of languages do without any notion of private. A lot, like the entire ALGOL family up to Oberon-2, Go, Rust, Lua, Haskell, and Node.js, use exported and unexported symbols instead, and that's per module. A fair number just don't have a notion of public and private symbols.

> The amount of push back in the D community on this idea, is really odd to me. I'm still trying to understand why that is. Are D programmers just hackers, insterested in getting their code to work, no matter what? Are their not enough Java/C# programmers coming to D - and bringing their design skills with them?

There are plenty of language designers that didn't think it obvious. Might be better to consider why instead of implying that no D programmers are familiar with or care about good design. I mean, if there are popular languages from the 1960s through the 2010s that do things the same way as D, that sounds like a pretty good indication that it's not an obviously bad idea. It's not rock-solid; actual evidence from the industry would be superior. But I think you would have presented that evidence already.
November 02, 2018
On Friday, 2 November 2018 at 00:53:52 UTC, H. S. Teoh wrote:
>
> And along that line, recent wisdom is that it's better to move things *out* of classes (and structs) if they don't need access to private members. (Sorry, I wanted to include a link for this, but I couldn't find the article -- the title eludes me and google isn't turning up anything useful.)  Class and struct APIs should be as minimal as possible -- just enough to do what needs to be done and no more, and the rest of the syntactic sugar (that makes it more palatable to your users) belongs outside as optional convenience functions.
>

Maybe you are thinking of the "Prefer non-member non-friend functions to member functions" rule from Herb Sutter's "Effective C++" books?


November 02, 2018
On Fri, Nov 02, 2018 at 10:18:11AM +0000, ShadoLight via Digitalmars-d-announce wrote:
> On Friday, 2 November 2018 at 00:53:52 UTC, H. S. Teoh wrote:
> > 
> > And along that line, recent wisdom is that it's better to move things *out* of classes (and structs) if they don't need access to private members. (Sorry, I wanted to include a link for this, but I couldn't find the article -- the title eludes me and google isn't turning up anything useful.)  Class and struct APIs should be as minimal as possible -- just enough to do what needs to be done and no more, and the rest of the syntactic sugar (that makes it more palatable to your users) belongs outside as optional convenience functions.
> > 
> 
> Maybe you are thinking of the "Prefer non-member non-friend functions to member functions" rule from Herb Sutter's "Effective C++" books?
[...]

Ah yes, that's the one.  Thanks!


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond
November 02, 2018
On Friday, 2 November 2018 at 05:29:39 UTC, Neia Neutuladh wrote:
>It's also not obviously so useful as to merit inclusion.

No. I don't say it merits inclusion.

I do say it merits discussion, as to its merits.

But from what I see, so far, is D, Go, Rust...they are seem to have a love affair with C like programming, but are to afraid to call it C.

So they call it... modules. ... they're the wild west, where anything goes....(sorry, that thought had to come....just playing too much RDR2 at the moment...and as much fun as it looks, gee I'm glad we have the law now...well.. not always..)

November 03, 2018
On Thursday, 1 November 2018 at 22:37:59 UTC, unprotected-entity wrote:
> On Thursday, 1 November 2018 at 03:10:22 UTC, H. S. Teoh wrote:
>>
>> Actually, code within a module *should* be tightly coupled and cohesive -- that's the whole reason to put that code inside a single module in the first place.  If two pieces of code inside a module are only weakly coupled or completely decoupled, that's a sign that they should not be in the same module at all.  Or at the very least, they should belong in separate submodules that are isolated from each other.
>
> How does one determine, whether a 10,000 line module, is tightly coupled and cohesive?


You take a look through it and make a judgement.

> Only the author can make that statement - which they naturally will, even if it's not true.

?

> An outsider, seeking to verify that statement, has a hell of a job on their hands...(I for one, think code smell immediately).

There is a basic question in life.  Do you believe in discernment (if possible informed by data) or are you someone who makes decisions on the basis of evidence and believes that anything else is completely arbitrary and nothing more than a matter of opinion.

My impression is that the values of the D community tend more in the direction of recognising the importance of discernment. If someone is somebody who believes more in 'evidence', policy and rules then it probably isn't going to be satisfying expecting one's values to be shared on a wide scale here.

People also differ in their working memory and the degree to which they naturally think associatively.  Chopping up everything into small pieces favours those who have a smaller working memory and who think more analytically but it's a disadvantage for those who have a large working memory and think associatively.  For a private project that's something to be resolved between the relevant people, but I don't think it's reasonable to say that large files per se are wrong, just because they aren't your cup of tea personally.

I think that lots of things seem clear in theory but the difference between theory and practice is often quite large.

In practice Phobos is very readable.

And on the other hand, I have seen an experienced and capable C# programmer struggle to understand an intranet site written in the approved way by an experienced person who was well-trained at Microsoft.  I couldn't understand it either so I concatenated all the little itty bitty files, pulled out the data structures and then it was easy.

> I don't use a particular language. I'm more interested in design and architecture.

Can one really speak of that kind of design in the abstract ?  Language features shape your choices and lead to large shifts in the optimum.  If you don't have design by introspection as a possibility you are going to pick something else.

> In the age of 'lock-down-everything', increased modularity is becoming more important. A monolithic module approach, is already outdated, and risky, in terms of developing secure, maintainable software....

If you can't understand the program does that make you more or less secure?  Security requires also to understand the behaviour of the system as a coherent whole.  I think D programs are pretty easy from that perspective.

> Is that really such a bad idea? Are there no programmers out there in the D world that might think this could be a good, additional tool, to give programmers, so they can better architect their solution?

Burden of proof is on you to write a DIP and make an argument for it.  I am not sure you would find it easier to get a change into C++.  Look at how difficult it is for Walter sometimes ; and he has just a little earned credibility.  Same thing for Guido - he had such little fun with a recent PEP he decided to retire from BDFL of python.

> The amount of push back in the D community on this idea, is really odd to me. I'm still trying to understand why that is.

Persuading people isn't easy even if it's a good idea.  Look at the pushback from C++ over static if.  They crippled it when they finally did relent.  It's a bit entitled to think that if you can't persuade people without having written a DIP that it's them not you!

> Are D programmers just hackers, insterested in getting their code to work, no matter what? Are their not enough Java/C# programmers coming to D - and bringing their design skills with them?

Would you mind explaining why you think that people from mass communities have design skills by virtue of having come from a mass community?  Walter and Andrei are just a little bit known for their design capabilities so the bar is quite high.  I think it's possible you might have things topsy turvy.

Making D code work is rarely a problem.

Every nation has its own strengths and weaknesses and the same is true of language communities.

Having worked with D professionally since 2015 and with a decent size codebase in relation to what I have heard from others, my experience of working with D programmers is that I have never had to say please don't make this dirty hack.  Whereas you know you could copy memory just this once, please don't bother making this bit fast, I really think we can just make a start on a prototype before doing a full design, do we really need to refactor this part of the code now - those things whilst not common are more frequent sources of disagreement with D programmers.

So like I say I think you have it upside down.  The D programmers I work with have a strong focus on design and doing things in the right way.  Take a look at Atila Neves work on excel-d or dpp.  It's a model of clean code.  There are some outstanding Java and C# programmers but they are much sparser than in the D community.

I spoke to the then CTO of the company where I am now involved.  I was telling him about my project and he said ooh you want to spend a couple of weeks mapping out your class hierarchy.  I said I don't have any classes.  Well I did have two, but I took out one and John Colvin got rid of the other.  The expression on his face was priceless.

You can write D like Java or C# if you like.  Depends what you are doing - you certainly don't need to, and many people who have been using D for a while often find writing code using structs much better for their goals and given their values.

When one encounters a new idea that's unfamiliar sometimes it's easy to think that because it's unfamiliar it must be unsound.  That can be a mistake.  It might be better to suspend judgement for a while and keep an open mind.

November 02, 2018
On 11/02/2018 03:18 AM, ShadoLight wrote:

> Maybe you are thinking of the "Prefer non-member non-friend functions to
> member functions" rule from Herb Sutter's "Effective C++" books?

Scott Meyers.

Ali