On 2016-07-22 10:28, Dicebot wrote:

> .. which naturally leads to watching about Benjamin DConf talk about
> fixing "export" and that is where everything clicks together. Organizing
> large projects as bunch of small static libraries per package and
> defining public API of those via `export` (and not just public) would
> achieve this topic goal and much more, all without changing the language.

How does this relate to templates? Or is the suggestion to now use templates on API boundaries?

-- 
/Jacob Carlborg

On Wednesday, 20 July 2016 at 19:59:42 UTC, Jack Stouffer wrote:
> I concur. If the root problem is slow compilation, then there are much simpler, non-breaking changes that can be made to fix that.

I don't think compilation time is a problem, actually. It more has to do with dependency management and encapsulation.

Speeding up compilation should never be considered as an acceptable solution here, as it's not scalable: it just pushes the problem away, until your project size increases enough.

Here's my understanding of the problem:

// client.d
import server;
void f()
{
  Data x;
  // Data.sizeof depends on something in server_private.
  x.something = 3; // offset to 'something' depends on privateStuff.sizeof.
}

// server.d
private import server_private;
struct Data
{
  Opaque someOtherThing;
  int something;
}

// server_private.d
struct Opaque
{
  byte[43] privateStuff;
}

I you're doing separate compilation, your dependency graph has to express that "client.o" depends on "client.d", "server.d", but also "server_private.d".

GDC "-fdeps" option properly lists all transitive imported files (disclaimer: this was my pull request). It's irrelevant here that imports might be private or public, the dependency is still here.

In other words, changes to "server_private.d" must alway trigger recompilation of "client.d".

I believe the solution proposed by the OP doesn't work, because of voldemort types. It's always possible to return a struct whose size depends on something deeply private.

// client.d
import server;
void f()
{
  auto x = getData();
  // Data.sizeof depends on something in server_private.
  x.something = 3; // offset to 'something' depends on privateStuff.sizeof.
}

// server.d
auto getData()
{
  private import server_private;

  struct Data
  {
    Opaque someOtherThing;
    int something;
  }

  Data x;
  return x;
}

// server_private.d
struct Opaque
{
  byte[43] privateStuff;
}

My conclusion is that maybe there's no problem in the language, nor in the dependency generation, nor in the compiler implementation.
Maybe it's just a design issue.

On Sun, 24 Jul 2016 12:53:50 +0000, Sebastien Alaiwan wrote:
> Speeding up compilation should never be considered as an acceptable solution here, as it's not scalable: it just pushes the problem away, until your project size increases enough.

In order to get an equivalent speedup by refactoring, I need small modules. Probably no more than a handful of functions or one type. I also need to ensure that my types are as simple as possible -- free functions instead of methods, just so I can put them into different modules.

That's busywork for me and an inconvenience for anyone who needs to import anything I wrote.

Look at std.algorithm. Tons of methods, and I imported it just for `canFind` and `sort`.

Look at std.datetime. It imports eight or ten different modules, and it needs every one of those for something or other. Should we split it into a different module for each type, one for formatting, one for parsing, one for fetching the current time, etc? Because that's what we would have to do to work around the problem in user code.

That would be terribly inconvenient and would just waste everyone's time. There is no reason to do it when the compiler could do it for us automatically.

On Saturday, 23 July 2016 at 19:22:09 UTC, Jacob Carlborg wrote:
> How does this relate to templates? Or is the suggestion to now use templates on API boundaries?

Benjamin proposed to stop considering `export` a protection attribute and mark all functions called from templates as `export` (allowing `export private` if necessary).

On 2016-07-25 01:12, Dicebot wrote:

> Benjamin proposed to stop considering `export` a protection attribute
> and mark all functions called from templates as `export` (allowing
> `export private` if necessary).

Ah, forgot that detail.

-- 
/Jacob Carlborg

On Sunday, 24 July 2016 at 15:33:04 UTC, Chris Wright wrote:
> Look at std.algorithm. Tons of methods, and I imported it just for `canFind` and `sort`.
>
> Look at std.datetime. It imports eight or ten different modules, and it needs every one of those for something or other. Should we split it into a different module for each type, one for formatting, one for parsing, one for fetching the current time, etc? Because that's what we would have to do to work around the problem in user code.
>
> That would be terribly inconvenient and would just waste everyone's time.

I agree with you, but I think you got me wrong.

Modules like std.algorithm (and nearly every other, in any standard library) have very low cohesion. As you said, most of the time, the whole module gets imported, although only 1% of it is going to be used.

(selective imports such as "import std.algorithm : canFind;" help you reduce namespace pollution, but not dependencies, because a change in the imported module could, for example, change symbol names.)

I guess low cohesion is OK for standard libraries, because splitting this into lots of files would result in long import lists on the user side, e.g:

import std.algorithm.canFind;
import std.algorithm.sort;
import std.algorithm.splitter;

(though, this seems a lot like most of us already do with selective imports).

But my point wasn't about the extra compilation time resulting from the unwanted import of 99% of std.algorithm.

My point is about the recompilation frequency of *your* modules, due to changes in one module.

Although std.algorithm has low cohesion, it never changes (upgrading one's compiler doesn't count, as everything needs to be recompiled anyway).

However, if your project has a "utils.d" composed of mostly unrelated functions, that is imported by almost every module in your project, and that is frequently changed, then I believe you have a design issue.

Any compiler is going to have a very hard time trying to avoid recompiling modules which only imported something in the 99% of utils.d which wasn't modified (and, by the way, it's not compatible with the separate compilation model).

Do you think I'm missing something here?