Jump to page: 1 2 3
Thread overview
proposal: private module-level import for faster compilation
Jul 20, 2016
Timothee Cour
Jul 20, 2016
Dicebot
Jul 21, 2016
Kagamin
Jul 21, 2016
Kagamin
Jul 22, 2016
Jacob Carlborg
Jul 22, 2016
Kagamin
Jul 22, 2016
Dicebot
Jul 23, 2016
Jacob Carlborg
Jul 24, 2016
Dicebot
Jul 25, 2016
Jacob Carlborg
Jul 20, 2016
ketmar
Jul 20, 2016
Jack Stouffer
Jul 20, 2016
ketmar
Jul 20, 2016
Timothee Cour
Jul 20, 2016
ketmar
Jul 20, 2016
ketmar
Jul 20, 2016
Timothee Cour
Jul 20, 2016
Dicebot
Jul 20, 2016
ketmar
Jul 20, 2016
deadalnix
Jul 20, 2016
ketmar
Jul 20, 2016
Jack Stouffer
Jul 21, 2016
Johnjo Willoughby
Jul 24, 2016
Sebastien Alaiwan
Jul 24, 2016
Chris Wright
Jul 25, 2016
Sebastien Alaiwan
July 20, 2016
currently, top-level imports in a module A are visible by other modules B importing A, and are visited (recursively) during compilation of A, slowing down compilation and increasing dependencies (eg with separate compilation model, a single file change will trigger a lot of recompilations).

I propose a private import [1] to mean an import that's only used inside function definitions, not on the outside scope. It behaves exactly as if it the import occurred inside each scope (function and template definitions). This is applicable for the common use case where an import is only used for symbols inside functions, not for types in function signature.

----
module A;
private import util;
void fun1(){
// as if we had 'import util;'
}

void fun2(){
// as if we had 'import util;'
}

// ERROR: we need 'import util' to use baz in function declaration
void fun3(baz a){}

----
module util;
void bar(){}
struct baz{}
----
module B;
import A;
----

The following should not list 'util' as a dependency of B, since it's a
'private import'
dmd -c -o- -deps A.d


The benefits: faster compilation and recompilation (less dependencies).

NOTE [1] on syntax: currently private import just means import, we could use a different name if needed, but the particular syntax to use is a separate discussion.


July 20, 2016
I think this is a wrong approach patching a problem instead of fixing it. Real solution would be to improve and mature .di header generation and usage by compilers so that it can become the default way to import packages/libraries.
July 20, 2016
i can't see what problem this thing is trying to solve.

did you ever measured time taken by building AST of imported module?

use separate compilation and/or templates to avoid codegen. problem solved.
July 20, 2016
On Wednesday, 20 July 2016 at 07:45:12 UTC, Timothee Cour wrote:
> ...

This, and function local imports, are hacks around the actual problem: the compilers spending time on codegen on things your program will never use.

IIRC compiler also spends extra work on templates because it doesn't cache the result, so things like isInputRange!(string) could be evaluated hundreds of times for your program.

Fixing those two things are the actual solution.


July 20, 2016
On Wednesday, 20 July 2016 at 17:05:11 UTC, Jack Stouffer wrote:
> IIRC compiler also spends extra work on templates because it doesn't cache the result, so things like isInputRange!(string) could be evaluated hundreds of times for your program.

it does cache that (see template merging), it even causing some bugs. yet it is using linear search to find something in cache.
July 20, 2016
this simple example shows this feature would provide a 16X speedup.

time dmd -c -o- -version=A -I$code main.d
0.16s

time dmd -c -o- -version=B -I$code main.d
0.01s


---main.d:
module tests.private_import.main;
import tests.private_import.fun;
void test(){}
---

---fun.d:
module tests.private_import.fun;
version(A) import std.datetime;
//version(C) private import std.datetime;
void foo(){
// same as version(C) if this feature were implemented
version(B) import std.datetime;
}
---


July 20, 2016
On Wednesday, 20 July 2016 at 18:09:06 UTC, Timothee Cour wrote:
> this simple example shows this feature would provide a 16X speedup.
100 ms speedup in exchange of creating another special case in language? no, thank you, won't buy.

that was exactly what i meant: if we'll look at *real* numbers instead of scales, we'll find that all amazing "speedups" are measured in terms of milliseconds for most projects, and in terms of seconds for 100mb+ projects. breaking language orthogonality for this is something i can't see as improvement. sorry.
July 20, 2016
p.s. the sole improvement in symbol lookup mechanics can speed up the things by several factors, and without breaking the language. current dmdfe symbol/template lookup mechanics is not really fast.
July 20, 2016
On Wednesday, 20 July 2016 at 18:21:46 UTC, ketmar wrote:
> p.s. the sole improvement in symbol lookup mechanics can speed up the things by several factors, and without breaking the language. current dmdfe symbol/template lookup mechanics is not really fast.

If this example weren't enough, here's the other even more compelling argument: speedup up recompilation for makefile-like tools that trigger recompilation when a dependency is modified:

---
module fun1;
import fun2;
void test1(){}
---
module fun2;
version(A) import std.datetime;
//version(proposed_feature) private import std.datetime;
void test2(){}
---

dmd -c -o- -version=proposed_feature -deps fun1.d
would show following dependencies:
fun2

dmd -c -o- -version=A -deps fun1.d
shows following 68 dependencies (68!)

That means that a change in any single dependency would trigger recompilations in many files.

fun2 core.attribute core.bitop core.exception core.internal.string core.internal.traits core.memory core.stdc.config core.stdc.errno core.stdc.inttypes core.stdc.signal core.stdc.stdarg core.stdc.stddef core.stdc.stdint core.stdc.stdio core.stdc.stdlib core.stdc.string core.stdc.time core.stdc.wchar_ core.sys.osx.mach.kern_return core.sys.posix.config core.sys.posix.dirent core.sys.posix.fcntl core.sys.posix.inttypes core.sys.posix.signal core.sys.posix.stdlib core.sys.posix.sys.select core.sys.posix.sys.stat core.sys.posix.sys.time core.sys.posix.sys.types core.sys.posix.sys.wait core.sys.posix.time core.sys.posix.unistd core.sys.posix.utime core.time core.vararg object std.algorithm std.algorithm.comparison std.algorithm.iteration std.algorithm.mutation std.algorithm.searching std.algorithm.setops std.algorithm.sorting std.array std.ascii std.bitmanip std.conv std.datetime std.exception std.file std.format std.functional std.internal.cstring std.internal.unicode_tables std.meta std.path std.range std.range.interfaces std.range.primitives std.stdio std.stdiobase std.string std.system std.traits std.typecons std.typetuple std.uni

July 20, 2016
Same answer : http://forum.dlang.org/post/nmngk8$inm$1@digitalmars.com
« First   ‹ Prev
1 2 3