I don't know that much about compilers, and I've been wondering why is
the linker necessary. Is it really necessary to first compile the code into
an
.obj file and then link it with the .lib files and other .obj files in the
program? I believe that the source code of a program has the necessary
information to build an executable, except for DLLs and other Libraries but
that can be implemented like Borland did in Delphi.

what about compiled modules instead of .obj files, imported with the "import" keyword.

library function calls can be implemented like this...
extern(Windows, "dllname.dll") name(pars);

it would make the compilation process extremely easy. you could just type:

dmc main.d (-flags...) to compile a program with multiple modules and
library calls instead of
dmc module1.d module2.d module3.d main.d library1.lib library2.lib
library3.lib .. etc.

these are just some ideas.

Good idea.  Internally there will be a link step, otherwise separate compilation and incremental compilation would never work.  But we shouldn't have to deal with it.

Along with that comes some other stuff:  A module has to be a file, the file has to be in the right place, someplace very predictable so it can find it. Maybe modules could "incorporate" some other module, which would then extend it into multiple files.  This is like public imports.

This could slow compilation down somewhat.  Obviously it would be more than made up for by not having to parse text headers.  But in order to discover that in order to build module A, it must first build module B, but by the time it finds out it needs B, it has already started compiling A.  It would not want to discard the useful work it has done on A, so it chains out and compiles be and comes back and finishes compiling A.  But that means you could run low on memory as the compiler crawls thru potentially hundreds of files figuring out what to do.  Once it's finished it should definitely write out dependency files so it knows whether anything needs recompiled without having to do the whole thing again.

Anyway I would love to be spared the details of the linking.  ;)

Sean

"Giancarlo Bellido" <vicentico1@hotmail.com> wrote in message news:bi41d2$26ip$1@digitaldaemon.com...
> I don't know that much about compilers, and I've been wondering why is the linker necessary. Is it really necessary to first compile the code
into
> an
> .obj file and then link it with the .lib files and other .obj files in the
> program? I believe that the source code of a program has the necessary
> information to build an executable, except for DLLs and other Libraries
but
> that can be implemented like Borland did in Delphi.
>
> what about compiled modules instead of .obj files, imported with the "import" keyword.
>
> library function calls can be implemented like this...
> extern(Windows, "dllname.dll") name(pars);
>
> it would make the compilation process extremely easy. you could just type:
>
> dmc main.d (-flags...) to compile a program with multiple modules and
> library calls instead of
> dmc module1.d module2.d module3.d main.d library1.lib library2.lib
> library3.lib .. etc.
>
> these are just some ideas.

Giancarlo Bellido wrote:
> I don't know that much about compilers, and I've been wondering why is
> the linker necessary. Is it really necessary to first compile the code into
> an
> .obj file and then link it with the .lib files and other .obj files in the
> program? I believe that the source code of a program has the necessary
> information to build an executable, except for DLLs and other Libraries but
> that can be implemented like Borland did in Delphi.
> 
> what about compiled modules instead of .obj files, imported with the
> "import" keyword.
> 
> library function calls can be implemented like this...
> extern(Windows, "dllname.dll") name(pars);
> 
> it would make the compilation process extremely easy. you could just type:
> 
> dmc main.d (-flags...) to compile a program with multiple modules and
> library calls instead of
> dmc module1.d module2.d module3.d main.d library1.lib library2.lib
> library3.lib .. etc.
> 
> these are just some ideas.

Well, once upon a time, compiling was a very lengthy procedure.  It could take HOURS to compile your program AND the source for all the library functions it uses.  If the library functions were precompiled, and then (say) LINKED with your program, then youd only have to compile your source, which is much faster (only 30 minutes or so).

Of course, computers have gotten tens of thousands of times faster since then.  It might not be so impractical to build everything from source each time.

But there's still one concern, that the libraries you use might not be your own.  In this case, the owner of the library might not want you to be able to see the source, but they're happy to sell you a pre-compiled library.

Still... there's dynamic linking these days, too.  Maybe all the pre-compiled libraries would be .DLL (or .so) libraries, and everything else would need to be compiled from source.  Each and every time you change one line in one file of 70, all 70 get recompiled.  Or all 170. Or all 700.

Nah.  I think we should keep the linker, for a while anyway.

Giancarlo Bellido wrote:
> I don't know that much about compilers, and I've been wondering why is
> the linker necessary. Is it really necessary
...

I like to say (just because it's fun) that "The C linker is the Root of All Evil."  Being compatible with it leads to hacked C++ template support.  Most C++ programmers know about name mangling.  Optimization across files still generally sucks (there are some fairly new compilers that globally optimize).

The reason for the linker is mainly historical.  Back in high school, I had a job programming a PDP-11/45.  It had a whopping 128K of core memory (actual magnetic cores used for each bit).  We shared it between 8 users at a time.  My max memory allocation was 16K bytes.

The compilers had to run off disk, in little pieces.  We had "4 pass compilers", which were slower than "3 pass compilers."  That meant that the source code had to be read 3 times instead of 4, which was the bottleneck in compilation speed.

After creating all the obect files, the linker ran.  That was always fun.  We had these washing machine sized hard disks, and the linker sensitized an oscillation mode.  Whenever we linked, one of these huge things would rock back and forth making a terrible niose, and then start walking around the room!

I guess I haven't been fond of linkers ever since.

Bill

"Giancarlo Bellido" <vicentico1@hotmail.com> wrote in message news:bi41d2$26ip$1@digitaldaemon.com...
> I don't know that much about compilers, and I've been wondering why is the linker necessary. Is it really necessary to first compile the code
into
> an
> .obj file and then link it with the .lib files and other .obj files in the
> program? I believe that the source code of a program has the necessary
> information to build an executable, except for DLLs and other Libraries
but
> that can be implemented like Borland did in Delphi.

if you have a big project incremental compilation saves a lot of time when
making changes
i.e. a make file that compiles d->obj then make libs then link the libs +
program

also there was a limit on command line length so you can't pass the whole project as one command. (Some OS's limit command line length anyway)

>
> what about compiled modules instead of .obj files, imported with the "import" keyword.
>
> library function calls can be implemented like this...
> extern(Windows, "dllname.dll") name(pars);
I did write and post a templated lib that allow almost this behaviour. (explicit loading rather than implicit though)

> it would make the compilation process extremely easy. you could just type:
>
> dmc main.d (-flags...) to compile a program with multiple modules and
> library calls instead of
> dmc module1.d module2.d module3.d main.d library1.lib library2.lib
> library3.lib .. etc.
>
> these are just some ideas.

I guess ppl are all thinking a long similar lines ....

I would also like to be able to do `dmd -build -Imylib -I/shared/otherlib -o myapp.exe main.d` and have the d compiler find and compile all files into myapp.exe

I believe that the D compiler should use obj files in the same way C
compiler use precompiled headers, so there should be a tempary dir (defaults
to c:\dmd\inter\<mangled-path> or similar) which contains the precompiled
Objs the D compiler just has to check that <path>file.d is older than
c:\dmd\inter\<mangled-path>file.obj so work out if it need to read in the d
file or the obj
there is an issue with templates the easiest solution is that the obj file
gets marked in someway to say its incomplete and its original source file
will be read in every time.

the required options are build me an exe, build a shared object (.so/.dll), build me a lib or for backward compatibility `-c` => build me an object file.

this removes the requirement to have make files that contain the dependancy info, or tools that scan the D files looking for imports (he says wound up again 'cos adding a new member function to a class requires all the objects to be rebuild that use it or the prog crashes ... (obviously). and tracking dependacies is a pain).

it would also seem in these days of IDE's that the D compiler should export
an interface to allow it to be intergrated into an IDE
again why write a tool to workout what the D compiler imports, when the D
compiler is doing all that work anyway.
dmd should just be a front end that loads dcompiler.dll (.so) and passes it
the start file etc.

the required interface (C or COM) should allow the following
send a file to D to build in to an exe, .dll/.so /lib or obj
the D compiler then calls back to request source or object (flags to say
either or source only)
and callbacks to store intermedite and final data.
I would invisage that searching and intermediate storage are all processed
in the frontend, so the D compiler does not have to bother with where things
came from just that they are the right thing.

this could be taken a step further (as D is intended to be useable as a
backend lang) to perform jit like (well just ahead of time) compilation. and
if an added callback (DGetProcAddress or sililar was added then a script
could be run and link to the exe that started the dcompiler up).

my vision ....
extern (C) bool DGetDCompiler( out IDCompiler dc ); // in dcompiler.dll
interface IDCompiler : IUnknown {
       enum OutputFormat { RAW, EXE, SHARED, LIB }
        bool beginCompilation( wchar[] filename, int tag, OutputFormat
format, IDCompilerHelper helper );
}
interface IDCompilerHelper : IUnknown {
        enum FileType { ANY = 0xF, SOURCE=1, OBJECT=2, LIB=3 }
        bool getFileItem( wchar[] name, int tag, FileType type, IDCompiler
from, out IDInputStream src );
        bool setFileItem( wchar[] name, int tag, FileType type, IDCompiler
from, IDOutputStream src );
}
// tag values >0 are helper side identifiers, values <0 are compiler
generated tags for items
// this allows you to send beginCompilation( "internal", 1, .... )
// then when item with tag 1 is requested you return generated D rather than
a file.

interface IDStream : IUnknown {
       void close();
        bit valid();
}
interface IDInputStream : IDStream {
       int read( byte[] buf, int max_len );
// returns the number of bytes read <0 == error
}
interface IDOutputStream : IDStream {
       int write( byte[] buf, int len );
  // returns the number of bytes written <0 == error can write less than len
}

Like Walter has said, he's trying to reuse the few resources he has at hand.
That's why he uses the linker.
However, I also think there should be a way for not having those really long
lines when linking. Maybe, in some way, the compiler could look for all the
needed .obj or .lib and pass directly to the linker. I mean, I have 2
versions of the same program: a console mode one and a graphics mode, but
both use the same core. Why should I be compiling each time that core? I
know I can pass just the .obj, but I'd rather just do
dmd -c core.d
dmd textmode.d
dmd guimode.d
And both compilations would automagically link with core.obj too. (I don't
use digc much because I don't know what the effect of that .res file is. I
only use it for compiling dig programs.)
And that idea of having some sort of a central repository could be really
useful too.

————————————————————————— Carlos Santander


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.512 / Virus Database: 309 - Release Date: 2003-08-19

"Richard Krehbiel" <rich@kastle.com> wrote in message news:bi4u6j$gld$1@digitaldaemon.com...
> Well, once upon a time, compiling was a very lengthy procedure.  It could take HOURS to compile your program AND the source for all the library functions it uses.  If the library functions were precompiled, and then (say) LINKED with your program, then youd only have to compile your source, which is much faster (only 30 minutes or so).
>
> Of course, computers have gotten tens of thousands of times faster since then.  It might not be so impractical to build everything from source each time.
>
> But there's still one concern, that the libraries you use might not be your own.  In this case, the owner of the library might not want you to be able to see the source, but they're happy to sell you a pre-compiled library.

A more basic point is that object modules can be created by different compilers/assemblers. It's no help to see the source if it isn't buildable with your (single) tool.

Sam