View mode: basic / threaded / horizontal-split · Log in · Help
January 05, 2007
Re: D, Java? To D or not to D?
Frits van Bommel wrote:
> Alexander Panek wrote:
>> TPJ wrote:

>>> 2) From FAQ: Since D can call C functions, any GUI library with a
>>> C interface is accessible from D. Does it mean that I can access
>>> *any* C library from D? If so, it a great feature!
>> 
>> Yes, any C library. 
> 
> You need to convert C headers to D import modules to use them though.
> It shouldn't be too hard unless too much preprocessor tricks are
> used, but it might be a bit of work for large files.

It's actually a lot easier than it sounds. :-)

And, there's no use converting everything, just convert the things in a 
C header that your program _actually_ uses! (Most people feel an 
inexplicable urge to convert the whole header file "just in case", or 
"later I'm going to need it anyway". Don't.)

Also, if the header file looks like "an exuberant show-off of 
preprocessor wizardry", then you might want to run it thorugh the 
preprocessor, and compare it with the original.
January 05, 2007
Re: D, Java? To D or not to D?
Walter Bright wrote:
> TPJ wrote:
>> 1) In order to run a program written in D I have to compile it on the 
>> target OS.
> 
> I haven't talked about this much, but it is technically feasible to 
> create a D runtime that enables "compile once, run everywhere". It is 
> not necessary to have a VM to do it.

How would you implement a garbage collector in an 
architecture-independent way?

Or are you talking about using version(arch_1) to version(arch_N)? [1]
Because I'm pretty sure the garbage collector has to know some 
architecture-specific things. Start of stack, direction of stack, 
location of static data, threading API (if any), that sort of thing.
So you'd need to perform at least _some_ porting to get it to run on a 
new architecture, if only to tell the GC what its properties are.

Though I guess stack direction could be experimentally determined at 
runtime/install time, as could endianness... Threading API is going to 
be hard to determine in that way, I'd think.


[1] or "version(StackGrowsUp), version(BigEndian), version(HasFeatureX), 
version(HasFeatureY), ..." or something like that.
January 05, 2007
Re: D, Java? To D or not to D?
Frits van Bommel wrote:
> Walter Bright wrote:
>> TPJ wrote:
>>> 1) In order to run a program written in D I have to compile it on the 
>>> target OS.
>>
>> I haven't talked about this much, but it is technically feasible to 
>> create a D runtime that enables "compile once, run everywhere". It is 
>> not necessary to have a VM to do it.
> 
> How would you implement a garbage collector in an 
> architecture-independent way?

I don't think you would.  Rather, I think you'd have a separate runtime 
library for each platform that handled all the necessary back-end tasks, 
but which then called through to the same user-level D program.  This 
would allow you to deal with platform-specific startup requirements, 
memory allocation scheme, executable file layout, etc.  So long as the 
ABI or whatever were sufficiently strict regarding how exception 
handling occurs and a few other details, I think the compiled user code 
would be portable across OSes.  But I'm not sure how this would interact 
with things like calling C libraries built by other compilers.  For 
example, DMD/Linux appears to pad doubles differently than DMD/Win32, so 
how would the D user code know what to do re: pushing the values onto 
the stack?


Sean
January 05, 2007
Re: D, Java? To D or not to D?
Walter Bright wrote:
> TPJ wrote:
> 
>> 1) In order to run a program written in D I have to compile it on the 
>> target OS.
> 
> I haven't talked about this much, but it is technically feasible to 
> create a D runtime that enables "compile once, run everywhere". It is 
> not necessary to have a VM to do it.

1. To run on any OS on one specific processor, either those OSes have to 
have different entry points to the "exe", or there has to be a way of 
probing the OS within "common" code, the result of which would cause a 
jump to an OS specific routine.

2. To run on different processor architectures, (most probably with some 
of the same OSes), the worst case scenario would be that the same OS 
with the same jump target gets invocated on a different processor. This 
would mean that one (er, Walter) would have to invent a series of binary 
instructions that would not cause an (unwanted) exception on any of the 
processors, would not have any harmful side effects, and would 
eventually result in a jump to some processor specific version of the code.

Both of these seem like formidable tasks. OTOH, there's nothing that 
directly says they'd be theoretically impossible. So, in practice, the 
fewer processors we want to support, the less likely this is to be 
impossible. But, the more processors, specifically: for each additional 
processor (of a different architecture philosophy) we want to support, 
the harder it gets. And IMHO exponentially. (In English: pretty hard, 
and it gets _vastly_ harder for each single new processor!)

----

Not right now having the additional patience to think this entirely 
through, I got to thinking about the following:

At first sight ("hearing" ?), it sounds too good to be true. Then one 
starts to suspect that either Walter has figured out a runtime gimmick 
that does it (thus rendering this (btw, patentable idea) independent of 
D), or it is some kind of result of the specific way D and its source 
code (more to the point, syntax domain) relate to the semantics of the 
source code, taken as a whole.

----

If there's any remotest chance that this may be patentable, I DEFINITELY 
don't want to hear a word about it for the next 18 months!

If not, I'd kill to hear more! :-)
January 06, 2007
Re: D, Java? To D or not to D?
Sean Kelly wrote:
> Frits van Bommel wrote:
>> Walter Bright wrote:
>>> TPJ wrote:
>>>> 1) In order to run a program written in D I have to compile it on 
>>>> the target OS.
>>>
>>> I haven't talked about this much, but it is technically feasible to 
>>> create a D runtime that enables "compile once, run everywhere". It is 
>>> not necessary to have a VM to do it.
>>
>> How would you implement a garbage collector in an 
>> architecture-independent way?
> 
> I don't think you would.  Rather, I think you'd have a separate runtime 
> library for each platform that handled all the necessary back-end tasks, 
> but which then called through to the same user-level D program.  This 
> would allow you to deal with platform-specific startup requirements, 
> memory allocation scheme, executable file layout, etc.  So long as the 
> ABI or whatever were sufficiently strict regarding how exception 
> handling occurs and a few other details, I think the compiled user code 
> would be portable across OSes.  But I'm not sure how this would interact 
> with things like calling C libraries built by other compilers.  For 
> example, DMD/Linux appears to pad doubles differently than DMD/Win32, so 
> how would the D user code know what to do re: pushing the values onto 
> the stack?

Oh, I must've read it wrong. I thought this was about a runtime that 
would automagically compile on any platform... <blush>

That's different from "independent of OS" though, which seems to be what 
you're thinking of. ("run everywhere" does not equal "run on any x86". 
Not yet anyway ;) )

So this is about a single compiled form that'll work on any supported 
platform. Well, then it comes down to your definition of VM and 'compiled'.
You'd need some representation of the program that's independent of the 
platform. There are two basic forms of this: some form of bytecode or 
source code.
Since it's "compile once", I find it reasonable to assume there should 
be some compiling involved ;). That would indicate byte code. Unless you 
would care to argue that Winzip is a compiler? :P

So then you have several choices:
- Interpret the bytecode. This is basically the simplest VM possible. 
(Is this what Python does? At the very least, I'm pretty sure it used to 
do this)
- Perform another compilation at runtime: bytecode to machine code. This 
would basically be a JIT-compiling VM.
- Perform above compilation[1] at install time. This is the only option 
I see that doesn't require a VM... (For Java, this is possible through 
the Java bytecode frontend for GCC IIRC)
- {{ Did I miss anything? }}
- Some mixture of the above (such as: interpret uncommon parts, 
runtime-compile the most-ran code)

All options but the first require that the user environment has some 
form of compiler though, which might not be optimal either.


[1]: For this purpose, unpacking the source and performing a normal 
compilation would also work. It might take longer though.
January 06, 2007
Re: D, Java? To D or not to D?
Frits van Bommel wrote:
> Sean Kelly wrote:
>> Frits van Bommel wrote:
>>> Walter Bright wrote:
>>>> TPJ wrote:
>>>>> 1) In order to run a program written in D I have to compile it on 
>>>>> the target OS.
>>>>
>>>> I haven't talked about this much, but it is technically feasible to 
>>>> create a D runtime that enables "compile once, run everywhere". It 
>>>> is not necessary to have a VM to do it.
>>>
>>> How would you implement a garbage collector in an 
>>> architecture-independent way?
>>
>> I don't think you would.  Rather, I think you'd have a separate 
>> runtime library for each platform that handled all the necessary 
>> back-end tasks, but which then called through to the same user-level D 
>> program.  This would allow you to deal with platform-specific startup 
>> requirements, memory allocation scheme, executable file layout, etc.  
>> So long as the ABI or whatever were sufficiently strict regarding how 
>> exception handling occurs and a few other details, I think the 
>> compiled user code would be portable across OSes.  But I'm not sure 
>> how this would interact with things like calling C libraries built by 
>> other compilers.  For example, DMD/Linux appears to pad doubles 
>> differently than DMD/Win32, so how would the D user code know what to 
>> do re: pushing the values onto the stack?
> 
> Oh, I must've read it wrong. I thought this was about a runtime that 
> would automagically compile on any platform... <blush>
> 
> That's different from "independent of OS" though, which seems to be what 
> you're thinking of. ("run everywhere" does not equal "run on any x86". 
> Not yet anyway ;) )

True :)  But it still means quite a bit of portability.  And I could 
have misunderstood as well.  I simply couldn't imagine a binary app 
compiled for, say, x86 (one instruction set, big-endian) actually 
working on PPC (another instruction set, little-endian) without some 
sort of JIT compile step.

> So this is about a single compiled form that'll work on any supported 
> platform. Well, then it comes down to your definition of VM and 'compiled'.
> You'd need some representation of the program that's independent of the 
> platform. There are two basic forms of this: some form of bytecode or 
> source code.

I'm far from an expert here, but I think the bulk of the actual binary 
program is probably consistent across OSes for the same architecture. 
The differences are probably in the executable startup code, the thread 
code, memory allocation, some portions of garbage collection, etc, all 
of which could theoretically be abstracted to a runtime library (in a 
sense, they already are: Phobos).  The issue would be figuring out how 
to go from the command prompt to running the proper version of 
everything.  Would a link step be required first?  Would the runtime 
library actually contain multiple versions of the code for different 
OSes and call the appropriate one dynamically?  Or perhaps the runtime 
could be in a dynamic lib that's loaded by some startup code linked into 
the app?  Note that none of these are Virtual Machine situations because 
the code is running natively in all cases.


Sean
January 06, 2007
Re: D, Java? To D or not to D?
Sean Kelly wrote:
> Frits van Bommel wrote:
>> Oh, I must've read it wrong. I thought this was about a runtime that 
>> would automagically compile on any platform... <blush>
>>
>> That's different from "independent of OS" though, which seems to be 
>> what you're thinking of. ("run everywhere" does not equal "run on any 
>> x86". Not yet anyway ;) )
> 
> True :)  But it still means quite a bit of portability.  And I could 
> have misunderstood as well.  I simply couldn't imagine a binary app 
> compiled for, say, x86 (one instruction set, big-endian) actually 
> working on PPC (another instruction set, little-endian) without some 
> sort of JIT compile step.

x86 is little-endian, PPC can IIRC be set to either, but is typically 
(i.e. on PPC Macs) set to big-endian.

>> So this is about a single compiled form that'll work on any supported 
>> platform. Well, then it comes down to your definition of VM and 
>> 'compiled'.
>> You'd need some representation of the program that's independent of 
>> the platform. There are two basic forms of this: some form of bytecode 
>> or source code.
> 
> I'm far from an expert here, but I think the bulk of the actual binary 
> program is probably consistent across OSes for the same architecture. 

I was talking about multiple architectures here. Sorry if that wasn't 
clear enough.

> The differences are probably in the executable startup code, the thread 
> code, memory allocation, some portions of garbage collection, etc, all 
> of which could theoretically be abstracted to a runtime library (in a 
> sense, they already are: Phobos).  The issue would be figuring out how 
> to go from the command prompt to running the proper version of 
> everything.  Would a link step be required first?  Would the runtime 
> library actually contain multiple versions of the code for different 
> OSes and call the appropriate one dynamically?  Or perhaps the runtime 
> could be in a dynamic lib that's loaded by some startup code linked into 
> the app?  Note that none of these are Virtual Machine situations because 
> the code is running natively in all cases.

Before you consider how the runtime is linked to the app, first consider 
how the app starts in the first place. Let's say the bare minimum is x86 
Linux and Windows compatibility.

I don't think there's currently any way to have it run directly on 
Windows and Linux simply because there's AFAIK no executable format 
that's supported by both OSs (ELF is supported on multiple OSs, but 
Windows doesn't support it). Furthermore, all executable formats I know 
of (except for plain binary, which AFAIK Linux doesn't support[1]) have 
some (unique) signature in the first few bytes, so a file can't be 
multiple of these formats at the same time.

So you'd need to either using some sort of custom loader binary for each 
OS or modify OSs to support a common binary format (i.e. a custom 
loader, but integrated into the OS).


[1]: Windows "supports" these for programs that fit in 64k: .com files 
are plain binaries. They're not really useful for anything but backwards 
compatibility though.
January 06, 2007
Re: D, Java? To D or not to D?
On Fri, 05 Jan 2007 11:54:09 -0800, Walter Bright wrote:

> TPJ wrote:
>> 1) In order to run a program written in D I have to compile it on the target OS.
> 
> I haven't talked about this much, but it is technically feasible to 
> create a D runtime that enables "compile once, run everywhere". It is 
> not necessary to have a VM to do it.


Isn't this something that LLVM can be used for?

http://llvm.org/

-JJR
January 06, 2007
Re: D, Java? To D or not to D?
Frits van Bommel wrote:
> Sean Kelly wrote:
>> Frits van Bommel wrote:
>>> Oh, I must've read it wrong. I thought this was about a runtime that 
>>> would automagically compile on any platform... <blush>
>>>
>>> That's different from "independent of OS" though, which seems to be 
>>> what you're thinking of. ("run everywhere" does not equal "run on any 
>>> x86". Not yet anyway ;) )
>>
>> True :)  But it still means quite a bit of portability.  And I could 
>> have misunderstood as well.  I simply couldn't imagine a binary app 
>> compiled for, say, x86 (one instruction set, big-endian) actually 
>> working on PPC (another instruction set, little-endian) without some 
>> sort of JIT compile step.
> 
> x86 is little-endian, PPC can IIRC be set to either, but is typically 
> (i.e. on PPC Macs) set to big-endian.

Oops, I had it backwards, you're right.  A bigger issue is 64 vs. 32-bit 
code.  At some point, the architectural differences are simply too great 
for binary portability of even user code.

>> The differences are probably in the executable startup code, the 
>> thread code, memory allocation, some portions of garbage collection, 
>> etc, all of which could theoretically be abstracted to a runtime 
>> library (in a sense, they already are: Phobos).  The issue would be 
>> figuring out how to go from the command prompt to running the proper 
>> version of everything.  Would a link step be required first?  Would 
>> the runtime library actually contain multiple versions of the code for 
>> different OSes and call the appropriate one dynamically?  Or perhaps 
>> the runtime could be in a dynamic lib that's loaded by some startup 
>> code linked into the app?  Note that none of these are Virtual Machine 
>> situations because the code is running natively in all cases.
> 
> Before you consider how the runtime is linked to the app, first consider 
> how the app starts in the first place. Let's say the bare minimum is x86 
> Linux and Windows compatibility.
> 
> I don't think there's currently any way to have it run directly on 
> Windows and Linux simply because there's AFAIK no executable format 
> that's supported by both OSs (ELF is supported on multiple OSs, but 
> Windows doesn't support it). Furthermore, all executable formats I know 
> of (except for plain binary, which AFAIK Linux doesn't support[1]) have 
> some (unique) signature in the first few bytes, so a file can't be 
> multiple of these formats at the same time.

Hrm, right.  So there would probably have to be either a configuration 
step before the initial run or some sort of translation or linking that 
took place.  I'll be curious to see what comes of this.  If Walter can 
sort it out in a nice clean manner, it would be a killer feature.


Sean
January 06, 2007
Re: D, Java? To D or not to D?
Sean Kelly wrote:
> Walter Bright wrote:
>> TPJ wrote:
>>> 1) In order to run a program written in D I have to compile it on the 
>>> target OS.
>>
>> I haven't talked about this much, but it is technically feasible to 
>> create a D runtime that enables "compile once, run everywhere". It is 
>> not necessary to have a VM to do it.
> 
> This would still be limited to a specific architecture, wouldn't it?

Nope. But it would require one to code in a portable subset of D.

> ie. 
> "This app runs on x86, regardless of OS."  Given that there are really 
> very few architectures in the marketplace though, this is still a killer 
> feature.
> 
> 
> Sean
1 2 3 4 5 6 7
Top | Discussion index | About this forum | D home