View mode: basic / threaded / horizontal-split · Log in · Help
May 16, 2010
Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
Hi all, I'm toying around with the idea of porting my raytracer codebase to D.
But before committing, I have a few rookie questions:

1: What kind of license is the D compiler under? I'm thinking of shipping a
commercial, close sourced (for now) program with the D compiler (so that users
can compile within the GUI). Is this possible to do, or can I least pay for the
priviledge?

2: Is it possible to use D with the Visual C++ IDE? Preferably, I would like
the apprepriate compiler and D options listed in the options (in place of the
usual c/c++ options).

3: I need my program to be as fast as possible. The Visual C++ compiler has
features such as "link-time code generation" and "Profile guided optimization".
Does D have equivalents?

4: Does D play nicely with QT, SDL, Lua?

5: How about compatibility with GPGPU stuff like CUDA and OpenCL? Can I just as
easily write GPGPU programs which run as fast as I can with C/C++?
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
Dan W.:

> 3: I need my program to be as fast as possible. The Visual C++ compiler has
> features such as "link-time code generation" and "Profile guided optimization".
> Does D have equivalents?

You can't ask a new open source language to have the features of a ten+ years old commercial compiler.
If you compile D1-Tango code on Linux 32 bit using LDC using all the correct compile switches you can get performance comparable to C code compiled with GCC. LDC does not have vectorization (that gcc has) but has link-time optimization that GCC 4.5 has just in part. This is the very best performance you can hope with D.


General note: a trap D has put itself into: a significant group of people seem interested in D only as a high performance language. But history shows that nearly no new language starts its life being very fast. High performance, especially if you mean it as compared to quite mature C++ implementations, is something that can only come some years after a language has already reached some form of success and people start to use weeks, months or years just tuning the GC, creating whole new kinds of GC, inventing and implementing other D-specific optimizations, implementing a good escape analysis, implementing a good devirtualization+inlining of virtual functions, implementing various different kinds of efficient vectorizations, implementing a good pointer alias analysis, and so on.

Today some kind of Java programs running on HotSpot have a performance comparable to C++ programs. JavaScript running on V8 is often less than ten times slower than well compiled C. But for years both Java and JavaScript were dog-slow. Most things in D are designed to require a simple enough compiler, it doesn't need an advanced JIT just to be efficient. So even naively compiled D programs aren't 50 times slower than equivalent C++ programs. Yet, the performance is not the same as commercial C++ compilers, and it will not be like that unless groups of serious people set as their main/only purpose the creation of a efficient D2 compiler. "Performance" is not something that just happens, you need lot of focused work to gain it.

Bye,
bearophile
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
On 16/05/10 15:27, Dan W wrote:
> Hi all, I'm toying around with the idea of porting my raytracer codebase to D.
> But before committing, I have a few rookie questions:
>
> 1: What kind of license is the D compiler under? I'm thinking of shipping a
> commercial, close sourced (for now) program with the D compiler (so that users
> can compile within the GUI). Is this possible to do, or can I least pay for the
> priviledge?

dmd is under 2 (3) licenses, one for the front end and one for the 
backend. I won't go into details, you can find the details in the 
archives though. Long story short if you want to redistribute dmd you 
have to ask Walter for the priviledge. LDC and GDC have no such 
restrictions, you can include them as long as you don't modify the 
source, and if you do then you distribute the source as well as the 
binaries.

> 2: Is it possible to use D with the Visual C++ IDE? Preferably, I would like
> the apprepriate compiler and D options listed in the options (in place of the
> usual c/c++ options).

Try VisualD, which was released about a month ago. I haven't tried it 
yet, I believe it still has some way to go... This said its current 
feature list looks impressive.

http://dsource.org/projects/visuald/

> 3: I need my program to be as fast as possible. The Visual C++ compiler has
> features such as "link-time code generation" and "Profile guided optimization".
> Does D have equivalents?

If you want LTO you'll need to use LDC with some fancy compilation steps 
(I believe bearophile, our resident benchmarker should be able to 
provide you with these). The downside to LDC is that it does not support 
exceptions on windows (it will support them as soon as llvm does).

> 4: Does D play nicely with QT, SDL, Lua?

See:
http://dsource.org/projects/qtd/ - Qt bindings
http://dsource.org/projects/luad/ - Lua bindings
http://dsource.org/projects/derelict/ - Various bindings for 
multimedia/game apps including SDL, OpenGL, OpenAL etc

> 5: How about compatibility with GPGPU stuff like CUDA and OpenCL? Can I just as
> easily write GPGPU programs which run as fast as I can with C/C++?

I don't know what the status of this is, I think a couple of people have 
written some initial bindings for either CUDA or OpenCL, perhaps someone 
else can enlighten you as to their status. As for their speed it will be 
just as fast as the equivilant code in C/C++.

I hope things go well for you, there's a lot of initial hurdles for 
getting into D, but once you find your way around them you'll learn to 
love this great language! There are lots of people that have written ray 
tracers in D, so should you need assistance there's people who can help.
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
Dan W wrote:
> Hi all, I'm toying around with the idea of porting my raytracer codebase to D.

Hi,

May I ask you why are you planning to port an existing codebase to D?
What kind of benefits specifically(except comparable to C performance) 
you expect from D?

Thank you.



-- 
Alex Makhotin,
the founder of BITPROX,
http://bitprox.com
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
Hello Dan,

> Hi all, I'm toying around with the idea of porting my raytracer
> codebase to D. But before committing, I have a few rookie questions:
> 
> 1: What kind of license is the D compiler under? I'm thinking of
> shipping a commercial, close sourced (for now) program with the D
> compiler (so that users can compile within the GUI). Is this possible
> to do, or can I least pay for the priviledge?

The front end is under an Open Source (R) license. The backbend is open source 
but only in that you can see the source. Several projects combine the front 
end with a FOOS back end than can b shipped but you can't ship copies of 
the official exe without Walters ok but he's been know to give it at no cost 
if you ask really nicely.

> 2: Is it possible to use D with the Visual C++ IDE? Preferably, I
> would like the apprepriate compiler and D options listed in the
> options (in place of the usual c/c++ options).

There is a D plugin that recently got posted that allows that. I've never 
got it working but I think that's me FUBARing VS.

> 3: I need my program to be as fast as possible. The Visual C++
> compiler has features such as "link-time code generation" and "Profile
> guided optimization". Does D have equivalents?

For link time code generation: you might get the same effect via templates 
(they are way easier under D than C++). As for the other, I think DMD can 
do some of that but I don't remember the details.

> 5: How about compatibility with GPGPU stuff like CUDA and OpenCL?

I remember seeing some work in that direction about 2-3 years ago. If you 
can get a C API to that stuff, you can do it in D. There might be a wrapper 
somewhere that gives an API that's cleaner to use from D.

> Can I just as easily write GPGPU programs which run as fast as I can with 
C/C++?

Assuming a reasonable API, you should be able to whip out D code to interact 
with CUDA/OpenCL at least as fast as you can write the same in C/C++.


-- 
... <IXOYE><
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
Dan W.:

> 3: I need my program to be as fast as possible. The Visual C++ compiler has
> features such as "link-time code generation"

This page explains this topic:
http://msdn.microsoft.com/en-us/magazine/cc301698.aspx

Bye,
bearophile
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
bearophile wrote:
> You can't ask a new open source language to have the features of a ten+ years
> old commercial compiler.

Right, but dmd is using an optimizer and code generator that has been around for 
25 years now. It's optimization is competent and reasonably advanced - the usual 
data flow optimizations are there, and the expected back end optimizations like 
register allocation using live range analysis and instruction scheduling are all 
there.

The back end can be improved for floating point, but for integer/pointer work it 
is excellent.

It does not do link time code generation nor profile guided optimization, 
although in my experiments such features pay off only in a small minority of 
cases. In my experiments on vector array operations, the improvement from the 
CPU's vector instructions is disappointing. It always seems to get hopelessly 
bottlenecked by memory bandwidth.

The dmd does have a built-in profiling tool, which is extremely effective in 
pinpointing trouble spots in the source code. For example, in the recent issue 
where the spell checker was slow, the profiler pointed the damning finger at 
exactly where the problem was. (It was an algorithmic problem, not an 
optimization problem.)

Just to brag about how good it can be, DMC++ remains by far the fastest C++ 
compiler available, and DMD is incredibly fast at compiling. Both are built with 
the same optimizer and code generator that DMD uses.
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
bearophile wrote:
>> 3: I need my program to be as fast as possible. The Visual C++ compiler has
>> features such as "link-time code generation"
> 
> This page explains this topic:
> http://msdn.microsoft.com/en-us/magazine/cc301698.aspx

What's actually happening is interprocedural analysis, and inlining across 
source modules. In C++ this needs to happen at link time because the C++ 
compilation module is each source file is completely independent of other source 
files.

This is not true of D. In D, the compiler can (at the option of how it is 
compiled and how the programmer sets up the source modules) look at all the 
source to the program. Hence, a lot of inlining can (and does) happen across 
modules without needing any support from the linker.
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG, QT, SDL
"Dan W" <twinbee42@skytopia.com> wrote in message 
news:hsovdd$1s1j$1@digitalmars.com...
>
> 2: Is it possible to use D with the Visual C++ IDE? Preferably, I would 
> like
> the apprepriate compiler and D options listed in the options (in place of 
> the
> usual c/c++ options).
>

Other people mentioned the recent D plugin for Visual Studio. If that isn't 
mature enough for you, there's a very mature plugin for Eclipse called 
Descent: http://www.dsource.org/projects/descent

> 3: I need my program to be as fast as possible.
>

Optimization often seems to be a mixed bag across any two modern languages.

On one hand, there are some cases where D can be a little slower than 
average. For instance, I've heard that the GC isn't great at handling lots 
of small objects. Bearophile can probably tell you a lot about any slow 
spots of D, he's done a lot of testing in that area.

On the other hand, there's plenty that D is fast with. Other people have 
mentioned a lot about this already. But I'll also add that the design of D 
has a few things that can allow certain things to be done in a more 
efficient way than can easily be done in C/C++. Array slicing (combined with 
GC), for example, has been shown to go a long way in helping to make a 
ridiculously fast (and memory-efficient) XML parser with less effort than it 
would take in C/C++:

http://dotnot.org/blog/archives/2008/03/10/xml-benchmarks-parsequerymutateserialize/
http://dotnot.org/blog/archives/2008/03/10/xml-benchmarks-updated-graphs-with-rapidxml/
http://dotnot.org/blog/archives/2008/03/12/why-is-dtango-so-fast-at-parsing-xml/
May 16, 2010
Re: Misc questions:- licensing, VC++ IDE compatible, GPGPU, LTCG,
Walter Bright:

>This is not true of D. In D, the compiler can<

Thank you for your answers.
At the moment D compilers aren't doing this, I think. (LDC performs an optimization at link time). But it's nice that D leaves this optimization opportunity to future D compilers.

----------------------

If you have noticed that Html page lists three optimizations. The first one is the one you have explained.

The second optmizations it talks about is custom calling conventions:

>Normally, all functions are either cdecl, stdcall, or fastcall. With custom calling conventions, the back end has enough knowledge that it can pass more values in registers, and less on the stack. This usually cuts code size and improves performance.<


I have translated his demo code to D:

int foo(int i, int* j, int* k, int l) {
   *j = *k;
   *k = i + l;
   return i + *j + *k + l;
}
int main(char[][] args) {
   int i, j, k, l;
   l = i = args.length;
   int x = foo(i, &j, &k, l);
   return x * args.length;
}


This is how dmd compiles foo() (-O -release):

_D7stdcall3fooFiPiPiiZi	comdat
		push	EAX
		mov	ECX,8[ESP]
		mov	EDX,[ECX]
		push	EBX
		mov	EBX,010h[ESP]
		push	ESI
		mov	ESI,018h[ESP]
		push	EDI
		lea	EDI,[EAX][ESI]
		mov	[EBX],EDX
		mov	[ECX],EDI
		mov	EAX,[EBX]
		add	EAX,ESI
		add	EAX,EDI
		add	EAX,0Ch[ESP]
		pop	EDI
		pop	ESI
		pop	EBX
		pop	ECX
		ret	0Ch


This is how LDC compiles foo() with -O3 -release:

_D4test3fooFiPiPiiZi:
   pushl   %esi
   movl    8(%esp), %ecx
   movl    (%ecx), %edx
   movl    12(%esp), %esi
   movl    %edx, (%esi)
   addl    16(%esp), %eax
   movl    %eax, (%ecx)
   addl    %eax, %eax
   addl    (%esi), %eax
   popl    %esi
   ret $12


This is the asm of foo() shown in that article:

_foo:
   mov         ecx,dword ptr [eax]
   mov         dword ptr [esi],ecx
   lea         ecx,[edi+edx]
   mov         dword ptr [eax],ecx
   mov         eax,dword ptr [esi]    // *j
   add         eax,ecx                // *k sub-expression (from
   add         eax,edi                // l
   add         eax,edx                // i
   ret

It seems LDC isn't performing this optimization.

----------------------

The third optimizations it talks about is 'Small TLS Encoding':

>When you use __declspec(thread) variables, the code generator stores the variables at a fixed offset in each per-thread data area. Without LTCG, the code generator has no idea of how many __declspec(thread) variables there will be. As such, it must generate code that assumes the worst, and uses a four-byte offset to access the variable. With LTCG, the code generator has the opportunity to examine all __declspec(thread) variables, and note how often they're used. The code generator can put the smaller, more frequently used variables at the beginning of the per-thread data area and use a one-byte offset to access them.<

This is the C++ example code he uses:

__declspec(thread) int i = 1;
int main() {
   i = 4;
   return i;
}


The asm he shows without this optimization:
_main:
   mov         eax,dword ptr [__tls_index]
   mov         ecx,dword ptr fs:[2Ch]
   mov         ecx,dword ptr [ecx+eax*4]
   push        4
   pop         eax
   mov         dword ptr [ecx+4],eax
   ret



The asm he shows with this optimization:
_main:
   mov         eax,dword ptr fs:[0000002Ch]
   mov         ecx,dword ptr [eax]
   mov         eax,4
   mov         dword ptr [ecx+8],eax
   ret


I have translated that last C++ example in this D code:

int i = 1;
int main() {
   i = 4;
   return i;
}


I think I can't test this with LDC because it doesn't have TLS/__gshared.
dmd compiles it to:

__Dmain
		mov	ECX,FS:__tls_array
		mov	EDX,[ECX]
		mov	EAX,4
		mov	_D4test1ii[EDX],EAX
		ret


On this little example dmd seems to produce similar asm.

Bye,
bearophile
« First   ‹ Prev
1 2 3
Top | Discussion index | About this forum | D home