On 7 January 2012 15:24, Paulo Pinto <pjmlp@progtools.org> wrote:

> Hi Manu,
>
> nice to hear about your experience with C#.
>
> I am a bit of half-insider in the game industry, being
> a IGDA member for some years and also attended two
> GDCE so far.
>
> Several reasons have kept me from getting a job in the industry, but I still follow what's happening quite closely.
>
> From what I know here in Europe, many studios have been slowly migrating to C# for tools, and many that target mostly Windows are also experimenting with it in their engines.
>
> How does it look like from your side?
>
> I think this is important to know, because in what concerns game development, C# might eventually superseed C++, especially with good quality AOT compilers. Not sure how good Mono's AOT code quality is. From the public information C# is the default language for the PlayStation VITA and the PlayStation Suite.
>
> And game development is probably one of the few areas where D could get an entry to.


Most gamedev studios are VS-centric, and from that reason alone, C# is the
obvious choice if you want to remove your tools from C++/MFC/whatever.
In my experience, most first time C# coders have a great experience off the
bat, and tend to keep on that wagon.
I think it started like this; some coder decides he hates writing tools in
C++. They click new->project-> oh look, C#, everyone's talking about it.
They give it a go (...see my previous post).

I don't know if you're on the mark saying studios that 'target mostly
windows' are migrating engines to C#. Actually, I've never heard of an
instance of that outside indy-games.
The most interesting trend I think is the Unity effect. They embed mono
into their engine (ie, NON-windows machines), and all high level game code
is written in C#.
This seems an amazingly sensible idea, and most people seem to agree. I
love the idea of Unity, and on small games, handhelds and stuff, it seems
to work really well.

I don't realisticslly see C# 'taking over' gamedev. It's too closely
affiliated with Microsoft, although I am very surprised by Sony's move, so
maybe I'm way off the mark.
Also, for AAA mainstream titles, I don't think there's any choice other
than C++. Game engines are aggressively tuned at very low level to the
hardware. I don't think any non-C++-like systems language can take that
away (D may have a chance here, but the GC might be a problem at the engine
level).

I think it's likely that the shift more towards mono (or some other script-ish language) taking over the high level logic code will continue though. Mono just happens to have a great AOT experience, familiar IDE's/integration into workflow, and familiarity to most devs now via working on tools.

One of the biggest challenges for the gamedev world is adapting high-level
game code to scalable SMP systems. Engine is easy to make scale, game
logic, not so much. Strong OOP is the problem here.
With the current trend of using an auxilliary language to write this logic
code, there's opportunity for a language that totally nails SMP scalability
to really make a dent... but it's hard to predict :)

I don't think I can emphasise enough though the point I made in my last post where the successful language will likely NOT be the language with the best merits towards that end. It will be the one that is most easily integrated into developers workflows, and I think this is the key reason C# is so popular atm.

All that said, I'm just an engine programmer. I write C++ to the metal. I'm not the best authority on the topic :)

Thanks for the explanation.

I am convinced that eventually AAA game engines can be
made in a GC enabled language like D, they just need to
be coded in a different way, more GC-friendly.

But this is just a wish about progress, and I don't have
any real industry experience to say it will really happen.

Thanks again for the feedback,
Paulo

Am 07.01.2012 15:34, schrieb Manu:
> On 7 January 2012 15:24, Paulo Pinto <pjmlp@progtools.org
> <mailto:pjmlp@progtools.org>> wrote:
>
>     Hi Manu,
>
>     nice to hear about your experience with C#.
>
>     I am a bit of half-insider in the game industry, being
>     a IGDA member for some years and also attended two
>     GDCE so far.
>
>     Several reasons have kept me from getting a job in the
>     industry, but I still follow what's happening quite closely.
>
>      >From what I know here in Europe, many studios have been slowly
>     migrating to C# for tools, and many that target mostly Windows
>     are also experimenting with it in their engines.
>
>     How does it look like from your side?
>
>     I think this is important to know, because in what concerns
>     game development, C# might eventually superseed C++, especially
>     with good quality AOT compilers. Not sure how good Mono's AOT
>     code quality is. From the public information C# is the default
>     language for the PlayStation VITA and the PlayStation Suite.
>
>     And game development is probably one of the few areas where D could get
>     an entry to.
>
>
> Most gamedev studios are VS-centric, and from that reason alone, C# is
> the obvious choice if you want to remove your tools from C++/MFC/whatever.
> In my experience, most first time C# coders have a great experience off
> the bat, and tend to keep on that wagon.
> I think it started like this; some coder decides he hates writing tools
> in C++. They click new->project-> oh look, C#, everyone's talking about
> it. They give it a go (...see my previous post).
>
> I don't know if you're on the mark saying studios that 'target mostly
> windows' are migrating engines to C#. Actually, I've never heard of an
> instance of that outside indy-games.
> The most interesting trend I think is the Unity effect. They embed mono
> into their engine (ie, NON-windows machines), and all high level game
> code is written in C#.
> This seems an amazingly sensible idea, and most people seem to agree. I
> love the idea of Unity, and on small games, handhelds and stuff, it
> seems to work really well.
>
> I don't realisticslly see C# 'taking over' gamedev. It's too closely
> affiliated with Microsoft, although I am very surprised by Sony's move,
> so maybe I'm way off the mark.
> Also, for AAA mainstream titles, I don't think there's any choice other
> than C++. Game engines are aggressively tuned at very low level to the
> hardware. I don't think any non-C++-like systems language can take that
> away (D may have a chance here, but the GC might be a problem at the
> engine level).
>
> I think it's likely that the shift more towards mono (or some other
> script-ish language) taking over the high level logic code will continue
> though. Mono just happens to have a great AOT experience, familiar
> IDE's/integration into workflow, and familiarity to most devs now via
> working on tools.
>
> One of the biggest challenges for the gamedev world is adapting
> high-level game code to scalable SMP systems. Engine is easy to make
> scale, game logic, not so much. Strong OOP is the problem here.
> With the current trend of using an auxilliary language to write this
> logic code, there's opportunity for a language that totally nails SMP
> scalability to really make a dent... but it's hard to predict :)
>
> I don't think I can emphasise enough though the point I made in my last
> post where the successful language will likely NOT be the language with
> the best merits towards that end. It will be the one that is most easily
> integrated into developers workflows, and I think this is the key reason
> C# is so popular atm.
>
> All that said, I'm just an engine programmer. I write C++ to the metal.
> I'm not the best authority on the topic :)

On 7 January 2012 17:27, Paulo Pinto <pjmlp@progtools.org> wrote:

> Thanks for the explanation.
>
> I am convinced that eventually AAA game engines can be
> made in a GC enabled language like D, they just need to
> be coded in a different way, more GC-friendly.
>

I tend to disagree. I see it trending in the opposite direction.
Tight pooling of resources and updating in finer grained (probably
threaded) tight loops across neatly (cache friendly) packed bundles of like
objects. That's where engines are going.
Engines are becoming more and more memory-layout aware, not less... GC is
the opposite of rigid memory layout.
It's great for high level sugar that glues the whole thing together, but
the engine is heading the opposite way; everything works more like a video
card, small repetitive streamed jobs, with memory carefully managed.

Manu:

> and updating in finer grained (probably
> threaded) tight loops across neatly (cache friendly) packed bundles of like
> objects. That's where engines are going.

I agree. Something related:

http://research.scee.net/files/presentations/gcapaustralia09/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf

A language designed for games has to help this style of coding.

Bye,
bearophile

On 7 January 2012 17:55, bearophile <bearophileHUGS@lycos.com> wrote:

> Manu:
>
> > and updating in finer grained (probably
> > threaded) tight loops across neatly (cache friendly) packed bundles of
> like
> > objects. That's where engines are going.
>
> I agree. Something related:
>
>
> http://research.scee.net/files/presentations/gcapaustralia09/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf
>
> A language designed for games has to help this style of coding.
>

Hehe, Tony's a good mate of mine (that lecture) :)
He's become quite a prominent figure in Australia after taking the Sony
job. He quit now though.
Amusingly, he managed to convince Mike Acton to speak at the recent GCAP,
even though the Australian industry has disintegrated ;)

This is my biggest fear with investing in D. It's not incompatible with D, but liberal use of the GC in the language and/or in the libraries scares me. The tendency to encourage use of dynamic arrays will be a major problem.

I don't think it's reached a point of no return yet, but it needs to be carefully considered, and I really would like support for allocators in D. I *need* to be able to assign class memory out of pools, ideally without making my code archaic and/or breaking the language usage paradigms.

Am 07.01.2012 16:44, schrieb Manu:
> On 7 January 2012 17:27, Paulo Pinto <pjmlp@progtools.org
> <mailto:pjmlp@progtools.org>> wrote:
>
>     Thanks for the explanation.
>
>     I am convinced that eventually AAA game engines can be
>     made in a GC enabled language like D, they just need to
>     be coded in a different way, more GC-friendly.
>
>
> I tend to disagree. I see it trending in the opposite direction.
> Tight pooling of resources and updating in finer grained (probably
> threaded) tight loops across neatly (cache friendly) packed bundles of
> like objects. That's where engines are going.
> Engines are becoming more and more memory-layout aware, not less... GC
> is the opposite of rigid memory layout.
> It's great for high level sugar that glues the whole thing together, but
> the engine is heading the opposite way; everything works more like a
> video card, small repetitive streamed jobs, with memory carefully managed.


The reason I am conviced is that C++ was the last systems programming language with manual memory management. All the languages people are
developing as possible C++ replacement do have some form of automatic
memory management.

Meanwhile smart pointers and a GC API became part of the C++ language standard. In the future, if you want to target Windows 8 Metro games, you will need to use WinRT which requires C++/CX and makes use of reference counting in the language.

Apple added GC to Objective-C, which has some problems with existing frameworks, so their next step was to introduce ARC.

This is why I think that eventually even games engines will be developed
in a language that supports some kind of automatic memory management be
it GC or reference counting.

Naturally such language also needs to provide the possibility to disable
the memory management in critical code paths.

If you go to my web site, the mini games I have there are a joke, they
cannot even be called games, maybe half-finished prototypes. I just spent too much time in the university with compiler programming and
UNIX system programming, only to start looking at game development when life start making me busy with other stuff.

Anyway, I still remember the days when C was too high level to be used for games.

Then again, I don't have any experience in the game industry, so you are much better qualified to say if what I say makes any sense.

--
Paulo

Manu:

> The tendency to encourage use of dynamic arrays will be a major problem.

I don't know how much big that problem will be, D dynamic arrays are quite handy, but I agree that static arrays need to be encouraged more in D (and currently most algorithms of Phobos don't work with static arrays (you need to slice them first)).

Currently even this code with a stack-allocated fixed size array causes a heap allocation (DMD):

void main() {
    int[2] a = [1, 2];
}

ASM, optimized build:

__Dmain comdat
    push EAX
    push EAX
    mov EAX,offset FLAT:_D12TypeInfo_xAi6__initZ
    push EBX
    push 8
    push 2
    push EAX
    call near ptr __d_arrayliteralTX ; heap allocation
    add ESP,8
    mov EBX,EAX
    mov dword ptr [EAX],1
    mov ECX,EBX
    push EBX
    lea EDX,0Ch[ESP]
    mov dword ptr 4[EBX],2
    push EDX
    call near ptr _memcpy
    add ESP,0Ch
    xor EAX,EAX
    pop EBX
    add ESP,8
    ret


I have suggested to add a safer version of VLAs to avoid some heap-allocated dynamic arrays: http://d.puremagic.com/issues/show_bug.cgi?id=5348

Most people here seem to not care of the Ada language, but if you take a look at Ada code you see how often it doesn't use heap allocations.

Bye,
bearophile

> The reason I am conviced is that C++ was the last systems programming language with manual memory management. All the languages people are
> developing as possible C++ replacement do have some form of automatic
> memory management.
>
> Meanwhile smart pointers and a GC API became part of the C++ language standard. In the future, if you want to target Windows 8 Metro games, you will need to use WinRT which requires C++/CX and makes use of reference counting in the language.
>
> Apple added GC to Objective-C, which has some problems with existing frameworks, so their next step was to introduce ARC.
>
> This is why I think that eventually even games engines will be developed
> in a language that supports some kind of automatic memory management be
> it GC or reference counting.



 Well games often already integrate scripting languages, generally Lua or Mono, which are both GC based. But this is for the game logic, not for the high performance parts of the code.


  For a GC to be used at the C++/D layer it would need to be

1) optional always, D makes it optional but you loose a solid chunk of the language if you ditch GC, what remains is in some ways inferior to C++(no escaping lambda without GC, /cry)
2) no long pauses ever


 I think most game engine stuff will continue to be written without GC, as it doesn't really add much if what you are after is raw performance. And C++ does have smart pointers which perform much of what a GC does, but without the long pauses or non deterministic destruction.

Am 07.01.2012 19:59, schrieb bearophile:
> Manu:
>
>> The tendency to encourage use of dynamic arrays will be a major problem.
>
> I don't know how much big that problem will be, D dynamic arrays are quite handy, but I agree that static arrays need to be encouraged more in D (and currently most algorithms of Phobos don't work with static arrays (you need to slice them first)).
>
> Currently even this code with a stack-allocated fixed size array causes a heap allocation (DMD):
>
> void main() {
>      int[2] a = [1, 2];
> }
>
> ASM, optimized build:
>
> __Dmain comdat
>      push EAX
>      push EAX
>      mov EAX,offset FLAT:_D12TypeInfo_xAi6__initZ
>      push EBX
>      push 8
>      push 2
>      push EAX
>      call near ptr __d_arrayliteralTX ; heap allocation
>      add ESP,8
>      mov EBX,EAX
>      mov dword ptr [EAX],1
>      mov ECX,EBX
>      push EBX
>      lea EDX,0Ch[ESP]
>      mov dword ptr 4[EBX],2
>      push EDX
>      call near ptr _memcpy
>      add ESP,0Ch
>      xor EAX,EAX
>      pop EBX
>      add ESP,8
>      ret
>
>
> I have suggested to add a safer version of VLAs to avoid some heap-allocated dynamic arrays:
> http://d.puremagic.com/issues/show_bug.cgi?id=5348
>
> Most people here seem to not care of the Ada language, but if you take a look at Ada code you see how often it doesn't use heap allocations.
>
> Bye,
> bearophile

The Ada standard also allows for some form of optional automatic memory management, even if only some of the compilers offer it.


http://en.wikibooks.org/wiki/Ada_Programming/Types/access

On 1/7/2012 11:00 AM, Froglegs wrote:
> For a GC to be used at the C++/D layer it would need to be
>
> 1) optional always, D makes it optional but you loose a solid chunk of the
> language if you ditch GC, what remains is in some ways inferior to C++(no
> escaping lambda without GC, /cry)
> 2) no long pauses ever
>
>
> I think most game engine stuff will continue to be written without GC, as it
> doesn't really add much if what you are after is raw performance. And C++ does
> have smart pointers which perform much of what a GC does, but without the long
> pauses or non deterministic destruction.

There are more options with D:

3) Disable GC from collecting during critical parts of the code, i.e. you can still new all you want, it just won't run collection cycles.

4) Pre-allocate all necessary data before entering the critical sections. You have to do this anyway with C++ if you have hard realtime constraints, as C++ makes no guarantees about how long new or malloc() will take (and it cannot).

BTW, in C++ a lambda cannot escape, so C++ has no advantage there.