On Tuesday, 6 May 2014 at 10:58:14 UTC, Manu via Digitalmars-d wrote:
> On 6 May 2014 16:33, Jacob Carlborg via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On 06/05/14 08:07, HaraldZealot wrote:
>>
>>> I notice that I view only part of problem, can anybody link or describe
>>> me completely state and problems of current garbage collection and other
>>> resource management? It help me in finding of existence solution (at
>>> least theoretical).
>>
>>
>> The major issue with the garbage collector is that it's not guaranteed to
>> run a collection. When a collection is run the GC will call the destructors
>> for the objects it collects. If there's no guarantee a collection is run
>> there can be no guarantee that destructors are called. A collection is
>> usually run when allocating new memory and there's not enough memory
>> available.
>
> I think it's also an important consideration that GC is incompatible
> with low-memory and real-time environments.
>
> ...

I guess outside the gaming world, embedded and real-time seem to be getting lots of Java and .NET love:

https://www.aicas.com/cms/

http://www.is2t.com/products/

http://www.mountaineer.org/netmf-for-stm32/

Just a small sample of the partners providing the said support.


--
Paulo

On 6 May 2014 21:39, Paulo Pinto via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On Tuesday, 6 May 2014 at 10:58:14 UTC, Manu via Digitalmars-d wrote:
>>
>> On 6 May 2014 16:33, Jacob Carlborg via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>>>
>>> On 06/05/14 08:07, HaraldZealot wrote:
>>>
>>>> I notice that I view only part of problem, can anybody link or describe me completely state and problems of current garbage collection and other resource management? It help me in finding of existence solution (at least theoretical).
>>>
>>>
>>>
>>> The major issue with the garbage collector is that it's not guaranteed to
>>> run a collection. When a collection is run the GC will call the
>>> destructors
>>> for the objects it collects. If there's no guarantee a collection is run
>>> there can be no guarantee that destructors are called. A collection is
>>> usually run when allocating new memory and there's not enough memory
>>> available.
>>
>>
>> I think it's also an important consideration that GC is incompatible with low-memory and real-time environments.
>>
>> ...
>
>
> I guess outside the gaming world, embedded and real-time seem to be getting lots of Java and .NET love:
>
> https://www.aicas.com/cms/
>
> http://www.is2t.com/products/
>
> http://www.mountaineer.org/netmf-for-stm32/
>
> Just a small sample of the partners providing the said support.

True, much embedded work isn't realtime. There are a lot of purpose devices where performance is not particularly important, probably correctness is, hence Java may be popular in this environment, and @safe D may have a great application here.

In another realm, if we're talking about really small systems (microcontrollers with memory in the megabytes), the tendency to rely on libraries is reduced significantly, since you probably can't afford the memory for the libs. This environment is the one where it's realistic to say "tag main() with @nogc, and work from there", ie, ban the GC throughout the whole project.

However, another very popular use for embedded systems IS realtime software.
Games consoles are the obvious one here, but also POS/retail devices,
kiosks, televisions, PVR's/dvd/bluray and other entertainment related
devices, cars and automobile HUD's, music players, advertising
displays, etc. There's no shortage of realtime devices in the embedded
space.
Notably, I didn't say 'phones'. Although I think they do generally
fall into this category, I think they're drifting away. Since they run
full OS stack's, it's typical to have unknown amounts of free memory
for user-space apps and virtual memory managers that can page swap, so
having excess memory overhead probably isn't such a big deal. It's
still a major performance hazard though. Stuttering realtime
applications is never a professional look, and Android suffers
chronically in this department compared to iOS.

I spent my last weekend trying to get a PS4 game (built with Unity;
uses mono as a scripting environment) running at an acceptable frame
rate. I didn't succeed, and a significant factor was the JS/C# garbage
collector. We'll probably need to cut content from the game, such that
it leaves plenty of 'excess' resource available to absorb the spikes
it causes when they occur. What a waste of the machine!
Note, C#'s GC is much better than D's. It does seem to run in the
realm of single-digit milliseconds.

On Tuesday, 6 May 2014 at 06:39:45 UTC, Manu via Digitalmars-d wrote:
>
> The Obj-C thing as an example. Granted, it's a huge feature and
> has extensive implications. The Authors have said themselves
> that they agree it's not 'ready' for inclusion... so, what? It
> sits and rots?  I think it needs an experimental place to live
> and have people make use of it for what it is.

This bit right here reminds me of something the Linux kernel has called "staging" [1].  It's basically what you describe: a subtree within main source tree for things to be publicly available while they finish baking, with the understanding that you're going to continue working on it and hopefully get it promoted to first-class citizen.

> I'm plenty vocal and active on things I do feel I know about,
> but they're often pretty radical, unpopular, and rarely come
> even close to turning into code. I'm pretty certain that nothing
> left on my short list that I personally *really* care about will
> ever get pulled, even if I did do the work.

I know I and several other people are still interested in std.simd... :(

-Wyatt

[1] http://lwn.net/Articles/324279/

On Tuesday, 6 May 2014 at 12:05:10 UTC, Manu via Digitalmars-d wrote:
> On 6 May 2014 21:39, Paulo Pinto via Digitalmars-d
> <digitalmars-d@puremagic.com> wrote:
>> On Tuesday, 6 May 2014 at 10:58:14 UTC, Manu via Digitalmars-d wrote:
>>>
>>> On 6 May 2014 16:33, Jacob Carlborg via Digitalmars-d
>>> <digitalmars-d@puremagic.com> wrote:
>>>>
>>>> On 06/05/14 08:07, HaraldZealot wrote:
>>>>
>>>>> I notice that I view only part of problem, can anybody link or describe
>>>>> me completely state and problems of current garbage collection and other
>>>>> resource management? It help me in finding of existence solution (at
>>>>> least theoretical).
>>>>
>>>>
>>>>
>>>> The major issue with the garbage collector is that it's not guaranteed to
>>>> run a collection. When a collection is run the GC will call the
>>>> destructors
>>>> for the objects it collects. If there's no guarantee a collection is run
>>>> there can be no guarantee that destructors are called. A collection is
>>>> usually run when allocating new memory and there's not enough memory
>>>> available.
>>>
>>>
>>> I think it's also an important consideration that GC is incompatible
>>> with low-memory and real-time environments.
>>>
>>> ...
>>
>>
>> I guess outside the gaming world, embedded and real-time seem to be getting
>> lots of Java and .NET love:
>>
>> https://www.aicas.com/cms/
>>
>> http://www.is2t.com/products/
>>
>> http://www.mountaineer.org/netmf-for-stm32/
>>
>> Just a small sample of the partners providing the said support.
>
> True, much embedded work isn't realtime. There are a lot of purpose
> devices where performance is not particularly important, probably
> correctness is, hence Java may be popular in this environment, and
> @safe D may have a great application here.
>
> In another realm, if we're talking about really small systems
> (microcontrollers with memory in the megabytes), the tendency to rely
> on libraries is reduced significantly, since you probably can't afford
> the memory for the libs. This environment is the one where it's
> realistic to say "tag main() with @nogc, and work from there", ie, ban
> the GC throughout the whole project.
>
> However, another very popular use for embedded systems IS realtime software.
> Games consoles are the obvious one here, but also POS/retail devices,
> kiosks, televisions, PVR's/dvd/bluray and other entertainment related
> devices, cars and automobile HUD's, music players, advertising
> displays, etc. There's no shortage of realtime devices in the embedded
> space.

I can provide more examples for JVMs in all areas you list.

However those JVMs do cheat a bit, as they provide a fine grain
control over GC and also allow for region based allocations and immortal memory.

> Notably, I didn't say 'phones'. Although I think they do generally
> fall into this category, I think they're drifting away. Since they run
> full OS stack's, it's typical to have unknown amounts of free memory
> for user-space apps and virtual memory managers that can page swap, so
> having excess memory overhead probably isn't such a big deal. It's
> still a major performance hazard though. Stuttering realtime
> applications is never a professional look, and Android suffers
> chronically in this department compared to iOS.
>
> I spent my last weekend trying to get a PS4 game (built with Unity;
> uses mono as a scripting environment) running at an acceptable frame
> rate. I didn't succeed, and a significant factor was the JS/C# garbage
> collector. We'll probably need to cut content from the game, such that
> it leaves plenty of 'excess' resource available to absorb the spikes
> it causes when they occur. What a waste of the machine!
> Note, C#'s GC is much better than D's. It does seem to run in the
> realm of single-digit milliseconds.

Just as info regarding your experience, please note that both Android and Unity have not so good GC implementations.

Dalvik's GC is hardly touched since Android 2.3, hopefully it will be improved with the new ART runtime.

Unity's Mono implementation is  a legacy one, based on the old Novell Mono implementation as Unity is not willing to pay the royalties Xamarin is asking for modern Mono implementations.


As for the gaming world, I don't really have the required experience to talk much about it. Just doing the heads up that D GC != GC that many of us use.

--
Paulo

On 2014-05-06 12:04:55 +0000, Manu via Digitalmars-d <digitalmars-d@puremagic.com> said:

> Notably, I didn't say 'phones'. Although I think they do generally
> fall into this category, I think they're drifting away. Since they run
> full OS stack's, it's typical to have unknown amounts of free memory
> for user-space apps and virtual memory managers that can page swap, so
> having excess memory overhead probably isn't such a big deal. It's
> still a major performance hazard though. Stuttering realtime
> applications is never a professional look, and Android suffers
> chronically in this department compared to iOS.

Note that iOS has no page swap. Apps just get killed if there isn't enough memory (after being sent a few low memory signals they can react on, clearing caches, etc.). Apps that takes a lot of memory cause other apps in the background to be killed (and later restarted when they come to the foreground).

-- 
Michel Fortin
michel.fortin@michelf.ca
http://michelf.ca

On Tuesday, 6 May 2014 at 11:35:53 UTC, Manu via Digitalmars-d wrote:
> Please, make suggestions! It's been years, and I'm still waiting to hear another realistic proposal.

1. Use @nogc and C++ style allocation for hard real time threads (render logic).

2. Use segmented GC for soft real time threads (world logic).

Lobby for more restrictions/better runtime support from the compiler for fast precise single-pass regional  (local) garbage collection. As a bonus you will get faster exception handling too if identifying stack frames is faster.

Lobby for restrictions:

- C may only call back to @nogc D-functions so you can collect without C-frames on the stack.

- Allow  omitting framepointers, global call optimization and stack optimizations for @nogc @nothrow threads.

etc.

The key to performance is to allow specifying constraints that the compiler and runtime can assume to hold.

The problem with the D community is a lack of priority of system level programming which is at odds with "safe", "no discipline needed" and "Java/C# convinience".

No constraints == no performance.

On Tuesday, 6 May 2014 at 12:21:55 UTC, Paulo Pinto wrote:
> As for the gaming world, I don't really have the required experience to talk much about it. Just doing the heads up that D GC != GC that many of us use.

I very much appreciate the links you provide about different GC solutions, and GC can be appropriate for real time situations where you don't want to maximize resource utilization or where you have predictable downtime to collect.

However the approach taken by appliances (where the application reliability is more important than getting the highest possible execution speed) or business solutions such as Azul Zing (where scalability is more important than hardware cost) is not really suitable for system level programming. It is suitable for a wide range of applications, that is true.

You can have GC with predictable destructors, but you have to design the language and runtime around it. You can have real time GC, but you need a dedicated GC and structure your program to realize the potential.

I think the only realistic road for D is to imposing/infer constraints that makes certain that execution paths is compatible with desirable runtime properties and available runtime support for that specific thread. There is no silver bullet, but the implementation driven design that D seems to be undergoing is affecting the resulting language/run-time design in a bad way, IMO.

On 5/5/14, 11:07 PM, HaraldZealot wrote:
> I have to say that all this discussion (more precisely the understanding
> on the side of key developers) make me very upset.

This is a misunderstanding of the situation. This is brainstorming. There has to be a public place in which ideas can be discussed freely, no matter how radical. Otherwise we'd need to create a cabal and only present finalized decisions to the public. We can't get to a community censorship so strong so as to disallow unencumbered discussion of radical ideas. All those who got up in arms about this thread, I compel you to reconsider.

Andrei

On Tuesday, 6 May 2014 at 14:29:01 UTC, Andrei Alexandrescu wrote:
> On 5/5/14, 11:07 PM, HaraldZealot wrote:
>> I have to say that all this discussion (more precisely the understanding
>> on the side of key developers) make me very upset.
>
> This is a misunderstanding of the situation. This is brainstorming. There has to be a public place in which ideas can be discussed freely, no matter how radical. Otherwise we'd need to create a cabal and only present finalized decisions to the public. We can't get to a community censorship so strong so as to disallow unencumbered discussion of radical ideas. All those who got up in arms about this thread, I compel you to reconsider.
>
> Andrei

+1

with(devilsAdvocateMode)
{
   I think people are a bit used to the situation you describe,
   hence the misunderstanding w.r.t. the approach you're trying
   here
}

> This is a misunderstanding of the situation. This is brainstorming. There has to be a public place in which ideas can be discussed freely, no matter how radical.

This is another misunderstanding :)). Not radicalism upset me, but proposal to create one more hole instead exist hole from man, that touched me solidness in code by his book. It were surprising. But it's only emotion.