Am 25.05.2013 07:52, schrieb Manu:
> On 25 May 2013 15:29, deadalnix <deadalnix@gmail.com
> <mailto:deadalnix@gmail.com>> wrote:
>
>     On Saturday, 25 May 2013 at 05:18:12 UTC, Manu wrote:
>
>         On 25 May 2013 15:00, deadalnix <deadalnix@gmail.com
>         <mailto:deadalnix@gmail.com>> wrote:
>
>             On Saturday, 25 May 2013 at 01:56:42 UTC, Manu wrote:
>
>                 Understand, I have no virtual-memory manager, it won't
>                 page, it's not a
>                 performance problem, it will just crash if I
>                 mis-calculate this value.
>
>
>             So the GC is kind of out.
>
>
>         Yeah, I'm wondering if that's just a basic truth for embedded.
>         Can D implement a ref-counting GC? That would probably still be
>         okay, since
>         collection is immediate.
>
>
>     This is technically possible, but you said you make few allocations.
>     So with the tax on pointer write or the reference counting, you'll
>     pay a lot to collect very few garbages. I'm not sure the tradeoff is
>     worthwhile.
>
>
> But it would be deterministic, and if the allocations are few, the cost
> should be negligible.
>
>
>     Paradoxically, when you create few garbage, GC are really goos as
>     they don't need to trigger often. But if you need to add a tax on
>     each reference write/copy, you'll probably pay more tax than you get
>     out of it.
>
>
> They're still non-deterministic though. And unless (even if?) they're
> precise, they might leak.
>
> What does ObjC do? It seems to work okay on embedded hardware (although
> not particularly memory-constrained hardware).
> Didn't ObjC recently reject GC in favour of refcounting?

Yes, but is was mainly for not being able to have a stable working GC able to cope with the Objective-C code available in the wild. It had quite a few issues.

Objective-C reference counting requires compiler and runtime support.

Basically it is based in how Cocoa does reference counting, but instead of requiring the developers to manually write the [retain], [release] and [autorelease] messages, the compiler is able to infer them based on
Cocoa memory access patterns.

Additionally it makes use of dataflow analysis to remove superfluous use of those calls.

There is a WWDC talk on iTunes where they explain that. I can look for it if there is interest.

Microsoft did the same thing with their C++/CX language extensions and COM for WinRT.

--
Paulo

On Saturday, 25 May 2013 at 05:52:23 UTC, Manu wrote:
> But it would be deterministic, and if the allocations are few, the cost
> should be negligible.
>

You'll pay a tax on pointer write, not on allocations ! It won't be negligible !

> They're still non-deterministic though. And unless (even if?) they're
> precise, they might leak.
>

Not if they are precise. But this is another topic.

> What does ObjC do? It seems to work okay on embedded hardware (although not
> particularly memory-constrained hardware).
> Didn't ObjC recently reject GC in favour of refcounting?

ObjC is an horrible three headed monster in that regard, and I don't think this is the way to go.

Am 25.05.2013 03:29, schrieb Manu:
> On 25 May 2013 04:20, Benjamin Thaut <code@benjamin-thaut.de
> <mailto:code@benjamin-thaut.de>> wrote:
> [...]
> See, I have spend a decade on core tech/engine code meticulously
> worrying about memory allocation. I don't think a GC is an outright no-go.
> But we certainly don't have a GC that fits the bill.

Given that Android, Windows Phone 7/8 and PS Vita have system languages with GC, it does not seem to bother those developers.

Yes I know that most AAA studios are actually bypassing them and using C and C++ directly, but already having indie developers using D would be a great win.

One needs to start somehere.

>
>     - Better windows support. All of the developement we do happens on
>     windows and most of D's community does not care about windows
>     support. I'm curious how long it will take until D will get propper
>     DLL support.
>
>

Yeah, this is partially why I lost the train for game development. I was too much focused in FOSS issues, instead of focusing in doing a game.


--
Paulo

Am 25.05.2013 03:29, schrieb Manu:
>
>
> Win64 works for me out of the box... ?

For me dmd produces type names like modulename.typename.subtypename which will causes internal errors within the visual studio debugger in some cases. Also debugging of static / global variabels is not possible (even when gshared) because they are also formatted like modulename.variablename;

Kind Regards
Benjamin Thaut

On Saturday, 25 May 2013 at 05:29:31 UTC, deadalnix wrote:

> This is technically possible, but you said you make few allocations. So with the tax on pointer write or the reference counting, you'll pay a lot to collect very few garbages. I'm not sure the tradeoff is worthwhile.
>

Incidentally, I ran across this paper that talks about a reference counted garbage collector that claims to address this issue.  MIght be of interest to this group.

http://researcher.watson.ibm.com/researcher/files/us-bacon/Bacon03Pure.pdf

From the paper:

There are two primary problems with reference counting, namely:
(1) run-time overhead of incrementing and decrementing the reference count each time a
pointer is copied, particularly on the stack; and
(2) inability to detect cycles and consequent necessity of including a second garbage collection technique to deal with cyclic garbage.
In this paper we present new algorithms that address these problems and describe a
new multiprocessor garbage collector based on these techniques that achieves maximum
measured pause times of 2.6 milliseconds over a set of eleven benchmark programs that
perform significant amounts of memory allocation.

On 05/24/2013 04:33 PM, Manu wrote:
>     But anyway, after fixing the obvious Phobos offenders, another huge
>     step would be to get TempAlloc into druntime and used wherever
>     possible in Phobos.
>
>
> How does that work?
>
> One pattern I've used a lot is, since we have a regular 60hz timeslice
> and fairly a regular pattern from frame to frame, we use a temp heap
> which pushes allocations on the end like a stack, then wipe it clean at
> the state of the next frame.
> Great for any small allocations that last no longer than a single frame.
> It's fast (collection is instant), and it also combats memory
> fragmentation, which is also critically important when working on memory
> limited systems with no virtual memory/page file.

Yes, that is basically it.


https://github.com/dsimcha/TempAlloc/blob/master/std/allocators/region.d

On 25 May 2013 21:03, Benjamin Thaut <code@benjamin-thaut.de> wrote:

> Am 25.05.2013 03:29, schrieb Manu:
>
>>
>>
>> Win64 works for me out of the box... ?
>>
>
> For me dmd produces type names like modulename.typename.**subtypename which will causes internal errors within the visual studio debugger in some cases. Also debugging of static / global variabels is not possible (even when gshared) because they are also formatted like modulename.variablename;
>

True, sadly there are holes in the debug experience, which are pretty important to have fixed at some point.

On 5/25/13 6:28 PM, Manu wrote:
> On 25 May 2013 21:03, Benjamin Thaut <code@benjamin-thaut.de
> <mailto:code@benjamin-thaut.de>> wrote:
>
>     Am 25.05.2013 03:29, schrieb Manu:
>
>
>
>         Win64 works for me out of the box... ?
>
>
>     For me dmd produces type names like
>     modulename.typename.__subtypename which will causes internal errors
>     within the visual studio debugger in some cases. Also debugging of
>     static / global variabels is not possible (even when gshared)
>     because they are also formatted like modulename.variablename;
>
>
> True, sadly there are holes in the debug experience, which are pretty
> important to have fixed at some point.

Bugzilla links?

On Sat, 25 May 2013 01:52:10 -0400, Manu <turkeyman@gmail.com> wrote:

> What does ObjC do? It seems to work okay on embedded hardware (although not
> particularly memory-constrained hardware).
> Didn't ObjC recently reject GC in favour of refcounting?

Having used ObjC for the last year or so working on iOS, it is a very nice memory management model.

Essentially, all objects (and only objects) are ref-counted automatically by the compiler.  In code, whenever you assign or pass a pointer to an object, the compiler automatically inserts retains and releases extremely conservatively.

Then, the optimizer comes along and factors out extra retains and releases, if it can prove they are necessary.

What I really like about this is, unlike a library-based solution where every assignment to a 'smart pointer' incurs a release/retain, the compiler knows what this means and will factor them out, removing almost all of them.  It's as if you inserted the retains and releases in the most optimized way possible, and it's all for free.

Also, I believe the compiler is then free to reorder retains and releases since it understands how they work.  Of course, a retain/release is an atomic operation, and requires memory barriers, so the CPU/cache cannot reorder, but the compiler still can.

I asked David Nadlinger at the conference whether we could leverage this power in LDC, since LLVM is the compiler back-end used by Apple, but he said all those optimization passes are in the Objective-C front-end.

It would be cool/useful to have compiler-native reference counting.  The only issue is, Objective-C is quite object-heavy, and it's statically checkable whether a pointer is an Object pointer or not.  In D, you would have to conservatively use retains/releases on every pointer, since any memory block could be ref-counted.  But just like Objective-C most of them could be factored out.  Add in that D has the shared-ness of the pointer built into the type system, and you may have something that is extremely effective.

-Steve

On Tuesday, 28 May 2013 at 13:33:39 UTC, Steven Schveighoffer wrote:
> I asked David Nadlinger at the conference whether we could leverage this power in LDC, since LLVM is the compiler back-end used by Apple, but he said all those optimization passes are in the Objective-C front-end.

Hm, apparently I was imprecise or I slightly misunderstood your question:

The actual optimizations _are_ done in LLVM, and are part of its source tree (see lib/Transforms/ObjCARC). What I meant to say is that they are tied to the ObjC runtime function calls emitted by Clang – there is no notion of a "reference counted pointer" on the LLVM level.

Thus, we could definitely base a similar implementation D on this, which would recognize D runtime calls (potentially accompanied by D-specific LLVM metadata) instead of Objective-C ones. It's just that there would be quite a bit of adjusting involved, as the ObjC ARC implementation isn't designed to be language-agnostic.

David