November 20, 2006
Dave wrote:
> But the whole concern centers around two canards: a) GC is really slow and b) malloc/free offer deterministic performance for real-time appplications.
> 
> I actually think that the best defense is dispelling those two myths. a) for D will come in time and b) is just plain not true for general purpose malloc/free implementations on modern operating systems.

Although I don't think these are big problems, I'm not convinced myself that they are myths.

As for the slow GC getting fixed, I hope that will come but it still is a valid point in response to the idea that D is usable *right now*, and it will be when D get's the 1.0 label. To say the problem will be fixed doesn't mean it is not there.

The point about deterministic memory allocation (as in realtime) may be a myth, but a lot of C++ argue that C++ has a lot more means to control memory allocation, not just the C way of malloc/free.
Boost offers some allocators that can be used with the STL out of the box, for instance. Techniques of resource management based on scope is just better supported in C++ (not in C!) than in D. I see no myth in that point, it's just a different way of handling the poblem. Well this is actually a different point than you adressed but it is raised sometimes.
November 20, 2006
Don Clugston wrote:
> However, the use of GC instead of malloc enables advanced language constructs (especially, more powerful array syntax), which greatly reduce the number of memory allocations which need to be made.

Could you explain precisely what GC enables that is not possible with malloc? This just doesn't compute for me.
November 20, 2006
Walter Bright wrote:
> The ones who don't want to use D will find the first excuse, valid or not. Fix the excuse, and they'll just move on to the next excuse, valid or not. It's a fool's game. I've been around that circle before. The people we should listen to are the people who actually *use* D, not the ones who just glanced at a chart looking for fault.

People who are using any tool today are people that benefit from its features and are not affected by its shortcomings. Any other person won't use that tool because it is not appropriate.

It is the same with D. If you keep listening only to people who actually use D, D will end as a niche programming language: nobody outside that niche will ever use it because D doesn't fit their needs, and their needs are of no concern to the current users of D.

In my example, I would like to unplug the GC from D sometimes, or have a more predictable reference-count-based GC (even if it would mean disabling unions with pointers and a few other constructs).

> The poster who claimed that conservative gc is somehow incompatible with cryptographic software is misinformed. Even if he were correct, the cryptographic buffers could be allocated with malloc() and would then have no effect whatsoever on the gc.

Using two allocation methods in the same process address space looks really bad, not to say hackish. And you don't need cryptographic buffers or multimedia data, a single int variable is enough to hold a large block of unused data in memory, and the larger the block is, the easier it is for this to happen. Even if it was 1/2^32 of chances of this happening, it still will happen.
November 20, 2006
Walter Bright wrote:
> Boris Kolar wrote:
>> I don't understand why a good RIAA is not already a part of D. C++ has it, so
>> it obviously can be done. The new 'scoped' keyword is insufficient for me,
>> because I'd like to make all my classes scoped. I can usually develop
>> 99% of my C++ code without a single 'new' or 'malloc'. Most of my classes
>> are small (average ~2 fields and ~4 methods) and only used locally, so I'm
>> really angry when I think about negative performance impact they will have
>> in D simply because a decent RIAA is missing.
> 
> Have you considered using structs instead of classes? They are allocated on the stack.

Can you do RAII with them?
I thought that a struct cannot have a destructor, but reading the spec again I notice there's an entry for StructAllocator and StructDeallocator, but no indication of how to use it.
November 20, 2006
Miles wrote:
> Don Clugston wrote:
>> However, the use of GC instead of malloc enables advanced language
>> constructs (especially, more powerful array syntax), which greatly
>> reduce the number of memory allocations which need to be made.
> 
> Could you explain precisely what GC enables that is not possible with
> malloc? This just doesn't compute for me.

Array slicing, for one.
While technically possible with malloc, it would require maintaining a reference count and an extra pointer to the start of the allocated memory area to call free() on. And that still doesn't solve the problem of cyclic references (if the array doesn't just contain primitive data), as well as requiring synchronization in multi-threaded applications.
November 20, 2006
== Quote from Walter Bright (newshound@digitalmars.com)'s article
> Have you considered using structs instead of classes? They are allocated on the stack.

Yes, but I need to override things too. There is often some functionality that I need to keep abstract. I actually use structs quite often. Sometimes I use structs in combination with interfaces, like:

  struct Foo {
    interface Model {
      void foo();
    }
    static Foo opCall(Model foo) {
      Foo result;
      result._foo = foo;
      return result;
    }
    void foo() {
      if (_foo) _foo.foo();
    }
    private Model _foo;
  }

... but that doesn't solve the problem, because there are no destructors in structs (and it's too verbose for my taste too).
November 20, 2006
Frits van Bommel wrote:
> Array slicing, for one.
> While technically possible with malloc, it would require maintaining a
> reference count and an extra pointer to the start of the allocated
> memory area to call free() on. And that still doesn't solve the problem
> of cyclic references (if the array doesn't just contain primitive data),
> as well as requiring synchronization in multi-threaded applications.

Ok, I see.

But looking at the problem, being someone from algorithms, I think it is possible and feasible to implement array slicing using malloc allocation. An extra field will be needed anyway since you need reference count for the array, just put it along with the beginning of the array and only a single pointer will be needed for both reference count and beginning of array.

Cyclic references is a problem only when using GC. If the programmer wants to use a malloc-based allocation, he knows he should handle cyclic references himself. Not a problem.

As for synchronization, I think this is more a problem when GC is used than when it is not. Malloc-based allocation needs synchronization, of course, but I think GC-based does also. Deallocation is very atomic and can be implemented without synchronization and still be thread-safe. GC-based, OTOH, needs to freeze the whole application (not just the threads doing allocation) in order to collect.
November 20, 2006
Miles wrote:
> An extra field will be needed anyway since you need
> reference count for the array, just put it along with the beginning of
> the array and only a single pointer will be needed for both reference
> count and beginning of array.

But with GC, the extra field is not "needed anyway".. Furthermore, you can "slice" an arbitrary region of memory, and it still behaves like any other slice; with malloc, you can only slice regions specifically enabled for slicing (i.e. those that have the reference counter)


> Cyclic references is a problem only when using GC. If the programmer
> wants to use a malloc-based allocation, he knows he should handle cyclic
> references himself. Not a problem.

Cyclic references are a problem only when using reference counting. And you can't just say it's not a problem, because one knows it needs to be dealt with.. That's like saying it's really not a problem to win a lottery, because you know you need to have the right numbers on your ticket.


> As for synchronization, I think this is more a problem when GC is used
> than when it is not. Malloc-based allocation needs synchronization, of
> course, but I think GC-based does also. 

Well, for allocation it depends on implementation in both cases, but with reference counting you also need to synchronize on every reference count update, which can be very often.


> GC-based, OTOH, needs to freeze the whole application (not just the
> threads doing allocation) in order to collect.

It's not strictly necessary, though the current implementation does..


xs0
November 20, 2006
Don Clugston wrote:
> Walter Bright wrote:
>> Boris Kolar wrote:
>>> I don't understand why a good RIAA is not already a part of D. C++ has it, so
>>> it obviously can be done. The new 'scoped' keyword is insufficient for me,
>>> because I'd like to make all my classes scoped. I can usually develop
>>> 99% of my C++ code without a single 'new' or 'malloc'. Most of my classes
>>> are small (average ~2 fields and ~4 methods) and only used locally, so I'm
>>> really angry when I think about negative performance impact they will have
>>> in D simply because a decent RIAA is missing.
>>
>> Have you considered using structs instead of classes? They are allocated on the stack.
> 
> Can you do RAII with them?
> I thought that a struct cannot have a destructor, but reading the spec again I notice there's an entry for StructAllocator and StructDeallocator, but no indication of how to use it.

They are the same as class allocators and deallocators (which are not *ctors):
http://www.digitalmars.com/d/class.html#allocators

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
November 20, 2006
Miles wrote:
> Frits van Bommel wrote:
>> Array slicing, for one.
>> While technically possible with malloc, it would require maintaining a
>> reference count and an extra pointer to the start of the allocated
>> memory area to call free() on. And that still doesn't solve the problem
>> of cyclic references (if the array doesn't just contain primitive data),
>> as well as requiring synchronization in multi-threaded applications.
> 
> Ok, I see.
> 
> But looking at the problem, being someone from algorithms, I think it is
> possible and feasible to implement array slicing using malloc
> allocation. An extra field will be needed anyway since you need
> reference count for the array, just put it along with the beginning of
> the array and only a single pointer will be needed for both reference
> count and beginning of array.

With malloc/refcounting you need:
For every allocation:
* Reference count (possibly in array)
* Some kind of synchronization structure. (may be just a bit in the refcount, since 31 bits is probably enough :) )
In every reference:
* Pointer to array (for refcount & free when refcount == 0)
* Pointer to start of slice
* Length of slice or pointer to end of slice (or one byte further)

Current D implementation has (AFAIK):
* No extra overhead per allocation
In every reference:
* Pointer to start of slice
* Length of slice

So malloc/refcount takes an extra 4 bytes per allocation (assuming 32-bit refcount + synch) plus an extra 4 bytes per _reference_ (assuming 32-bit pointer). (on top of synch issues discussed below)

> Cyclic references is a problem only when using GC. If the programmer
> wants to use a malloc-based allocation, he knows he should handle cyclic
> references himself. Not a problem.

Actually, cyclic references are not a problem for GC. Not having to handle them is in fact one of the benefits of GC, or (depending on how you look at it), having to handle them is a problem with reference counting.

> As for synchronization, I think this is more a problem when GC is used
> than when it is not. Malloc-based allocation needs synchronization, of
> course, but I think GC-based does also. Deallocation is very atomic and
> can be implemented without synchronization and still be thread-safe.
> GC-based, OTOH, needs to freeze the whole application (not just the
> threads doing allocation) in order to collect.

I wasn't talking about allocation, I was talking about slicing arrays, copying of references, passing references as parameters, deleting objects containing references, returning from functions holding references and returning references. (though return-value optimization may remove that last one, if only one reference to it existed in the function)