----- Original Message ----
> From: Fawzi Mohamed <fawzi at gmx.ch>

> >  http://d.puremagic.com/issues/show_bug.cgi?id=4572
> 
> also tango has this  problem, and I had discussed about it with kris exactly
>because
> I was worried  about the NO_SCAN bits would be wrong, and so I found ubyte[] a
>better
> choice.
> But it turns out that (at least in tango) void[] is almost  only used to pass
>in a buffer where one then
> either reads or writes, and  never resized, so it was almost never a problem. So the fact that one can  cast to void[] any array easily it makes it a better
>choice than  ubyte[].
> 
> Still load and get methods that load a whole file have this  problem, and
>should allocate their
> buffers as ubyte[] (something read by a  file should *not* contain pointers).

Yes, this is somewhat equivalent to allocating via gc_malloc with the correct bits set.

> Not sure if the return type of those  methods should be ubyte[] after all (I
>lean a bit on yes
> because calling  those methods you will need to cast the void[] away, not cast
>to void[]
>  as  all other cases), but for sure the internal allocation should be of an
>ubyte[].

I proposed that you be able to specify the type of the return array via a template.  a la std.file.read!ubyte()  And have this statically disable allowing types that contain pointers.

The thing about ubyte is that you still need to cast it if ubyte is not what you are looking for.  For example, if I expect a file is a utf-8, I want to read it as char[].  At least void[] makes no assumptions as to the type.  I like it in the pure sense that it does the Right Thing.  But ubyte is probably the more practical thing to do.

Thankfully, the template solution makes all this moot since you can simply choose the type you want the function to return ;)

> > OK, I'll go about making the  change for dup.
> ok great, it seems that sometime it is possible to reach a  conclusion to a
>discussion quickly,
> somehow refreshing  :)

hehe :)  Often times, people start "discussing" with the expected conclusion in mind, and cannot seem to release that goal.  I try to keep an open mind but I'm certainly guilty of explaining how it will be vs. discussing what it should be.

-Steve

On 6-ago-10, at 14:48, Steve Schveighoffer wrote:

> [...]

> Thankfully, the template solution makes all this moot since you can
> simply
> choose the type you want the function to return ;)

yep :)

>>> OK, I'll go about making the  change for dup.
>> ok great, it seems that sometime it is possible to reach a
>> conclusion to a
>> discussion quickly,
>> somehow refreshing  :)
>
> hehe :)  Often times, people start "discussing" with the expected
> conclusion in
> mind, and cannot seem to release that goal.  I try to keep an open
> mind but I'm
> certainly guilty of explaining how it will be vs. discussing what it
> should be.

well to be fair I also fall ino that trap, and actually some lengthly
discussions
can be interesting, but not everything has to be dissected forever,
there are
diminishing returns, even if all contributions were well tought out,
and a given
point it often it is just about choosing which tradeoffs to make
(which is always
a bit arbitrary).

ok now I stop before I succesfully transform this in another lengthly discussion ;)

Fawzi

On Aug 6, 2010, at 5:03 AM, Steve Schveighoffer wrote:
> 
> Well, let me get to the root of the problem then :)
> 
> The issue is that phobos deals with void[] for things like I/O, which to me seems like the correct type.  It allows you to pass in any array type without casting, and it says "I don't know what type this is".  One of these functions is std.file.read() which reads the entire file into an array and returns that array as a void[].  Given that all the C functions use void *, it seems like the right call.  But problems come in because void[] has the NO_SCAN bit cleared, but this clearly has no pointers in it (the source is a file!).  So what to do?

The buffer can be passed around as void[] but the type that's allocated should be byte[] or ubyte[].  Allocating a void[] type will make the block scannable, which isn't desirable.

> std.file.read gets around this by calling gc_malloc directly.  However, it was rightfully pointed out that if one simply dups the array, or uses concatenation with it, it magically loses the NO_SCAN bit.  See the bug report here: http://d.puremagic.com/issues/show_bug.cgi?id=4572

Is it really necessary to call gc_malloc() here?  Is the reason simply to avoid the zero initialization?

----- Original Message ----
> From: Sean Kelly <sean at invisibleduck.org>
> To: D's runtime library developers list <d-runtime at puremagic.com>
> Sent: Fri, August 6, 2010 3:20:02 PM
> Subject: Re: [D-runtime] dup attributes?
> 
> On Aug 6, 2010, at 5:03 AM, Steve Schveighoffer wrote:
> > 
> > Well,  let me get to the root of the problem then :)
> > 
> > The issue is that  phobos deals with void[] for things like I/O, which to me

> > seems like  the correct type.  It allows you to pass in any array type
>without
>
> >  casting, and it says "I don't know what type this is".  One of these
>functions
>
> > is std.file.read() which reads the entire file into an array  and returns
>that
>
> > array as a void[].  Given that all the C  functions use void *, it seems like
>the
>
> > right call.  But problems  come in because void[] has the NO_SCAN bit
>cleared,
>
> > but this clearly  has no pointers in it (the source is a file!).  So what to
>do?
> 
> The  buffer can be passed around as void[] but the type that's allocated should
>be  byte[] or ubyte[].  Allocating a void[] type will make the block scannable, which isn't desirable.

I believe gc_malloc is used, with the NO_SCAN bit set.  Essentially, this is the same as allocating a ubyte[].

> 
> > std.file.read gets around this by calling  gc_malloc directly.  However, it
>was
>
> > rightfully pointed out that  if one simply dups the array, or uses
>concatenation
>
> > with it, it  magically loses the NO_SCAN bit.  See the bug report here:
> >  http://d.puremagic.com/issues/show_bug.cgi?id=4572
> 
> Is it really necessary  to call gc_malloc() here?  Is the reason simply to
>avoid the zero  initialization?

Hm... good point.  I hadn't thought of the "zero-initialization" issue.  So yes, it is necessary to use gc_malloc.  Do you have a good reason why gc_malloc shouldn't be used?

-Steve

On Aug 6, 2010, at 12:38 PM, Steve Schveighoffer wrote:
> 
> ----- Original Message ----
>> From: Sean Kelly <sean at invisibleduck.org>
>> 
>> Is it really necessary  to call gc_malloc() here?  Is the reason simply to avoid the zero  initialization?
> 
> Hm... good point.  I hadn't thought of the "zero-initialization" issue.  So yes, it is necessary to use gc_malloc.  Do you have a good reason why gc_malloc shouldn't be used?

Nope :-)