I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.

Is there an efficient way of converting this information to a D array of type T[] or even T[n]?

On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
>
> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?

Slice it:

T[] = t[0 .. n];

On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:
> On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
>> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
>>
>> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?
>
> Slice it:
>
> T[] = t[0 .. n];

 You can do that??? I thought slices weren't allowed on raw pointers. You just blew my mind away!

On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:
> On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
>> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
>>
>> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?
>
> Slice it:
>
> T[] = t[0 .. n];

That! ...is amazing!

On Tuesday, May 24, 2016 19:52:17 Era Scarecrow via Digitalmars-d-learn wrote:
> On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:
> > On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
> >> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
> >>
> >> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?
> >
> > Slice it:
> >
> > T[] = t[0 .. n];
>
>   You can do that??? I thought slices weren't allowed on raw
> pointers. You just blew my mind away!

A dynamic array is basically just

struct Array(T)
{
    size_t length;
    T* ptr;
}

It can point to any memory. Much as its the default, a dynamic array does not need to be backed by GC-allocated memory. It's just that unlike when it's backed by GC-allocated memory, you have to worry about keeping track of whether that memory is currently being used and when it needs to be freed, and the dynamic array is guranteed to not have enough capacity to grow in place, so any kind of array operation that would require growing the array (e.g. calling reserve or appending) would be guaranteed to reallocate, whereas when the dynamic array is backed by GC-allocated memory, it may or may not have to reallocate when you do something like append to it or call reserve. But aside from worrying about keeping track of its original memory, having a dynamic array backed by malloc-ed memory is basically semantically identical to having it backed by the GC. T[] really doesn't care what memory it refers to. It works either way as long as the memory it refers to continues to be valid.

- Jonathan M Davis

On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
>
> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?

BTW, the simplest way to use C malloc to allocate and initialize a D array is with http://dlang.org/phobos/std_experimental_allocator#makeArray:

struct MyObj { /* ... */ }

{
    MyObj[] arr = Mallocator.instance
        .makeArray!MyObj(10, ctorArgs);
    scope (exit) Mallocator.instance.dispose(arr);

    // use arr...

} // arr is freed at the end of the scope

Which is rougly equivalent to:

void* p = malloc(10 * MyObj.sizeof);
T[] arr = (cast(T*)p)[0 .. 10];
foreach (ref elem; arr)
    emplace(elem, ctorArgs);

The benfit of using make* and dispose is that they allow you to change the allocator if you so need. You can also use the GC:
int[] arr = GCAllocator.instance.makeArray!int(128);
// no need to free arr

The common idiom is that the user of a function that produces an array should supply the allocator as a parameter through which the function will make allocations. This useful when you know for example that the allocation would be small and short-lived in which case it would better to use a stack allocator, rather than a heap allocator.

Am Tue, 24 May 2016 20:58:14 +0000
schrieb Gary Willoughby <dev@nomad.so>:

> On Tuesday, 24 May 2016 at 18:43:22 UTC, Adam D. Ruppe wrote:
> > On Tuesday, 24 May 2016 at 18:42:41 UTC, Gary Willoughby wrote:
> >> I have a T* pointer to the start of a malloc'd chunk of memory, the type T and the number of T's stored in the chunk.
> >>
> >> Is there an efficient way of converting this information to a D array of type T[] or even T[n]?
> >
> > Slice it:
> >
> > T[] = t[0 .. n];
> 
> That! ...is amazing!

How did you _ever_ read structs from files without this knowledge?

-- 
Marco

On Tuesday, 24 May 2016 at 19:52:17 UTC, Era Scarecrow wrote:
>  You can do that??? I thought slices weren't allowed on raw pointers. You just blew my mind away!

Slicing and indexing are both allowed (actually, that's a way to disable bounds checking on an individual expression, see tip: http://arsdnet.net/this-week-in-d/dec-06.html ) on pointers, though it isn't considered @safe.

Be careful though: it is still manually managed, so if you pass it to some function that keeps a slice to your slice, you open yourself up to a use-after-free bug!

On Wednesday, 25 May 2016 at 12:06:58 UTC, Adam D. Ruppe wrote:
> Slicing and indexing are both allowed (actually, that's a way to disable bounds checking on an individual expression, see tip: http://arsdnet.net/this-week-in-d/dec-06.html ) on pointers, though it isn't considered @safe.
>
> Be careful though: it is still manually managed, so if you pass it to some function that keeps a slice to your slice, you open yourself up to a use-after-free bug!

 If I knew about this trick in the past, it's been years so it would have been easy to forget. Still awesome to rediscover. Rereading a lot of stuff.

 As for passing slices to functions, I generally only do that when I know it won't hold onto it (or try to resize it or anything weird), which then lets me use stack memory rather than heap memory for temporaries.