I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a "regular" string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance.

So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated.

On Wednesday, 7 May 2014 at 18:26:08 UTC, Maxime Chevalier-Boisvert wrote:
> I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a "regular" string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance.
>
> So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated.

Unless I'm misunderstanding it should be as simple as:

wchar[100] stackws; // alloca() if you need it to be dynamically sized.

A slice of this static array behaves just like a slice of a dynamic array.

On Wednesday, 7 May 2014 at 18:29:23 UTC, Brad Anderson wrote:
> Unless I'm misunderstanding it should be as simple as:
>
> wchar[100] stackws; // alloca() if you need it to be dynamically sized.
>
> A slice of this static array behaves just like a slice of a dynamic array.

But you should avoid slicing the static array unless it's via arr.dup.

> Unless I'm misunderstanding it should be as simple as:
>
> wchar[100] stackws; // alloca() if you need it to be dynamically sized.
>
> A slice of this static array behaves just like a slice of a dynamic array.

I do need it to be dynamically sized. I also want to avoid copying my string data if possible. Basically, I just want to create a wstring "view" on an existing "raw" buffer that exists in memory somewhere, based on a pointer to this buffer and its length.

Side note: wouldn't alloca just produce a wchar*, not an array (wouldn't have length information)?

Meta:

> But you should avoid slicing the static array unless it's via arr.dup.

Slicing fixed size arrays is often necessary, because many Phobos functions don't accept fixed size arrays, they force you to lose the compile-time knowledge of the length.

Bye,
bearophile

Maxime Chevalier-Boisvert:

> I do need it to be dynamically sized.

But often you can determine statically a maximum length of the string, so you can use a fixed size stack buffer and slice it with a dynamic length. If this is not acceptable, then use alloca.


> Basically, I just want to create a wstring "view" on an existing "raw" buffer that exists in memory somewhere, based on a pointer to this buffer and its length.

This is named slicing. You can also slice a global/static/__gshared buffer.


> Side note: wouldn't alloca just produce a wchar*, not an array (wouldn't have length information)?

alloca returns a void*, then you can cast it to the pointer type you want, and then you slice the pointer:

auto ptr = cast(wchar*)alloca(wchar.sizeof * len);
if (ptr == null) throw new Error("...");
auto mySlice = ptr[0 .. len];

Bye,
bearophile

> This is named slicing. You can also slice a global/static/__gshared buffer.

> alloca returns a void*, then you can cast it to the pointer type you want, and then you slice the pointer:
>
> auto ptr = cast(wchar*)alloca(wchar.sizeof * len);
> if (ptr == null) throw new Error("...");
> auto mySlice = ptr[0 .. len];

Is the slice going to be allocated on the stack? (I imagine the answer is yes)

Thanks for your help Mr Bear.

Maxime Chevalier-Boisvert:

> Is the slice going to be allocated on the stack? (I imagine the answer is yes)

Slicing doesn't change where the data is allocated. Slicing means just creating a new struct that contains a length and pointer to the data (and the struct itself is allocated in-place. So it's allocated on the stack, or inside a struct instance, or inside a class instance, or on the data segment, etc).


> Thanks for your help Mr Bear.

You are welcome :-)

Bye,
bearophile

>> Is the slice going to be allocated on the stack? (I imagine the answer is yes)
>
> Slicing doesn't change where the data is allocated. Slicing means just creating a new struct that contains a length and pointer to the data (and the struct itself is allocated in-place. So it's allocated on the stack, or inside a struct instance, or inside a class instance, or on the data segment, etc).

Indeed. It's the struct representing the slice I was asking about. Off to test out the performance impact :)

On Wednesday, 7 May 2014 at 18:41:17 UTC, Maxime Chevalier-Boisvert wrote:
> Basically, I just want to create a wstring "view" on an existing "raw" buffer that exists in memory somewhere, based on a pointer to this buffer and its length.

Looks like you actually don't need to allocate anything at all. Any slice is just a struct with 2 fields, it does not own data and can be used as a view to anything:

void* some_external_data;
size_t external_length;

void foo()
{
    auto str = cast(wstring[])(some_external_data[0..external_length]);
    // this will create a stack instance of slice struct with `.ptr` field pointing to same address as `some_external_data`
}