Is it possible to have a structure with a dynamic size? The structure would contain an array.

I know I can use templates, but the size won't be known at compile time. I also know I could just put a dynamic array into it, but that way it would just be a pointer.

I know there would be major issues like how to pass the struct to a function, as it has an unknown size, but to be quite honest I just want pretty code. I'm doing network related operations. I was hoping there'd still be a way to do this using templates or so? I just don't want to go through the hassle of writing a constructor for it to fill in all the fields, and a toByteArray method to convert it back to raw data.

struct MyStruct {
        ulong length;
        ubyte[length] data; // obv won't compile
}

On Wed, 16 Apr 2014 23:15:40 +0000, Jeroen Bollen wrote:

> Is it possible to have a structure with a dynamic size? The structure would contain an array.
> 
> I know I can use templates, but the size won't be known at compile time. I also know I could just put a dynamic array into it, but that way it would just be a pointer.
> 
> I know there would be major issues like how to pass the struct to a function, as it has an unknown size, but to be quite honest I just want pretty code. I'm doing network related operations. I was hoping there'd still be a way to do this using templates or so? I just don't want to go through the hassle of writing a constructor for it to fill in all the fields, and a toByteArray method to convert it back to raw data.
> 
> struct MyStruct {
>          ulong length;
>          ubyte[length] data; // obv won't compile
> }

No, you can't make the structs dynamic.  I had a similar situation reading shapefiles a while ago and approached it like this:

struct lengthOf
{
   string fieldName;
}

struct Polygon
{
   ulong numParts;
   @lengthOf("numParts") Part[] parts;

   ulong numPoints;
   @lengthOf("numPoints") Point[] points;
}

T read(T)(...)
    if (/** T is one of your wire-format structs **/)
{
    T ret;
    foreach (I, field; ret.tupleof)
    {
        static if (isDynamicArray!(typeof(field)))
        {
            // use __traits(getAttributes) on the field to get the
lengthOf attribute
            // mixin a line like this:
            mixin(`field.length = ret.`~lengthOf.fieldName~`;`);
            rawRead(field);

        } else {
            // Assume the field is a primitive and read it
        }
    }
    return ret;
}

Can't find the actual code at the moment, but that's the gist of it.

Jeroen Bollen:

> Is it possible to have a structure with a dynamic size?

See an usage example I have written here:
http://rosettacode.org/wiki/Sokoban#Faster_Version

But that code requires a very updated compiler. Otherwise you will need a little different code.

Bye,
bearophile

On Wednesday, 16 April 2014 at 23:15:43 UTC, Jeroen Bollen wrote:
> Is it possible to have a structure with a dynamic size? The structure would contain an array.
>
> I know I can use templates, but the size won't be known at compile time. I also know I could just put a dynamic array into it, but that way it would just be a pointer.
>
> I know there would be major issues like how to pass the struct to a function, as it has an unknown size, but to be quite honest I just want pretty code. I'm doing network related operations. I was hoping there'd still be a way to do this using templates or so? I just don't want to go through the hassle of writing a constructor for it to fill in all the fields, and a toByteArray method to convert it back to raw data.
>
> struct MyStruct {
>         ulong length;
>         ubyte[length] data; // obv won't compile
> }

Dynamic structs are impossible with D's static type system. However I suspect vibe's BSON module [1] is pretty close to what you're looking for; it has a nice API and can easily be sent in binary form.

[1] http://vibed.org/api/vibe.data.bson/

On Wednesday, 16 April 2014 at 23:36:05 UTC, bearophile wrote:
> Jeroen Bollen:
>
>> Is it possible to have a structure with a dynamic size?
>
> See an usage example I have written here:
> http://rosettacode.org/wiki/Sokoban#Faster_Version
>
> But that code requires a very updated compiler. Otherwise you will need a little different code.
>
> Bye,
> bearophile

Just in case, the key line to pay attention to in that example is this one:

CellIndex[0] c_;

It is a commonly used C idiom for dynamically sized structures that D also supports.

On Thursday, 17 April 2014 at 00:55:19 UTC, Dicebot wrote:
> Just in case, the key line to pay attention to in that example is this one:
>
> CellIndex[0] c_;
>
> It is a commonly used C idiom for dynamically sized structures that D also supports.

Absolutely. However, from a technical point of view, it doesn't make the structure "dynamically sized". It is only a hack to access data past the end of the struct.

You still have to manually and dynamically allocate the struct:
auto p = cast(State*)malloc(blockSize * stateSize);

And accessing data is done via .ptr, to avoid out of bounds.

    CellIndex get(in size_t i) inout pure nothrow {
        return c_.ptr[i];
    }

Dicebot:

> Just in case, the key line to pay attention to in that example is this one:
>
> CellIndex[0] c_;

When you define dynamically sized structs it's also a good idea to wrap the items access with some kind of get/set functions, to make the code less bug-prone (and sometimes there is also a length available somewhere that those functions can use to verify the bounds in non-release builds).


> It is a commonly used C idiom for dynamically sized structures that D also supports.

D supports zero length fixed size arrays for this usage too. And recently their .ptr was changed from null to their start addreess to improve this usage.

Bye,
bearophile

Bound checked version of variable size struct: http://dpaste.dzfl.pl/fcd91d6912d3

On Thursday, 17 April 2014 at 17:15:15 UTC, Kagamin wrote:
> Bound checked version of variable size struct: http://dpaste.dzfl.pl/fcd91d6912d3

Well, indexes getter could take cellCount into account...

Kagamin:

> Bound checked version of variable size struct: http://dpaste.dzfl.pl/fcd91d6912d3

I think you are missing one of the main points of a variable sized struct, that is to reduce by 1 the number of indirection levels.

Bye,
bearophile