So I'm experimenting and want to override how arrays are managed for a short duration, this is for optimization of not having to make another array to keep track of lengths and then shorten them when i can in fact just manage it myself *very* briefly.

 So, I need to make sure the structure is compatible with the built-in array structure (since the remainder of the calls are via normal convention and no allocation/resizing is done except by progressively smaller slices, so it should be compatible as long as it's a valid pointer).

 But i seem to have an issue trying to test if ptr is at offset 0 or not. Having the wrong order would be catastrophic, and if the source changes (for whatever reason) i don't want to have to fix it.

 So... the obvious test: static if ([].ptr.offsetof == 0)

 If the ptr is at offset 0, we declare it first, otherwise second, simple... Except this fails since "no property 'offsetof' for type 'void*'". SO... I make a template to test for it instead.

template isArrayPtrOffsetZero() {
  auto check() {
    static assert([].sizeof == (size_t.sizeof*2));
    size_t[] arr = new size_t[1];      //length == 1.
    return *(cast(size_t*) &arr) != 1;
  }
  enum isArrayPtrOffsetZero = check();
}

struct X {
  static if (isArrayPtrOffsetZero!()) {
    int* ptr;
    size_t length;
  } else { ... }
}

 Except this blows up and always gives the wrong answer... ==1 length test reverses it but is now the wrong test, and changing it to isArrayLengthOffsetZero will in turn give the wrong answer again...

 Am i going about this the wrong way?

try this:

struct X
{
	byte[] data;
	alias data this;
}

On Thursday, 26 May 2016 at 09:16:48 UTC, Kagamin wrote:
> try this:
>
> struct X
> {
> 	byte[] data;
> 	alias data this;
> }

 This doesn't help me with what I want, and effectively does nothing as far as I can tell. Although testing is showing managing the pointer & length directly is a little slower for whatever reason.

On 05/26/2016 09:51 AM, Era Scarecrow wrote:
>   So I'm experimenting and want to override how arrays are managed for a
> short duration, this is for optimization of not having to make another
> array to keep track of lengths and then shorten them when i can in fact
> just manage it myself *very* briefly.
>
>   So, I need to make sure the structure is compatible with the built-in
> array structure (since the remainder of the calls are via normal
> convention and no allocation/resizing is done except by progressively
> smaller slices, so it should be compatible as long as it's a valid
> pointer).

I don't follow. Why can't you use a built-in array? What can you do with the stand-in that you can't do with the array itself?

[...]
> template isArrayPtrOffsetZero() {
>    auto check() {
>      static assert([].sizeof == (size_t.sizeof*2));
>      size_t[] arr = new size_t[1];      //length == 1.
>      return *(cast(size_t*) &arr) != 1;
>    }
>    enum isArrayPtrOffsetZero = check();
> }
[...]
>   Except this blows up and always gives the wrong answer... ==1 length
> test reverses it but is now the wrong test, and changing it to
> isArrayLengthOffsetZero will in turn give the wrong answer again...

Here's how I understand that code:
isArrayPtrOffsetZero is supposed to check if the offset of ptr is 0. You do that by setting the length of some array to 1, and then checking if the first half of the array structure is not 1. If it's 1, then length's offset is 0, so ptr's offset can't be 0. If it's not 1, then length's offset isn't 0, so ptr's offset must be 0.

Seems a bit back and forth, but as far as I can see it works as you want. Why do you think the answer it gives is wrong? What answer does it give you?

On Thursday, 26 May 2016 at 07:51:46 UTC, Era Scarecrow wrote:
> ...

This smells like an XY problem [0]. Why exactly do you need the internal layout of the array structure?

The line "not having to make another array to keep track of lengths and then shorten them" is fairly vague. "Shortening" an array via slicing is basically free (it's just some integer arithmetic), but I'm not sure if that's what you meant.

[0]: http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem

On Thursday, 26 May 2016 at 07:51:46 UTC, Era Scarecrow wrote:
>  If the ptr is at offset 0, we declare it first, otherwise second, simple... Except this fails since "no property 'offsetof' for type 'void*'". SO... I make a template to test for it instead.


Built-in arrays are kinda magical and thus their members aren't ordinary members you can inspect like that.

>     return *(cast(size_t*) &arr) != 1;

And I'm pretty sure that cast is illegal in CTFE. Perhaps the bug here is that dmd isn't throwing an error when it is supposed to...

The value being compared gives 0 when I try it, which wouldn't make sense for either .length or .ptr.

>  Am i going about this the wrong way?

I don't think there's a need for a test because the ABI spec says length is always defined to be at offset zero anyway:

http://dlang.org/spec/abi.html#arrays

If you want to test it, a runtime assert would probably be enough, at least then it won't do the wrong thing and your message can tell the user to recompile.

Runtime assert is prolly better anyway just in case ctfe disagrees with the execution platform.

On Thursday, 26 May 2016 at 12:30:30 UTC, Alex Parrill wrote:
> The line "not having to make another array to keep track of lengths and then shorten them" is fairly vague. "Shortening" an array via slicing is basically free (it's just some integer arithmetic), but I'm not sure if that's what you meant.


Indeed, creating a slice from a static array, another slice, or a pointer is free too...

On Thursday, 26 May 2016 at 11:47:13 UTC, ag0aep6g wrote:
> I don't follow. Why can't you use a built-in array? What can you do with the stand-in that you can't do with the array itself?

 I can do the job with the built-in arrays, however the need for another temporary array to keep track of lengths so I can slice them after the fact is what i want to avoid. I'm also slicing static arrays. I'm trying to see if this code would be more efficient or not (if it is, it's not by much, which I'm sure has to do with the cache more than anything else).

 To do what I want currently it's something like...

  enum Size = 1024, Other = 128;
  Data[Size][Other] staticarray;  //stack allocation
  Data[][] sliced = staticarray[];
  scan(sliced, condition);

  void scan(ref Data[][] data, Condition cond) {
    int lengths[Size];

    foreach(i; ...) {
      if (cond)
        data[i][lengths[i]++] = ...
    }

    //cleanup/shrink
    foreach(i, l; lengths)
      data[i] = data[i][0 .. l];
  }

  By adding a struct overload for opOpAssign I can shrink it all down to this, and avoid lengths entirely... As such internally the length starts at 0, and checks are in place to ensure the bounds are never exceeded.

  void scan(ref Data[][] data, Condition cond) {
    foreach(i; ...) {
      if (cond)
        data[i] ~= ...
    }
  }


> Seems a bit back and forth, but as far as I can see it works as you want. Why do you think the answer it gives is wrong? What answer does it give you?

 The answer is wrong _because_ the code blows up. I inverted the check to check if length is at offset 0 since (the length can only be 1 and a better test), however it _still_ gives the wrong answer. Curiously CTFE is fine with this casting as long as the pointer stays in that one spot.

template isArrayLengthOffsetZero() {
  bool check() {
    size_t[] arr = new size_t[1];
    return *(cast(size_t*) &arr) == 1;
  }

  enum isArrayLengthOffsetZero = check(); //wrong answer
//  enum isArrayLengthOffsetZero = !check(); //right answer but logically wrong
}

//test struct
struct S(T) {
  static if (isArrayLengthOffsetZero!()) {
    size_t length;
    T* ptr;
  } else {
    T* ptr;
    size_t length;
  }

  this(T* p) { ptr = p; }
  //opOpAssign
}

unittest {
  import std.stdio;

  //union to get our overlapping data types
  union X { string str; S!(immutable char) s_str;}
  X x;
  x.s_str = S!(immutable char)("t".ptr); //only ptr assigned

  //writeln(x.str);          //blows up
  writeln(x.str.length);     //not 0...
  assert(x.str.length == 0); //fails
}


 Most likely the internal array structure CTFE uses for the array is inverted, and if that's the case the proper answer to how to get the simple offset of ptr (or length) is a mess.

 Forcing the inverted answer works, but this is a lot of overhead to try and make an algorithm faster. Speedups are minimal at best, so I'm about ready to drop this line of thought.

On Thursday, 26 May 2016 at 12:33:31 UTC, Adam D. Ruppe wrote:
> On Thursday, 26 May 2016 at 12:30:30 UTC, Alex Parrill wrote:
>> The line "not having to make another array to keep track of lengths and then shorten them" is fairly vague. "Shortening" an array via slicing is basically free (it's just some integer arithmetic), but I'm not sure if that's what you meant.
>
>
> Indeed, creating a slice from a static array, another slice, or a pointer is free too...

 But managing the lengths separately and then re-slicing it afterwards is not free. It does have a low constant cost though; but can I remove that cost entirely (including the foreach loop overhead)?

 There's also the annoyance that on windows machines the stack size seems to be fairly small. Sometimes in my tests the program will crash before it starts because the requested size of a static array on the stack is too big. This means if I have a big stack allocation already, I want to try and avoid further allocations if I can avoid it.

 byte[1024*1024] buffer; //on my machine, it just crashes when this is allocated.

On 05/26/2016 11:13 PM, Era Scarecrow wrote:
>    By adding a struct overload for opOpAssign I can shrink it all down
> to this, and avoid lengths entirely... As such internally the length
> starts at 0, and checks are in place to ensure the bounds are never
> exceeded.
>
>    void scan(ref Data[][] data, Condition cond) {
>      foreach(i; ...) {
>        if (cond)
>          data[i] ~= ...
>      }
>    }

Sorry, I'm still lost. Why can't you do whatever you're doing in opOpAssign directly there, or in a free function? Does the pseudo-array contain any additional data? Would a wrapper around a built-in array not work?

[...]
>   The answer is wrong _because_ the code blows up. I inverted the check
> to check if length is at offset 0 since (the length can only be 1 and a
> better test), however it _still_ gives the wrong answer. Curiously CTFE
> is fine with this casting as long as the pointer stays in that one spot.

Oh, I see. Yeah, the answer is garbage. Seeing the spec section that Adam D. Ruppe linked, the length field actually comes first.

That CTFE errors out when you move the pointer to a variable screams that something is wrong. The code seems to be seriously misinterpreted in CTFE.

----
enum x = () {
    size_t[] arr = [13];
    return *(cast(size_t*) &arr);
} ();
pragma(msg, x);
----

That prints "13LU". Apparently &arr is mistaken for arr here.

I've filed an issue:
https://issues.dlang.org/show_bug.cgi?id=16081