On Sun, 21 Aug 2005 21:45:44 -0400, Ben Hinkle <ben.hinkle@gmail.com> wrote:
> I say special case because it would be the only construct in D that takes an
> expression (the indexing and assignment expression) and forces them to
> appear in their special statement form. Other expressions in D have types
> and can be assigned to variables and passed to functions or used as
> subexpressions. Compared to other expressions and statements in D it is very special.

I need a concrete example to understand the problem here.

>> It would be _much_ nicer than using  functions as it would be allow
>> shorter more concise yet more descriptive  code.
>>
>> For example, a challenge:
>>
>> What would the following code look like using your proposed functions?
>>
>> //take certain bits from 3 different variables, build a new variable
>> uint newVar;
>> ubyte varOne;
>> ushort varTwo;
>> uint  varThree;
>>
>> newVar[0..8] = varOne[3..11];
>> newVar[8..16] = varTwo[5..13];
>> newVar[16..32] = varThree[7..23];
>
> The proposal written above didn't mention copying one var[a..b] to another var[a..b] so that's another case to add.

It's just using 2 of the cases that were mentioned at the same time. How is it "another case to add"?

>> I expect the code above to:
>>  - look nicer
>>  - be clearer
>>  - be more concise
>
> sigh. There's more to adding features than making a small set of user code "more concise".

The goal of this suggestion is to make working with bits easier. Part of that is being able to do things without strange and/or complicated code. Part of that is then having a nice concise syntax. Part of that is having a syntax that is clear as to what it's doing.

As you've already mentioned there are many pitfalls when dealing with bits eg. little/big endian byte ordering, packing, etc. Having the compiler (or functions) handle them is a huge bonus. Having the compiler handle it with a built-in syntax is a bit (no pun intended) nicer than using functions.

I have yet to understand the problem with the "special case" you keep mentioning. The only special case I see so far is the fact that copying/assigning is required if slicing over byte boundaries.

>> than anything you can come up with. However I am wary about you being able to come up with something ;)
>
> The most obvious API that comes to mind using a single function is
>   copybits(newVar,0,8,varOne,3,11);
> The newVar is 'inout'. The varOne is 'in'. Seems like a trivial rewrite of your syntax to me. The benefits of these functions are:
> 1) no special case
> 2) users can immediately use them without having to learn a special syntax

Users either have to learn:

 - the existance of the module AND
   the function names AND
   what they do.

 - the fact that you can slice a basic type to obtain bits AND
   the restriction that you must copy or assign over byte boundaries

The latter requires one or two sentences of explanation (assuming they know what slicing is, which they will or will need to in order to use D anyway).

> 3) no confusion with mistaking newVar and varOne with an array

I can't see how this is a problem at all. A variable _is_ (effectively) an array when you can slice it for bits. It's no different(*) to any other array (or class pretending to be an array).

*the differences play no part when you're reading existing code rather than writing new code.

Out of context you'll assume (correctly) that it's an array, no problem. In context you'll (correctly) see it as an array of bit[], again, no problem.

Regan

On Mon, 22 Aug 2005 12:26:49 +1000, Derek Parnell <derek@psych.ward> wrote:
> On Mon, 22 Aug 2005 13:14:39 +1200, Regan Heath wrote:
>
> [snip]
>
>> What would the following code look like using your proposed functions?
>>
>> //take certain bits from 3 different variables, build a new variable
>> uint newVar;
>> ubyte varOne;
>> ushort varTwo;
>> uint  varThree;
>>
>> newVar[0..8] = varOne[3..11];
>> newVar[8..16] = varTwo[5..13];
>> newVar[16..32] = varThree[7..23];
>>
>> I expect the code above to:
>>   - look nicer
>>   - be clearer
>>   - be more concise
>>
>> than anything you can come up with. However I am wary about you being able
>> to come up with something ;)
>
> My issue with this idea is that just by looking at 'newVar[0..8]' it gives the impression that newVar is an array.

And it is.

> And I don't mean an array of bits

But that's exactly what it is.

> ;-) Of course all data ultimately is stored as array(s) of bits so this
> syntax is a tad misleading.

What problems can you see it causing?

> I can see a point to the argument that it would be useful to have the
> compiler help coders who need to access variables at the bit level. But it would need a different syntax to avoid ambiguities.

I don't yet see any problem with "ambiguities". A variable is a collection of bits.

> Seeing that all data is ultimately a set of bits, and is thus intrinsic to the variable, maybe an in-built property of '.bits' might be a useful idea.
>
>  newVar.bits[0..8] = varOne.bits[3..11];
>  newVar.bits[8..16] = varTwo.bits[5..13];
>  newVar.bits[16..32] = varThree.bits[7..23];

That's 4 more chars I have to type, for no gain. IMO.

> And the compiler can do the Endian transformations on the coder's behalf.

That is a crucial part of the idea it makes bit manipulation much easier.

Regan

On Mon, 22 Aug 2005 15:00:45 +1200, Regan Heath wrote:


[snip]
>> My issue with this idea is that just by looking at 'newVar[0..8]' it gives the impression that newVar is an array.
> 
> And it is.
> 
>> And I don't mean an array of bits
> 
> But that's exactly what it is.
> 
>> ;-) Of course all data ultimately is stored as array(s) of bits so this
>> syntax is a tad misleading.

All data is a set of bits, but nearly all data is never used this way. Data is used in conceptual ways rather than implementational ways. And therefore, assuming we give common usage better support than rare usage, we should use special syntax to indicate rare usage.

> What problems can you see it causing?

Confusion.

What is the mistake here ...

 uint newVar;
 newVar[0..8] = 255;

Its hard to tell, no? I actually meant to code ...

 uint[8] newVar;
 newVar[0..8] = 255;

Now how would the compiler know that?

But consider this ...

 uint newVar;
 newVar.bits[0..8] = 255;

Anyone see the ambiguity here? Neither can I.

>> I can see a point to the argument that it would be useful to have the compiler help coders who need to access variables at the bit level. But it would need a different syntax to avoid ambiguities.
> 
> I don't yet see any problem with "ambiguities". A variable is a collection of bits.

Yes, but its not as commonly thought of as that, is it? Sure, there are cases when one *has* to bit twiddle, but these are rare.

As a code reader,  if I saw 'RecordCount[0]' I would be thinking the it was referring to the first RecordCount value and not the first bit in the record count value. It is visually confusing, so why bother with it.

>> Seeing that all data is ultimately a set of bits, and is thus intrinsic to the variable, maybe an in-built property of '.bits' might be a useful idea.
>>
>>  newVar.bits[0..8] = varOne.bits[3..11];
>>  newVar.bits[8..16] = varTwo.bits[5..13];
>>  newVar.bits[16..32] = varThree.bits[7..23];
> 
> That's 4 more chars I have to type, for no gain. IMO.

Legibility is a huge gain! Making code unambiguous and/or easier to read is saving maintenance costs.  Code is read much more frequently than written, so the 'pain' of typing four characters is paid for many times over because people read it better.

>> And the compiler can do the Endian transformations on the coder's behalf.
> 
> That is a crucial part of the idea it makes bit manipulation much easier.

Yes!

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
22/08/2005 1:03:59 PM

On Mon, 22 Aug 2005 13:19:13 +1000, Derek Parnell <derek@psych.ward> wrote:
>>> My issue with this idea is that just by looking at 'newVar[0..8]' it
>>> gives the impression that newVar is an array.
>>
>> And it is.
>>
>>> And I don't mean an array of bits
>>
>> But that's exactly what it is.
>>
>>> ;-) Of course all data ultimately is stored as array(s) of bits so this
>>> syntax is a tad misleading.
>
> All data is a set of bits, but nearly all data is never used this way. Data is used in conceptual ways rather than implementational ways.

I agree totally. When it 'conceptually' makes sense you will use [] and slice an int, otherwise you wont.

> And therefore, assuming we give common usage better support than rare usage,

We already do. 'a' is easier to use and type than 'a[x..y]'

> we should use special syntax to indicate rare usage.

Sure, that's what I am suggesting. [] _is_ special syntax. It indicates slicing, which is exactly what we're doing.

>> What problems can you see it causing?
>
> Confusion.
>
> What is the mistake here ...
>
>  uint newVar;
>  newVar[0..8] = 255;

What is the context?

> Its hard to tell, no?

Without context, yes.

> I actually meant to code ...
>
>  uint[8] newVar;
>  newVar[0..8] = 255;
>
> Now how would the compiler know that?

It wont, nor is it supposed to. The compiler verifies the code is legal, in some situations it can provide additional help. This example, like _many_ others, is not one of them.

> But consider this ...
>
>  uint newVar;
>  newVar.bits[0..8] = 255;
>
> Anyone see the ambiguity here? Neither can I.

A clunky syntax is unlikely to collide with any existing stuff by it's very nature ;)

I understand your point, I just disagree that there is any(*) more likelihood of problems here than other features where a typo can and does mean something other than the programmer intended.

*where 'any' is defined as some really small number/value.

>>> I can see a point to the argument that it would be useful to have the
>>> compiler help coders who need to access variables at the bit level. But
>>> it would need a different syntax to avoid ambiguities.
>>
>> I don't yet see any problem with "ambiguities". A variable is a collection
>> of bits.
>
> Yes, but its not as commonly thought of as that, is it? Sure, there are
> cases when one *has* to bit twiddle, but these are rare.

Sure, bit twiddling is rare.

> As a code reader,  if I saw 'RecordCount[0]' I would be thinking the it was referring to the first RecordCount value

And you'd be correct. The 'value' is a bit and RecordCount is a collection of bits.

> and not the first bit in the
> record count value. It is visually confusing, so why bother with it.

It's not visually confusing because it's idential to slicing any other sort of array, and further it will only occur in situations where it makes sense conceptually.

In other words you're only going to be using it when you want to reference the bits of something. Thus the variable will be named appropriately i.e. 'fields' or 'rawData' or something and "if (fields[IS_BINARY])" would make perfect sense.

You cannot ignore context.

>>> Seeing that all data is ultimately a set of bits, and is thus intrinsic
>>> to the variable, maybe an in-built property of '.bits' might be a useful
>>> idea.
>>>
>>>  newVar.bits[0..8] = varOne.bits[3..11];
>>>  newVar.bits[8..16] = varTwo.bits[5..13];
>>>  newVar.bits[16..32] = varThree.bits[7..23];
>>
>> That's 4 more chars I have to type, for no gain. IMO.
>
> Legibility is a huge gain! Making code unambiguous and/or easier to read is saving maintenance costs. Code is read much more frequently than written, so the 'pain' of typing four characters is paid for many times over because people read it better.

Sure, but, I still see 'no gain' in the above because I don't agree that it 'solves' any problem that the original syntax had. I believe the original syntax is as legible and unambiguous as any other instance of slicing.

Regan

On Mon, 22 Aug 2005 15:51:38 +1200, Regan Heath wrote:


> Sure, but, I still see 'no gain' in the above because I don't agree that it 'solves' any problem that the original syntax had. I believe the original syntax is as legible and unambiguous as any other instance of slicing.

The joke is that Walter's not going to do either anyway. ;-)

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
22/08/2005 2:23:18 PM

On Mon, 22 Aug 2005 14:24:38 +1000, Derek Parnell <derek@psych.ward> wrote:
> On Mon, 22 Aug 2005 15:51:38 +1200, Regan Heath wrote:
>
>
>> Sure, but, I still see 'no gain' in the above because I don't agree that
>> it 'solves' any problem that the original syntax had. I believe the
>> original syntax is as legible and unambiguous as any other instance of
>> slicing.
>
> The joke is that Walter's not going to do either anyway. ;-)

We can but hope.

Regan

On Mon, 22 Aug 2005 14:54:43 +1200, Regan Heath <regan@netwin.co.nz> wrote:
> On Sun, 21 Aug 2005 21:45:44 -0400, Ben Hinkle <ben.hinkle@gmail.com> wrote:
>> I say special case because it would be the only construct in D that takes an
>> expression (the indexing and assignment expression) and forces them to
>> appear in their special statement form. Other expressions in D have types
>> and can be assigned to variables and passed to functions or used as
>> subexpressions. Compared to other expressions and statements in D it is very special.
>
> I need a concrete example to understand the problem here.

Are you saying that:
  uint i;

cannot be assigned to a variable, eg.
  bit[] j = i[0..3];

cannot be a sub-expresion, eg.
  bit[] j = i[0..3] = 1;

or passed to a function, eg.
  foo(i[0..3]);


If so, you're quite correct about the assigned to a variable part. Due to byte boundary issues this:
  bit[] j = i[0..3];

is illegal, however:
  bit[3] j;
  j[] = i[0..3];

and:
  bit[] j;
  j.length = 3;
  j[] = i[0..3];

are both legal.

That said, if the slice started on a byte boundary it would be _possible_ to allow it. I doubt many people would like/accept the inconsistency there however.

As for the others, an expression like:
  bit[] j = i[0..3] = 1;

is processed like any other expression, in 2 sub-expressions.

The first sub-expression "i[0..3] = 1" sets the bits in the variable 'i' to 1. (as previously described)
The seccond "j = i[0..3]" is illegal (as previously described). It could instead be written as:

bit[3] j;
j[] = i[0..3] = 1;

The same applies to the function call it creates a bit[] or bit[3] on the stack copying the bits from the source and passing it to the function.

Regan

On Mon, 22 Aug 2005 17:23:11 +1200, Regan Heath wrote:

> On Mon, 22 Aug 2005 14:54:43 +1200, Regan Heath <regan@netwin.co.nz> wrote:
>> On Sun, 21 Aug 2005 21:45:44 -0400, Ben Hinkle <ben.hinkle@gmail.com> wrote:
>>> I say special case because it would be the only construct in D that
>>> takes an
>>> expression (the indexing and assignment expression) and forces them to
>>> appear in their special statement form. Other expressions in D have
>>> types
>>> and can be assigned to variables and passed to functions or used as
>>> subexpressions. Compared to other expressions and statements in D it is
>>> very special.
>>
>> I need a concrete example to understand the problem here.
> 
> Are you saying that:
>    uint i;
> 
> cannot be assigned to a variable, eg.
>    bit[] j = i[0..3];
> 
> cannot be a sub-expresion, eg.
>    bit[] j = i[0..3] = 1;
> 
> or passed to a function, eg.
>    foo(i[0..3]);
> 
> If so, you're quite correct about the assigned to a variable part. Due to
> byte boundary issues this:
>    bit[] j = i[0..3];
> 
> is illegal, however:
>    bit[3] j;
>    j[] = i[0..3];
> 
> and:
>    bit[] j;
>    j.length = 3;
>    j[] = i[0..3];
> 
> are both legal.
> 
> That said, if the slice started on a byte boundary it would be _possible_ to allow it. I doubt many people would like/accept the inconsistency there however.
> 
> As for the others, an expression like:
>    bit[] j = i[0..3] = 1;
> 
> is processed like any other expression, in 2 sub-expressions.
> 
> The first sub-expression "i[0..3] = 1" sets the bits in the variable 'i'
> to 1. (as previously described)
> The seccond "j = i[0..3]" is illegal (as previously described). It could
> instead be written as:
> 
> bit[3] j;
> j[] = i[0..3] = 1;
> 
> The same applies to the function call it creates a bit[] or bit[3] on the stack copying the bits from the source and passing it to the function.

The more I see examples of your proposal, the more I like another way of doing it. Its just *too* misleading in my mind. I keep seeing arrays of integers and not arrays of bits ;-)

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
22/08/2005 4:30:20 PM

On Mon, 22 Aug 2005 16:35:48 +1000, Derek Parnell <derek@psych.ward> wrote:
> On Mon, 22 Aug 2005 17:23:11 +1200, Regan Heath wrote:
>
>> On Mon, 22 Aug 2005 14:54:43 +1200, Regan Heath <regan@netwin.co.nz> wrote:
>>> On Sun, 21 Aug 2005 21:45:44 -0400, Ben Hinkle <ben.hinkle@gmail.com>
>>> wrote:
>>>> I say special case because it would be the only construct in D that
>>>> takes an
>>>> expression (the indexing and assignment expression) and forces them to
>>>> appear in their special statement form. Other expressions in D have
>>>> types
>>>> and can be assigned to variables and passed to functions or used as
>>>> subexpressions. Compared to other expressions and statements in D it is
>>>> very special.
>>>
>>> I need a concrete example to understand the problem here.
>>
>> Are you saying that:
>>    uint i;
>>
>> cannot be assigned to a variable, eg.
>>    bit[] j = i[0..3];
>>
>> cannot be a sub-expresion, eg.
>>    bit[] j = i[0..3] = 1;
>>
>> or passed to a function, eg.
>>    foo(i[0..3]);
>>
>> If so, you're quite correct about the assigned to a variable part. Due to
>> byte boundary issues this:
>>    bit[] j = i[0..3];
>>
>> is illegal, however:
>>    bit[3] j;
>>    j[] = i[0..3];
>>
>> and:
>>    bit[] j;
>>    j.length = 3;
>>    j[] = i[0..3];
>>
>> are both legal.
>>
>> That said, if the slice started on a byte boundary it would be _possible_
>> to allow it. I doubt many people would like/accept the inconsistency there
>> however.
>>
>> As for the others, an expression like:
>>    bit[] j = i[0..3] = 1;
>>
>> is processed like any other expression, in 2 sub-expressions.
>>
>> The first sub-expression "i[0..3] = 1" sets the bits in the variable 'i'
>> to 1. (as previously described)
>> The seccond "j = i[0..3]" is illegal (as previously described). It could
>> instead be written as:
>>
>> bit[3] j;
>> j[] = i[0..3] = 1;
>>
>> The same applies to the function call it creates a bit[] or bit[3] on the
>> stack copying the bits from the source and passing it to the function.
>
> The more I see examples of your proposal, the more I like another way of
> doing it. Its just *too* misleading in my mind. I keep seeing arrays of
> integers and not arrays of bits ;-)

The question is then, where does the problem lie, in the proposal or in your mind ;)
I don't have the same problem with it, but then perhaps _I_ am strange.

You're seeing arrays (correctly, because they are in this case) but you're assuming integers, so, if bit literals had some sort of suffix (I guess the binary suffix is it?) then you could, if you so chose, make it more obvious i.e.

i[0..3] = 1b
i[3..8] = 0b

So, you could code like that, I could code without, both of us are happy (until you have to read my code perhaps)

Regan

"Regan Heath" <regan@netwin.co.nz> wrote in message news:opsvv88xxi23k2f5@nrage.netwin.co.nz...
> On Mon, 22 Aug 2005 14:54:43 +1200, Regan Heath <regan@netwin.co.nz> wrote:
>> On Sun, 21 Aug 2005 21:45:44 -0400, Ben Hinkle <ben.hinkle@gmail.com> wrote:
>>> I say special case because it would be the only construct in D that
>>> takes an
>>> expression (the indexing and assignment expression) and forces them to
>>> appear in their special statement form. Other expressions in D have
>>> types
>>> and can be assigned to variables and passed to functions or used as
>>> subexpressions. Compared to other expressions and statements in D it is
>>> very special.
>>
>> I need a concrete example to understand the problem here.
>
> Are you saying that:
>   uint i;
>
> cannot be assigned to a variable, eg.
>   bit[] j = i[0..3];
>
> cannot be a sub-expresion, eg.
>   bit[] j = i[0..3] = 1;
>
> or passed to a function, eg.
>   foo(i[0..3]);
>
>
> If so, you're quite correct about the assigned to a variable part. Due to
> byte boundary issues this:
>   bit[] j = i[0..3];
>
> is illegal, however:
>   bit[3] j;
>   j[] = i[0..3];
>
> and:
>   bit[] j;
>   j.length = 3;
>   j[] = i[0..3];
>
> are both legal.
>
> That said, if the slice started on a byte boundary it would be _possible_ to allow it. I doubt many people would like/accept the inconsistency there however.

There is a big difference between j[] = i[a..b] and j = i[a..b] no matter
what the byte boundaries are. Consider big-endian machines with trying to
slice shorts and longs where the bit[] packing unit is a byte. Short is
positioned in memory as
  1 0
where the number "1" means that byte stores the data starting at bit 8*1.
Long as
  7 6 5 4 3 2 1 0
Assuming bit[] is made of a pointer and length with the pointer at the low
memory value it is reverse of the endian order (bit[] is currently
independent of endianness). Implementing bit[] with the pointer at the high
byte and decreasing might work for setting bits but it wouldn't mesh with
D's existing array semantics of increasing indices mapping to increasing
memory locations. Plus resizing a bit[] with decreasing contents would be
very different than resizing a standard array. That's why I think the most
practical suggestion was your original restriction of only allowing the
slicing if it is immediately copied and never exists as a bit[]. But then I
think that's too large a restriction to be useful.