http://d.puremagic.com/issues/show_bug.cgi?id=6652

           Summary: foreach parameter with number range is always ref
           Product: D
           Version: D2
          Platform: Other
        OS/Version: FreeBSD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody@puremagic.com
        ReportedBy: dawg@dawgfoto.de


--- Comment #0 from dawg@dawgfoto.de 2011-09-12 10:19:15 PDT ---
void main() {
  size_t cnt;
  foreach(ulong n; 0 .. 10)
  {
    ++n;
    ++cnt;
  }
  assert(cnt == 10);

  cnt = 0;
  foreach(ref ulong n; 0 .. 10)
  {
    ++n;
    ++cnt;
  }
  assert(cnt == 5);
}

---

As this is rewritten in terms of a for loop all writes to n will
alter the loop.
A hidden copy of n is needed for non-ref parameters to match the range
foreach semantic.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652


bearophile_hugs@eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs@eml.cc


--- Comment #1 from bearophile_hugs@eml.cc 2011-09-12 12:24:41 PDT ---
Recently I have started a very long thread about this problem:

http://www.digitalmars.com/d/archives/digitalmars/D/About_foreach_loops_138630.html

http://www.digitalmars.com/d/archives/digitalmars/D/Re_About_foreach_loops_138666.html


An alternative and probably a bit better solution is to think of 0..10 as a immutable entity (just like the integer number "1" is immutable, likewise a range of numbers is immutable), so the foreach index is a const (the compiler keeps only one index, for efficiency, and modifies this const index):


foreach (int i; 0 .. 10) {
    i++; // forbidden, i is const
}


If you want to modify the index variable inside the loop, then you use a for()
loop.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #2 from dawg@dawgfoto.de 2011-09-12 14:35:51 PDT ---
Making a const/immutable copy is not the right solution to this.
Instead a mutable copy of a hidden loop variable should be made.
Being a copy is the common behavior for non-ref foreach arguments,
to my surprise it has even become my intuitive assumption of what's happening.
The old behavior can be achieved through a ref argument.

not possible using const:
foreach(i; 1 .. 10)
  while(i--) { do some }

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #3 from bearophile_hugs@eml.cc 2011-09-12 15:10:08 PDT ---
(In reply to comment #2)
> Making a const/immutable copy is not the right solution to this.

Keep in mind that foreach(i;0..10) must have *zero* abstraction penalty over a for loop even with non-optimizing D compilers, because it's meant to replace for loops everywhere possible. The more abstraction you put into foreach there higher the probability it will not have zero abstraction penalty (currently it has a bit of penalty for nested loops, sometimes).


> not possible using const:
> foreach(i; 1 .. 10)
>   while(i--) { do some }

Use a for loop :-)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #4 from dawg@dawgfoto.de 2011-09-13 00:02:29 PDT ---
https://github.com/D-Programming-Language/dmd/pull/377

Foreach arguments behave like function arguments. Here they don't.
The variable can be optimized out, if no altering happens.
This will not happen in a debug build, where it is irrelevant in comparison to
every variable being accessed through the stack.

You can use ref, if you're having too expensive copies
foreach(ref const i; iter(0) .. iter(10)) as with every other foreach argument.

Most important it has an explicit rule, that one can alter the loop index
through using a ref index.
foreach(ref idx, v; [0, 1, 2, 3, 4, 5])
  idx += stride - 1;

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #5 from bearophile_hugs@eml.cc 2011-09-13 03:59:49 PDT ---
(In reply to comment #4)
> https://github.com/D-Programming-Language/dmd/pull/377

> - I've double checked that a simple size_t index is optimized out if unaltered

I suggest you to check it fifteen more times, using 4 nested foreach, with some code inside the bodies, with other data types (ubyte, short, ulong, real, double, etc), etc.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #6 from dawg@dawgfoto.de 2011-09-13 21:49:08 PDT ---
It really is not that much an issue of performance.
The compiler should be able to eliminate dead assignments
and *& for ref parameters.

The issue is that of breaking code. I don't know any feasible solution to attach a deprecation to this now and change it later.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652


Masahiro Nakagawa <repeatedly@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |repeatedly@gmail.com


--- Comment #7 from Masahiro Nakagawa <repeatedly@gmail.com> 2012-06-11 08:47:41 PDT ---
I hit this issue today.

Current behavior is different from foreach semantics.
So, I agree dawg opinion.
We should fix this bug!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652


Kenji Hara <k.hara.pg@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
           Platform|Other                       |All
         OS/Version|FreeBSD                     |All


--- Comment #8 from Kenji Hara <k.hara.pg@gmail.com> 2012-06-15 10:22:02 PDT ---
I've opened three pulls: https://github.com/D-Programming-Language/dmd/pull/1008 https://github.com/D-Programming-Language/dmd/pull/1009 https://github.com/D-Programming-Language/dmd/pull/1010

To reduce breaking of existing codes,
1. Warn to modifying loop variable in foreach body.
   It is shown only when -w switch is specified.
2. Deprecate modifying loop variable in foreach body.
   If user specifies -d switch, it is allowed.
3. Allow modifying loop variable in foreach body, and it does not affect to
   the number of iterations of the loop.

How about?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=6652



--- Comment #9 from bearophile_hugs@eml.cc 2012-06-15 11:56:35 PDT ---
(In reply to comment #8)

> To reduce breaking of existing codes,
> 1. Warn to modifying loop variable in foreach body.
>    It is shown only when -w switch is specified.
> 2. Deprecate modifying loop variable in foreach body.
>    If user specifies -d switch, it is allowed.
> 3. Allow modifying loop variable in foreach body, and it does not affect to
>    the number of iterations of the loop.

This is great, you are the best Kenji Hara.

I prefer the number 2. I think it breaks none of my programs.

The number 3 is a trap, because it silently changes the semantics of old D code. And it's bug-prone for new D programmers too because they can change the variable by mistake. Generally immutable variables are safer.


Are you able and willing to compile the whole Phobos with the option number 2? So we can see how often Phobos code change the foreach-on-range iteration variable.


What's the semantics of this, now?
foreach (ref int i; 0 .. 10) i++;

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------