Bruce Adams wrote:
> D1.0 is frozen. Bug fixes (to the spec) are permitted but major additions
> are not.

Major additions in the 1.0 series:
- string mixins, string imports in 1.005
- CTFE in 1.006
- CTFE changes in 1.007 and 1.014
- addition of the "ref" and "macro" keywords in 1.011
- struct/AA literals in 1.014
- array operations in 1.034

All of the above broke backwards compatibility with earlier 1.0 versions.

Many minor changes that could be argued as being more than just bug fixes:
- removal of === and !== in 1.005
- "final" changes in 1.011
- the infamous .init change in 1.017

All of the above broke code that compiled with earlier 1.0 versions.

The suggestion under discussion doesn't seem so bad to me, considering all that...

-- 
E-mail address: matti.niemenmaa+news, domain is iki (DOT) fi

On Sat, Sep 20, 2008 at 1:05 PM, Matti Niemenmaa <see_signature@for.real.address> wrote:
> Bruce Adams wrote:
>> D1.0 is frozen. Bug fixes (to the spec) are permitted but major additions
>> are not.
>
> Major additions in the 1.0 series:
> - string mixins, string imports in 1.005
> - CTFE in 1.006
> - CTFE changes in 1.007 and 1.014
> - addition of the "ref" and "macro" keywords in 1.011
> - struct/AA literals in 1.014
> - array operations in 1.034
>
> All of the above broke backwards compatibility with earlier 1.0 versions.
>
> Many minor changes that could be argued as being more than just bug fixes:
> - removal of === and !== in 1.005
> - "final" changes in 1.011
> - the infamous .init change in 1.017
>
> All of the above broke code that compiled with earlier 1.0 versions.
>
> The suggestion under discussion doesn't seem so bad to me, considering all that...

But see, every time we mention these things to Walter he says "no more, that was the last thing."  I.. somehow don't buy it, especially with array ops coming out of the blue so recently.

The fact remains that Walter is horrid at communication and no one knows for sure whether anything will be implemented, either soon or in the distant future.  Feature requests like this might go completely unnoticed, or he might be feverishly working on implementing them as we speak.  No one knows.  The development of D is a black box, and even after five years of experience with it, I _still_ have not determined what inputs you have to poke and prod to effect the desired outputs.

On Sat, 20 Sep 2008 15:18:38 +0100, Jason House <jason.james.house@gmail.com> wrote:

> <posted & mailed>
>
> Bruce Adams wrote:
>
>> On Wed, 17 Sep 2008 18:31:02 +0400, Jason House
>> <jason.james.house@gmail.com> wrote:
>>
>> > I'd like to see a smooth transition for Tango, not just the users! I
>> > made several posts on the topic. The latest was titled "forward
>> > compatibility" -- Making the D1 compiler tolerant of code that looks
>> > like D2, but not adding new functionality...
>>
>> The thing is what you are asking for is a major change to the D1.0 spec.
>> D1.0 is frozen. Bug fixes (to the spec) are permitted but major additions
>> are not.
>
> That means almost nothing to me.  Is the spec on how users should write
> their D1 code or a spec on how compiler writers should handle D1 code?  If
> it's the former, my suggestion poses no problems because old D1 code will
> still compile and execute the exact same way it always did.  If it's the
> latter, I don't see much of a problem since all alternative D1 compilers
> base their execution on dmd's front end source code.
>
Maybe I've misunderstood but what I thought was being asked for was for the D1.0
language to be changed to accept D2.0 syntax and ignore it at some point within blocks
marked as D1.0 versions.
This would mean that for a D1.0 compiler vendor to comply with the standard the
D1.0 compiler would have to extended to parse D2.0.
A bit like asking a C compiler to ignore C++ class definitions.
Having D2.0 compilers accept D1.0 syntax in D1.0 version sections makes more sense.

Now I think I understand what people are saying. You want different version sections
to not report errors for versions of the language they do not support.
By default the compiler parses code in all version sections.
One purpose of the version statement is so that you can always tell your code is syntactically
correct for all versions.
Another possible solution is to add an option to the compiler that modifies this behaviour so
the compiler doesn't bother parsing the other version block. This defeats the purpose above.
Still I think it is probably the correct thing to do (by default) for D1 compilers to accept
D2 code and for D2 compilers to accept D3 code.
For a D2 compiler it would be down to the vendor as to whether to have a D1.0 parser builtin
as well and switch modes within version blocks or simply ignore them. But there is going to be
significant code bloat in having a compiler support every variant of the language so at some
point your going to have to say ignore code in other version blocks.
The parser needs a way of identifying the end of the version block without doing too much
analysis on the code inbetween which is a bit gnarly.

>
>> Moreoever this addition does not add any functionality. So while
>> it
>> may seem like a sensible thing to do from a pragmatic programmer's point
>> of view
>>  from a language design point of view its very silly indeed.
>
> You're absolutely right that it depends on the point of view.  While I may
> like the pragmatic programmer's point of view, I'm motivated by the library
> maintainer's point of view and the library users' point of view.
>
> Users would want freedom to choose their D version and the libraries that
> they use.  The more incompatibilities that exist, the more it splits apart
> the D community.  Such splits leave a sour taste for current users and for
> prospective users of D.  Users will also want to read through library code
> from time to time, and the last thing they want to see is code that's
> littered with preprocessor directives.
>
> Library maintainers are left with the task of maintaining code.  They need
> to have code with little or no duplication.  Library maintainers are also
> users, and I bet that they were attracted to D by its elegance and
> simplicity.
>
A separate pre-processing step actually helps this. Users can look at the
source code in the version of the language they are using.
Library developers have the choice of using version statements or some kind
of annotation whichever they find cleaner.
There is overhead and complexity which may not be justified but if you want
flexibility sometimes you have to pay for it.
>
>
>> You need to find a difficult way to tackle the problem.
>
> One time difficulty or increased difficulty for all library writers and
> users?  Either way, a solution needs to be found that the community at
> large supports.
>
I think I meant different not difficult. The tricky unconscious at work there :)
>
>
>> * strip const's and invariants from the source in appropriate contexts
>
> I've suggested this before too :)
>
I thought someone had. I only piped up because I sick of seeing it discussed
without being resolved :).
>
>
>> * use source code annotations specific to your Doneificator program
>
> You mean add an unofficial D preprocessor?  IMHO, if there's a need for
> this, it should be part of the D specification.
>
I agree that's a bad idea. If you want to do something that your tool doesn't
support and your tool vendor doesn't look likely to implement then your options are
limited.
Source code transformation isn't inherantly evil though. You just need a very good
reason to justify the expense of doing it.

>
>
>> Have a configure script which checks which versions of the compiler you
>> have available.
>
> ick.  I can guarantee there are many here who have explicitly tried to avoid
> that kind of thing when coming to D.

Regardless of how you handle multiple language versions in the source code you need some way
to set the flags that say what you are going to build and how you are going to build it.
You can only avoid having something doing the job of a configure script if you only plan on
having a single configuration. If you know you will have a D2.0 compiler you shouldn't need
to worry.

Bruce Adams wrote:

> Maybe I've misunderstood but what I thought was being asked for was for
> the D1.0
> language to be changed to accept D2.0 syntax and ignore it at some point
> within blocks
> marked as D1.0 versions.
> This would mean that for a D1.0 compiler vendor to comply with the
> standard the
> D1.0 compiler would have to extended to parse D2.0.
> A bit like asking a C compiler to ignore C++ class definitions.
> Having D2.0 compilers accept D1.0 syntax in D1.0 version sections makes
> more sense.

I've proposed many things :)  My ultimate goal is for a smooth transition between D1.0 and D2.0.  For that to happen requires providing a path for Tango users and therefore Tango.  Obviously, that's not the only important project, but it does have the largest user base.  I have to assume that a smooth transition for a project as large as Tango means it would exist for other D1 projects.

There are many ways to achieve that ultimate goal:
1. Backward compatibility (D2 parsing of all D1 code)
2. Forward compatibility (D1 parsing of a subset of D2 code)
3. version(D1), version(D2), with lightened parsing restrictions
4. Replace D's pre-processor
5. 3rd party pre-processor
6. D2 to D1 converter

Below I tried to quickly make a list of pros and cons of each approach before having to run out the door.  I've probably missed an alternative and many pros and cons.  I hope we can find an alternative, or combination of alternatives, that allow this to happen.

#1: (backward compatibility)
pros:
* Enables a single D1 code base that compiles with either D dialect
* Allows legacy code to compile and run without modification
* Least effort for library maintainers
cons:
* Requires allowing exceptions to const/invariant rules
* Requires Walter to commit to the approach
* Walter has previously stated that he felt C was held back by backwards
  compatibility.  It's unlikely that he'd support this approach.

#2: (forward compatibility)
pros:
* Enables a single "D1" code base that compiles with either D dialect
* Partial adoption makes other alternatives easier/simpler
cons:
* Allows D1 to look like D2 code
* Requires Walter's to commit (at least partially) to the approach

#3: (version(D1), version(D2))
pros:
* Easy to spot code intended for D1 and D2
* Allows gradual phasing out of old D versions
cons:
* Can't wrap function prototypes in a version statement by themselves
* Will cause massive code duplication and restrict code maintainability
* Requires Walter to commit to one of two altrnatives:
  * Parsing of alternate D dialects in one D dialect
  * Ignoring of code within these special version blocks
* Fraction of code that must be covered with version statements

#4: (Replace D's pre-processor)
pros:
* Allows versioned function prototypes without duplicating all containing
  code.
cons:
* Requires Walter to commit to this approach
* Pre-processor is more C-like and Walter has already stated he's against
  that

#5: (3rd party pre-processor)
pros:
* Allows versioned function prototypes without duplicating all containing
  code.
* Unrestricted by D language design or Walter's choices
cons:
* Non-standard and unlikely to gain wide adoption
* Makes use of library code tougher because of extra tools and compilation
  steps.  Could be alleviated by distributing 3 code bases.

#6: (D2 to D1 converter)
pros:
* Enables a single D2 code base that compiles with either D dialect
cons:
* Impossible to downconvert all code automatically.
* Must be combined with another approach

On Sun, 21 Sep 2008 19:55:48 +0400, Jason House <jason.james.house@gmail.com> wrote:

> Bruce Adams wrote:
>
>> Maybe I've misunderstood but what I thought was being asked for was for
>> the D1.0
>> language to be changed to accept D2.0 syntax and ignore it at some point
>> within blocks
>> marked as D1.0 versions.
>> This would mean that for a D1.0 compiler vendor to comply with the
>> standard the
>> D1.0 compiler would have to extended to parse D2.0.
>> A bit like asking a C compiler to ignore C++ class definitions.
>> Having D2.0 compilers accept D1.0 syntax in D1.0 version sections makes
>> more sense.
>
> I've proposed many things :)  My ultimate goal is for a smooth transition
> between D1.0 and D2.0.  For that to happen requires providing a path for
> Tango users and therefore Tango.  Obviously, that's not the only important
> project, but it does have the largest user base.  I have to assume that a
> smooth transition for a project as large as Tango means it would exist for
> other D1 projects.
>
> There are many ways to achieve that ultimate goal:
> 1. Backward compatibility (D2 parsing of all D1 code)
> 2. Forward compatibility (D1 parsing of a subset of D2 code)
> 3. version(D1), version(D2), with lightened parsing restrictions
> 4. Replace D's pre-processor
> 5. 3rd party pre-processor
> 6. D2 to D1 converter
>
> Below I tried to quickly make a list of pros and cons of each approach
> before having to run out the door.  I've probably missed an alternative and
> many pros and cons.  I hope we can find an alternative, or combination of
> alternatives, that allow this to happen.
>
> #1: (backward compatibility)
> pros:
> * Enables a single D1 code base that compiles with either D dialect
> * Allows legacy code to compile and run without modification
> * Least effort for library maintainers
> cons:
> * Requires allowing exceptions to const/invariant rules
> * Requires Walter to commit to the approach
> * Walter has previously stated that he felt C was held back by backwards
>   compatibility.  It's unlikely that he'd support this approach.
>
> #2: (forward compatibility)
> pros:
> * Enables a single "D1" code base that compiles with either D dialect
> * Partial adoption makes other alternatives easier/simpler
> cons:
> * Allows D1 to look like D2 code
> * Requires Walter's to commit (at least partially) to the approach
>
> #3: (version(D1), version(D2))
> pros:
> * Easy to spot code intended for D1 and D2
> * Allows gradual phasing out of old D versions
> cons:
> * Can't wrap function prototypes in a version statement by themselves
> * Will cause massive code duplication and restrict code maintainability
> * Requires Walter to commit to one of two altrnatives:
>   * Parsing of alternate D dialects in one D dialect
>   * Ignoring of code within these special version blocks
> * Fraction of code that must be covered with version statements
>
> #4: (Replace D's pre-processor)
> pros:
> * Allows versioned function prototypes without duplicating all containing
>   code.
> cons:
> * Requires Walter to commit to this approach
> * Pre-processor is more C-like and Walter has already stated he's against
>   that
>
> #5: (3rd party pre-processor)
> pros:
> * Allows versioned function prototypes without duplicating all containing
>   code.
> * Unrestricted by D language design or Walter's choices
> cons:
> * Non-standard and unlikely to gain wide adoption
> * Makes use of library code tougher because of extra tools and compilation
>   steps.  Could be alleviated by distributing 3 code bases.
>
> #6: (D2 to D1 converter)
> pros:
> * Enables a single D2 code base that compiles with either D dialect
> cons:
> * Impossible to downconvert all code automatically.
> * Must be combined with another approach

I would like to show some other aspect of the issue.

D has support for inline assembly. Different processors might have different assembly syntax and it is unlikely that every future D compiler would support all the syntaxes and recognize all command. Take a look at the tango.core.Atomic module. It contains x86 and x64 assembly instructions so far. However, it is likely that implementation for other platforms will be added soner or later (ARM, PowerPC and others). And they should somehow co-exist in one source code, like this:

bool cas(ref int value, int newValue, int oldValue) {
  version (X86) {
    asm {
      mov EDX, newValue;
      mov EAX, oldValue;
      mov ECX, value;
      lock;
      cmpxchg [ECX], EDX;
      setz AL;
    }
  } else version (PowerPC) {
    asm {
      // an implementaion using load-linked/store-conditional
    }
  }
}

(I am no expert in assembly, so I copied x86 code and left ppc code blank :)).

Current parsing rules state that the code in version block should be semantically correct, even if it is ignored. But since DMD knows basically nothing about those ll/sc op-codes, the code can't be placed in one source file, which is bad for maintainability.

Besides, I foresee "Smoth D2 to D3 transition", "Smooth D3 to D4 transition" etc topics in future, so we should solve the issue once and for all.

On Sun, 21 Sep 2008 21:13:04 +0100, Denis Koroskin <2korden@gmail.com> wrote:
>
> I would like to show some other aspect of the issue.
>
> D has support for inline assembly. Different processors might have different assembly syntax and it is unlikely that every future D compiler would support all the syntaxes and recognize all command. Take a look at the tango.core.Atomic module. It contains x86 and x64 assembly instructions so far. However, it is likely that implementation for other platforms will be added soner or later (ARM, PowerPC and others). And they should somehow co-exist in one source code, like this:
>
> bool cas(ref int value, int newValue, int oldValue) {
>    version (X86) {
>      asm {
>        mov EDX, newValue;
>        mov EAX, oldValue;
>        mov ECX, value;
>        lock;
>        cmpxchg [ECX], EDX;
>        setz AL;
>      }
>    } else version (PowerPC) {
>      asm {
>        // an implementaion using load-linked/store-conditional
>      }
>    }
> }
>
> (I am no expert in assembly, so I copied x86 code and left ppc code blank :)).
>
> Current parsing rules state that the code in version block should be semantically correct, even if it is ignored. But since DMD knows basically nothing about those ll/sc op-codes, the code can't be placed in one source file, which is bad for maintainability.
>
> Besides, I foresee "Smoth D2 to D3 transition", "Smooth D3 to D4 transition" etc topics in future, so we should solve the issue once and for all.

The inline assembly and language variant cases both demonstrate there are at minimum 2
incompatible use cases for the version statement. The language currently only supports
the case where the code is syntactically correct for all versions. Though if the alternate
assembly versions work in Tango there must already be a special case that asm blocks are
ignored outside the currently active version.
To handle it on the language/compiler side two things are required.
 1) a way to indicate the differences between the two kinds of version block
 2) a way to handle 'non'-parsing blocks.
I don't think its unreasonable to expect that braces will be balanced in all future versions of the
language. So a compiler could re-synchronise following the next balanced code brace. The tokeniser will
choke on unidentified keywords unless they are parsed as strings and ignored rather than producing an
unknown token. The lexer is going to have to know enough to handle braces and detect braces that do not
apply because they are within strings or comment whatever. It would take a lot of care to get right.
However its done, compiler, preprocessor or whatever its going to be a bit yucky somewhere. Maybe a
different kind of bracket would make things easier. Say version @{  /* don't parse me */ @}, though that's
dangerously close to #endif :). I cant' help thinking the safest thing is some kind of source 2 source translator.
D1 to D2 would help ease the transition greatly but result in a lot of crappy quality D2 code out there.
D2 to D1 might be a bigger project but possibly more useful.
I previously suggested annotatations as they might simplify the implementation of a converter significantly. They
could be deprecated as time went on and the project grew more mature.
But what do I know, I'm just tha window cleaner :-)

Bruce Adams <tortoise_74@yeah.who.co.uk> wrote:
> I don't think its unreasonable to expect that braces will be balanced in all future versions of the language.

Braces can be in strings or comments.  It's quite likely to have braces in strings in many kinds of parser programs.

If you want code hidden from a compiler, put it in a string and mix in.

enum veryD2Code = `
  auto foo(T)(T x) if (__traits(hasMember, "opCall"))
  {
    return x();
  }
`;

version (D_Version2)
{
  mixin(veryD2Code);
}

In D2 you can use token strings for that purpose, so that the code will even be highlighted in your editor, but will not interfere with the compiler.

"Sergey Gromov" wrote
> Bruce Adams <tortoise_74@yeah.who.co.uk> wrote:
>> I don't think its unreasonable to expect that braces will be balanced in all future versions of the language.
>
> Braces can be in strings or comments.  It's quite likely to have braces in strings in many kinds of parser programs.
>
> If you want code hidden from a compiler, put it in a string and mix in.
>
> enum veryD2Code = `
>  auto foo(T)(T x) if (__traits(hasMember, "opCall"))
>  {
>    return x();
>  }
> `;
>
> version (D_Version2)
> {
>  mixin(veryD2Code);
> }
>
> In D2 you can use token strings for that purpose, so that the code will even be highlighted in your editor, but will not interfere with the compiler.

oops, D1 doesn't support that type of enum ;)

If the problem can't be solved with the current compilers, then the solution provided to make it work should be easy to use.  I don't see this type of solution as easy to use, nor the 'version(D2)' type of solution.  What if you have const member functions?  If we had a preprocessor, I'd say:

#ifdef Dv2
   const int f()
#else
   int f()
#endif

But we don't, so the reality is you must duplicate the function just to change the signature.  Unless you do some hacky stuff like the mixin above.

The two best solutions I've read are the original post (make D1 compile D2 as if it were D1), and a D2toD1 translator program.

-Steve

On Tue, 23 Sep 2008 02:00:36 +0400, Steven Schveighoffer <schveiguy@yahoo.com> wrote:

>
> "Sergey Gromov" wrote
>> Bruce Adams <tortoise_74@yeah.who.co.uk> wrote:
>>> I don't think its unreasonable to expect that braces will be balanced
>>> in all future versions of the language.
>>
>> Braces can be in strings or comments.  It's quite likely to have braces
>> in strings in many kinds of parser programs.
>>
>> If you want code hidden from a compiler, put it in a string and mix in.
>>
>> enum veryD2Code = `
>>  auto foo(T)(T x) if (__traits(hasMember, "opCall"))
>>  {
>>    return x();
>>  }
>> `;
>>
>> version (D_Version2)
>> {
>>  mixin(veryD2Code);
>> }
>>
>> In D2 you can use token strings for that purpose, so that the code will
>> even be highlighted in your editor, but will not interfere with the
>> compiler.
>
> oops, D1 doesn't support that type of enum ;)
>
> If the problem can't be solved with the current compilers, then the solution
> provided to make it work should be easy to use.  I don't see this type of
> solution as easy to use, nor the 'version(D2)' type of solution.  What if
> you have const member functions?  If we had a preprocessor...

We do have some kind of it:

#!/usr/bin/dmd -run
#line 42 "foo.d"
#pragma(msg, "bar");
...

C# has also support for the following "preprocessor" directives (although it doesn't have a preprocessor):

#if
#else
#elif
#endif
#warning
#error
#line
#region
#endregion
#define
#undef

These are widely known and understandable to many programmers, so D could extend the preprocessor directives list with some of them.

An import point is that unlike C and C++, you cannot use these directives to create macros!

D ought to have a way to mark some section of code so that it won't get parsed and semantically analyzed. Balancing of the curly braces is one of the solutions:

version (Dv2) {
    const int f()
} else {
    int f()
}

#if/#else/#elif/#endif is another, also a good one:

>
> #ifdef Dv2
>    const int f()
> #else
>    int f()
> #endif
>

Steven Schveighoffer <schveiguy@yahoo.com> wrote:
> oops, D1 doesn't support that type of enum ;)

I think you've got the idea. The code

version (D_Version2)
{
  mixin("some very specific code");
}

works in any version of D, current or future.