11-May-2013 22:15, Daniel Murphy пишет:
>> If we aren't confident that we can write and maintain a large real-world
>> application in D just yet, we must pull the emergency brakes on the whole
>> DDDMD effort, right now.
>>
>> David
>
> I'm confident in D, just not in phobos.  Even if phobos didn't exist, we'd
> still be in better shape using D than C++.  What exactly are we going to
> need from phobos?  sockets?  std.datetime? std.regex? std.container?
>

Sockets may come in handy one day. Caching compiler daemon etc.
std.container well ... mm ... eventually.

> If we use them in the compiler, we effectively freeze them.  We can't use
> the new modules, because the old toolchains don't have them.  We can't fix
> old broken modules because the compiler depends on them.  If you add code to
> work around old modules being gone in later versions, you pretty much end up
> moving the source into the compiler after all.
>

I propose a different middle ground:

Define a minimal subset of phobos, compilable and usable separately.
Then full phobos will depend on it in turn (or rather contain it). Related to my recent thread on limiting inter-dependencies - we will have to face that problem while make a subset of phobos.

It has some operational costs but will limit the frozen surface.

-- 
Dmitry Olshansky

On Saturday, May 11, 2013 20:40:46 deadalnix wrote:
> Except that now, it is a pain to migrate old haskell stuff to newer haskelle stuff if you missed several compile release.
> 
> You ends up building recursively from the native version to the version you want.

Yeah. And I'm stuck with the opposite problem at the moment. I have to be able to build old haskell code without updating it, but I don't have an older version of ghc built currently, and getting a version old enough to compile my code has turned out to be a royal pain, because the old compiler won't compile with the new compiler. I don't even know if I'm going to be able to do it.

If you're always moving forward, you're okay, but if you have to deal with older code, then you quickly run into trouble if the compiler is written in an up-to-date version of the language that it's compiling. At least at this point, if you needed something like 2.059 for some reason, you can just grab 2.059, compile it, and use it with your code. But if the compiler were written in D, and the version of D with 2.059 was not fully compatible with the current version, then compiling 2.059 would become a nightmare.

The situation between a normal program and the compiler is quite different. With a normal program, if your code isn't going to work with the current compiler due to language or library changes, then you just grab an older version of the compiler and use that (possibly upgrading your code later if you intend to maintain it long term). But if it's the compiler that you're trying to compile, then you're screwed by any language or library changes that affect the compiler, because it could very well become impossible to compile older versions of the compiler.

Yes, keeping language and library changes to a minimum reduces the problem, but unless they're absolutely frozen, you risk problems. Even changes with high ROI (like making implicit fall-through on switch statements illegal) could make building older compilers impossible.

So, whatever we do with porting dmd to D, we need to be very careful. We don't want to lock ourselves in so that we can't make changes to the language or libraries even when we really need to, but we don't want to make it too difficult to build older versions of the compiler for people who have to either. At the extreme, we could end up in a situation where you have to grab the oldest version of the compiler which was written in C++, and then build each newer version of the compiler in turn until you get to the one that you want.

- Jonathan M Davis

On Saturday, 11 May 2013 at 21:09:57 UTC, Jonathan M Davis wrote:
> On Saturday, May 11, 2013 20:40:46 deadalnix wrote:
>> Except that now, it is a pain to migrate old haskell stuff to
>> newer haskelle stuff if you missed several compile release.
>> 
>> You ends up building recursively from the native version to the
>> version you want.
>
> Yeah. And I'm stuck with the opposite problem at the moment. I have to be able
> to build old haskell code without updating it, but I don't have an older
> version of ghc built currently, and getting a version old enough to compile my
> code has turned out to be a royal pain, because the old compiler won't compile
> with the new compiler. I don't even know if I'm going to be able to do it.
>
> If you're always moving forward, you're okay, but if you have to deal with
> older code, then you quickly run into trouble if the compiler is written in an
> up-to-date version of the language that it's compiling. At least at this
> point, if you needed something like 2.059 for some reason, you can just grab
> 2.059, compile it, and use it with your code. But if the compiler were written
> in D, and the version of D with 2.059 was not fully compatible with the
> current version, then compiling 2.059 would become a nightmare.
>
> The situation between a normal program and the compiler is quite different.
> With a normal program, if your code isn't going to work with the current
> compiler due to language or library changes, then you just grab an older
> version of the compiler and use that (possibly upgrading your code later if
> you intend to maintain it long term). But if it's the compiler that you're
> trying to compile, then you're screwed by any language or library changes that
> affect the compiler, because it could very well become impossible to compile
> older versions of the compiler.
>
> Yes, keeping language and library changes to a minimum reduces the problem,
> but unless they're absolutely frozen, you risk problems. Even changes with
> high ROI (like making implicit fall-through on switch statements illegal)
> could make building older compilers impossible.
>
> So, whatever we do with porting dmd to D, we need to be very careful. We don't
> want to lock ourselves in so that we can't make changes to the language or
> libraries even when we really need to, but we don't want to make it too
> difficult to build older versions of the compiler for people who have to either.
> At the extreme, we could end up in a situation where you have to grab the
> oldest version of the compiler which was written in C++, and then build each
> newer version of the compiler in turn until you get to the one that you want.
>
> - Jonathan M Davis

Can't this be eased with readily available binaries and cross compilation?

E.g. We drop the C++ version in 2.7. You want DMD version 2.8.2. The minimum needed to compile 2.8.2 is 2.7.5:

You can download a binary of 2.7.5 for any common system, cross compile 2.8.2 for your development system, viola! If there are binaries available for your development system, then it becomes almost trivial.


Even if this wasn't possible for some reason, recursively building successive versions of the compiler is a completely automatable process. dmd+druntime+phobos compiles quickly enough that it's not a big problem.

Am 11.05.2013 23:43, schrieb John Colvin:
> On Saturday, 11 May 2013 at 21:09:57 UTC, Jonathan M Davis wrote:
>> On Saturday, May 11, 2013 20:40:46 deadalnix wrote:
>>> Except that now, it is a pain to migrate old haskell stuff to
>>> newer haskelle stuff if you missed several compile release.
>>>
>>> You ends up building recursively from the native version to the
>>> version you want.
>>
>> Yeah. And I'm stuck with the opposite problem at the moment. I have to
>> be able
>> to build old haskell code without updating it, but I don't have an older
>> version of ghc built currently, and getting a version old enough to
>> compile my
>> code has turned out to be a royal pain, because the old compiler won't
>> compile
>> with the new compiler. I don't even know if I'm going to be able to do
>> it.
>>
>> If you're always moving forward, you're okay, but if you have to deal
>> with
>> older code, then you quickly run into trouble if the compiler is
>> written in an
>> up-to-date version of the language that it's compiling. At least at this
>> point, if you needed something like 2.059 for some reason, you can
>> just grab
>> 2.059, compile it, and use it with your code. But if the compiler were
>> written
>> in D, and the version of D with 2.059 was not fully compatible with the
>> current version, then compiling 2.059 would become a nightmare.
>>
>> The situation between a normal program and the compiler is quite
>> different.
>> With a normal program, if your code isn't going to work with the current
>> compiler due to language or library changes, then you just grab an older
>> version of the compiler and use that (possibly upgrading your code
>> later if
>> you intend to maintain it long term). But if it's the compiler that
>> you're
>> trying to compile, then you're screwed by any language or library
>> changes that
>> affect the compiler, because it could very well become impossible to
>> compile
>> older versions of the compiler.
>>
>> Yes, keeping language and library changes to a minimum reduces the
>> problem,
>> but unless they're absolutely frozen, you risk problems. Even changes
>> with
>> high ROI (like making implicit fall-through on switch statements illegal)
>> could make building older compilers impossible.
>>
>> So, whatever we do with porting dmd to D, we need to be very careful.
>> We don't
>> want to lock ourselves in so that we can't make changes to the
>> language or
>> libraries even when we really need to, but we don't want to make it too
>> difficult to build older versions of the compiler for people who have
>> to either.
>> At the extreme, we could end up in a situation where you have to grab the
>> oldest version of the compiler which was written in C++, and then
>> build each
>> newer version of the compiler in turn until you get to the one that
>> you want.
>>
>> - Jonathan M Davis
>
> Can't this be eased with readily available binaries and cross compilation?
>
> E.g. We drop the C++ version in 2.7. You want DMD version 2.8.2. The
> minimum needed to compile 2.8.2 is 2.7.5:
>
> You can download a binary of 2.7.5 for any common system, cross compile
> 2.8.2 for your development system, viola! If there are binaries
> available for your development system, then it becomes almost trivial.
>
>
> Even if this wasn't possible for some reason, recursively building
> successive versions of the compiler is a completely automatable process.
> dmd+druntime+phobos compiles quickly enough that it's not a big problem.


I also don't understand the problem. This is how compilers get botstraped all the time.

You just use toolchain X to build toolchain X+1.

--
Paulo

On May 11, 2013 6:35 PM, "David Nadlinger" <see@klickverbot.at> wrote:
>
> On Saturday, 11 May 2013 at 17:23:53 UTC, Daniel Murphy wrote:
>>
>> That... doesn't sound very nice to me.  How much of phobos are we realistically going to need?
>
>
> All of it? Well, not quite, but large parts at least.
>
> If we are going to stick to the C subset of the language, there is little
point in translating it to D in the first place.
>
> Of course, there will be some restrictions arising from the fact that the
code base needs to work with D versions from a year back or so. But to me duplicating the whole standard library inside the compiler source seems like maintenance hell.
>
> David

I don't think it would be anything in the slightest at all.   For instance,  Bigint implementation is big,  BIG.  :)

What would be ported to the compiler may be influenced by BigInt,  but would be a limited subset of its functionality tweaked for the purpose of use in the front end.

I am more concerned from GDC's perspective of things.  Especially when it comes to building from hosts that may have phobos disabled (this is a configure switch).

Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';

On Saturday, May 11, 2013 23:43:19 John Colvin wrote:
> Can't this be eased with readily available binaries and cross compilation?
> 
> E.g. We drop the C++ version in 2.7. You want DMD version 2.8.2. The minimum needed to compile 2.8.2 is 2.7.5:
> 
> You can download a binary of 2.7.5 for any common system, cross compile 2.8.2 for your development system, viola! If there are binaries available for your development system, then it becomes almost trivial.

Sure, but that assumes that you have access to a compatible binary. That's not always easy, and it can be particularly nasty in *nix. A binary built a few years ago stands a good chance of being completely incompatible with current systems even if all it depends on is glibc, let alone every other dependency that might have changed. It's even harder when your language is not one included by default in distros. For Windows, this probably wouldn't be an issue, but it could be a big one for *nix systems.

> Even if this wasn't possible for some reason, recursively building successive versions of the compiler is a completely automatable process. dmd+druntime+phobos compiles quickly enough that it's not a big problem.

Sure, assuming that you can get an old enough version of the compiler which you can actually compile. It's by no means an insurmountable problem, but you _do_ very much risk being in a situation where you literally have to compile the last C++ version of D's compiler and then compile every version of the compiler since then until you get to the one you want. And anyone who doesn't know that they could go to an older compiler which was in C++ (let alone which version it was) is going to have a lot of trouble.

I don't know how much we want to worry about this, but it's very much a real world problem when you don't have a binary for an older version of the compiler that you need, and the current compiler can't build it. It's been costing me a lot of time trying to sort that out in Haskell thanks to the shift from the 98 standard to 2010.

- Jonathan M Davis

On 5/11/2013 2:09 PM, Jonathan M Davis wrote:
> I have to be able
> to build old haskell code without updating it,

I guess this is the crux of the matter. Why can't you update the source?

On Saturday, May 11, 2013 17:51:24 Walter Bright wrote:
> On 5/11/2013 2:09 PM, Jonathan M Davis wrote:
> > I have to be able
> > to build old haskell code without updating it,
> 
> I guess this is the crux of the matter. Why can't you update the source?

Well, in this particular case, it has to do with work on my master's thesis, and I have the code in various stages of completion and need to be able to look at exactly what it was doing at each of those stages for writing the actual paper. Messing with the code risks changing what it does, and it wasn't necessarily in a great state anyway given that I'm basically dealing with snapshots of the code over time, and not all of the snapshots are necessarily fully functional.

In the normal case, I'd definitely want to update my code, but I still might need to get the old code working before doing that so that I can be sure of how it works before changing it. Obviously, things like solid unit testing help with that, but if you're dealing with code that hasn't been updated in a while, it's not necessarily a straightforward task to update it, especially when it's in a language that you're less familiar with. It's even worse if it's code written by someone else entirely, and you're just trying to get it working (which isn't my current situation, but that's often the case when building old code).

Ultimately, I don't know how much we need to care about situations where people need to compile an old version of the compiler, and all they have is the new compiler. Much as its been causing me quite a bit of grief in haskell, for the vast majority of people, it's not likely to come up. But I think that it at least needs to be brought up so that it can be considered when deciding what we're doing with regards to porting the front-end to D. I think that main reason that C++ avoids the problem is that it's so rarely updated (which causes a whole different set of problems). And while we obviously want to minimize breakage caused by changes to the library, language, or just due to bugs, they _are_ going to have an effect with regards to building older compilers if the compiler itself is affected by them.

So, we might be better of restricting how much the compiler depends on - or we may decide that the workaround is to simply build the last C++ version of the compiler and then move forward from there. But I think that the issue should at least be raised.

- Jonathan M Davis

On 5/11/2013 6:09 PM, Jonathan M Davis wrote:
> So, we might be better of restricting how much the compiler depends on - or we
> may decide that the workaround is to simply build the last C++ version of the
> compiler and then move forward from there. But I think that the issue should
> at least be raised.

Last month I tried compiling an older 15 line D utility, and 10 of those lines broke due to phobos changes.

I discussed this a bit with Andrei, and proposed that we keep around aliases for the old names, and put them inside a:

   version (OldNames) {
      alias newname oldname;
      ....
   }

or something like that.

On Saturday, May 11, 2013 18:18:27 Walter Bright wrote:
> On 5/11/2013 6:09 PM, Jonathan M Davis wrote:
> > So, we might be better of restricting how much the compiler depends on - or we may decide that the workaround is to simply build the last C++ version of the compiler and then move forward from there. But I think that the issue should at least be raised.
> 
> Last month I tried compiling an older 15 line D utility, and 10 of those lines broke due to phobos changes.
> 
> I discussed this a bit with Andrei, and proposed that we keep around aliases for the old names, and put them inside a:
> 
>     version (OldNames) {
>        alias newname oldname;
>        ....
>     }
> 
> or something like that.

Well, that particular problem should be less of an issue in the long run. We renamed a lot of stuff in an effort to make the naming more consistent, but we haven't been doing much of that for a while now. And fortunately, those changes are obvious and quick.

But in theory, the way to solve the problem of your program not compiling with the new compiler is to compile with the compiler it was developed with in the first place, and then if you want to upgrade your code, you upgrade your code and use it with the new compiler. The big problem is when you need to compile the compiler. You have a circular dependency due to the compiler depending on itself, and have to break it somehow. As long as newer compilers can compiler older ones, you're fine, but that's bound to fall apart at some point unless you freeze everything. But even bug fixes could make the old compiler not compile anymore, so unless the language and compiler (and anything they depend on) is extremely stable, you risk not being able to compile older compilers, and it's hard to guarantee that level of stability, especially if the compiler is not restricted in what features it uses or in what it uses from the standard library.

- Jonathan M Davis