From the recent PR-related discussion I gather that FreeBSD 12 has made some breaking changes to file primitives such as stat.

Do binaries built for pre-v12 crash on v12? If that's the case we should have two distinct build platforms, FreeBSD "legacy" and FreeBSD 12. They would be selected statically.

IF older binaries don't crash, what mechanism is FreeBSD 12 using to identify old API calls?


Thanks,

Andrei

On Thursday, November 02, 2017 00:25:26 Andrei Alexandrescu via Digitalmars- d wrote:
>  From the recent PR-related discussion I gather that FreeBSD 12 has made
> some breaking changes to file primitives such as stat.
>
> Do binaries built for pre-v12 crash on v12? If that's the case we should have two distinct build platforms, FreeBSD "legacy" and FreeBSD 12. They would be selected statically.
>
> IF older binaries don't crash, what mechanism is FreeBSD 12 using to identify old API calls?

Technically speaking, with how FreeBSD deals with its API/ABI, it promises compatibility across major releases but not minor ones, so _any_ time you go up a major version, there's a risk of breakage (though I don't know how often APIs get changed in practice). In that respect, the fact that we've been doing FreeBSD releases that don't target a particular version of FreeBSD is kind of crazy, though for the most part, I think that the API and ABI have not changed sufficiently in the areas that matter to dmd or Phobos for it to be a big issue, though occasionally, fixes have had to be made because something worked on one version of FreeBSD but not another. But we don't currently have a way to version code for a particular version of FreeBSD even if we wanted to. So, while we could do separate builds for different versions of FreeBSD (and thus handle ABI changes), the source code would have to be the same. AFAIK, the closest that we can currently come to versioning the source code based on the version of FreeBSD is to check the uname at runtime and run different code depending on the version, but that obviously doesn't work in all situations.

Right now, what we have mostly works on FreeBSD 10.x and 11.x (it looks like 64-bit has a dmd test failure, and 32-bit has a Phobos test failure) as well as older versions of 12-to-be (that's what I run locally, since that's what TrueOS is based on), but several months ago, FreeBSD did make changes to stat and related functions so that inodes are 64-bit, and that obviously changed the API. They did a lot of work to make that compatible so that programs didn't just break, but the compatibility stuff that they did was for C/C++, and we don't use C/C++ headers; we duplicate their declarations in D. So, we have to do work of some kind to be compatible, and I'm not familiar enough with the details to know what needs doing. The related bug report is

https://issues.dlang.org/show_bug.cgi?id=17596

and some work has been done to figure out what needs doing including a suggested way to wrap the API changes at least until we're only supporting FreeBSD 12+ (which is quite a ways off, since it hasn't even been released yet). I don't know exactly what the situation with that currently is. I keep hoping that it will be sorted out by December, since that's when the next TrueOS update happens, and then it'll be based on a new enough version of FreeBSD 12 that D will break if I upgrade, but I haven't had time to look into it, and clearly those who looked into it previously haven't finished doing whatever needs doing.

- Jonathan M Davis

On Thursday, 2 November 2017 at 04:25:26 UTC, Andrei Alexandrescu wrote:
> From the recent PR-related discussion I gather that FreeBSD 12 has made some breaking changes to file primitives such as stat.
>
> Do binaries built for pre-v12 crash on v12? If that's the case we should have two distinct build platforms, FreeBSD "legacy" and FreeBSD 12. They would be selected statically.
>
> IF older binaries don't crash, what mechanism is FreeBSD 12 using to identify old API calls?
>
>
> Thanks,
>
> Andrei


Targetted versioning should be implemented. Full stop ;-)

Pre ino64 ABI compatibility can be obtained in 12, **if** the kernel is compiled with option COMPAT_FREEBSD11 (presumably this option is enabled in the DEFAULT kernel)...

That option enables 'several techniques' that can mitigate for ABI breakage, but it won't work for all cases - really depends on what the application is doing with struct stat.


https://svnweb.freebsd.org/base/head/sys/sys/stat.h?view=markup#l158

On 11/2/17 1:11 AM, Jonathan M Davis wrote:

> 
> Right now, what we have mostly works on FreeBSD 10.x and 11.x (it looks like
> 64-bit has a dmd test failure, and 32-bit has a Phobos test failure) as well
> as older versions of 12-to-be (that's what I run locally, since that's what
> TrueOS is based on), but several months ago, FreeBSD did make changes to
> stat and related functions so that inodes are 64-bit, and that obviously
> changed the API. They did a lot of work to make that compatible so that
> programs didn't just break, but the compatibility stuff that they did was
> for C/C++, and we don't use C/C++ headers;

Well, that is only header compatibility, not binary compatibility. In other words, that still means if you use C or C++, you need to recompile if you are on FreeBSD 12. You can't have one binary that supports both platforms.

If this is the only compatibility they support, then I think we do need to have multiple versions.

-Steve

On 11/02/2017 08:46 AM, Steven Schveighoffer wrote:
> Well, that is only header compatibility, not binary compatibility. In other words, that still means if you use C or C++, you need to recompile if you are on FreeBSD 12. You can't have one binary that supports both platforms.

Thanks, this is the kind of hard data we're looking for. Do you have a link to support this?

> If this is the only compatibility they support, then I think we do need to have multiple versions.

That's exactly right. Thanks!


Andrei

On 11/2/17 10:56 AM, Andrei Alexandrescu wrote:
> On 11/02/2017 08:46 AM, Steven Schveighoffer wrote:
>> Well, that is only header compatibility, not binary compatibility. In other words, that still means if you use C or C++, you need to recompile if you are on FreeBSD 12. You can't have one binary that supports both platforms.
> 
> Thanks, this is the kind of hard data we're looking for. Do you have a link to support this?

Sorry, I don't have any experience with FreeBSD. I was reacting to Jonathan's claim that "They did a lot of work to make that compatible so that programs didn't just break, but the compatibility stuff that they did was for C/C++, and we don't use C/C++ headers"

That lead me to believe that marking the headers differently is all they did. As I said, this is not binary compatibility.

Sure, we can do this too, we don't use C/C++ headers, but we can port our headers. The question to answer is, "can we create a binary that runs on both versions of FreeBSD?".

The answer to that question, I don't know. Is it possible on FreeBSD to detect this at runtime and call the right binding? Not sure. It might not even be detectable (see codephantom's post on how FreeBSD 12 may be compiled with the older binding anyway!)

Another possible option, if it existed, would be to simply call a function that does the right thing. Perhaps a .so that can be swapped out based on which version of the OS you are on?

But the only thing I can think of currently, that will definitely "solve" the problem, is to treat each version as a different OS with a new version statement. This doesn't scale well, so I'm not sure if that's the road we want to go down.

-Steve

On Thursday, 2 November 2017 at 15:33:39 UTC, Steven Schveighoffer wrote:
>
> But the only thing I can think of currently, that will definitely "solve" the problem, is to treat each version as a different OS with a new version statement. This doesn't scale well, so I'm not sure if that's the road we want to go down.
>

That seems to be the road version statements are already going down...There is Playstation and Playstation4. There is SysV3 and SysV4. There are 5 different ARM versions. Posix groups together FreeBSD, OSX, Solaris, etc, which also exist on their own.

On Thursday, November 02, 2017 11:33:39 Steven Schveighoffer via Digitalmars-d wrote:
> On 11/2/17 10:56 AM, Andrei Alexandrescu wrote:
> > On 11/02/2017 08:46 AM, Steven Schveighoffer wrote:
> >> Well, that is only header compatibility, not binary compatibility. In other words, that still means if you use C or C++, you need to recompile if you are on FreeBSD 12. You can't have one binary that supports both platforms.
> >
> > Thanks, this is the kind of hard data we're looking for. Do you have a link to support this?
>
> Sorry, I don't have any experience with FreeBSD. I was reacting to Jonathan's claim that "They did a lot of work to make that compatible so that programs didn't just break, but the compatibility stuff that they did was for C/C++, and we don't use C/C++ headers"
>
> That lead me to believe that marking the headers differently is all they did. As I said, this is not binary compatibility.
>
> Sure, we can do this too, we don't use C/C++ headers, but we can port our headers. The question to answer is, "can we create a binary that runs on both versions of FreeBSD?".
>
> The answer to that question, I don't know. Is it possible on FreeBSD to detect this at runtime and call the right binding? Not sure. It might not even be detectable (see codephantom's post on how FreeBSD 12 may be compiled with the older binding anyway!)

The bug report I linked to has an example of how it might be fixed with a wrapper function that calls a different version of stat depending on what version of FreeBSD it looks like is running at runtime. But I have no idea how complete that is. I keep meaning to find the time to look into this in detail but haven't gotten around to it yet. I believe that what codephantom's post means is that with that build option, the compatibility stuff is in place which allows the 32-bit inode stuff to be used but in spite of the fact that they've now switched to 64-bit inodes (certainly, that's my understanding of what the compatibility stuff does), but I haven't dug deep enough to figure out the details.

Given that this issue hasn't been sorted out yet and that if it's not sorted out by next month, I'm going to be stuck between not upgrading my desktop and not having a working D setup (due to the TrueOS-stable update to a newer version of FreeBSD 12), it's looking like I'm going to have to find the time to dig deeper.

> Another possible option, if it existed, would be to simply call a function that does the right thing. Perhaps a .so that can be swapped out based on which version of the OS you are on?
>
> But the only thing I can think of currently, that will definitely "solve" the problem, is to treat each version as a different OS with a new version statement. This doesn't scale well, so I'm not sure if that's the road we want to go down.

Well, in the case of FreeBSD, it's not terrible, especially since it doesn't make much sense to support versions of FreeBSD that the FreeBSD guys don't support. And in that case, the most you'd get is probably 3 versions (currently FreeBSD10, FreeBSD11, and FreeBSD12), and they would shift over time, so while there would be more in the compiler if they weren't removed, druntime or Phobos wouldn't get an ever growing number of them. But from what I can tell, the API changes rarely enough that it usually wouldn't even be needed. I think that it's usually a greater risk that you'd need to build for a newer system because of ABI changes than that the API changed in a way that requires versioning druntime or Phobos. This inode situation is an extreme case, and they've been working for years on getting it done in a way that minimizes the level of disruption and breakage that it causes.

The bigger problem with separate versions is that the autotester would then really need to be running at least a FreeBSD 10.x and FreeBSD 11.x box (and later 11.x and 12.x once 12 is released), and we'd need to release different packages for each (though we could arguably just rely on the ports tree for the releases so long as someone is maintaining the FreeBSD port). The fact that we don't test multiple versions of FreeBSD has occasionally been a problem already, but the bigger problem has been that the auto-testers just haven't gotten updated often (IIRC they're on 8.x and not even 9.x, whereas 10.3 is currently the oldest supported release), and now several things have needed to get fixed towards being able to update them, because several things broke.

If we _can_ continue to do just do one release of dmd for FreeBSD that works across several versions, then that's ideal, but the way that FreeBSD is released, I think that it's normally expected that you're going to have to rebuild stuff for each major release even if APIs don't change, and if we're not testing multiple versions, then we're going to miss it when we manage to break something.

In any case, we need a clear picture of exactly what needs to be done for the inode64 breakage to know whether we absolutely need to have separate builds or whether we can get away with playing tricks at runtime to work around it. The bug report makes it sound like we _might_ be able to work around this with runtime tricks.

- Jonathan M Davis

On 11/1/2017 10:11 PM, Jonathan M Davis wrote:
> But we
> don't currently have a way to version code for a particular version of
> FreeBSD even if we wanted to.

The FreeBSD version can be detected at runtime, and different code executed.

On Thursday, November 02, 2017 13:46:11 Walter Bright via Digitalmars-d wrote:
> On 11/1/2017 10:11 PM, Jonathan M Davis wrote:
> > But we
> > don't currently have a way to version code for a particular version of
> > FreeBSD even if we wanted to.
>
> The FreeBSD version can be detected at runtime, and different code executed.

Sure, but you can't actually compile different code based on the FreeBSD version, and if a type or function API changes, then that doesn't work. Wouldn't you get a linker error for the stuff that isn't actually defined on the C side of things? If so, then you can't really provide both the old and new versions of a function or type and switch between code that uses one or the other based on the version of the OS at runtime. You'd have to be able to have the source code differ depending on the FreeBSD version.

Depending on what was done with the compatibility layer for the 64-bit inode stuff, we may be able to do something in this case to choose which to use based on the version of the OS at runtime, because you'd have both the direct API and the compatibility API to use and thus there are different symbols to link against, but in the general case, if an API is simply updated, then you only have one version to link against. So, while we may be able to deal with this particular issue with something that makes the choice at runtime, I don't see how that would work in general any time that there's a breaking API change. Any time that happens, we'd need to be able to statically differentiate which version of FreeBSD we'd be compiling for.

And that may not actually ever come up, since it's not like the FreeBSD folks change APIs for the fun of it, but it still looks to me like not being able to version the source code based on the version of FreeBSD your compiling for is potentially a showstopper for having code work with multiple versions of FreeBSD. I think that we're more likely to get away with it with dmd, druntime, and Phobos, because we mostly use POSIX APIs, which they aren't going to be changing much, but if you're dealing with something that isn't POSIX, then the odds are that it would matter more.

In such a case, you'd probably be forced to either provide different versions of a module depending on the OS version, and have the build system differentiate for you, or you'd have to provide custom version identifiers as part of the build so that the code could be versioned appropriately.

- Jonathan M Davis