Jump to page: 1 24  
Page
Thread overview
[Issue 17596] dmd d 2.073.2 and 2.074.1 interim generated dmd segfaults on FreeBSD 12-CURRENT
Jul 05, 2017
Cy Schubert
Jul 05, 2017
Cy Schubert
Jul 06, 2017
Vladimir Panteleev
Jul 06, 2017
Cy Schubert
Jul 06, 2017
Vladimir Panteleev
Jul 09, 2017
Cy Schubert
Jul 10, 2017
anonymous4
Jul 10, 2017
Vladimir Panteleev
Jul 14, 2017
Nemanja Boric
Jul 19, 2017
Cy Schubert
Jul 20, 2017
Vladimir Panteleev
Sep 23, 2017
Cy Schubert
Oct 16, 2017
Cy Schubert
Oct 18, 2017
Walter Bright
Oct 18, 2017
Walter Bright
Oct 18, 2017
Walter Bright
Nov 03, 2017
anonymous4
Feb 23, 2018
Walter Bright
Feb 23, 2018
Cy Schubert
May 24, 2018
anonymous4
Aug 15, 2018
Jonathan M Davis
Aug 15, 2018
Jonathan M Davis
Aug 16, 2018
Martin Nowak
Aug 18, 2018
Jonathan M Davis
Aug 11, 2019
Cy Schubert
Nov 13, 2020
Dlang Bot
Nov 24, 2020
Dlang Bot
Nov 24, 2020
Iain Buclaw
Nov 24, 2020
Iain Buclaw
Nov 24, 2020
Dlang Bot
July 05, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

Cy Schubert <cy@FreeBSD.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P1                          |P2

--
July 05, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #1 from Cy Schubert <cy@FreeBSD.org> ---
Silly me, thinking kernel instead of application. s/panic/segfault/.

--
July 06, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #2 from Vladimir Panteleev <dlang-bugzilla@thecybershadow.net> ---
I don't know much about FreeBSD, but do I understand correctly that this is a breaking ABI change? Meaning that it's not possible to compile a binary that will work on both FreeBSD systems before the change and FreeBSD systems after it?

--
July 06, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #3 from Cy Schubert <cy@FreeBSD.org> ---
Correct.

The implementer of inode64 has done a great job of avoiding ABI breakage. Applications built before inode64 continue to run due to code in libc to detect "old" v.s. "new". Most newly compiled code builds and runs okay. DMD is different however because many data structures are replicated in DMD. For example stat_t and dirent_t headers have issues.

I wasn't able to find a stub that DMD D might use to make the actual call. If you can point me in the right direction I should be able to do the work myself (when I can find the time).

Having said that, any change would be incompatible with older releases of FreeBSD so, a possible solution might be a version check for version(FreeBSD12+) -- (FreeBSD uses a version check in /usr/include/sys/param.h.) Is there a way to determine which release of FreeBSD within DMD D? Such as version(something)?

So summarize my thoughts above:

1. The data structures and files that use inodes system calls to stat(), fstat, and others need to be identified. I can do some of the work or we can work together.

2. Some way of identifying "old" FreeBSD v.s. "new" FreeBSD through a version check.

Hopefully this is not too much information (as I tend to get into the weeds).

~cy

--
July 06, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #4 from Vladimir Panteleev <dlang-bugzilla@thecybershadow.net> ---
(In reply to Cy Schubert from comment #3)
> The implementer of inode64 has done a great job of avoiding ABI breakage. Applications built before inode64 continue to run due to code in libc to detect "old" v.s. "new". Most newly compiled code builds and runs okay. DMD is different however because many data structures are replicated in DMD. For example stat_t and dirent_t headers have issues.

How does the old/new detection work? Does it look at the ELF timestamp, or something like that?

Because D has binary releases for FreeBSD (and compiling it from source is a bit onerous as it requires bootstrapping through an older version of D), we will probably want to ensure, if possible, that the binary D releases will work on both new and old versions of the system for a while. This likely means that we would want to use the old fstat ABI exclusively for a few years, so if it's possible to opt in to the old ABI, that would probably be the way to go for now.

Is the new functionality available via a new function akin to fstat64 on Linux?

> I wasn't able to find a stub that DMD D might use to make the actual call. If you can point me in the right direction I should be able to do the work myself (when I can find the time).

dmd/src/root/file.d is now located at dmd/src/ddmd/root/file.d:

https://github.com/dlang/dmd/blob/master/src/ddmd/root/file.d#L97

For FreeBSD, fstat is defined here:

https://github.com/dlang/druntime/blob/master/src/core/sys/posix/sys/stat.d#L1207

and stat_t here:

https://github.com/dlang/druntime/blob/master/src/core/sys/posix/sys/stat.d#L730-L758

Finally, ino_t is defined here:

https://github.com/dlang/druntime/blob/master/src/core/sys/posix/sys/types.d#L135

I believe that fstat is called by linking to the respective libc, i.e. D does not perform the syscall directly.

> Having said that, any change would be incompatible with older releases of FreeBSD so, a possible solution might be a version check for version(FreeBSD12+) -- (FreeBSD uses a version check in /usr/include/sys/param.h.) Is there a way to determine which release of FreeBSD within DMD D? Such as version(something)?

Not as far as I know. I guess it would require changing the compiler to detect the FreeBSD release and enable a corresponding version, which would also need to be added.

--
July 09, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #5 from Cy Schubert <cy@FreeBSD.org> ---
Thanks. I'll develop a patch (will take some time -- I have a number of projects on the fly at the moment). The patch will will work for FreeBSD 12 after r318736 but break all FreeBSD releases prior to that. This is not optimal. Is there a version() statement that will allow me to test for FreeBSD version or can DMD read /usr/include/sys/param.h for the C pre-processor define of __FreeBSD_version?

Also, it appears that ldc (DMD D with an llvm backend, see http://wiki.dlang.org/LDC) has the same segfault under FreeBSD 12. I don't maintain it for the FreeBSD project but it too suffers the same segfault. Maybe a more holistic approach needs to be taken than a simple point patch. Eventually when FreeBSD 12 is released to the public there will be two different stat_t and dirent_t data structures (pre- and post-inode64). What do you think if DMD D and LDC D provided a facility to test __FreeBSD_version or if not that the major.minor version number? No matter what we do we will break old or new systems.

If you can provide a version() statement that can test for either __FreeBSD_version in /usr/include/sys/param.h (preferred) or the FreeBSD major.minor version number, any patch I produce will work for both. (I kind of think this might be useful for most other O/S's too, e.g. RHEL 6 v.s. RHEL 7, though the test would be very different for each.)

--
July 10, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #6 from anonymous4 <dfj1esp02@sneakemail.com> ---
see https://github.com/freebsd/freebsd/blob/master/lib/libc/gen/gen-compat.h

--
July 10, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #7 from Vladimir Panteleev <dlang-bugzilla@thecybershadow.net> ---
(In reply to Cy Schubert from comment #5)
> What do you think if DMD D and LDC D provided a facility to test __FreeBSD_version or if not that the major.minor version number?

Cy, you mentioned that the patch came with careful backwards compatibility provisions so as not to break older binaries. In my previous comment I asked how that compatibility check works, and whether we can take advantage of it. Have you looked into that at all yet?

--
July 14, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

Nemanja Boric <4burgos@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |4burgos@gmail.com

--
July 19, 2017
https://issues.dlang.org/show_bug.cgi?id=17596

--- Comment #8 from Cy Schubert <cy@FreeBSD.org> ---
Sorry for not replying sooner, way too many projects on the go here.

Here is how kib supported older apps prior to inode64:

https://svnweb.freebsd.org/base?view=revision&revision=318736

Forward compatibility:

https://svnweb.freebsd.org/base?view=revision&revision=320278

Stat.c, below, relies on versioned symbols (see https://svnweb.freebsd.org/base/head/lib/libprocstat/Symbol.map?r1=318736&r2=318735&pathrev=318736)

int
stat(const char *path, struct stat *sb)
{
        struct freebsd11_stat stat11;
        int rv;

        if (__getosreldate() >= INO64_FIRST)
                return (__sys_fstatat(AT_FDCWD, path, sb, 0));
        rv = syscall(SYS_freebsd11_stat, path, &stat11);
        if (rv == 0)
                __stat11_to_stat(&stat11, sb);
        return (rv);
}

IMO implementing it in this way would require architectural changes to D's implementation of some syscalls, similar to how Perl might interface with the O/S. Discussing this with the maintainer of the ldc port, the short term solution might be for he and I to patch the ports, however that would be a fragile solution as depending on the options used at FreeBSD build time, stat_t and dirent_t may vary.

--
« First   ‹ Prev
1 2 3 4