[Issue 2842] New: std.file.listdir on OSX produces invalid UTF-8 sequence

Apr 16, 2009

d-bugmail

Jul 17, 2009

Mike Kinney

Nov 09, 2009

Aug 12, 2010

May 15, 2011

May 15, 2011

May 15, 2011

May 15, 2011

May 15, 2011

May 28, 2011

http://d.puremagic.com/issues/show_bug.cgi?id=2842 Summary: std.file.listdir on OSX produces invalid UTF-8 sequence Product: D Version: 1.043 Platform: PC OS/Version: Mac OS X Status: NEW Severity: minor Priority: P2 Component: Phobos AssignedTo: bugzilla@digitalmars.com ReportedBy: af@liquidstate.eu std.file.listdir throws an "invalid UTF-8 sequence" exception. please see the source file and the produced output below. thanks. ------------------------------ import std.stdio, std.file; int main(char[][] args) { foreach (file; listdir(".")) writefln("%s", file); return 0; } ------------------------------ blah:misc user$ dmd listdir.d blah:misc user$ ./listdir .. some_directory Error: 1invalid UTF-8 sequence blah:misc user$ --

http://d.puremagic.com/issues/show_bug.cgi?id=2842 Mike Kinney <mike.kinney@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mike.kinney@gmail.com --- Comment #1 from Mike Kinney <mike.kinney@gmail.com> 2009-07-16 23:07:16 PDT --- What does the "ls -al" show for that directory? Is it happening on all files? It works ok on v2.031 when run from the samples dir. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

November 09, 2009

[Issue 2842] std.file.listdir on OSX produces invalid UTF-8 sequence

Posted by Justin Whear
in reply to d-bugmail

Permalink

Justin Whear

Posted in reply to d-bugmail

Permalink

http://d.puremagic.com/issues/show_bug.cgi?id=2842


Justin Whear <mrjnewt@hotmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mrjnewt@hotmail.com


--- Comment #2 from Justin Whear <mrjnewt@hotmail.com> 2009-11-09 13:15:23 PST ---
I can confirm this bug using DMD 1.050 on OS X 10.5.
Running program by OP:
emsi-testings-mac-mini:ratchet emsitesting$ ./listdir
.
.
ogout
.
e
istory
.
s
.
_central.d
t
re
sion
.
.
ac.sh
inux.sh
_central.o
_central
.
.d
.o
.


The function seems to be trimming the names arbitrarily. Here's ls -al in same
directory:
emsi-testings-mac-mini:ratchet emsitesting$ ls -al
total 1324
drwx------   0 emsitesting  staff   16384 Nov  9 13:10 .
drwxrwxrwt@  5 root         admin     170 Nov  5 11:35 ..
-rw--w----   1 emsitesting  staff    6148 Nov  5 11:27 .DS_Store
-rw-------   1 emsitesting  staff    7467 Nov  9 12:36 .bash_history
-rw-r--r--   1 emsitesting  staff     220 Nov  4 11:11 .bash_logout
-rw-r--r--   1 emsitesting  staff    3211 Nov  4 13:32 .bashrc
drwxr-xr-x   2 emsitesting  staff   16384 Nov  4 13:27 .cache
-rw-r--r--   1 emsitesting  staff     675 Nov  4 11:11 .profile
-rw-r--r--   1 emsitesting  staff      81 Nov  6 12:20 .ratchet
drwx------   2 emsitesting  staff   16384 Nov  4 13:53 .ssh
drwxr-xr-x   3 emsitesting  staff   16384 Nov  6 10:50 .subversion
-rwxrwxr-x   1 emsitesting  staff     228 Nov  6 13:41 build_linux.sh
-rwxr-xr-x   1 emsitesting  staff     187 Nov  6 14:17 build_mac.sh
lrwxrwxrwx   1 emsitesting  staff       6 Nov  6 10:50 ccb -> d/ccb/
lrwxrwxrwx   1 emsitesting  staff      16 Nov  6 10:54 chains ->
d/chains/chains/
drwxr-xr-x  26 emsitesting  staff   16384 Nov  9 12:02 d
-rwxr-xr-x   1 emsitesting  staff  261192 Nov  9 13:10 listdir
-rw-rw-r--   1 emsitesting  staff     138 Nov  9 13:10 listdir.d
-rw-rw-r--   1 emsitesting  staff    2748 Nov  9 13:10 listdir.o
drwxr-xr-x   6 emsitesting  staff   16384 Nov  9 12:34 projects
-rwxr-xr-x   1 emsitesting  staff  260223 Nov  6 13:37 ratchet_central
-rw-r--r--   1 emsitesting  staff    2331 Nov  6 13:37 ratchet_central.d
-rw-r--r--   1 emsitesting  staff    9456 Nov  6 13:37 ratchet_central.o

This one's rather critical as I have code that I'm trying to get running on OS X.


(In reply to comment #1)
> What does the "ls -al" show for that directory?
> 
> Is it happening on all files?
> 
> It works ok on v2.031 when run from the samples dir.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 Justin Whear <mrjnewt@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|minor |major --- Comment #3 from Justin Whear <mrjnewt@hotmail.com> 2010-08-12 09:20:03 PDT --- This bug still exists in 1.063. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 --- Comment #5 from Jakob Bornecrantz <wallbraker@gmail.com> 2011-05-15 15:05:16 PDT --- Still exists in 1.064, 1.065 and 1.068. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 Jakob Bornecrantz <wallbraker@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P2 |P1 Version|1.043 |D1 Severity|major |critical --- Comment #6 from Jakob Bornecrantz <wallbraker@gmail.com> 2011-05-15 15:19:47 PDT --- Looks like the cause of this is due to std.c.linux.linux is being used in std.file where dirent is defined to this: struct dirent { uint d_ino; // this is int on some linuxes off_t d_off; ushort d_reclen; ubyte d_type; // this field isn't there on some linuxes char[256] d_name; } While in /usr/include/sys/dirent.h it is defined to this: struct dirent { ino_t d_ino; /* file number of entry */ __uint16_t d_reclen; /* length of this record */ __uint8_t d_type; /* file type, see below */ __uint8_t d_namlen; /* length of string in d_name */ char d_name[__DARWIN_MAXNAMLEN + 1]; /* name must be no longer than this */ }; Cheers Jakob. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 --- Comment #7 from Jakob Bornecrantz <wallbraker@gmail.com> 2011-05-15 15:20:31 PDT --- Created an attachment (id=976) Patch that should fix the problem, untested. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 --- Comment #8 from Jakob Bornecrantz <wallbraker@gmail.com> 2011-05-15 15:22:58 PDT --- Created an attachment (id=977) Test program from other bug. Here is the test program from issue 6011. Cheers Jakob. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

http://d.puremagic.com/issues/show_bug.cgi?id=2842 Walter Bright <bugzilla@digitalmars.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED CC| |bugzilla@digitalmars.com Resolution| |FIXED --- Comment #9 from Walter Bright <bugzilla@digitalmars.com> 2011-05-27 21:04:16 PDT --- https://github.com/D-Programming-Language/phobos/commit/149bab20505637a2a77a13081c2b0fb93b27f959 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------

Forums