I'm working on listdir and stat functions for the file module in Phobos, which naturally requires compiling Phobos once I'm done.  The makefile is out of date, however, and the directories are all wrong.

There's more.  array, memset, obj, and switcherr are not included in the distribution but symbols from them are used.  interface.c, intrinsic.d, iunknown.d, and time.d are not touched by the makefile. None of them provide the necessary symbols, so I'm guessing there's files missing.

Finally, sc compiles the .asm files incorrectly (it adds a semicolon to the filename, bizarrely enough), and the version of masm and link I got don't like the options used.  It'd be nice if the obj file were just stuck in the distribution.

Now, onto listdir, statdir, and stat.  This code is in the public domain blah blah blah and it follows.  I tried to follow Walter's coding style, except for the no-indent-for-trivial-blocks, which I've never seen before and hopefully will never see again.

This requires "ERROR_NO_MORE_FILES = 18," in the appropriate place in windows.d and FileTimeToSystemTime using "export BOOL FileTimeToSystemTime(FILETIME* lpFileTime, SYSTEMTIME* lpSystemTime);".

-- /* For the string module */ --

/***************************************************
 * Determine whether the string ends with another.
 */

bit endswith(char[] string, char[] match)
{
    if (string.length < match.length)
        return 0;
    if (memcmp(&string [string.length - match.length], &match [0],
match.length))
        return 0;
    return 1;
}

/***************************************************
 * Determine whether the string begins with another.
 */

bit startswith(char[] string, char[] match)
{
    if (string.length < match.length)
        return 0;
    if (memcmp(&string [0], &match [0], match.length))
        return 0;
    return 1;
}

-- /* For the file module */ --

import date;
import path;

/***************************************************
 * Structure of a file stat.
 */

class Stat
{
    char[] filename; /* Filename plus full path */
    char[] filebase; /* Filename without path */
    char[] alternate; /* Alternate (8.3) filename, no path */
    d_time creation; /* When this file was created */
    d_time access; /* When this file was last used (this will probably
be writeTime in Linux) */
    d_time write; /* When this file was last written */
    long filesize; /* File size in bytes */

    bit archive; /* Is an archive file */
    bit directory; /* Is a directory */
    bit hidden; /* Is hidden (Linux should probably flag this if the
file starts with .) */
    bit readonly; /* Is read-only (Linux should set this if the
process can't write this file) */
    bit system; /* Is a system file */

    this (char[] path, WIN32_FIND_DATA finder)
    {
        SYSTEMTIME systemtime;
        FILETIME filetime;

        /* Get the filename and file size */
        filebase = (char []) finder.cFileName;
        filename = path ~ filebase;
        alternate = ((char []) finder.cAlternateFileName).dup;
        filesize = (long) finder.nFileSizeHigh * (long) 0x100000000 +
finder.nFileSizeLow;

        /* Get the attributes */
        if (finder.dwFileAttributes & FILE_ATTRIBUTE_ARCHIVE)
            archive = 1;
        if (finder.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
            directory = 1;
        if (finder.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN)
            hidden = 1;
        if (finder.dwFileAttributes & FILE_ATTRIBUTE_READONLY)
            readonly = 1;
        if (finder.dwFileAttributes & FILE_ATTRIBUTE_SYSTEM)
            system = 1;

        /* Get the times */

        filetime = finder.ftCreationTime;
        if (filetime.dwLowDateTime == 0 && filetime.dwHighDateTime ==
0)
            filetime = finder.ftLastWriteTime;
        if (!FileTimeToSystemTime(&filetime, &systemtime))
            throw new FileError(filename, GetLastError());
        creation = SYSTEMTIME2d_time(&systemtime, 0);

        filetime = finder.ftLastAccessTime;
        if (filetime.dwLowDateTime == 0 && filetime.dwHighDateTime ==
0)
            filetime = finder.ftLastWriteTime;
        if (!FileTimeToSystemTime(&filetime, &systemtime))
            throw new FileError(filename, GetLastError());
        access = SYSTEMTIME2d_time(&systemtime, 0);

        filetime = finder.ftLastWriteTime;
        if (!FileTimeToSystemTime(&filetime, &systemtime))
            throw new FileError(filename, GetLastError());
        write = SYSTEMTIME2d_time(&systemtime, 0);
    }
}

/***************************************************
 * Collate all files in a directory, returning an
 * array of strings.  The strings do not contain the
 * path.  Throws FileError if the directory does not
 * exist, if there was some file problem during the
 * collation, or if the path is malformed (generally
 * trying to use nonportable features).
 */

char[][] listdir(char[] path)
{
    WIN32_FIND_DATA finder;
    HANDLE handle;
    char[][] list;
    bit done = 0;

    /* Since we want to flatten out platform differences,
       first we cry if the user tried to use Windows. */

    if (find (path, (char) "*") >= 0 || find (path, (char) "?") >= 0)
        throw new FileError(path, "Path contains invalid characters");

    /* Form the search path for Windows */
    if (endswith(path, "/") || endswith(path, "\\"))
        path ~= "*";
    else
        path ~= "/*";

    /* Start the file search */
    handle = FindFirstFileA(toStringz(path ~ "*"), &finder);
    if (handle == (HANDLE)-1)
        throw new FileError(path, GetLastError());

    while (!done)
    {
        char[] filename;

        /* Handle this file in our own inimitable fashion */
        filename = (char[]) finder.cFileName;
        list ~= filename.dup;

        /* Go to the next file or bust */
        if (!FindNextFileA(handle, &finder))
        {
            if (GetLastError() == ERROR_NO_MORE_FILES)
                done = 1;
            else
            {
                FindClose(handle);
                throw new FileError(path, GetLastError());
            }
        }
    }

    /* Finished, clean up */
    if (!FindClose(handle))
        throw new FileError(path, GetLastError());

    return list;
}

/***************************************************
 * Collate all files in a directory, returning an
 * array of stats.  Throws FileError if the
 * directory does not exist, if there was some file
 * problem during the collation, or if the path is
 * malformed (generally trying to use nonportable
 * features).
 */

Stat[] statdir(char[] path)
{
    WIN32_FIND_DATA finder;
    HANDLE handle;
    Stat[] list;
    bit done = 0;

    /* Since we want to flatten out platform differences,
       first we cry if the user tried to use Windows. */

    if (find (path, (char) "*") >= 0 || find (path, (char) "?") >= 0)
        throw new FileError(path, "Path contains invalid characters");

    /* Form the search path for Windows */
    if (!endswith(path, "/") && !endswith(path, "\\"))
        path ~= "/";

    /* Start the file search */
    handle = FindFirstFileA(toStringz(path ~ "*"), &finder);
    if (handle == (HANDLE)-1)
        throw new FileError(path, GetLastError());

    while (!done)
    {
        /* Handle this file in our own inimitable fashion */
        list ~= new Stat (path, finder);

        /* Go to the next file or bust */
        if (!FindNextFileA(handle, &finder))
        {
            if (GetLastError() == ERROR_NO_MORE_FILES)
                done = 1;
            else
            {
                FindClose(handle);
                throw new FileError(path, GetLastError());
            }
        }
    }

    /* Finished, clean up */
    if (!FindClose(handle))
        throw new FileError(path, GetLastError());

    return list;
}

/***************************************************
 * Attempt to stat a single file, returning the
 * file details.  This will throw FileError when the
 * file cannot be found or when the path is badly
 * formed.
 */

Stat stat(char[] filename)
{
    WIN32_FIND_DATA finder;
    HANDLE handle;
    char[] path;
    Stat stat;

    /* Since we want to flatten out platform differences,
       first we cry if the user tried to use Windows. */

    if (find (filename, (char) "*") >= 0 || find (filename, (char)
"?") >= 0)
        throw new FileError(filename, "Path contains invalid
characters");

    /* Start the file search */
    handle = FindFirstFileA(toStringz(filename), &finder);
    if (handle == (HANDLE)-1)
        throw new FileError(filename, GetLastError());

    /* Construct the path */
    if (filename [0] != "/" && filename [0] != "\\")
        path = curdir;
    else
        path = getDirName(filename);

    /* Statify the sucker */
    stat = new Stat(path, finder);

    /* Finish up */
    if (!FindClose(handle))
        throw new FileError(path ~ pathsep, GetLastError());

    return stat;
}

Not really on the same subject, but I've put up a pre-alpha version of a quick program maker for D; most Deed files (as the program is called) are a single line, but have inherited options, dependencies checking, and multitargets.  You can grab it at:

http://prdownloads.sourceforge.net/willow/deed.zip

The only extra symbol I needed was system:

"extern (C) { export int system(char* string); }
int system(char[] string) { return system ((char *) &(string ~ "\0")
[0]); }"

There apparently aren't many solid programs using D, so here's my observations.

It's about as easy to prototype in D as it is in Python, if a little easier (and that's saying something)... particularly now that I have Deed, of course.

Array slicing should use default arguments; [ .. ] refers to the full array, [x .. ] refers to x to the end of the array, [ .. x] refers to the start of the array to x.  It's convenient.  I also like Python's negative array indexing, but I can see many reasons to oppose it.

I want operator overloading; something like:

operator a + b (qbert a, int b) { qbert (a.x + b); }

Clean and unambiguous; C++'s method is way too confusing.

I want control of the parser and compiler at runtime.  I also want to be able to take a class instance and reconstruct its class definition perfectly.  I then want to be able to pick up dynamic modules and link them into the running program myself.  This is all for my never-ending quest for a scripting language that can act as a system language.

Having runtime compiler control is not only useful for modularization and scripting, but also for dynamic code generation of OpenGL shaders and utility parsers that always work with the language.

Otherwise things are either working good or are on a clear path to getting good.  Nothing has made me feel that such is the wrong way to go, although I'm opposed to templates.

"Burton Radons" <loth@users.sourceforge.net> wrote in message news:4kekcu4p1pbtp790rvugkbatvln5b3ra2l@4ax.com...

> It's about as easy to prototype in D as it is in Python, if a little easier (and that's saying something)... particularly now that I have Deed, of course.

Agreed. D is definitely much better in this area than C or C++, and this is a great achievement.

> Array slicing should use default arguments; [ .. ] refers to the full array, [x .. ] refers to x to the end of the array, [ .. x] refers to the start of the array to x.  It's convenient.  I also like Python's negative array indexing, but I can see many reasons to oppose it.

Default arguments for slicing would be very nice as well. It's nothing more than syntactic sugar, but it's convenient, why not add it?

> I want control of the parser and compiler at runtime.  I also want to be able to take a class instance and reconstruct its class definition perfectly.  I then want to be able to pick up dynamic modules and link them into the running program myself.  This is all for my never-ending quest for a scripting language that can act as a system language.

Wouldn't it bloat the resulting programs too much? I can understand the need for RTTI and dynamic construction of objects of unknown classes. But how are you going to control PARSER at run-time??? And linking modules dynamically... I think it's a bit too much... DLLs are there, isn't it enough?

On Sat, 27 Apr 2002 11:06:00 +0400, "Pavel Minayev" <evilone@omen.ru> wrote:

>"Burton Radons" <loth@users.sourceforge.net> wrote in message news:4kekcu4p1pbtp790rvugkbatvln5b3ra2l@4ax.com...
>
>> It's about as easy to prototype in D as it is in Python, if a little easier (and that's saying something)... particularly now that I have Deed, of course.
>
>Agreed. D is definitely much better in this area than C or C++, and this is a great achievement.
>
>> Array slicing should use default arguments; [ .. ] refers to the full array, [x .. ] refers to x to the end of the array, [ .. x] refers to the start of the array to x.  It's convenient.  I also like Python's negative array indexing, but I can see many reasons to oppose it.
>
>Default arguments for slicing would be very nice as well. It's nothing more than syntactic sugar, but it's convenient, why not add it?

Sugar makes string processing go round.  Out of 14 string slices in Deed, 13 use either 0 or the string length on the appropriate side, pretty typical.

>> I want control of the parser and compiler at runtime.  I also want to be able to take a class instance and reconstruct its class definition perfectly.  I then want to be able to pick up dynamic modules and link them into the running program myself.  This is all for my never-ending quest for a scripting language that can act as a system language.
>
>Wouldn't it bloat the resulting programs too much? I can understand the need for RTTI and dynamic construction of objects of unknown classes. But how are you going to control PARSER at run-time??? And linking modules dynamically... I think it's a bit too much... DLLs are there, isn't it enough?

What do you mean by bloat?  If the parser's not linked in, it takes no space.  The parser part of the compiler shouldn't take much space at all even when it is linked in.

A parser class would mostly create tokens from an input stream, as well as have functions for reading in declarations and expressions and whatnot, plus a list of keywords.  It doesn't have to be a particularly complex interface, just consistent with the language.  So I could quickly pass through expressions and find any import statements no matter what future changes occur.  Good tools to parse source code are awful hard to make correct; it'd be nice if the language itself standardized this.

The compiler would produce Module, Class, or Function objects from a stream.  The common expression compile should be done by building functions.  It's for scripting.  I'm sick and/or tired of having to use two languages in a project; the reason I'm here is as part of my exploration to get out of the paradigm.  C/Python was the best I've ever tried, but getting the two languages to work together was a lot of work, and I had to put a lot of Python-applicable code in C merely because Python was so slow.

Now that I've thought through it some more, I can see a way to make D behave for the most part, so dynamic linking control is not necessary.

The only thing I need from here is a compiler and parser, actually, for in-game entities with code.  But I'll let it simmer until I'm actually at the point that I could use it.

Oh, Walter: Trying to compile mfile.d in Deed using -O causes an "Internal error: ..\ztc\gloop.c 1299".  It appears to be in the listdir function, specifically the line "filename = (char[]) finder.cFileName;"  Commenting this out removes the internal error, although a doppleganger of this line appears earlier in Stat's constructor.

"Burton Radons" <loth@users.sourceforge.net> wrote in message news:q5tkcus6dleacm81j2v6jl9h45fri6qbgg@4ax.com...

> What do you mean by bloat?  If the parser's not linked in, it takes no space.  The parser part of the compiler shouldn't take much space at all even when it is linked in.

Parser of what? The program is already compiled to machine code!

> A parser class would mostly create tokens from an input stream, as well as have functions for reading in declarations and expressions and whatnot, plus a list of keywords.  It doesn't have to be a particularly complex interface, just consistent with the language.  So I could quickly pass through expressions and find any import statements no matter what future changes occur.  Good tools to parse source code are awful hard to make correct; it'd be nice if the language itself standardized this.

So, you propose to integrate a D interpreter into each program?

> The compiler would produce Module, Class, or Function objects from a stream.  The common expression compile should be done by building functions.  It's for scripting.  I'm sick and/or tired of having to use two languages in a project; the reason I'm here is as part of my exploration to get out of the paradigm.  C/Python was the best I've ever tried, but getting the two languages to work together was a lot of work, and I had to put a lot of Python-applicable code in C merely because Python was so slow.

I guess it's better to have an external D interpreter module (DScript or something), available as a part of Phobos. Not something built into the language itself though.

> The only thing I need from here is a compiler and parser, actually, for in-game entities with code.  But I'll let it simmer until I'm actually at the point that I could use it.

Oh, now I see. Sorta UnrealScript, right? Well, then, as I proposed a standard interpreter module could be done. Seems like a worthy idea.

... wanna try?

"Burton Radons" <loth@users.sourceforge.net> wrote in message news:q5tkcus6dleacm81j2v6jl9h45fri6qbgg@4ax.com...
> On Sat, 27 Apr 2002 11:06:00 +0400, "Pavel Minayev" <evilone@omen.ru> wrote:
> >"Burton Radons" <loth@users.sourceforge.net> wrote in message news:4kekcu4p1pbtp790rvugkbatvln5b3ra2l@4ax.com...
> >> Array slicing should use default arguments; [ .. ] refers to the full array, [x .. ] refers to x to the end of the array, [ .. x] refers to the start of the array to x.  It's convenient.  I also like Python's negative array indexing, but I can see many reasons to oppose it.
> >Default arguments for slicing would be very nice as well. It's nothing more than syntactic sugar, but it's convenient, why not add it?
> Sugar makes string processing go round.  Out of 14 string slices in Deed, 13 use either 0 or the string length on the appropriate side, pretty typical.

This is a good idea.


> What do you mean by bloat?  If the parser's not linked in, it takes no space.  The parser part of the compiler shouldn't take much space at all even when it is linked in.

Are you thinking of something like javascript's exec function?

> Oh, Walter: Trying to compile mfile.d in Deed using -O causes an "Internal error: ..\ztc\gloop.c 1299".  It appears to be in the listdir function, specifically the line "filename = (char[]) finder.cFileName;"  Commenting this out removes the internal error, although a doppleganger of this line appears earlier in Stat's constructor.

Ok, could you email me please a reproducible example?

On Sat, 27 Apr 2002 09:50:25 -0700, "Walter" <walter@digitalmars.com> wrote:

>"Burton Radons" <loth@users.sourceforge.net> wrote in message news:q5tkcus6dleacm81j2v6jl9h45fri6qbgg@4ax.com...
[snip]
>> What do you mean by bloat?  If the parser's not linked in, it takes no space.  The parser part of the compiler shouldn't take much space at all even when it is linked in.
>
>Are you thinking of something like javascript's exec function?

You mean eval?  No - t'would be a pain to get the local scope into the dynamic code (and not really be desirable) and the type system would interfere.  Better would be to construct a function in text, compile that, then execute it.

For the parser, I just mean the ability to get tokens and parse declarations, nothing more.  Just a few judicious links into the frontend as it stands, with the frontend refactored into a library. So, for a minimal example:

class DLexer
{
    this (wchar[] text);
    int LineNo ();
    DToken Token ();
}

enum DTokenType { ... }

class DToken
{
    DTokenType type;
    DIdentifier ident;
}

class DIdentifier
{
    int value;
    wchar[] string;
}

With additional features added later.  I'm not proposing suddenly jumping into a full-fledged feature-complete parser, but just a start that I can add things to.  The only part of the work I can't do is putting the frontend into a library that the backend uses.  I'm volunteering my time for the rest.

>> Oh, Walter: Trying to compile mfile.d in Deed using -O causes an "Internal error: ..\ztc\gloop.c 1299".  It appears to be in the listdir function, specifically the line "filename = (char[]) finder.cFileName;"  Commenting this out removes the internal error, although a doppleganger of this line appears earlier in Stat's constructor.
>
>Ok, could you email me please a reproducible example?

This is the smallest fragment I could get it into:

void listdir()
{
    char[14] foo;
    char[] bar;

    while (1)
        bar = foo;
}

Using "\dmd\bin\dmd -I/dmd/src/phobos file.d -O" to compile, version 0.28.

Thanks, I can reproduce the error. I'll see about a fix.

"Burton Radons" <loth@users.sourceforge.net> wrote in message news:tg9jcu418udak1jp1g1d98g82fqjfk9or3@4ax.com...
> There's more.  array, memset, obj, and switcherr are not included in the distribution but symbols from them are used.

That's a mistake. I'll put them in.

>  interface.c,

Obsolete, I'll remove it.

> intrinsic.d,

Just declarations. No need to compile it.

> iunknown.d,

Needs to be added to the library, I'll fix it.

>> and time.d

Just declarations, no need to compile it.

> are not touched by the makefile.
>
> Finally, sc compiles the .asm files incorrectly (it adds a semicolon to the filename, bizarrely enough), and the version of masm and link I got don't like the options used.

Unfortunately, Microsoft keeps changing how MASM works. Older versions did require the ;. It's a constant problem, and one reason why D has its own inline assembler.

> It'd be nice if the obj file were
> just stuck in the distribution.

Good idea. I'll do it.

"Burton Radons" <loth@users.sourceforge.net> wrote in message news:tg9jcu418udak1jp1g1d98g82fqjfk9or3@4ax.com...
> This requires "ERROR_NO_MORE_FILES = 18," in the appropriate place in windows.d and FileTimeToSystemTime using "export BOOL FileTimeToSystemTime(FILETIME* lpFileTime, SYSTEMTIME* lpSystemTime);".

Done.