Generally, I think Phobos has picked a good abstraction level (threads, sockets, files, etc.).  It also has some excellent additional functionality (base64, utf, zlib, etc.).  But some of the modules -- while cool and useful -- don't strike me as universal (recls, uri, zip, etc.).  I'm concerned that Phobos will fracture, devolving from "universal, core functionality with rock-solid support" to "a collection of useful, albeit diverse functionality with varying support depending on the author".  Does anyone else share this concern?  Might we add a seventh rule to the "Phobos Philosophy", phrased something like:

"No special-purpose modules.  Don't add modules to Phobos that accomplish specific, high-level functions.  Rather, Phobos is reserved for modules that provide universal, low-level functionality that almost all applications require, irrespective of problem domain.  Phobos modules are supported with a priority equal to the compiler itself.  The namespace is big.  Use it."

In my opinion, the greatest value Java and C# bring over C++ is not any particular language feature, but a well-supported, well-defined platform on which to build applications.  For example, virtually all Java programmers use the same core classes for strings, files, threads, sockets, etc.  This vastly reduces the learning curve for integrating a new developer into your team, as well as reduces the "design space" of options to consider when creating a new application.

Given that D doesn't have nearly the resources put into it as Java or C#, I think we need to be extremely disciplined about putting only the fewest, most critical features into Phobos, and then supporting them to the best of our ability.  Thus don't put anything in that we can't afford to make as rock-solid as the compiler itself.

After a quick glance, I'd recommend reorganizing Phobos as follows:

Keep little more than a hardware abstraction layer in 'std', and maintain at
the highest priority
- std.compiler - Information about the D compiler implementation.
- std.ctype - Simple character classification
- std.file - Basic file operations like read, write, append.
- std.gc - Control the garbage collector.
- std.intrinsic - Compiler built in intrinsic functions
- std.math - Include all the usual math functions like sin, cos, atan, etc.
- std.mmfile - Memory mapped files.
- object - The root class of the inheritance hierarchy
- std.outbuffer - Assemble data into an array of bytes
- std.process - Create/destroy threads.
- std.random - Random number generation.
- std.socket - Sockets.
- std.stdio - Standard I/O.
- std.stream - Stream I/O.
- std.stdint - Integral types for various purposes.
- std.string - Basic string operations not covered by array ops.
- std.system - Inquire about the CPU, operating system.
- std.thread - One per thread. Operations to do on a thread.
- std.utf - Encode and decode utf character encodings.

Move general purpose, frequently useful code into 'util', and maintain
according to interest
- std.base64 - Encode/decode base64 format.
- std.conv - Conversion of strings to integers.
- std.date - Date and time functions. Support locales.
- std.format - Formatted conversions of values to strings.
- std.md5 - Compute MD5 digests.
- std.path - Manipulate file names, path names, etc.
- std.regexp - The usual regular expression functions.
- std.socketstream - Stream for a blocking, connected Socket.
- std.uri - Encode and decode Uniform Resource Identifiers (URIs).
- std.zlib - Compression / Decompression of data.

Move high-level, single purpose modules into "misc", and leave to authors to
maintain
- std.recls - Recursively search file system and (currently Windows only)
FTP sites.
- std.zip - Read/write zip archives.

Oh, and finally, I'd recommend ditching the name "Phobos" and just calling it "the D standard library".  By assigning it a separate name, we treat it as separate entity, divorced from the "official D development package".

-david

I would really like to agree with this post..

However, I suggest organizing the "Phobos" library differently...

object - No Complaints...
std    - This is good too...
compiler  - Keep the compiler stuff here...
intrinsic - Not like anything else is going to ever need this symbol.
string    - Move EVERYTHING that has to do with working with
strings in here. if you want the whole thing, use
std.string, if you want less, include them individually.
convert - Yes, I think we should use all 7 letters of the word
convert here... Why not? We used all 8 of compiler?
Are we saving people time, or making things easy to
understand? Honestly, this gets on my nerves.
test    - What's in here? C TypeDefs? No, they are tests...
Tests on characters. Yes I get it. Character TYPE.
But this is lame. Put all the character and string
tests in here, and move toupper and tolower to the
convert section.
format  - Perfect place for functions that format strings.
path    - Aren't these functions all just more string functions?
But move those matching functions to test, and make
them more general. C'mon, those are usefull for other
stuff, not just file names.
regexp  - I won't compain about the name, its pretty normal to call
it this, but this one still bothers me.
crypt   - This gets the contents of base64 and md5. Maybe even sub
set them as sub modules. Think ahead. How many algorithms
are there? Don't crowd the namespace.
uri     - Seems like a good place for these too.
utf
int  - I'm sorry, but std.stdint is as bad as Windows 2000's boot
message saying "Built on NT Technology"... (NT = New Technology)
math - First off, move the random stuff here too. Why wouldn't you?
Stop trying to make D like C, only.. kinda different!!
file - No need to change this. I love it here!
mmfile - If it's gonna throw a std.file.FileException, put it in
std.file...
search - recls is a bad name. This name is more universal, and we
can expand the module in the future to include other search
tools for files and folders.
buffer - outbuffer is too specific a name for something so usefull.
date   - This should stay here. What could be more universal?

net    - This is where network stuff should be.
socket - See how this tucks in nicely... As we get more networking
modules, put them in net too.
stream - This should be in socket. Let the compiler get rid
of what we don't use, and keep names clean. Besides,
as more socket helpers come, put them socket too.
Leave these...
gc
stream
system
process
thread

Remove these...
zlib  - This is a wrapper... Don't put it in the standard library namespace.
zip   - This is a good use of zlib, but, making the std library dependent on
zlib? Make a new library called compress or something and put the
old zlib interface in there, as well as other tools like this. Just
keep it out of the standard library...

And yes, Phobos, is a bad name. why do we access it as std. in actual code, and call it phobos otherwise? Look, I love giving things cute names, but as a usability consultant, I know that cute names are for marketing, and when they dtract from the ability to unserstand what is going on, in this case the relation of phobos and D, they are a VERY bad thing. I also have noticed the devvelopment of other standard libraries for D with other cute names. To really set the "Phobos" library apart as the one that is the native, D Standard Library, we should call it nothing more than just that. The "D Standard Library"

Let me know what you think... I know my opinions are drastically different in some ways to the original layout, but sometimes an outside perspective can really help to clean up something that was made as you went like D obviously was.

Thanks,
Trevor Parscal
www.trevorparscal.com

David Barrett wrote:

> I'm  concerned that Phobos will fracture, devolving from "universal, core functionality with rock-solid support" to "a collection of useful, albeit diverse functionality with varying support depending on the author".  Does anyone else share this concern? 

Considering that both D compilers now has their own version of Phobos,
I'd say it's a valid concern. And the recls support has been described
by the author of that particular package as... well, not rock-solid ?

There is more than one high-end alternative to Phobos, so the more of
a basic runtime library that it could become - the better (just IMHO).


The main objections against Phobos boils down to something like:
- any and all changes to it taking ages to appear (the "Cathedral")
- lack of object-orientation (usually the lack of a "class String")
- runtime library being tied to the DMD compiler (i.e. not portable)
- runtime library being tied to standard library (e.g. GC, callbacks)
- missing lib features, like a template library or a widget toolkit

There has been different "solutions" to this: Deimos, Dool, and so on.


I think the best would probably be if Phobos was split in three parts,
* lowlevel like _d_assert, object and std.gc (std.c too, probably ?)
* regular D standard library modules : std.stdio, std.utf, etc...
* highlevel add-on C libraries: recls, zlib, libSDL, OpenGL, etc...

At least on a "philosophical" level, it does not have to be visible ?


> After a quick glance, I'd recommend reorganizing Phobos as follows:

For some reason, you left out std.c.math, std.c.stdio and std.c.stdlib ?
Phobos still uses the C standard library to implement a lot of features.

There is also the veritable zoo of std.c.operatingsystemdujour modules,
which I can understand that you didn't want to mention either... :-)

But both of these are part of Phobos.

> Keep little more than a hardware abstraction layer in 'std', and maintain at the highest priority
> - std.compiler - Information about the D compiler implementation.

This file is just a stub at the moment, hardcoded as "DMD 0.0"...

> - object - The root class of the inheritance hierarchy

As you might be aware, this module now holds Exception and Error too.
(well, and "printf" and "bool" and the kitchen sink too - but anyway)

> - std.stdio - Standard I/O.

Currently lacking all forms of input, making it the "Standard O" :-)
(but code to implement readf and std.unformat have been submitted...)

> - std.stdint - Integral types for various purposes.

Mostly converting legacy C code, not for use with new D code...
(I think that it *could* be used, but I suggested std.stdutf too)

> - std.system - Inquire about the CPU, operating system.

Again, this module is just stub. Hardcoded as "Windows NT 4.0"...

> - std.zlib - Compression / Decompression of data.

Most of the functionality in this one comes from etc.c.zlib
(which in turn is a straight-forward translation of zlib.h)

Preferably, Phobos should use the system version of zlib -
and just use the bundled static zlib version as a fallback... ?

And then it could be moved to "std.c.zlib instead", as well.

> Move high-level, single purpose modules into "misc", and leave to authors to maintain

I think the chosen name for this hierarchy is already "etc",
and has been partly taken up by the Deimos project already...

http://dsource.org/projects/deimos/

Both recls and zlib also have different licensing than Phobos does,
if you disregard the handful of std modules with different licenss.

> - std.recls - Recursively search file system and (currently Windows only) FTP sites.

The (C++) implementation of this module being found under etc already...

I know that std.recls itself has a couple of version(GNU) patches, too.

> - std.zip - Read/write zip archives.

std.zip is not that bad, it's std.zlib that has the etc dependancies...

In the current implementation, they should move to etc.zip and etd.zlib

> Oh, and finally, I'd recommend ditching the name "Phobos" and just calling it "the D standard library".  By assigning it a separate name, we treat it as separate entity, divorced from the "official D development package".

I think the "D Runtime Library" is already the official name for it ?

http://www.digitalmars.com/d/phobos.html
> Phobos is the standard runtime library that comes with the D language compiler.

But it would be nice if:
1) The Phobos library source code could be shared between DMD and GDC
2) The "D runtime library" and the "D standard library" could be split
3) recls and zlib could move from the "std" to the "etc" hierarchy

I can contribute my adapations of SDL 1.2.8 and OpenGL 1.2.1 headers as
etc.c.sdl and etc.c.opengl, if wanted... ? (Different from Derelict's,
in that it doesn't use function pointers - just a C header translation)

--anders

I would really like to agree with this post..

However, I suggest organizing the "Phobos" library differently...
--------------------------------------------------------------------------------
object - No Complaints...

std.compiler - Keep the compiler stuff here...

std.intrinsic - Not like anything else is going to ever need this symbol.

std.string - Move EVERYTHING that has to do with working with strings in here. if you want the whole thing, use std.string, if you want less, include them individually.

std.string.convert - Yes, I think we should use all 7 letters of the word
convert here... Why not? We used all 8 of compiler?
Are we saving people time, or making things easy to
understand? Honestly, this gets on my nerves.

std.string.test - What's in here? C TypeDefs? No, they are tests... Tests on characters. Yes I get it. Character TYPE. But this is lame. Put all the character and string tests in here, and move toupper and tolower to the convert section.

std.string.format - Perfect place for functions that format strings.

std.string.path - Aren't these functions all just more string functions? But move those matching functions to test, and make them more general. C'mon, those are usefull for other stuff, not just file names.

std.string.regexp  - I won't compain about the name, its pretty normal to call it this, but this one still bothers me.

std.string.crypt - This gets the contents of base64 and md5. Maybe even sub set them as sub modules. Think ahead. How many algorithms are there? Don't crowd the namespace.

std.string.uri - Seems like a good place for these too. std.string.utf

std.int - I'm sorry, but std.stdint is as bad as Windows 2000's boot message saying "Built on NT Technology"... (NT = New Technology)

std.math - First off, move the random stuff here too. Why wouldn't you? Stop trying to make D like C, only.. kinda different!!

std.file - No need to change this. I love it here!

std.file.mm - If it's gonna throw a std.file.FileException, put it in std.file...

std.file.search - recls is a bad name. This name is more universal, and we can expand the module in the future to include other search tools for files and folders.

std.buffer - outbuffer is too specific a name for something so usefull.

std.date   - This should stay here. What could be more universal?

std.net    - This is where network stuff should be.

std.net.socket - See how this tucks in nicely... As we get more networking modules, put them in net too.

std.net.socket.stream - This should be in socket. Let the compiler get rid
of what we don't use, and keep names clean. Besides,
as more socket helpers come, put them socket too.

Leave these...
std.gc
std.stream
std.system
std.process
std.thread

Remove these...
std.zlib  - This is a wrapper... Don't put it in the standard library namespace.

std.zip   - This is a good use of zlib, but, making the std library dependent on
zlib? Make a new library called compress or something and put the
old zlib interface in there, as well as other tools like this. Just
keep it out of the standard library...

And yes, Phobos, is a bad name. why do we access it as std. in actual code, and call it phobos otherwise? Look, I love giving things cute names, but as a usability consultant, I know that cute names are for marketing, and when they dtract from the ability to unserstand what is going on, in this case the relation of phobos and D, they are a VERY bad thing. I also have noticed the devvelopment of other standard libraries for D with other cute names. To really set the "Phobos" library apart as the one that is the native, D Standard Library, we should call it nothing more than just that. The "D Standard Library"

Let me know what you think... I know my opinions are drastically different in some ways to the original layout, but sometimes an outside perspective can really help to clean up something that was made as you went like D obviously was.

Thanks,
Trevor Parscal
www.trevorparscal.com

Trevor Parscal wrote:

> std.string.regexp  - I won't compain about the name, its pretty normal to call
> it this, but this one still bothers me.

"standard.regularexpression" becomes kinda awkward :-)

> std.string.crypt - This gets the contents of base64 and md5. Maybe even sub set
> them as sub modules. Think ahead. How many algorithms are there? Don't crowd the
> namespace.

Although, base64 and md5 are not encryptions ? ("crypt")
And neither of them has much to do with strings, either ?

Grouping them is OK, just not as "string" or "crypt"...

There is also "crc", which kinda got lost ? (crc32.d)
Most likely, it also belongs in a crc32 module here.

> std.int - I'm sorry, but std.stdint is as bad as Windows 2000's boot message
> saying "Built on NT Technology"... (NT = New Technology)

stdint.h is a portability wrapper, stdint.d is the same.
So the naming for this particular module is due to legacy
interfacing with C, same as std.c.stdio and std.c.stdlib

But the same goes for std.stdio, of course... ("std.io" ?)

See earlier posts, on the same topic. Here is one sample thread:
http://www.digitalmars.com/d/archives/digitalmars/D/6450.html

> std.zlib  - This is a wrapper... Don't put it in the standard library namespace.

It belongs in "etc.zlib" (since it calls upon etc.c.zlib...)

> std.zip   - This is a good use of zlib, but, making the std library dependent on
> zlib? Make a new library called compress or something and put the
> old zlib interface in there, as well as other tools like this. Just
> keep it out of the standard library...

And if anything, it probably also should go into etc instead.

--anders

The crypt thing, I agree. sorry, it's late. I am not thinking straight. but grouping them is a good idea.

etc... As long as it's clean. That works..

Othwerwise, you get my idea. Clean it up. Names of things should have a strong meaning, not just, "Oh, now that you explain it, that kinda makes sense.."

Thanks
Trevor Parscal
www.trevorparscal.com

Trevor Parscal wrote:

> Othwerwise, you get my idea. Clean it up. Names of things should have a strong
> meaning, not just, "Oh, now that you explain it, that kinda makes sense.."

Most of the names have history reasons, since D has a strong C legacy.

Whether or not they need to be "cleaned up" for use with D, seems to be
a constant source of argument over the five years that D has existed...

--anders

David Barrett wrote:

> After a quick glance, I'd recommend reorganizing Phobos as follows:

You would also want to visit both Dsource and the Wiki4D, first ?

http://www.prowiki.org/wiki4d/wiki.cgi?StandardLibrary
> Every now and then it is said that the Phobos library (the
> current standard library) is preliminary and needs a redesign.

It's probably the place to discuss any Phobos re-engineering,
at least if you want the Bazaar rather than the Cathedral... ;-)
(see http://www.catb.org/~esr/writings/cathedral-bazaar/)


MAJOR D LIBRARIES:

Phobos (http://www.digitalmars.com/d/phobos.html)
------

"Phobos* is the standard runtime library that comes with the D
language compiler." (i.e. DMD, but has also been patched for GDC)
With a few exceptions, it is consisting of the std and std.c trees.


Deimos (http://www.dsource.org/forums/viewforum.php?f=26)
------
While the "std" tree is Phobos, the "etc" tree is called Deimos*.
Right now it mostly hosts etc.bigint and etc.unicode, which are
next to useless due to licensing and replaced by GNU MP and ICU.

Ares (http://dsource.org/forums/viewforum.php?f=31)
----
You should also check out "Ares - the second coming of Phobos":
There is some D code at: http://www.home.f4.ca/sean/d/ares.zip.
This project is about "fixing" the problems with the std tree...


Mango (http://www.dsource.org/projects/mango/)
-----
Ares also "includes" ports of ICU and Log4J, from Kris's Mango Tree -
there are standalone versions of those, without the rest of mango:
mango.icu and mango.log is the module namespace being used there)

Dool (http://dool.sourceforge.net/)
----
Another Phobos replacement project is Ant's Object Oriented Library,
with dool.io, dool.sql and the venerable "dool.String" class too...
(Ant, if you're lurking - can't String be made immutable? Please?)


None of these, except for the Phobos library that is,
are "official D" libraries, or from Digital Mars™...

And until "D 1.0" is finalized/released, it is kinda hard to tell
what went into the standard library and what instead became addons ?

--anders

* PS. Here is the background story of the somewhat odd library naming:

Walter Bright said: (note that the D is still red, just like Mars is)

> I tried calling D the "mars" language initially, but everyone kept calling
> it D. There's just no point in trying to swim up Niagra Falls <g>, so D it
> is. As you said, it's short, sweet, and to the point.
(being archived under http://www.digitalmars.com/d/archives/15721.html)

And Arcane Jill wrote, just before disappearing into the Great Unknown:

> The runtime library which comes with the D compiler is called Phobos,
> named after one of the moons of Mars. Deimos (named after the _other_ of
> Mars's moons) is the unofficial but commonly used name for a body of
> code which is provided by contributors to supplement the modules
> provided by Phobos.
(from http://svn.dsource.org/svn/projects/deimos/trunk/docs/intro.html)

Anders F Björklund wrote:
> Most of the names have history reasons, since D has a strong C legacy.

I thought D was supposed to be fun because it does not have to be compatible with C(++)?

> Whether or not they need to be "cleaned up" for use with D, seems to be
> a constant source of argument over the five years that D has existed...

Geez, you would think people would stop asking for reforms after being told off for five years. :D

> --anders

Benjamin

Benjamin Herr wrote:

>> Most of the names have history reasons, since D has a strong C legacy.
> 
> I thought D was supposed to be fun because it does not have to be compatible with C(++)?

It is (fun). Besides, who said C was boring? C++ on the other hand...

I just meant the D naming follows a C-like manner? Thus, "std.stdio".

> Geez, you would think people would stop asking for reforms after being told off for five years. :D

I wouldn't know, as I've only followed for it like one (i.e. since GDC)

Stopped asking somewhere after the futile "boolean" discussions earlier.

--anders