Andrei Alexandrescu wrote:
> IUnknown wrote:
>> Regarding Phobos + Tango, the minimum I expect is things like
>> containers, algorithm and common math stuff to be in one core module.
> 
> This is already bound to be an issue because there is disagreement on how e.g. containers should look like (Java-style vs. STL-style). Which should be chosen? This naturally influences how algorithms are defined.
> 
> 
> Andrei


The analogy with KDE vs Gnome doesn't seem valid to me -- most libraries will work regardless of which GUI library is chosen. Programmers can still rely on the Posix and C standard libraries.

Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).

By the way, Andrei, this is exactly the kind of attitude which I was ranting about. There are clear areas of duplicated code (fortunately much of it is in druntime now), and even Tango's containers still seem to be in a state of flux (the old container library is deprecated, and very little of Tango is currently using containers).
Everyone -- can we be productive instead of dismissive, please?

Don wrote:
> Andrei Alexandrescu wrote:
>> IUnknown wrote:
>>> Regarding Phobos + Tango, the minimum I expect is things like
>>> containers, algorithm and common math stuff to be in one core module.
>>
>> This is already bound to be an issue because there is disagreement on how e.g. containers should look like (Java-style vs. STL-style). Which should be chosen? This naturally influences how algorithms are defined.
>>
>>
>> Andrei
> 
> 
> The analogy with KDE vs Gnome doesn't seem valid to me -- most libraries will work regardless of which GUI library is chosen. Programmers can still rely on the Posix and C standard libraries.

I agree.

> Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).

That would be great. I don't think that's a major issue anyway. If I were you, to be compatible with today's state of affairs, I'd simply put in the makefile the code necessary for switching the module prefixes.

> By the way, Andrei, this is exactly the kind of attitude which I was ranting about. There are clear areas of duplicated code (fortunately much of it is in druntime now), and even Tango's containers still seem to be in a state of flux (the old container library is deprecated, and very little of Tango is currently using containers).
> Everyone -- can we be productive instead of dismissive, please?

I think you're reading significantly more into what I wrote than I meant to put in.


Andrei

Andrei Alexandrescu wrote:
> Don wrote:
>> Andrei Alexandrescu wrote:
>>> IUnknown wrote:
>>>> Regarding Phobos + Tango, the minimum I expect is things like
>>>> containers, algorithm and common math stuff to be in one core module.
>>>
>>> This is already bound to be an issue because there is disagreement on how e.g. containers should look like (Java-style vs. STL-style). Which should be chosen? This naturally influences how algorithms are defined.
>>>
>>>
>>> Andrei
>>
>>
>> The analogy with KDE vs Gnome doesn't seem valid to me -- most libraries will work regardless of which GUI library is chosen. Programmers can still rely on the Posix and C standard libraries.
> 
> I agree.
> 
>> Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).
> 
> That would be great. I don't think that's a major issue anyway. If I were you, to be compatible with today's state of affairs, I'd simply put in the makefile the code necessary for switching the module prefixes.

It means that any code which uses a library based on both Tango and a library based on Phobos will end up with two copies of all of the functions, and they'll have different name mangling etc. You end up with two incompatible Bigints, for example, even though they have identical code inside.

>> By the way, Andrei, this is exactly the kind of attitude which I was ranting about. There are clear areas of duplicated code (fortunately much of it is in druntime now), and even Tango's containers still seem to be in a state of flux (the old container library is deprecated, and very little of Tango is currently using containers).
>> Everyone -- can we be productive instead of dismissive, please?
> 
> I think you're reading significantly more into what I wrote than I meant to put in.

Good to hear.

> 
> 
> Andrei

IUnknown wrote:
> Agree. Which is why I said the problems you are facing seem to be non-technical. I'm suggesting that the D library developers should pick one and axe the other. *I* think what more important is to have one single set of containers in a single style rather than have two separate ones. There is going to be complaining for sure from the current developers, but in my opinion, the target of having a single standard library (with core and advanced modules to suit system/ app programming) is more important than having to make a difficult choice. 

Totally agree. While I personally prefer the Java-style containers, I'd gladly accept the STL-style containers if it meant unification of Phobos and Tango.

Having druntime is nice, sure, but application-level code and high-level libraries will bake the container API into their public interfaces, and any code that uses both the Phobos and Tango libraries would have to perform a zillion tedious conversions.

In my mind, the things that need a unified API are (in order of importance):

1. GC and TypeInfo
2. Data structures
3. Algorithms
4. String processing
5. Date & Time
6. IO

Everything else (encryption, compression, sockets, regular expressions, could have a totally different API in Tango & Phobos and I wouldn't care much.

Having a common runtime (GC and TypeInfo) is a neat trick, but pretty useless if the data structures and algorithms are entirely different.

And, while I'm perfectly willing to accept either Java-style or STL-style containers, I'd also really appreciate it if the design anticipates and supports custom implementations (because I almost always end up implementing my own multimaps, multisets, circular queues etc)

--benji

On Thu, 22 Jan 2009 09:18:52 +0300, Benji Smith <dlanguage@benjismith.net> wrote:

> IUnknown wrote:
>> Agree. Which is why I said the problems you are facing seem to be non-technical. I'm suggesting that the D library developers should pick one and axe the other. *I* think what more important is to have one single set of containers in a single style rather than have two separate ones. There is going to be complaining for sure from the current developers, but in my opinion, the target of having a single standard library (with core and advanced modules to suit system/ app programming) is more important than having to make a difficult choice.
>
> Totally agree. While I personally prefer the Java-style containers, I'd gladly accept the STL-style containers if it meant unification of Phobos and Tango.
>
> Having druntime is nice, sure, but application-level code and high-level libraries will bake the container API into their public interfaces, and any code that uses both the Phobos and Tango libraries would have to perform a zillion tedious conversions.
>
> In my mind, the things that need a unified API are (in order of importance):
>
> 1. GC and TypeInfo
> 2. Data structures
> 3. Algorithms
> 4. String processing
> 5. Date & Time
> 6. IO
>
> Everything else (encryption, compression, sockets, regular expressions, could have a totally different API in Tango & Phobos and I wouldn't care much.
>
> Having a common runtime (GC and TypeInfo) is a neat trick, but pretty useless if the data structures and algorithms are entirely different.
>
> And, while I'm perfectly willing to accept either Java-style or STL-style containers, I'd also really appreciate it if the design anticipates and supports custom implementations (because I almost always end up implementing my own multimaps, multisets, circular queues etc)
>
> --benji

I think believe we could take advantage of current state of both libraries in D2 - they are both incomplete and being redesigned to fit D2 better.
We could revisit both Tango and Phobos, and clean them up by removing outdated modules and modules with same functionality. This will make Phobos really small and much easier to learn.

On the other hand, Tango will continue providing all the extended functionality.

Here is a list of Phobos modules that I believe could be safely removed:

- crc32 and std.md5 - these should be deprecated in favor of tango.io.digest.Crc32 and tango.io.digest.Md5
Tango is better designed and has support for other algoriths (MD2, MD4, SHA256, SHA512, Tiger and more).
See http://www.dsource.org/projects/tango/wiki/ChapterEncodingAndCrypto for details.

- std.atomics - tango.core.Atomic is superior to it (std.atomics has nothing but CAS anyway).
- std.base64 - deprecate in favor of tango.io.encode.Base64
- std.cover - is it supposed to be visible to user? Should it be in Phobos?
- std.loader - deprecate in favor of tango.sys.SharedLib
- std.bitarray
- std.openrj
- std.process - merge with tango.sys.Process
- std.regexp - buggy, deprecate in favor of tango.text.Regex
- std.socket, std.socketstream - deprecate in favor of tango.net.*
- std.uni - deprecate in favor of tango.text.Unicode
- std.uri - deprecate in favor of tango.net.Uri
- std.xml - deprecate in favor of tango.text.xml.*
- std.zip and std.zlib - deprecate in favor of tango.io.compress.*

In almost *all* cases Tango has cleaner, faster and less buggy implementation of the same functionality.

Other modules - does anyone use any of these:
std.bitmanip
std.bind
std.boxer
std.outbuffer
std.stdint
std.syserror
std.system
?

There are a lot of modules that only exist in Phobos because of some historical reasons or because Walter wrote it. Is it a sufficient reason for them to stay?
They are no doubt useful sometimes, but are we going to put everything that is useful sometimes into Phobos?

I believe it would be better for everyone to keep Phobos simple and prevent the two libraries from competing by separating the functionality.
Once they don't compete anymore, users won't have to worry about what library to use how to stay compatible.

Denis Koroskin pisze:
> On Thu, 22 Jan 2009 09:18:52 +0300, Benji Smith <dlanguage@benjismith.net> wrote:
> 
>> IUnknown wrote:
>>> Agree. Which is why I said the problems you are facing seem to be non-technical. I'm suggesting that the D library developers should pick one and axe the other. *I* think what more important is to have one single set of containers in a single style rather than have two separate ones. There is going to be complaining for sure from the current developers, but in my opinion, the target of having a single standard library (with core and advanced modules to suit system/ app programming) is more important than having to make a difficult choice.
>>
>> Totally agree. While I personally prefer the Java-style containers, I'd gladly accept the STL-style containers if it meant unification of Phobos and Tango.
>>
>> Having druntime is nice, sure, but application-level code and high-level libraries will bake the container API into their public interfaces, and any code that uses both the Phobos and Tango libraries would have to perform a zillion tedious conversions.
>>
>> In my mind, the things that need a unified API are (in order of importance):
>>
>> 1. GC and TypeInfo
>> 2. Data structures
>> 3. Algorithms
>> 4. String processing
>> 5. Date & Time
>> 6. IO
>>
>> Everything else (encryption, compression, sockets, regular expressions, could have a totally different API in Tango & Phobos and I wouldn't care much.
>>
>> Having a common runtime (GC and TypeInfo) is a neat trick, but pretty useless if the data structures and algorithms are entirely different.
>>
>> And, while I'm perfectly willing to accept either Java-style or STL-style containers, I'd also really appreciate it if the design anticipates and supports custom implementations (because I almost always end up implementing my own multimaps, multisets, circular queues etc)
>>
>> --benji
> 
> I think believe we could take advantage of current state of both libraries in D2 - they are both incomplete and being redesigned to fit D2 better.
> We could revisit both Tango and Phobos, and clean them up by removing outdated modules and modules with same functionality. This will make Phobos really small and much easier to learn.
> 
> On the other hand, Tango will continue providing all the extended functionality.
> 
> Here is a list of Phobos modules that I believe could be safely removed:
> 
> - crc32 and std.md5 - these should be deprecated in favor of tango.io.digest.Crc32 and tango.io.digest.Md5
> Tango is better designed and has support for other algoriths (MD2, MD4, SHA256, SHA512, Tiger and more).
> See http://www.dsource.org/projects/tango/wiki/ChapterEncodingAndCrypto for details.
> 
> - std.atomics - tango.core.Atomic is superior to it (std.atomics has nothing but CAS anyway).
> - std.base64 - deprecate in favor of tango.io.encode.Base64
> - std.cover - is it supposed to be visible to user? Should it be in Phobos?
> - std.loader - deprecate in favor of tango.sys.SharedLib
> - std.bitarray
> - std.openrj
> - std.process - merge with tango.sys.Process
> - std.regexp - buggy, deprecate in favor of tango.text.Regex
> - std.socket, std.socketstream - deprecate in favor of tango.net.*
> - std.uni - deprecate in favor of tango.text.Unicode
> - std.uri - deprecate in favor of tango.net.Uri
> - std.xml - deprecate in favor of tango.text.xml.*
> - std.zip and std.zlib - deprecate in favor of tango.io.compress.*
> 
> In almost *all* cases Tango has cleaner, faster and less buggy implementation of the same functionality.
> 
> Other modules - does anyone use any of these:
> std.bitmanip
> std.bind
> std.boxer
> std.outbuffer
> std.stdint
> std.syserror
> std.system
> ?
> 
> There are a lot of modules that only exist in Phobos because of some historical reasons or because Walter wrote it. Is it a sufficient reason for them to stay?
> They are no doubt useful sometimes, but are we going to put everything that is useful sometimes into Phobos?
> 
> I believe it would be better for everyone to keep Phobos simple and prevent the two libraries from competing by separating the functionality.
> Once they don't compete anymore, users won't have to worry about what library to use how to stay compatible.

Also my thoughts. Several months ago I send similar proposition to divide libraries responsibilities:
- for Phobos - low level API
- for Tango - higher level API

Stuff which is in both libraries should be designed together by both teams. Andrei is talking about new design for IO in Phobos. Why not to prepare this new design together with Tango people?

Some differences between philosophies of libraries should not stop merging. E.g. Phobos uses IO synchronized with C, but Tango is not. Why to divide libraries based on this? I think that there are cases where users would want one approach and also cases where they would want the other. IMHO standard library should have API allowing one or other approach depending on what user wants... So this difference is purely rhetorical...

BR
Marcin Kuszczak
(aarti_pl)
www.zapytajmnie.com - my christian site

Don wrote:
> Andrei Alexandrescu wrote:
>> Don wrote:
>>> Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).
>>
>> That would be great. I don't think that's a major issue anyway. If I were you, to be compatible with today's state of affairs, I'd simply put in the makefile the code necessary for switching the module prefixes.
> 
> It means that any code which uses a library based on both Tango and a library based on Phobos will end up with two copies of all of the functions, and they'll have different name mangling etc. You end up with two incompatible Bigints, for example, even though they have identical code inside.

Interestingly, this might be less of a problem when using LDC. LLVM has an IPO pass to merge identical functions (opt -mergefunc).

Don wrote:
> Andrei Alexandrescu wrote:
>> Don wrote:
>>> Andrei Alexandrescu wrote:
>>>> IUnknown wrote:
>>>>> Regarding Phobos + Tango, the minimum I expect is things like
>>>>> containers, algorithm and common math stuff to be in one core module.
>>>>
>>>> This is already bound to be an issue because there is disagreement on how e.g. containers should look like (Java-style vs. STL-style). Which should be chosen? This naturally influences how algorithms are defined.
>>>>
>>>>
>>>> Andrei
>>>
>>>
>>> The analogy with KDE vs Gnome doesn't seem valid to me -- most libraries will work regardless of which GUI library is chosen. Programmers can still rely on the Posix and C standard libraries.
>>
>> I agree.
>>
>>> Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).
>>
>> That would be great. I don't think that's a major issue anyway. If I were you, to be compatible with today's state of affairs, I'd simply put in the makefile the code necessary for switching the module prefixes.
> 
> It means that any code which uses a library based on both Tango and a library based on Phobos will end up with two copies of all of the functions, and they'll have different name mangling etc. You end up with two incompatible Bigints, for example, even though they have identical code inside.

Oh, I see. You want your library to be usable whether the end user prefers Phobos or Tango. But then why not stick it into a namespace of your choosing? Let's say your libraries are general enough to warrant putting them in a common core, but then anyone who defines some library don't have to go to the "core ombudsman" to add it to the common namespace. They'd just create unique namespaces of their own. No?


Andrei

Andrei Alexandrescu wrote:
> Don wrote:
>> [snip]
>>
>> It means that any code which uses a library based on both Tango and a library based on Phobos will end up with two copies of all of the functions, and they'll have different name mangling etc. You end up with two incompatible Bigints, for example, even though they have identical code inside.
> 
> Oh, I see. You want your library to be usable whether the end user prefers Phobos or Tango. But then why not stick it into a namespace of your choosing? Let's say your libraries are general enough to warrant putting them in a common core, but then anyone who defines some library don't have to go to the "core ombudsman" to add it to the common namespace. They'd just create unique namespaces of their own. No?
> 
> 
> Andrei

I think he means this: let's say you're writing app A.  A depends on libraries B and C.  B depends on Phobos, and C depends on Tango.  Both B and C happen to use BigInts or IO or anything else that isn't shared.

All of a sudden, you've now got to link in TWO standard libraries instead of just one, each with potentially duplicated code.

From personal experience, the alternative isn't much better: writing code that switches between the two.

I have an XML library that can use either Phobos or Tango.  It does this by implementing all the calls it needs in a shim library, essentially using it's own standard library.

It gets really fun when you need to implement some call that's trivial in one library, but really hard in the other.  I remember having to build an IO layer so that both Phobos and Tango had the same semantics regarding EOF or something...

Urgh.

  -- Daniel

Andrei Alexandrescu wrote:
> Don wrote:
>> Andrei Alexandrescu wrote:
>>> Don wrote:
>>>> Andrei Alexandrescu wrote:
>>>>> IUnknown wrote:
>>>>>> Regarding Phobos + Tango, the minimum I expect is things like
>>>>>> containers, algorithm and common math stuff to be in one core module.
>>>>>
>>>>> This is already bound to be an issue because there is disagreement on how e.g. containers should look like (Java-style vs. STL-style). Which should be chosen? This naturally influences how algorithms are defined.
>>>>>
>>>>>
>>>>> Andrei
>>>>
>>>>
>>>> The analogy with KDE vs Gnome doesn't seem valid to me -- most libraries will work regardless of which GUI library is chosen. Programmers can still rely on the Posix and C standard libraries.
>>>
>>> I agree.
>>>
>>>> Can we work out the math stuff at least? There's no difference between Phobos and Tango there. All we need is an agreement on common module naming (eg, create core.math).
>>>
>>> That would be great. I don't think that's a major issue anyway. If I were you, to be compatible with today's state of affairs, I'd simply put in the makefile the code necessary for switching the module prefixes.
>>
>> It means that any code which uses a library based on both Tango and a library based on Phobos will end up with two copies of all of the functions, and they'll have different name mangling etc. You end up with two incompatible Bigints, for example, even though they have identical code inside.
> 
> Oh, I see. You want your library to be usable whether the end user prefers Phobos or Tango. But then why not stick it into a namespace of your choosing? Let's say your libraries are general enough to warrant putting them in a common core, but then anyone who defines some library don't have to go to the "core ombudsman" to add it to the common namespace. They'd just create unique namespaces of their own. No?

That's possible too. Originally, my code was in the 'mathextra' namespace, and I'm somewhat regretting moving it it out.

You still need an ombudsman, though, to determine which libraries are standard, and which are not, in order to prevent namespace collisions. Personally, I like the boost model -- all you need is a root level (like 'boost') to act as a namespace protection mechanism, and then a library standardisation policy.

But then you have the question as to the organisation of the 'std' namespace -- for example, it contains such obscure stuff as std.openrj which is actually a stand-alone library.

(I notice Dennis has just posted some similar sentiments, so I'll stop for now).

It could be that all that we need is to create a standard naming policy.