April 01, 2007
kris wrote:
> 
> FWIW: if some of those "Language Shootout" tests are IO-bound, perhaps tango.io might help? Can't imagine they'd apply that as a "language" test, but stranger things have happened before.
> 

FWIW: I think it's worth a lot <g> If D could come out convincingly on top for common I/O bound stuff, and do so with a "more modern" API than a C stdio clone, I would think that would be worth something to quite a few people.

IIRC, making the I/O as fast as C was time-consuming for quite a few of the current programs -- more so than even the GC. Even for the programs where I/O is not a "bottleneck" (but where I/O was involved), the faster implementation would still cut the overall time a bit anyway.

Alas, since Tango is not part of the std. distribution, I think they'd be very reluctant to add it - not only because of the extra workload for them, but simply because it's not considered std. for D. OTOH, they did add gmp4d for the pidigits test, but I think that was because it was easy to add via CVS, use through an altered command-line and there wasn't an alternative. If the same could be done for Tango, hmmm, perhaps...

- Dave
April 01, 2007
Andrei Alexandrescu (See Website For Email) wrote:
> Bill Baxter wrote:
>> kris wrote:
>>> Andrei Alexandrescu (See Website For Email) wrote:
>>
>>> - Tango is for D programmers; not C programmers.
>>
>> D programmers sometimes like to call 3rd party code written in other languages, and pretty much any interop in D has to happen via C compatibility.  E.g. pyD.  So I'm guessing if my D code calls on some Python code that prints to the console that somewhere down the line that eventually ends up on C's stdout.  I could be wrong, but at least

Well, it *all* ends up using file descriptor 0 on POSIX systems.

>> that's why I *think* Andrei and Walter keep saying that C compatibility is important.
>
> Yes.

Please help me to understand this issue... C's stdio predefined set of file handles or streams are simply wrappers around "pipes" into and out of the console, which in itself is just a part of another separate program (the shell). For the _great majority_ of cases where a D program interacts with another program though a shell (which it will even if spawned via the POSIX exec() family for example), as long as Tango can read from and write -- and eventually flush if/as the underlying shell requires -- to the console, why would a D program really need to read and write to the exact same handles? The same *file descriptors as the shell, yes,* but file handles?

Basically what I'm saying is this: Let the shell handle this stuff as it sees fit; that's what it is there for!

For more complex high-throughput (http://cr.yp.to/qmail.html) or multi-threaded C programs that could use the stdio API, you almost have to use a closer to the metal API (like unistd read()/write()) anyway. Somewhere in the docs for Qmail, security and performance issues were the main reason why it was written with a complete replacement for stdio. IIRC, that developer really rails on C's stdio as outdated, and as something that drastically needs to be replaced. BTW, I myself have written programs that interact very well (and easily) with Qmail though C's stdio even though the only common bond is the POSIX file descriptors.

And for the minority interop case, as long as Tango I/O offers the flexibility to follow the same conventions as the client or server process regardless of shell conventions, I would think that would be good enough, as this code often needs to be "hand-crafted" -- at least in my experience --, or use the shell native API anyway (i.e.: read()/write()). I don't know much about Tango, but it actually looks like it offers much broader functionality than phobos in this regard (for example, the http classes).

Can I get a concrete and preferably common example of where the current Tango implementation fails for stdin/stdout/stderr in this regard?

Tango does not fail with the benchmark/example you provided... Similar functionality (at its base) as your program is by far the most common use case for reading and writing to the console.

>
>> Andrei -- by "compatibility" does that mean if I rebind stdio/stdout to something different that both D and C's output go to the new place?  Or is it still necessary to rebind them individually?  I did this once for some legacy code in C++, and found that I had to rebind 3 things: the C streams, the C++ old-style streams from <iostream.h> (was under MSVC 6), and the new-style C++ streams from <iostream>.    And then I had to do the interleaving myself, which didn't really work (because all the streams were just writing to output buffers individually).  If what you're talking about with compatibility would avoid that kind mess, that is certainly be a good thing.
>
> Exactly. I'm hoping to save the D community from the iostreams mess, which requires the programmer to deal with two (or three as in your example) incompatible APIs. It's iostreams' amazing counterperformance that they managed to cut the speed so effectively in the process :o).
>
> I think a wise thing that Tango could do is to offer a backward-compatible stdio, and a high-performance I/O class using the latest and greatest from each I/O (zero-copy (see http://research.sun.com/techrep/1995/abstract-39.html), asynchronous, threaded, kernelized, teleported, you name it).
>

I really don't think it needs to be that complicated, especially since Tango apparently performs very well anyhow for the most common cases.

> The main desideratum is that fast/incompatible I/O is opt-in, not lock-in.
>

If phobos and Tango could be bundled together on equal footing with the D distributions, phobos could fill the 'stdio compatible' role.

- Where that is needed (the minority of cases, almost always developed by experienced programmers), D can easily do that. Tango and/or phobos could even provide a separate API for that, but not the default.
- The "C way" is not the "D way". D is supposed to be an improvement over C w/o all of the C++ corner cases. This paradigm should extend to the std. lib. as well.
- A lot of very experienced and smart people have contributed to Tango. I'd venture a guess that as a group they have all used C's stdio extensively and have also done various forms of interop in the past. They apparently don't have a problem with the current implementation, so I'd be very reluctant to question them on it w/o some very concrete examples of the most common use cases that fail with Tango.
- Speaking only for myself, at one time I was reluctant to accept anything like Tango simply because it seemed to be a split from Walter's direction, and because phobos was kind-of 'C like' so I was more comfortable with it. However, here is the Tango team willing and able to contribute, and more than happy (for the most part) to leave the compiler development and overall direction of the language to Walter. I think Walter's (and perhaps your?) time is best spent on that, and let the Tango team wrestle with the std. lib.

I'd like to suggest this: Distribute both phobos and Tango on equal footing with the D distributions, and have the compiler link both in by default (with a switch to link only one):

- Walter and David Friedman could automate the build process to grab the latest Tango.
- Some overlapping areas (like the GC) would have to be resolved somehow (Tango team - how tough would this be?).
- Add the Tango docs. to the DigitalMars site on "equal footing" as well.
- C++ distributions provide somewhat of a template and precedence for distributing more than one library with the compiler.

Thanks,

- Dave

>
> Andrei
April 02, 2007
Dave wrote:
> 
> I'd like to suggest this: Distribute both phobos and Tango on equal footing with the D distributions, and have the compiler link both in by default (with a switch to link only one):
> 
> - Walter and David Friedman could automate the build process to grab the latest Tango.
> - Some overlapping areas (like the GC) would have to be resolved somehow (Tango team - how tough would this be?).

The compiler runtime is a logically separate entity in Tango.  Some cooperation would be necessary for the development of new back-end functionality, etc, but the compiler runtime could theoretically be maintained entirely separate from the rest of the library if Walter and/or David chose to do so.  That said, updating the Tango runtime for new compiler releases is generally fairly straightforward, so a shift in responsibility would probably just change the update process rather than simplify it.


Sean
1 2 3 4 5 6 7 8 9 10 11
Next ›   Last »