Walter Bright wrote:
> 
> Those are good parallels, and it's good you brought that up. Working C++ legacy code is not going away and is not going to be translated to D. What will happen is *new* code will be done in D.

Walter,
I guess you have good reasons to think in this way, but my experience is different.

I've been in two different graduate schools and at two different departments (Civil Engineering and Earth Sciences). I think the situation in those departments in different to what people observe in a CS department. In both places I met people that buy expensive Fortran compilers not only to compile old libraries, but to write new code. I know people that write large parallel codes in Fortran and that use a C library (PETSc) for the core numerical routines. Most of them are young researchers, so they learned Fortran few years ago. So, I don't think Fortran is still around because of old legacy code or "old legacy programmers".

I think most of that people use Fortran because there wasn't anything better for that kind of code. I think/hope D can be a viable alternative.

Paulo.

Walter Bright wrote:
> Don Clugston wrote:
> 
>>> About native libraries
>>> ----------------------
>>> I think the way to go is to create QUALITY native D libraries. I think we usually overestimate the cost of porting/developing native libraries.
>>
>> I agree. When porting a library
>> (a) there isn't any algorithm development (not much 'thinking' time required);
>> (b) it's very easy to test (you can run test cases against the original version);
>> (c) the D version is frequently greatly superior to the original.
> 
> Hmm. Is there a 'canonical' Fortran numerics library with a solid test suite that could be translated to D as a showcase project?

Do you have some idea in mind?

For my work I need a good iterative (or fast sparse) linear solver. I used the Fortran SPARSKIT library in the past (http://www-users.cs.umn.edu/~saad/software/SPARSKIT/sparskit.html), but  I found much more comprehensive libraries in other languages.

For example, the MTJ Java library (http://rs.cipr.uib.no/mtj/. That is a very complete linear algebra library with dense and sparse matrix and vectors, and direct and iterative solvers. The library is very well designed and organized. Unfortunately, the performance of the pure Java version is not so good (see http://rs.cipr.uib.no/mtj/benchmark.html). MTJ can use native CBLAS and LAPACK, but that requires to "convince" Java to use those libraries (another example of problems with the use of different languages).

If we can make a port of that library to D and show that it performs close to  SPARSKIT, that would be a good demonstration of D capabilities for numerical computing.

Paulo.

In article <e4cvf6$pfi$1@digitaldaemon.com>, dennis luehring says...
>
>> // For simple template instantiation.  Creates a wrapper type in the cpp file
>> for this type.
>> cpptype VectorInt { std::vector<int> } MyVectorInt;
>
>have you any idea of how templates are generated and "instanciated" what should the compiler do at this step? (please the describe the low-level part a little bit more detailed)

What are you talking about?  I have no intention of intantiating any templates at all.  Templated classes are defered to the external cpp wrapper file:

To make it a bit clearer.. templates are instantiated in this way in the wrapper Cpp code file.

class __VectorInt : public std::vector<int> {};

D uses __VectorInt, which was the point of providing the cpptype {} keyword in the first place.  The code within the braces is not parsed by D, but placed in an external cpp file which is compiled by a Cpp compiler into a standard Cpp class that D CAN use.

>> // For instantiating constants
>> cppconst int NULL { NULL };

This is to permit the assiging of Cpp macros to D constants.  If the constants are simple, the value of the constant is directly assigned, if not, the constant expression is converted into an "extern int" reference, and the constant variable is placed in the Cpp file to be compiled by a Cpp compiler.

// Cpp file definition for a complex macro constant
int __COMPLEX_MACRO_CONSTANT = COMPLEX_MACRO_CONSTANT;

Likewise, a complex functional macro can be placed in the Cpp file and wrapped with a standard c function:

// In D if you the following...
int i = cppmacro(int,int) MIN(100, 200);

// Would cause a call to this function defined in the generated Cpp file below
extern "C" {
int __MIN(int a, int b)
{
return MIN(a,b);
}
}

>> // Creating a cpp class
>> MyClass c = new MyClass("A class\0");
>> // Calling a method on a cpp class
>> c.Method("String Argument\0");
>
>calling methods and instanciat classes through strings?
>this is joke, or? (use the COM interface)

I don't know.. is this a joke?  Let me ask you, do you find yourself personally offended by these ideas?

>> // Handling an exception
>> cpp {
>> try {
>> c->ThrowException(%%str);
>> }
>> catch (std::string ex)
>> {
>> %%ret = ex;
>> }
>> }
>
>is this your idea of an clean integration? (what does clean mean in your world)

Compared to what?  A multi-year project to convert the C Windows API to D so that Windows software can finally be written?

I would suggest if you don't like it, then don't feel that you have to use it.

>ciao dennis
>
>btw: how many years do you code c/c++?

21 years.  I've actually used to own a copy of Zortech.

How about you?

"Ben Cooley" <Ben_member@pathlink.com> wrote in message news:e4dprq$4ga$1@digitaldaemon.com...
> Compared to what?  A multi-year project to convert the C Windows API to D
> so
> that Windows software can finally be written?

Well, that's a pretty poor comparison, but the reason the Windows API (that is, just the header files) have taken so long is dues to copyright issues. We're not technically allowed to programmatically convert the official Windows headers over; that's been done, and it was done _ages_ ago.  But it's not legal.  So, instead, it has to be done the way it's being done, and as such, it's taken a while.  Or rather, it hasn't taken that long, considering it started just a month or two ago.

People have been writing Windows programs in D for a long time now.  I mean, there's even a section about it in the spec.  It's not like we've been diddling around on the console all this time.

Paulo Herrera wrote:
> Walter Bright wrote:
>> Don Clugston wrote:
>>
>>>> About native libraries
>>>> ----------------------
>>>> I think the way to go is to create QUALITY native D libraries. I think we usually overestimate the cost of porting/developing native libraries.
>>>
>>> I agree. When porting a library
>>> (a) there isn't any algorithm development (not much 'thinking' time required);
>>> (b) it's very easy to test (you can run test cases against the original version);
>>> (c) the D version is frequently greatly superior to the original.
>>
>> Hmm. Is there a 'canonical' Fortran numerics library with a solid test suite that could be translated to D as a showcase project?
> 
> Do you have some idea in mind?

I am not familiar with the various Fortran libraries out there, so I have no specific one in mind.


> If we can make a port of that library to D and show that it performs close to  SPARSKIT, that would be a good demonstration of D capabilities for numerical computing.

Want to get started on it? <g>

In article <e4cubh$nou$1@digitaldaemon.com>, Ben Cooley says...
>
>In article <e4cgup$2te2$1@digitaldaemon.com>, Bill Baxter says...
>>
>>In article <e4ca0r$2hvj$1@digitaldaemon.com>, Paulo Herrera says...
>>>
>>>Bill Baxter wrote:
>>>> 
>>>> Finally about whether it's worth it to be able to call C++ code, I'd like to point to the example of Fortran.  It was superceded by better languages with better syntax many many years ago, but people still use it.  I hear a lot of people saying "we're better off without all that old junky C++ code anyway". But the fact of the matter is that nobody in the real world has the time or budget to go back and rewrite all their old code (that already works and has been debugged) just because it would be easier to read in a new language.  Most numerical work is still based on lots of legacy fortran code, even if the front end code is now written in C or C++ or Python or what have you.   It just makes no sense to rewrite gobs and gobs of stuff that works.
>>>> 
>>>> Gotta go so there's no time to finish my point, but there are some interesting parallels in this debate with the migration away from Fortran, I think.
>>>> 
>>>> Bill Baxter
>>>Hi everybody,
>>>This is my first post to this mailing list.
>>
>>Howdy.
>>
>>>
>>>About the Fortran example
>>>-------------------------
>>>I think the Fortran example is a bad one. Which better languages were
>>>available long time ago? C or C++? Do you think C or C++ have a clean
>>>syntax? I don't think so, that's why I've been exploring D.
>>>There are a lot of C (e.g. GSL, PETSc) and C++ (e.g. ITL, Blitz++)
>>>numerical libraries, so I don't think that is the main reason people
>>>still use Fortran.
>>
>>
>>My point was that while some people still use Fortran (and others have moved on), LOTS of people still use old Fortran *libraries*.  I.e. C/C++ are to Fortran as D is to C/C++.  And just as with Fortran, there's just no value proposition in rewriting gobs of well-known, well-tested, well-debugged code, like that available at netlib.org.  Dreaming that everything of value that has ever been written in other languages (including Fortran, C, and C++) will get rewritten in D is just that, dreaming.  Heck, from what I hear there are still plenty of mainframes out there, cranking away on Cobol code.  Because it just doesn't make sense to spend a lot of resources fixing what ain't broke.
>>
>>Writing new libs from scratch is certainly the best in the long run.  But that takes time, and it takes a lot of raw man hours.  In the short term, it seems obvous to me that the best bet is to increase the number man-hours by increasing the number of 'mans' -- attracting as many folks as possible to D.  But starting off by saying "sorry you can't use your old code" is just going to put people like Ben off from the start, thereby shrinking the pool of potential contributors.   I don't think D needs compatibility with C++ -- I'm inclined to think it's pretty hopeless even with a working C++ compiler as a starting point -- but some tools to create wrappers for existing C++ code (a la SWIG), should be doable, and would be a huge help in luring people to D.
>>
>>I would venture to guess that the vast majority of C++ code that people write in real life is not at all like the STL, but more like the subset of C++ that's contained in  Java.  Those are the things you want to target (most bang for the buck), and, no co-incidence, those are precisely the things that SWIG targets. Libraries like Blitz++ and ITL that go crazy with C++ templates are so steeped in C++-specific issues that I doubt they could be wrapped reasonably anyway, whether done by hand or by a wrapper-generator.
>>
>>Apparently this idea to use SWIG has been discussed (and even worked on?) a bit
>>before:
>
>I agree. And I'm not interested anymore in arguing about "why" this should be done.  If it isn't made obvious to anyone by the post about "finally having the Windows API translated to D".. then it never will be.  <rolls eyes>
>
>I think the practical way to do this is to allow D to comprehend basic C++ vtable types, access C++ members and methods and be able to instantiate macro constants.  Other C++ interop would be handled using inline cpp { } blocks which could access D vars.

Yes, we're in agreement insofar as we both think it would be good if it were easier to use legacy C and C++ code with new D code.  I think the majority of readers of this list would agree there's nothing wrong with that.  They might prioritize it differently, but I think most everyone here can agree that if a magic wand could make C++ callable from D then it should be done.  But seeing Walter's Giant List of all the interoperability issues pretty much clenches it for me that making D work _directly_ with C/C++ is just a non-starter.  Not unless Google "buys" D tomorrow and throws 20 full time engineers on it.  On the other hand, SWIG, or SWIG-like, wrapping of C/C++ for consumption by D is a much more feasible, realistic goal.  SWIG makes the easy stuff easy (automatic, just %include "header.h") and the hard stuff doable (with manual annotations), but it doesn't magically make the impossible possible (like porting the STL to Python).

Forgive me if I'm misunderstanding you, but your talk of accessing vtables and such from D sounds like you're still thinking of making D understand C++ binaries.  I just don't think that's practical given the current D development situation.  But I think it would only take one or two determined souls to revive the SWIG approach and wrangle it into something useful.

Speaking of Google, and projects for one or two determined souls, were there any D proposals submitted to the Google Summer of Code?

Regards,

Bill Baxter

In article <e4cv4j$nqu$6@digitaldaemon.com>, Walter Bright says...
>
>Don Clugston wrote:
>
>>> About native libraries
>>> ----------------------
>>> I think the way to go is to create QUALITY native D libraries. I think we usually overestimate the cost of porting/developing native libraries.
>> 
>> I agree. When porting a library
>> (a) there isn't any algorithm development (not much 'thinking' time
>> required);
>> (b) it's very easy to test (you can run test cases against the original
>> version);
>> (c) the D version is frequently greatly superior to the original.
>
>Hmm. Is there a 'canonical' Fortran numerics library with a solid test suite that could be translated to D as a showcase project?


BLAS?  (Basic Linear Algebra Something-er-other) It's a set of very low-level routines for doing things like dot products and vector multiply-adds.

LAPACK? Basic linear algebra functions like LU decompostion and the like.

Lapack (and lots of other numerical libraries) rely on BLAS for low level array operations.  The idea is that BLAS libraries can be highly tuned for particular architectures by particular vendors, taking into account things like cache sizes or special vector instructions.  ATLAS is the canonical example of such a tuned BLAS library.  Given an optimized BLAS library then your LAPACK or whatever should also run pretty fast without having to specifically tune it for each architecture individually.

Not sure what porting either of those would showcase about D, though.  Their interfaces are very basic,  consisting of just a set of cryptically named functions to fit within ridiculous character limit of the original fortran compilers.  It's about as far away as you can get from a modern, easy-to-use API.

Porting (or reimplementing the concepts of) something like Blitz++ may be a better showcase for D.   But I wouldn't know.  I've never used Blitz++ despite the hype.  It simply wouldn't compile with my compiler back when I was interested in trying it out.

At any rate, any work on porting/writing a big numerical library should probably hold off until this new array stuff gets added to D.

Regards,

Bill Baxter

In article <e4dq4u$4ub$1@digitaldaemon.com>, Jarrett Billingsley says...
>
>"Ben Cooley" <Ben_member@pathlink.com> wrote in message news:e4dprq$4ga$1@digitaldaemon.com...
>> Compared to what?  A multi-year project to convert the C Windows API to D
>> so
>> that Windows software can finally be written?
>
>Well, that's a pretty poor comparison, but the reason the Windows API (that is, just the header files) have taken so long is dues to copyright issues. We're not technically allowed to programmatically convert the official Windows headers over; that's been done, and it was done _ages_ ago.  But it's not legal.  So, instead, it has to be done the way it's being done, and as such, it's taken a while.  Or rather, it hasn't taken that long, considering it started just a month or two ago.
>
>People have been writing Windows programs in D for a long time now.  I mean, there's even a section about it in the spec.  It's not like we've been diddling around on the console all this time.

Right.  My mistake then.

I knew that you could link with Windows libs, but I didn't know that you could actually code to a reasonably current version of the Windows API.

In article <e4dhnf$1l5n$1@digitaldaemon.com>, Paulo Herrera says...
>
>Walter Bright wrote:
>> 
>> Those are good parallels, and it's good you brought that up. Working C++ legacy code is not going away and is not going to be translated to D. What will happen is *new* code will be done in D.
>
>Walter,
>I guess you have good reasons to think in this way, but my experience is
>different.
>
>I've been in two different graduate schools and at two different departments (Civil Engineering and Earth Sciences). I think the situation in those departments in different to what people observe in a CS department. In both places I met people that buy expensive Fortran compilers not only to compile old libraries, but to write new code. I know people that write large parallel codes in Fortran and that use a C library (PETSc) for the core numerical routines. Most of them are young researchers, so they learned Fortran few years ago. So, I don't think Fortran is still around because of old legacy code or "old legacy programmers".
>
>I think most of that people use Fortran because there wasn't anything better for that kind of code. I think/hope D can be a viable alternative.

Yeh, my impression is that Fortran users care most about
A) the ability to easily use the gobs of existing Fortran code thats out there
and
B) having basic operations like C[i] = A[i] + B[i] run as fast as possible for
their given architecture.
I seem to recall Fortran also has some other syntactic sugar that makes various
array operations easier than with C.  Not sure how important that is.

For A) nothing beats the ease of calling legacy fortran code than fortran itself.  It's hard to beat there.

For B) C's lack of alias-free arrays means C compilers can't optimize as aggressively as Fortran ones.  C99 has the restrict keyword, but here it is seven years after the C99 standard, and the big compiler vendors still don't seem interested in implementing it.  And besides if you're using Fortran, you're doing numerics.  Fast numerical code is pretty much the only benchmark that matters in that world, so if you want to sell a fortran compiler, it better do a great job on numerical code.  C compiler writers sell to a much wider variety of folks, so the need to optimize numerics in order to make a profit is much less.

None of the above has anything to do with whether Fortran is a great language or not.  Its good enough to get their job done, and the costs of moving to a different language outweigh the benefits for some folks.

And even if young researchers are using it, I suspect that that also has a lot more do with what their professors were proficient with than the merits of the language -- i.e. it *is* a legacy/inertia issue.  I took a few classes in the Applied Math dept where the prof would provide a skeleton of Fortran code or some support code in fortran and you were supposed to fill in the rest.  He said we were free to use C or C++, but then we'd have to scrap together our own bootstrap code.  I went with C++ but every other student in the class went with Fortran, because it was the default path, and because the prof was more likely to be able to provide help with the Fortran.

Er, so summing up, I'm agreeing with you Paolo, that some people aren't going to give up on Fortran just because a better language comes along.  But mostly because of how easy it is to call Fortran from Fortran, and because that's what they teach to young numerical researchers.  But that's not everyone.  Some numerics people do see the light and long for better more scalable tools and aren't afraid to spend time learning something new.  I know at least one dyed-in-the-wool fortran numerics guy who's writing a massive new library in C++ because he wants the modularity maintainability provided by OO abstractions. But currently he has to put up with people saying his code is going to be slower than the equivalent Fortran code.  If D could match Fortran for raw numeric speed then at least that one argument would go away.  So I agree with Walter that if D can be as good at numerics as Fortran then some, but certainly not all, new code would be written in D.  This guy's library is a good example.  Big new C++ code base with virtually no external dependencies.  Probably would have been perfect for D if he had known about it -- and if D had been 1.0 when he started -- and if there had been sufficient resources (like books) out there to learn how to program in D.

Bill Baxter

>Yes, we're in agreement insofar as we both think it would be good if it were easier to use legacy C and C++ code with new D code.  I think the majority of readers of this list would agree there's nothing wrong with that.  They might prioritize it differently, but I think most everyone here can agree that if a magic wand could make C++ callable from D then it should be done.  But seeing Walter's Giant List of all the interoperability issues pretty much clenches it for me that making D work _directly_ with C/C++ is just a non-starter.  Not unless Google "buys" D tomorrow and throws 20 full time engineers on it.  On the other hand, SWIG, or SWIG-like, wrapping of C/C++ for consumption by D is a much more feasible, realistic goal.  SWIG makes the easy stuff easy (automatic, just %include "header.h") and the hard stuff doable (with manual annotations), but it doesn't magically make the impossible possible (like porting the STL to Python).
>
>Forgive me if I'm misunderstanding you, but your talk of accessing vtables and such from D sounds like you're still thinking of making D understand C++ binaries.  I just don't think that's practical given the current D development situation.  But I think it would only take one or two determined souls to revive the SWIG approach and wrangle it into something useful.

The majority of issues Walter listed simply don't apply to the strategy I'm going to try.  There are no issues with parsing because that task is handled using an external tool (GCC-XML).  There are no issues with specific C++ language details because any construct outside of cpp vtable calls or member access are simply punted to wrappers in the auto generated Cpp file.

As for swig, the parsing of headers in SWIG is not really adequate IMO.  It takes shortcuts which tend to break.  In any case, since GCC-XML exists, and it was actually written specifically to produce the precise sort of reflection info you need here, why bother with an imprecise parser like swig?  They're using GCC-XML for the current C++ reflection library at cern and I've personally parsed some of our own code here, and it works fine.  Of course Elsa also works on our code here after running it through the pre-processor.. which is quite remarkable given the extensive amount of template programming we use, but it doesn't parse some of the MS specific stuff which GCC-XML handles.

In any case, I really think you can do better than wrapping. If you can get an accurate vtable layout, macros, method list and params, and class/struct layout for standard classes, you pretty much have the information that you need to access members and call vtable methods.

For the rest: templates, complex macros, exception handling, rtti, etc.. yes, I'm saying it would use inline cpp {} and generate "wrapper code" for these the same way swig does.. and call this wrapper code via extern C function calls to an externally compiled and linked Cpp file.