"Ben Cooley" <Ben_member@pathlink.com> wrote in message news:e4dv0t$e18$1@digitaldaemon.com...

> Right.  My mistake then.
>
> I knew that you could link with Windows libs, but I didn't know that you
> could
> actually code to a reasonably current version of the Windows API.

And hey, if none of the existing Windows headers have a really new API function, you can always look it up on MSDN, and put in the external declaration yourself, like

extern(Windows) void SomeNewAPIFunc(char* lpszoifoiweoriworijName);

(I hate Hungarian notation)

Or, if the linker won't accept that, you can always.. _persuade_ it.

In article <e4e0v2$i78$1@digitaldaemon.com>, Ben Cooley says...
>
>The majority of issues Walter listed simply don't apply to the strategy I'm going to try.

That's truly excellent if you're really going to give this a shot.  I wish you luck.

> There are no issues with parsing because that task is handled
>using an external tool (GCC-XML).  There are no issues with specific C++ language details because any construct outside of cpp vtable calls or member access are simply punted to wrappers in the auto generated Cpp file.
>
>As for swig, the parsing of headers in SWIG is not really adequate IMO.  It takes shortcuts which tend to break.  In any case, since GCC-XML exists, and it was actually written specifically to produce the precise sort of reflection info you need here, why bother with an imprecise parser like swig?  They're using GCC-XML for the current C++ reflection library at cern and I've personally parsed some of our own code here, and it works fine.  Of course Elsa also works on our code here after running it through the pre-processor.. which is quite remarkable given the extensive amount of template programming we use, but it doesn't parse some of the MS specific stuff which GCC-XML handles.

Fair enough.  That sounds like an argument against using SWIG for any purpose, not just against using it to wrap D.  Maybe you'll create the startings of XNSWIG  ("XNSWIG's Not SWIG"), a new SWIG replacement based on GCC-XML. :-)

>In any case, I really think you can do better than wrapping. If you can get an accurate vtable layout, macros, method list and params, and class/struct layout for standard classes, you pretty much have the information that you need to access members and call vtable methods.
>
>For the rest: templates, complex macros, exception handling, rtti, etc.. yes, I'm saying it would use inline cpp {} and generate "wrapper code" for these the same way swig does.. and call this wrapper code via extern C function calls to an externally compiled and linked Cpp file.

If you really think you can make it happen then by all means jump to it.  If it works, many folks will sing your praises.

Regards,

Bill Baxter

Bill Baxter wrote:
> In article <e4dhnf$1l5n$1@digitaldaemon.com>, Paulo Herrera says...
>> Walter Bright wrote:
>>> Those are good parallels, and it's good you brought that up. Working C++ legacy code is not going away and is not going to be translated to D. What will happen is *new* code will be done in D.
>> Walter,
>> I guess you have good reasons to think in this way, but my experience is different.
>>
>> I've been in two different graduate schools and at two different departments (Civil Engineering and Earth Sciences). I think the situation in those departments in different to what people observe in a CS department. In both places I met people that buy expensive Fortran compilers not only to compile old libraries, but to write new code. I know people that write large parallel codes in Fortran and that use a C library (PETSc) for the core numerical routines. Most of them are young researchers, so they learned Fortran few years ago. So, I don't think Fortran is still around because of old legacy code or "old legacy programmers".
>>
>> I think most of that people use Fortran because there wasn't anything better for that kind of code. I think/hope D can be a viable alternative.
> 
> Yeh, my impression is that Fortran users care most about A) the ability to easily use the gobs of existing Fortran code thats out there
> and B) having basic operations like C[i] = A[i] + B[i] run as fast as possible for
> their given architecture.  I seem to recall Fortran also has some other syntactic sugar that makes various
> array operations easier than with C.  Not sure how important that is.
> 
> For A) nothing beats the ease of calling legacy fortran code than fortran
> itself.  It's hard to beat there.  
> 
> For B) C's lack of alias-free arrays means C compilers can't optimize as
> aggressively as Fortran ones.  C99 has the restrict keyword, but here it is
> seven years after the C99 standard, and the big compiler vendors still don't
> seem interested in implementing it.  And besides if you're using Fortran, you're
> doing numerics.  Fast numerical code is pretty much the only benchmark that
> matters in that world, so if you want to sell a fortran compiler, it better do a
> great job on numerical code.  C compiler writers sell to a much wider variety of
> folks, so the need to optimize numerics in order to make a profit is much less. 
> 
> None of the above has anything to do with whether Fortran is a great language or
> not.  Its good enough to get their job done, and the costs of moving to a
> different language outweigh the benefits for some folks.
> 
> And even if young researchers are using it, I suspect that that also has a lot
> more do with what their professors were proficient with than the merits of the
> language -- i.e. it *is* a legacy/inertia issue.  I took a few classes in the
> Applied Math dept where the prof would provide a skeleton of Fortran code or
> some support code in fortran and you were supposed to fill in the rest.  He said
> we were free to use C or C++, but then we'd have to scrap together our own
> bootstrap code.  I went with C++ but every other student in the class went with
> Fortran, because it was the default path, and because the prof was more likely
> to be able to provide help with the Fortran.
> 
> Er, so summing up, I'm agreeing with you Paolo, that some people aren't going to
> give up on Fortran just because a better language comes along.  But mostly
> because of how easy it is to call Fortran from Fortran, and because that's what
> they teach to young numerical researchers.  But that's not everyone.  Some
> numerics people do see the light and long for better more scalable tools and
> aren't afraid to spend time learning something new.  I know at least one
> dyed-in-the-wool fortran numerics guy who's writing a massive new library in C++
> because he wants the modularity maintainability provided by OO abstractions.
> But currently he has to put up with people saying his code is going to be slower
> than the equivalent Fortran code.  If D could match Fortran for raw numeric
> speed then at least that one argument would go away.  So I agree with Walter
> that if D can be as good at numerics as Fortran then some, but certainly not
> all, new code would be written in D.  This guy's library is a good example.  Big
> new C++ code base with virtually no external dependencies.  Probably would have
> been perfect for D if he had known about it -- and if D had been 1.0 when he
> started -- and if there had been sufficient resources (like books) out there to
> learn how to program in D.
> 
> Bill Baxter


Just curious: how does Fortran scale for use of parallel computations? Perhaps some Fortran compilers are more advanced in that field than C/C++?  If so, lack of intrinsic (computational) parallel support (ie, easily programmable) could be a short-coming that D might continue to share with C/C++?  Could this also be a reason Fortran still remains more attractive to some extent?

Of course, I'm not familiar enough with scientific uses of Fortran to comment on that language comparatively.  I just wonder if some of these advanced Fortran compilers also contain automatic parallelism of some of the computational tasks?

On a slightly different but related topic:

My observation of D is that, as wonderful a language as it is, it still inherits much of the same low-level concurrency difficulties of C/C++. There's no adoption of clean/provable parallel constructs as should be seen in languages hoping to take on the 21st century's multi-core systems.  We're still stuck with the tricky/deadlock-friendly mechanisms of mutex's and semaphores that require highly skilled programming and extremely astute/alert developers (even then deadlocks are often unpredictable).  We need something higher level (and provable) to lighten the load for D users and remove the nasty complexities from parallel programming. (see CSP -- Communicating Sequential Processes -- and Pi Calculus type constructs/constraints).

Such high-level parallelism support is usually most palatable and useful by integrating the constructs with keywords within the language (I'm not sure how attractive these features would be if integrated via mixins or templates in D, but it has been done to some degree with object inheritance in other languages [see JCSP for Java])  A month or so ago, Kris first pointed this out in a previous post to this newsgroup.

Granted, I think such a feature is a difficult task to integrate into D, but I also think it's one major detail that would really set D apart from other computer languages.  The addition would likely be considered hugely welcome in multiple scientific, engineering, and computational science sub-cultures.

-JJR

John Reimer wrote:
> Just curious: how does Fortran scale for use of parallel computations? Perhaps some Fortran compilers are more advanced in that field than C/C++?  If so, lack of intrinsic (computational) parallel support (ie, easily programmable) could be a short-coming that D might continue to share with C/C++?  Could this also be a reason Fortran still remains more attractive to some extent?

The C/C++ compiler vendors, to a large extent, just don't "get it" when it comes to numerics code. For evidence of that, you don't need to look further than dropping 80 bit floating point support. I just can't take seriously the numerics capability of a compiler that doesn't even support the basic FPU types that have been around for 20 years.

For another, you should see some of the postings people have sent to me about how C++'s <complex> class is good enough.

> D uses __VectorInt, which was the point of providing the cpptype {} keyword in
> the first place.  The code within the braces is not parsed by D, but placed in
> an external cpp file which is compiled by a Cpp compiler into a standard Cpp
> class that D CAN use.

due to the missing standart abi in c++ (there is no official and STABLE convention) - what is a "standart" cpp class that D CAN use? and even if
we can make one cpp class a "standart" how do we link to your current object code... (does there the magic thing happen)

if I don't get the point - can someone send me the c++ abi for some vc++ versions (5-2005) and borland/bcb c++ (3-2006)...

because im involved in an projects which uses vc5, vc2005 and bcb4
it would be nice to link them together without preparing them (using dlls, COM interface,...)

> Likewise, a complex functional macro can be placed in the Cpp file and wrapped
> with a standard c function:

how to detect functional macros?, what means complex to you?
can you help me convert this macro into an function

#define DEFINT(varname,value)\
##varname int = value;\

> I don't know.. is this a joke?  Let me ask you, do you find yourself personally
> offended by these ideas?

not offended but amused about your ideas (your first posts sounded like 21years of c/c++, your last posts becoming more and more 2.1 years of c/c++ or vb style...)

> Compared to what?  A multi-year project to convert the C Windows API to D so
> that Windows software can finally be written?

compared to what - a multi-year c++-to-D interface wrapper/compiler which wont ever become stable (because there is no stable abi)

in the pure source code world c++ is nice and nearly vendor unspezific
but c++ object code comes out of the hell...

btw: the winapi project is just a (yes manualy made) conversion of the headers - because D can directly link c object code

but yours is header conversion AND the linkage problem - the winapi port will be finished long, long time before

> I would suggest if you don't like it, then don't feel that you have to use it.

the problem is i would give it a try, but i don't think it would ever happen to exist...

> 21 years.  I've actually used to own a copy of Zortech.  
ok then your familar with assembler (me too)

can you please write a small class (template), compile your code with vc,gcc,bcb,bc (at least 2-3 different compilers)
and then try to use it in assembler (maybe all objectcodes together)

or try to find some good how-to-link-manuals

* Name mangling
In C++, names are mangled to encode the type of the symbol and allow
overloading.  So it may be called `MerryCallMe__voidfn__void' or
something equally cryptic.  You may have to look at the assembly to find
out what it is.
(vendor specific)

* Calling conventions
Member functions are often passed an invisible argument, which is (I
think) their `this'.  I think it's the first argument under GCC, but may
be wrong.  You'll have to pass it appropriately.
(vendor specific)

> How about you?
just 17 years

In article <e4drmd$6q5$1@digitaldaemon.com>, Walter Bright says...
>
[...]
>I am not familiar with the various Fortran libraries out there, so I have no specific one in mind.

FFTW (http://www.fftw.org/) is not written in fortran, but it's used by many fortran libraries. It could be a nice addition to D.

Ciao

---
http://www.mariottini.net/roberto/

Walter Bright wrote:
> Paulo Herrera wrote:
>> Walter Bright wrote:
>>> Don Clugston wrote:
>>>
>>>>> About native libraries
>>>>> ----------------------
>>>>> I think the way to go is to create QUALITY native D libraries. I think we usually overestimate the cost of porting/developing native libraries.
>>>>
>>>> I agree. When porting a library
>>>> (a) there isn't any algorithm development (not much 'thinking' time required);
>>>> (b) it's very easy to test (you can run test cases against the original version);
>>>> (c) the D version is frequently greatly superior to the original.
>>>
>>> Hmm. Is there a 'canonical' Fortran numerics library with a solid test suite that could be translated to D as a showcase project?
>>
>> Do you have some idea in mind?
> 
> I am not familiar with the various Fortran libraries out there, so I have no specific one in mind.
> 
> 
>> If we can make a port of that library to D and show that it performs close to  SPARSKIT, that would be a good demonstration of D capabilities for numerical computing.
> 
> Want to get started on it? <g>
OK, I take the challenge. However, I'm pretty new to D so I'd like to ask some questions before starting.

As someone else posted, most of numerical libraries in Fortran and C/C++ are based on BLAS and LAPACK. So, a first logic step is to evaluate if we should port those libraries to D. This weekend I took a look to the specification of those libraries (http://www.netlib.org/blas/). After going through the document I'm not sure what is the best way to write a similar library in a "D way". I will try to explain what I mean by "D way" ....
I'm sure many people are familiar with those libraries, but I include a brief explanation below to make my question clear.

Those libraries define functions to compute basic matrix/vector operations, such as:
- x = y, where x and y are vectors
- R = aAx, where A and R are matrices, a is a scalar, and x is a vector
- r = aAx + by; where a and b are scalars, A is a matrix, and r, x and y  are vectors.

Since they were originally developed as a specification for old Fortran77 and C, the declaration of the routines that implement those operations look like:
- x = y    => void copy(T *x, T *y)
- R = aAx  => void mult0(T a, T *A, T *x, T *r), etc.

Those declarations are not elegant and they are not easy to use. My first question is: Should we write a D library in that way to get best performance? I hope the answer is no, because that would be an advantage of D over other languages.
I believe we should overwrite operators to make the notation as natural as possible. This is something that the designers of BLAS also faced about Fortran95 that includes array operations: "Some of the functions ... can be replaced by simple array expressions and assignments in Fortran95, without loss of convenience or performance(assuming a reasonable degree of optimization by the compiler)...." (in "Basic Linear Algebra Subprograms Technical (BLAST) Forum Standard", 2001, p. 26)

My questions are:
1) Do you think it's possible to write vector and matrix classes with overloaded operators that perform as well as the primitive BLAS operations? What about temporary objects?
2) Do you think some of those operations can be accelerated by implementing them as part of the language (as described in future work in D) If yes, is there any time frame for those changes?
3) Since the type T on those expressions can change, what is the best way to implement those function without loosing performance?  templates?   Since in general people will use only one data type in their programs, could we use typedef or alias to get better performance?
4) Is there a real interest for this kind of libraries?

Thanks,
Paulo.

Paulo Herrera wrote:
> Walter Bright wrote:
>> Paulo Herrera wrote:
>>> Walter Bright wrote:
>>>> Don Clugston wrote:
>>>>
>>>>>> About native libraries
>>>>>> ----------------------
>>>>>> I think the way to go is to create QUALITY native D libraries. I think we usually overestimate the cost of porting/developing native libraries.
>>>>>
>>>>> I agree. When porting a library
>>>>> (a) there isn't any algorithm development (not much 'thinking' time required);
>>>>> (b) it's very easy to test (you can run test cases against the original version);
>>>>> (c) the D version is frequently greatly superior to the original.
>>>>
>>>> Hmm. Is there a 'canonical' Fortran numerics library with a solid test suite that could be translated to D as a showcase project?
>>>
>>> Do you have some idea in mind?
>>
>> I am not familiar with the various Fortran libraries out there, so I have no specific one in mind.
>>
>>
>>> If we can make a port of that library to D and show that it performs close to  SPARSKIT, that would be a good demonstration of D capabilities for numerical computing.
>>
>> Want to get started on it? <g>
> OK, I take the challenge. However, I'm pretty new to D so I'd like to ask some questions before starting.
> 
> As someone else posted, most of numerical libraries in Fortran and C/C++ are based on BLAS and LAPACK. So, a first logic step is to evaluate if we should port those libraries to D. This weekend I took a look to the specification of those libraries (http://www.netlib.org/blas/). After going through the document I'm not sure what is the best way to write a similar library in a "D way". I will try to explain what I mean by "D way" ....
> I'm sure many people are familiar with those libraries, but I include a brief explanation below to make my question clear.
> 
> Those libraries define functions to compute basic matrix/vector operations, such as:
> - x = y, where x and y are vectors
> - R = aAx, where A and R are matrices, a is a scalar, and x is a vector
> - r = aAx + by; where a and b are scalars, A is a matrix, and r, x and y  are vectors.
> 
> Since they were originally developed as a specification for old Fortran77 and C, the declaration of the routines that implement those operations look like:
> - x = y    => void copy(T *x, T *y)
> - R = aAx  => void mult0(T a, T *A, T *x, T *r), etc.
> 
> Those declarations are not elegant and they are not easy to use. My first question is: Should we write a D library in that way to get best performance? I hope the answer is no, because that would be an advantage of D over other languages.
> I believe we should overwrite operators to make the notation as natural as possible. This is something that the designers of BLAS also faced about Fortran95 that includes array operations: "Some of the functions ... can be replaced by simple array expressions and assignments in Fortran95, without loss of convenience or performance(assuming a reasonable degree of optimization by the compiler)...." (in "Basic Linear Algebra Subprograms Technical (BLAST) Forum Standard", 2001, p. 26)
> 
> My questions are:
> 1) Do you think it's possible to write vector and matrix classes with overloaded operators that perform as well as the primitive BLAS 

Not right now probably, but...

> operations? What about temporary objects?

...check out this thread regarding temporaries:

http://www.digitalmars.com/d/archives/digitalmars/D/35949.html

This hasn't been implemented (yet), but given Walter's comment looks promising. I'm assuming you agree this would be a great idea too.

> 2) Do you think some of those operations can be accelerated by implementing them as part of the language (as described in future work in D) If yes, is there any time frame for those changes?
> 3) Since the type T on those expressions can change, what is the best way to implement those function without loosing performance?  templates? 

Yes.

>   Since in general people will use only one data type in their programs, could we use typedef or alias to get better performance?

Typedef or alias shouldn't make a difference, but may be a convenience.

> 4) Is there a real interest for this kind of libraries?

I'm not a numerical programmer myself, so I'll gladly defer to the opinions of others, but I think there would be interest because there have been quite a few in that field participating in these NG's. There really seems to be great interest in a curly-brace language that "does numerics right" <g>

> 
> Thanks,
> Paulo.

BTW - I downloaded this code to get an idea of 'raw' matrix performance:

http://rs.cipr.uib.no/mtj/bench/JAVAGEMM.html

and modified it to gather an avg.

Here's what I get on a P4 2.2GHz, 512 KB L2, 512 MB RAM running FC5:

GDC v0.17, using dmd v0.140 w/ GCC v4.0.2 (GDC is the D port for GCC)
gdc -O3 -fomit-frame-pointer -funroll-loops -frelease dgemm.d -o dgemm
Max N = 500 avg. mfs: 632.507
Max N = 200 avg. mfs: 905.185

DMD v0.157
dmd -O -inline -release
Max N = 500 avg. mfs: 418.852
Max N = 200 avg. mfs: 485.995

Sun Java v1.5.0
java -server
Max N = 500 avg. mfs: 596.748
Max N = 200 avg. mfs: 797.121

DMD closes the gap where N > 300 and cache size is the bottleneck. In general, DMD does very well at integer stuff and I'm assuming floating point has yet to be fully optimized.

- Dave

What I'd do is do the most straightforward translation possible. Leave redesign for later. I've done this kind of thing before, and attempting to redesign during the translation makes it much, much more time consuming than doing it in two phases.

In article <e4eenc$18j6$1@digitaldaemon.com>, dennis luehring says...
>
>> D uses __VectorInt, which was the point of providing the cpptype {} keyword in the first place.  The code within the braces is not parsed by D, but placed in an external cpp file which is compiled by a Cpp compiler into a standard Cpp class that D CAN use.
>
>due to the missing standart abi in c++ (there is no official and STABLE convention) - what is a "standart" cpp class that D CAN use? and even if we can make one cpp class a "standart" how do we link to your current object code... (does there the magic thing happen)
>
>if I don't get the point - can someone send me the c++ abi for some vc++ versions (5-2005) and borland/bcb c++ (3-2006)...
>
>because im involved in an projects which uses vc5, vc2005 and bcb4 it would be nice to link them together without preparing them (using dlls, COM interface,...)

The format for OBJ files (OMF I believe) is on microsoft's msdn site. I don't know how current it is. However, I'm talking only about gcc's object file format.

>> Likewise, a complex functional macro can be placed in the Cpp file and wrapped with a standard c function:
>
>how to detect functional macros?, what means complex to you? can you help me convert this macro into an function
>
>#define DEFINT(varname,value)\
>##varname int = value;\

cppdecl { DEFINT(MyVar, 100) } int MyVar;

Basically, as long as you know what the C linkage of a particular construct will be, you can simply push it over into the generated cpp file for a standard Cpp compiler (i.e. GCC) to figure out.  In this case you would have the following code added to your generated file:

extern "C" {
DEFINT(MyVar,100)
}

..and an implicit..

extern (C) int MyVar;

. as if you manually wrote that code yourself.  Taking snippets of Cpp code and placing them over in the generated Cpp file, and adding the proper D C extern linkages is not particularly difficult.

>> I don't know.. is this a joke?  Let me ask you, do you find yourself personally offended by these ideas?
>
>not offended but amused about your ideas (your first posts sounded like 21years of c/c++, your last posts becoming more and more 2.1 years of c/c++ or vb style...)

Hmmmm...

>> Compared to what?  A multi-year project to convert the C Windows API to D so that Windows software can finally be written?
>
>compared to what - a multi-year c++-to-D interface wrapper/compiler which wont ever become stable (because there is no stable abi)

GDC and GCC are tightly linked.  When the abi of one changes, so does the other, likewise with GCC-XML.  Any CPP which can be parsed by GCC can be output to GCC-XML.

As far as the Windows C++ abi, it hasn't changed in years as far as I know.  My MSVC 6 code still works with MSVC 8.  There have been changes to the heap allocator, but not the way vtables are implemented or registers are used.

But since the D windows compiler isn't open source anyway.. it doesn't matter? You'd either use an OMF compatible version of GCC/GCD, or recompile your windows code with MinGW, or not use D at all.

>in the pure source code world c++ is nice and nearly vendor unspezific but c++ object code comes out of the hell...
>
>btw: the winapi project is just a (yes manualy made) conversion of the headers - because D can directly link c object code
>
>but yours is header conversion AND the linkage problem - the winapi port will be finished long, long time before

Wanna bet?  In any case, I only have to finish my system once and I'm finished. You're never going to be finished porting the Windows API headers.

>> I would suggest if you don't like it, then don't feel that you have to use it.
>
>the problem is i would give it a try, but i don't think it would ever happen to exist...

Of course not.  It is, after all, impossible as many people have explained to me in vauge detail.  Why would somebody ever even consider such a crazy idea, and who would want it anyway.

>> 21 years.  I've actually used to own a copy of Zortech.
>ok then your familar with assembler (me too)
>
>can you please write a small class (template), compile your code with
>vc,gcc,bcb,bc (at least 2-3 different compilers)
>and then try to use it in assembler (maybe all objectcodes together)
>
>or try to find some good how-to-link-manuals

Uh.. why would I want to do that?

>* Name mangling
>In C++, names are mangled to encode the type of the symbol and allow
>overloading.  So it may be called `MerryCallMe__voidfn__void' or
>something equally cryptic.  You may have to look at the assembly to find
>out what it is.
>(vendor specific)

I'm only interested in GCC.  Those other vendors can fend for themselves.

>* Calling conventions
>Member functions are often passed an invisible argument, which is (I
>think) their `this'.  I think it's the first argument under GCC, but may
>be wrong.  You'll have to pass it appropriately.
>(vendor specific)
>
>> How about you?
>just 17 years

Sure, Cpp calls also often pass silent return argument parameters in some cases. There are a number of details to consider.. but it's certainly not impossible, or improbably difficult as some of the people here would have you believe.