So I have this recurring pattern, it's really starting to annoy me.
It stems from the fact that a function prototype and the definition can not
appear in the same file in D (as it can in C/C++)
Eg,

void func(int x); // <-- declaration of function, informs type and
associated names, args, ...

//later
void func(int x) // <-- may be generated with magic (and may use the
prototype declaration for type information as declared by the prototype
above)
{
  ... do stuff
}

I really need this. Why is it illegal? Is there chance of having this supported? What are the problems?

My problem is essentially to do with supporting both static or dynamic
linkage optionally, but it also shows up in situations where I want to
perform comprehensive magic code generation, but want clear-readable user
declarations.
The simplest case:
  I have an extern that I may want to statically or dynamically link.
    In the case of static linkage, one just produces a prototype, and it
links, no problem.
    In the case of dynamic linkage, one must produce a stub for the
function, a function pointer to call through, and perhaps some code to
hook-up the function pointer at init.

I have a fairly comprehensive binding solution which automates the work in
the case of dynamic linkage, but the problem is the way the user defines
the functions that exist in the D code.
I'd like it if the users would just write the prototypes, they'd be easily
readable, simple to create by cutting and pasting straight from C code. And
when statically linking, it would just work, the module hook-up mixin does
nothing in this configuration.
In the case of dynamic linkage, a single hook-up mixin in the file
somewhere could scan the module for these prototypes and generate the
stubs, function pointers, and the boot-up code that would connect them
(while validating their signatures against the extern import table, etc).

This approach isn't supported though. What I do instead:

mixin( ImportFunction!( "functionName", int function( int arg1, float arg2,
ref in SomeStruct arg3 ), "pure nothrow" ) );
mixin( ImportFunction!( "functionName2", int function() ) );
mixin( ImportFunction!( "functionName3", int function( int x, int y ),
"nothrow" ) );
etc etc

These templates produce, from the information provided, either a prototype for static linkage, or the { stub, func pointer, initialisation info } for dynamic linkage.

I really don't like doing it this way for a number of reasons:
  *** This results in hundreds of mixins, which quickly lead to intolerable
compile times (to the point where D is losing it's appeal as a viable
choice for our needs). If done my preferred way, I could do the magic with
a single mixin at the bottom of the file.
  * It's barely readable.
  * Much more annoying to write and maintain when the counterpart API
changes, and is more error-prone.
  * IDE features don't work properly; syntax highlighting, hover info,
go-to-definition, etc.

I also have numerous more advanced cases of the same problem, and becomes even more unsightly when used in structs to emulate dynamic linkage to static methods of C++ classes.

The solution is seemingly trivial; allow function prototypes and definitions to exist in the same file, like in C/C++.


Go on, tear me apart... :)

Manu:

> I have a fairly comprehensive binding solution which automates the work in
> the case of dynamic linkage, but the problem is the way the user defines the functions that exist in the D code.

Maybe there are better ways to solve your problems (even changing D language in some ways), and keep avoiding the need of free floating ghosts of functions.

Bye,
bearophile

On 25 September 2012 15:21, bearophile <bearophileHUGS@lycos.com> wrote:

> Manu:
>
>
>  I have a fairly comprehensive binding solution which automates the work in
>> the case of dynamic linkage, but the problem is the way the user defines the functions that exist in the D code.
>>
>
> Maybe there are better ways to solve your problems (even changing D language in some ways), and keep avoiding the need of free floating ghosts of functions.
>

Can you suggest a more direct approach? Declaring prototypes seems completely natural, supports all the expected IDE features, and in the simple/common case (static linkage) requires absolutely no magic voodoo.

On Tue, 25 Sep 2012 07:53:17 -0400, Manu <turkeyman@gmail.com> wrote:

> So I have this recurring pattern, it's really starting to annoy me.
> It stems from the fact that a function prototype and the definition can not
> appear in the same file in D (as it can in C/C++)

Doing this is not illegal.

> Eg,
>
> void func(int x); // <-- declaration of function, informs type and
> associated names, args, ...
>
> //later
> void func(int x) // <-- may be generated with magic (and may use the
> prototype declaration for type information as declared by the prototype
> above)
> {
>   ... do stuff
> }

This compiles.  Do you have a better case to show the problem?

-Steve

On 25 September 2012 17:25, Steven Schveighoffer <schveiguy@yahoo.com>wrote:

> On Tue, 25 Sep 2012 07:53:17 -0400, Manu <turkeyman@gmail.com> wrote:
>
>  So I have this recurring pattern, it's really starting to annoy me.
>> It stems from the fact that a function prototype and the definition can
>> not
>> appear in the same file in D (as it can in C/C++)
>>
>
> Doing this is not illegal.
>
>
>  Eg,
>>
>> void func(int x); // <-- declaration of function, informs type and
>> associated names, args, ...
>>
>> //later
>> void func(int x) // <-- may be generated with magic (and may use the
>> prototype declaration for type information as declared by the prototype
>> above)
>> {
>>   ... do stuff
>> }
>>
>
> This compiles.  Do you have a better case to show the problem?
>

void blah();

void  blah()
{
int x = 0;
}

void f()
{
   blah(); // <- call it
}


W:\project\main\sourcedata\plugins\remedy\modules\test_module.d(38):Error:
function remedy.testmodule.blah called with argument types:
(())
matches both:
remedy.testmodule.blah()
and:
remedy.testmodule.blah()

Obviously, one is just a prototype, and the symbol is resolved within the local file.

On 2012-09-25 13:53, Manu wrote:

> I really don't like doing it this way for a number of reasons:
>    *** This results in hundreds of mixins, which quickly lead to
> intolerable compile times (to the point where D is losing it's appeal as
> a viable choice for our needs). If done my preferred way, I could do the
> magic with a single mixin at the bottom of the file.

I guess you already figured this out but a single mixin with a lot of code is a lot faster than several mixins with little code. Although this will decrease the readability even further.

-- 
/Jacob Carlborg

On Tue, 25 Sep 2012 10:40:41 -0400, Manu <turkeyman@gmail.com> wrote:

> On 25 September 2012 17:25, Steven Schveighoffer <schveiguy@yahoo.com>wrote:
>
>> On Tue, 25 Sep 2012 07:53:17 -0400, Manu <turkeyman@gmail.com> wrote:
>>
>>  So I have this recurring pattern, it's really starting to annoy me.
>>> It stems from the fact that a function prototype and the definition can
>>> not
>>> appear in the same file in D (as it can in C/C++)
>>>
>>
>> Doing this is not illegal.
>>
>>
>>  Eg,
>>>
>>> void func(int x); // <-- declaration of function, informs type and
>>> associated names, args, ...
>>>
>>> //later
>>> void func(int x) // <-- may be generated with magic (and may use the
>>> prototype declaration for type information as declared by the prototype
>>> above)
>>> {
>>>   ... do stuff
>>> }
>>>
>>
>> This compiles.  Do you have a better case to show the problem?
>>
>
> void blah();
>
> void  blah()
> {
> int x = 0;
> }
>
> void f()
> {
>    blah(); // <- call it
> }
>
>
> W:\project\main\sourcedata\plugins\remedy\modules\test_module.d(38):Error:
> function remedy.testmodule.blah called with argument types:
> (())
> matches both:
> remedy.testmodule.blah()
> and:
> remedy.testmodule.blah()

Oh, that is funny.  I simply compiled the functions and didn't call them, assuming if that compiled it was legal.

This is definitely a bug.  In fact you can implement a function *twice* and this isn't an error.

Check this out:

testme2.d:

module testme2;
import std.stdio;

void foo()
{
    writeln("first");
}

void foo()
{
    writeln("second");
}

testme2.di:

module testme2;

void foo();

testme.d:
import testme2;

void main()
{
    foo();
}

Compile like this:

dmd -c testme2.d
dmd testme.d testme2.o

links, and when I run, it displays:

first

So clearly there isn't something right here, that should definitely be an error.

I think there are two errors here.  First, the spec does not say you cannot prototype a function before declaring it.  All it says is:

Functions without bodies:

int foo();

that are not declared as abstract are expected to have their implementations elsewhere, and that implementation will be provided at the link step. This enables an implementation of a function to be completely hidden from the user of it, and the implementation may be in another language such as C, assembler, etc.

from: http://dlang.org/function.html

I don't see any rules that would preclude pre-declaring a prototype, even if it's not promoted to do this (after all, forward references should always work, right?).

Second, you should not be able to compile two identically prototyped functions with bodies into an object file,  I have no idea how that even works.

-Steve

Le 25/09/2012 13:53, Manu a écrit :
> So I have this recurring pattern, it's really starting to annoy me.
> It stems from the fact that a function prototype and the definition can
> not appear in the same file in D (as it can in C/C++)
> Eg,
>
> void func(int x); // <-- declaration of function, informs type and
> associated names, args, ...
>
> //later
> void func(int x) // <-- may be generated with magic (and may use the
> prototype declaration for type information as declared by the prototype
> above)
> {
>    ... do stuff
> }
>
> I really need this. Why is it illegal? Is there chance of having this
> supported? What are the problems?
>
> My problem is essentially to do with supporting both static or dynamic
> linkage optionally, but it also shows up in situations where I want to
> perform comprehensive magic code generation, but want clear-readable
> user declarations.
> The simplest case:
>    I have an extern that I may want to statically or dynamically link.
>      In the case of static linkage, one just produces a prototype, and
> it links, no problem.
>      In the case of dynamic linkage, one must produce a stub for the
> function, a function pointer to call through, and perhaps some code to
> hook-up the function pointer at init.
>
> I have a fairly comprehensive binding solution which automates the work
> in the case of dynamic linkage, but the problem is the way the user
> defines the functions that exist in the D code.
> I'd like it if the users would just write the prototypes, they'd be
> easily readable, simple to create by cutting and pasting straight from C
> code. And when statically linking, it would just work, the module
> hook-up mixin does nothing in this configuration.
> In the case of dynamic linkage, a single hook-up mixin in the file
> somewhere could scan the module for these prototypes and generate the
> stubs, function pointers, and the boot-up code that would connect them
> (while validating their signatures against the extern import table, etc).
>
> This approach isn't supported though. What I do instead:
>
> mixin( ImportFunction!( "functionName", int function( int arg1, float
> arg2, ref in SomeStruct arg3 ), "pure nothrow" ) );
> mixin( ImportFunction!( "functionName2", int function() ) );
> mixin( ImportFunction!( "functionName3", int function( int x, int y ),
> "nothrow" ) );
> etc etc
>
> These templates produce, from the information provided, either a
> prototype for static linkage, or the { stub, func pointer,
> initialisation info } for dynamic linkage.
>
> I really don't like doing it this way for a number of reasons:
>    *** This results in hundreds of mixins, which quickly lead to
> intolerable compile times (to the point where D is losing it's appeal as
> a viable choice for our needs). If done my preferred way, I could do the
> magic with a single mixin at the bottom of the file.
>    * It's barely readable.
>    * Much more annoying to write and maintain when the counterpart API
> changes, and is more error-prone.
>    * IDE features don't work properly; syntax highlighting, hover info,
> go-to-definition, etc.
>
> I also have numerous more advanced cases of the same problem, and
> becomes even more unsightly when used in structs to emulate dynamic
> linkage to static methods of C++ classes.
>
> The solution is seemingly trivial; allow function prototypes and
> definitions to exist in the same file, like in C/C++.
>
>
> Go on, tear me apart... :)

This is code duplication and is generally considered as a bad practice. I'm not convinced that the language should be modified to allow something that is known as bad practice.

What you need here is a more robust di generator IMO.

On Tuesday, 25 September 2012 at 17:48:46 UTC, deadalnix wrote:
> This is code duplication and is generally considered as a bad practice. I'm not convinced that the language should be modified to allow something that is known as bad practice.
>
> What you need here is a more robust di generator IMO.

This is neither a code duplication (there is certain distinction between prototype and body) nor a bad practice - this is a common pattern in C and C++. It is surprising that D cannot do such a simple thing which in addition is not harmless, so there is no sense in depreciating it.

Le 25/09/2012 20:57, Maxim Fomin a écrit :
> On Tuesday, 25 September 2012 at 17:48:46 UTC, deadalnix wrote:
>> This is code duplication and is generally considered as a bad
>> practice. I'm not convinced that the language should be modified to
>> allow something that is known as bad practice.
>>
>> What you need here is a more robust di generator IMO.
>
> This is neither a code duplication (there is certain distinction between
> prototype and body) nor a bad practice - this is a common pattern in C
> and C++. It is surprising that D cannot do such a simple thing which in
> addition is not harmless, so there is no sense in depreciating it.

This is code duplication. And this has been considered a problem in C/C++ for a while now.