Greetings, I'm a newcomer in D and I've been very happy with the language so far. There's one topic that I can't figure out though; how am I supposed to distribute libraries for D.

Let's suppose I want to program a /closed-source/ library for D. People should be able to use that library but without access to the source code.

When using Java I can provide a .jar package with the necessary .class files. Users can can import this package and specify the .jar file as a compiler argument.

In .NET I provide the generated .dll file and the .xml documentation file. These can be plugged in and used in Visual Studio very easily.

In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header files. No source code required so far (although Bytecode and MSIL are easy to decompile).

It seems that the D compiler requires the full source code for D libraries. Whenever I import a package in D source code as such:

#  import mylib;

The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument doesn't do the trick. Am I missing something here? Since I want to provide directly usable libraries for D I really wouldn't like to limit myself to that export(C) assembler stuff.

So can I build full-featured (not exported) libraries with D, for D, without opening the source?

On Thu, 29 Jul 2004 02:41:30 +0300, Niko Korhonen wrote:

> Greetings, I'm a newcomer in D and I've been very happy with the language so far. There's one topic that I can't figure out though; how am I supposed to distribute libraries for D.
> 
> Let's suppose I want to program a /closed-source/ library for D. People should be able to use that library but without access to the source code.
> 
> When using Java I can provide a .jar package with the necessary .class files. Users can can import this package and specify the .jar file as a compiler argument.
> 
> In .NET I provide the generated .dll file and the .xml documentation file. These can be plugged in and used in Visual Studio very easily.
> 
> In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header files. No source code required so far (although Bytecode and MSIL are easy to decompile).
> 
> It seems that the D compiler requires the full source code for D libraries. Whenever I import a package in D source code as such:
> 
> #  import mylib;
> 
> The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument doesn't do the trick. Am I missing something here? Since I want to provide directly usable libraries for D I really wouldn't like to limit myself to that export(C) assembler stuff.
> 
> So can I build full-featured (not exported) libraries with D, for D, without opening the source?

It can be done!

I worked it out with the help of a few people here. I was *not* intuitive and involved a bit of /smoke & mirrors/. Unfortunately, I can't remember all the steps involved. I'll try to nut it out again and this time write down the process. ;-)

-- 
Derek
Melbourne, Australia
29/Jul/04 10:18:26 AM

"Niko Korhonen" <niktheblak@hotmail.com> wrote in message news:ce9dgi$phm$1@digitaldaemon.com...
> Greetings, I'm a newcomer in D and I've been very happy with the language so far. There's one topic that I can't figure out though; how am I supposed to distribute libraries for D.
>
> Let's suppose I want to program a /closed-source/ library for D. People should be able to use that library but without access to the source code.
>
> When using Java I can provide a .jar package with the necessary .class files. Users can can import this package and specify the .jar file as a compiler argument.
>
> In .NET I provide the generated .dll file and the .xml documentation file. These can be plugged in and used in Visual Studio very easily.
>
> In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header
> files. No source code required so far (although Bytecode and MSIL are
> easy to decompile).
>
> It seems that the D compiler requires the full source code for D libraries. Whenever I import a package in D source code as such:
>
> #  import mylib;
>
> The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument doesn't do the trick. Am I missing something here? Since I want to provide directly usable libraries for D I really wouldn't like to limit myself to that export(C) assembler stuff.
>
> So can I build full-featured (not exported) libraries with D, for D,
> without opening the source?

Yes, make a stripped D file just for importing. Given the actual source file is foo.d, make foo_imp.d:

module foo; // Not foo_imp.
class Foo
{
   int publicfield;
   void stuff();
}
int bar();
// etc ,,..,

Niko Korhonen wrote:
> Greetings, I'm a newcomer in D and I've been very happy with the language so far. There's one topic that I can't figure out though; how am I supposed to distribute libraries for D.
> 
> Let's suppose I want to program a /closed-source/ library for D. People should be able to use that library but without access to the source code.
> 
> So can I build full-featured (not exported) libraries with D, for D, without opening the source?

The D compiler needs declarations, but doesn't care about definitions.

Except for templates, you can just leave out the definition:

    class Foo : Bar {
        this();
        void thing();
        int x;
        etc();
    }

    int nonmember_function();

The linker will sort it all out.

 -- andy

"Niko Korhonen" <niktheblak@hotmail.com> wrote in message news:ce9dgi$phm$1@digitaldaemon.com...
> So can I build full-featured (not exported) libraries with D, for D,
> without opening the source?

Sure! Look at phobos/object.d and phobos/internal/object.d for an example. The trick is to remove the function bodies from the shipped version of the source, and leave the bodies in for the compiled library that you ship.

On Thu, 29 Jul 2004 10:21:32 +1000, Derek Parnell wrote:

> On Thu, 29 Jul 2004 02:41:30 +0300, Niko Korhonen wrote:
> 
>> Greetings, I'm a newcomer in D and I've been very happy with the language so far. There's one topic that I can't figure out though; how am I supposed to distribute libraries for D.
>> 
>> Let's suppose I want to program a /closed-source/ library for D. People should be able to use that library but without access to the source code.
>> 
>> When using Java I can provide a .jar package with the necessary .class files. Users can can import this package and specify the .jar file as a compiler argument.
>> 
>> In .NET I provide the generated .dll file and the .xml documentation file. These can be plugged in and used in Visual Studio very easily.
>> 
>> In C/C++ I provide the binaries (.obj, .a, .lib etc) and the header files. No source code required so far (although Bytecode and MSIL are easy to decompile).
>> 
>> It seems that the D compiler requires the full source code for D libraries. Whenever I import a package in D source code as such:
>> 
>> #  import mylib;
>> 
>> The compiler wants mylib.d. Passing mylib.obj or mylib.lib as argument doesn't do the trick. Am I missing something here? Since I want to provide directly usable libraries for D I really wouldn't like to limit myself to that export(C) assembler stuff.
>> 
>> So can I build full-featured (not exported) libraries with D, for D, without opening the source?
> 
> It can be done!
> 
> I worked it out with the help of a few people here. I was *not* intuitive and involved a bit of /smoke & mirrors/. Unfortunately, I can't remember all the steps involved. I'll try to nut it out again and this time write down the process. ;-)

Okay. This is what I did...

** Created a folder called 'IMP'
** Stored my source code in here. These were implementation code, and not
'headers'.
   example:
   --- file: mfoo.d ---
   module mfoo;
   import std.stdio;

   class Foo
   {
    void foo(int x){ writef("FOO\n");}
   }

   --- file: mbar.d ---
   module mbar;
   import std.stdio;
   import mfoo;
   class Bar: Foo
   {
    alias Foo.foo foo;
    void foo(char[] x){writef("BAR\n");};
   }

** Compile these with the -c switch.
** Move the resulting *.obj files into a library file (eg. via the
DigitalMars lib program)
** Copy the source files. eg. mfoo.d --> mfoo_exp.d mbar.d --> mbar_exp.d
** Edit the _exp.d files and remove all the implementation stuff.
   example:
   --- file: mfoo_exp.d ---
   module mfoo;
   import std.stdio;

   class Foo
   {
    void foo(int x);
   }

   --- file: mbar.d ---
   module mbar;
   import std.stdio;
   import mfoo;
   class Bar: Foo
   {
    alias Foo.foo foo;
    void foo(char[] x);
   }
** Create another folder. eg. EXPORT
** Copy the library from IMP to EXPORT
** Copy the _exp.d files from IMP to EXPORT
** Rename the copied _exp.d files in EXPORT to their .d names (remove the
_exp) part.
** The EXPORT folder is now your distribution package.

To use the distribution package. Create your application code as normal and
compile it using the .d files and .lib file from the distribution.
   example:
   --- file: test.d ---
   import mfoo;
   import mbar;
   void main()
   {
    Foo f = new Foo;
    Bar b = new Bar;
    int i;
    char[] s = "test";

    b.foo(i);
    b.foo(s);
   }
   --------------------------

and compiled with ...

   dmd test mylib.lib


So the net result is that the user of your library only gets the .LIB file and the stripped down source code files, containing definitions but no implementation details.

-- 
Derek
Melbourne, Australia
29/Jul/04 11:13:49 AM

On 28 Jul 2004, "Walter" <newshound@digitalmars.com> wrote:

> "Niko Korhonen" <niktheblak@hotmail.com> wrote in message news:ce9dgi$phm$1@digitaldaemon.com...
>> So can I build full-featured (not exported) libraries with D, for
>> D, without opening the source?
> 
> Sure! Look at phobos/object.d and phobos/internal/object.d for an example. The trick is to remove the function bodies from the shipped version of the source, and leave the bodies in for the compiled library that you ship.

I think this is an important issue and not just for keeping source code closed.  One feature I have not yet found in D is a clean solution for separating interface from implementation in the way C prototypes and forward declarations do.  So far, this method of stripping out the implementation seems to be the *only* offered solution, and it seems kludgy.

Would it be possible to do something like this?  A module could (optionally) use 2 files, say x.d and x.i.  x.d would be the "stripped" interface file.  x.i would be the "full" implementation file.  x.d would have a statement like:

module x;
implementation x;

When building the object x.obj, the compiler would "bring in" the implementation and compile the "combined" file.  When compiling any other object using "import x" it would not.  It would just see the .d interface file.  You'd just ship the .lib containing the .obj and .d files.  Just a thought.

BTW, do you have a tool or script of some kind that strips the function bodies?  It looks to me like that might be hard to do reliably without actually parsing the code to distinguish function bodies from other types of blocks.

Dave

-- 
D.a.v.i.d  T.i.k.t.i.n
t.i.k.t.i.n [at] a.d.v.a.n.c.e.d.r.e.l.a.y [dot] c.o.m

>I think this is an important issue and not just for keeping source code closed.  One feature I have not yet found in D is a clean solution for separating interface from implementation in the way C prototypes and forward declarations do.  So far, this method of stripping out the implementation seems to be the *only* offered solution, and it seems kludgy.

Exactly the thing I've been thinking about recently.

>  x.i would be the "full" implementation
>file.  x.d would have a statement like:
>
>module x;
>implementation x;

That I haven't thought about ;)
It might get better:

module implementation x;

and I think it wouldn't be too painful to parse if that's concerned

David Tiktin schrieb:

> I think this is an important issue and not just for keeping source code closed.  One feature I have not yet found in D is a clean solution for separating interface from implementation in the way C prototypes and forward declarations do.  So far, this method of stripping out the implementation seems to be the *only* offered solution, and it seems kludgy.

It's really nice for development to have it together. Like, there are different opinions, and i do program in C++ primarily, but i still feel much more comfortable with both implementation and declarations in one file. Nice for people coming from Delphi (like myself) or Java originally.

What else can probably be done, make 2 sections within one file, the upper for (public) declarations, the lower for implementation. I don't know whether it always works, but i think it should, so it's worth a try. And where it doesn't, report a bug.

> Would it be possible to do something like this?  A module could (optionally) use 2 files, say x.d and x.i.  x.d would be the "stripped" interface file.  x.i would be the "full" implementation file.  x.d would have a statement like: 

Yikes! It was such a relief that D doesn't force one to do that. but... you really want it back?

Walters opinion to this is, is that interface mixed with implementation is OK, but one needs some documentation tool. Nice HTMLs are far easier to read than stripped source or headers.

> module x;
> implementation x;
> 
> When building the object x.obj, the compiler would "bring in" the implementation and compile the "combined" file.  When compiling any other object using "import x" it would not.  It would just see the .d interface file.  You'd just ship the .lib containing the .obj and .d files.  Just a thought.

If you can use separate interface from implementation as above, one could expand on it. For example, by using a version statement and a simple tool to strip out the contents of it, or by chaining the preprocessor into the build process.

> BTW, do you have a tool or script of some kind that strips the function bodies?  It looks to me like that might be hard to do reliably without actually parsing the code to distinguish function bodies from other types of blocks.

I recall dig compiler driver (look for undig at dsource) would usually be able to do the trick, but someone was working on a better utility.

-eye

On 29 Jul 2004, Ilya Minkov <minkov@cs.tum.edu> wrote:

> David Tiktin schrieb:
> 
>> I think this is an important issue and not just for keeping source code closed.  One feature I have not yet found in D is a clean solution for separating interface from implementation in the way C prototypes and forward declarations do.  So far, this method of stripping out the implementation seems to be the *only* offered solution, and it seems kludgy.
> 
> It's really nice for development to have it together. Like, there are different opinions, and i do program in C++ primarily, but i still feel much more comfortable with both implementation and declarations in one file. Nice for people coming from Delphi (like myself) or Java originally.
> 
> What else can probably be done, make 2 sections within one file, the upper for (public) declarations, the lower for implementation. I don't know whether it always works, but i think it should, so it's worth a try. And where it doesn't, report a bug.

You'd still have to physically remove the lower part when shipping a library if you wanted to keep the source code closed.

>> Would it be possible to do something like this?  A module could (optionally) use 2 files, say x.d and x.i.  x.d would be the "stripped" interface file.  x.i would be the "full" implementation file.  x.d would have a statement like:

> Yikes! It was such a relief that D doesn't force one to do that. but... you really want it back?

I did say "optionally" :-).  But yes, I guess I do want something *like* this and not just for code privacy.  (Yes, Java and C# do the same thing, and have the same problem.  They have spawned a code obfuscation industry so people can ship classes that can't be decompiled.  Yuck!)  I'm not suggesting that D require this, just that it support it.  I think conceptual separation of interface and implmenatation is the first principle of good software engineering. So I think there should be language constructs which support good abstraction barriers.  But I agree that forcing people to be good isn't nice :-).

> Walters opinion to this is, is that interface mixed with implementation is OK, but one needs some documentation tool. Nice HTMLs are far easier to read than stripped source or headers.

I agree with the last part, but that also introduces lots of redundancy in the code.  You have the code *and* HTML comments that explain it.  And we know how easy it is for the 2 to get out of synch.

I think this is a really *hard* problem.  No one wants the same thing in 2 places.  Having the same thing in 2 places is the first principle of bad software engineering!

>> module x;
>> implementation x;
>> 
>> When building the object x.obj, the compiler would "bring in" the implementation and compile the "combined" file.  When compiling any other object using "import x" it would not.  It would just see the .d interface file.  You'd just ship the .lib containing the .obj and .d files.  Just a thought.
> 
> If you can use separate interface from implementation as above, one could expand on it. For example, by using a version statement and a simple tool to strip out the contents of it, or by chaining the preprocessor into the build process.

Yes, there could be a solution along these lines.  But does D allow you to "reopen" a class definition?

[pseudo code]

class X
{
  this(int);
}

version (implementation)
{
  class X
  {
     this(int x)
     {
        x_ = x;
     }
     private int x_;
  }
}

[/pseudo code]

If it did, I suppose you could just do:

[pseudo code]

version (implementation)
{
  import x.impl;
}

[/pseudo code]

Or something like that.

>> BTW, do you have a tool or script of some kind that strips the function bodies?  It looks to me like that might be hard to do reliably without actually parsing the code to distinguish function bodies from other types of blocks.
> 
> I recall dig compiler driver (look for undig at dsource) would usually be able to do the trick, but someone was working on a better utility.

Thanks, I'll look there.

Dave

-- 
D.a.v.i.d  T.i.k.t.i.n
t.i.k.t.i.n [at] a.d.v.a.n.c.e.d.r.e.l.a.y [dot] c.o.m