This is only losely related to D, but I don't fully understand the separation of component programming and OOP (cf. https://en.wikipedia.org/wiki/Component-based_software_engineering#Differences_from_object-oriented_programming). In an OO framwork, the objects are basically components. See also

"Brad Cox of Stepstone largely defined the modern concept of a software component.[4] He called them Software ICs and set out to create an infrastructure and market for these components by inventing the Objective-C programming language." (see link above)

Walter's example (http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321)

void main() {
        stdin.byLine(KeepTerminator.yes)    // 1
        map!(a => a.idup).                  // 2
        array.                              // 3
        sort.                               // 4
        copy(                               // 5
            stdout.lockingTextWriter());    // 6
    }

This is more or less how mature OO programs look like. Ideally each class (component) does one thing (however small the class might be) and can be used or called to perform this task. All other classes or components can live independently. From my experience this is exactly what Objective-C does. Rather than subclassing, it uses other classes to get a job done.

On Wed, Jul 31, 2013 at 12:20:56PM +0200, Chris wrote:
> This is only losely related to D, but I don't fully understand the
> separation of component programming and OOP (cf. https://en.wikipedia.org/wiki/Component-based_software_engineering#Differences_from_object-oriented_programming).
> In an OO framwork, the objects are basically components. See also
> 
> "Brad Cox of Stepstone largely defined the modern concept of a software component.[4] He called them Software ICs and set out to create an infrastructure and market for these components by inventing the Objective-C programming language." (see link above)
> 
> Walter's example (http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321)
[...]

Thanks for the link to Walter's article, it was a very insightful read.

I can't say I'm that clear about the difference between component programming and OO, so I'll decline comment.

One question that the article did raise, though, was what to do when your algorithms require non-linear interconnectivity between components. For example, say I have an expression evaluator that takes an object that represents a math expression, and another object representing a set of identifier-to-value mappings, and returns the value of the expression given those mappings:

	Value evalExpr(Expr,Ident,Value)(Expr e, Value[Ident] mappings)
	{
		...
	}

In the spirit of component programming, one would conceivably have an expression parsing component that takes, say, an input stream of characters and returns an expression object:

	Expr parseExpr(InputRange)(InputRange input)
		if (is(ElementType!InputRange : dchar))
	{
		...
	}

And conceivably, one would also have a variable assignment parser that parses an input stream of characters containing user-typed value assignments, and returns a Value[Ident] hash (obviously, this last bit can be generalized to any AA-like interface, but let's keep it simple for now):

	Value[Ident] parseBindings(InputRange)(InputRange input) { ... }

So now, my main code would look like this:

	void main(string[] args) {
		assert(args.length == 3);
		parseExpr(args[1])
			.evalExpr(parseBindings(args[2]))
			.copy(stdout);
	}

Which is not as pretty, because of the non-linear dependence on args[1] and arg[2]. I've a hard time turning this into its own reusable component, because it requires multiple disparate inputs. It's also easy to come up with examples with multiple outputs, and, in the general case, components with n-to-m input/output connectivity. How do we still maximize reusability in those cases?


T

-- 
There are two ways to write error-free programs; only the third one works.

On Wed, 31 Jul 2013 12:20:56 +0200, Chris wrote:

> This is only losely related to D, but I don't fully understand the separation of component programming and OOP (cf. https://en.wikipedia.org/wiki/Component-
based_software_engineering#Differences_from_object-oriented_programming).
> In an OO framwork, the objects are basically components. See also
> 
> "Brad Cox of Stepstone largely defined the modern concept of a software component.[4] He called them Software ICs and set out to create an infrastructure and market for these components by inventing the Objective-C programming language." (see link above)
> 
> Walter's example (http://www.drdobbs.com/architecture-and-design/component-programming-
in-d/240008321)
> 
> void main() {
>          stdin.byLine(KeepTerminator.yes)    // 1 map!(a => a.idup).
>                       // 2 array.                              // 3
>          sort.                               // 4 copy(
>                       // 5
>              stdout.lockingTextWriter());    // 6
>      }
> 
> This is more or less how mature OO programs look like. Ideally each class (component) does one thing (however small the class might be) and can be used or called to perform this task. All other classes or components can live independently. From my experience this is exactly what Objective-C does. Rather than subclassing, it uses other classes to get a job done.

A few things:
1) The functions used in Walter's example are not methods, they are
generic free functions.  The "interfaces" they require are not actual OOP
interfaces, but rather a description of what features the supplied type
must supply.
2) The avoidance of actual objects, interfaces, and methods means that
the costly indirections of OOP are also avoided.  The compiler is free to
inline as much of the pipeline as it wishes.
3) Component programming simplifies usage requirements, OOP frameworks
complicate usage requirements (e.g. you must inherit from this class).

If anything, component programming is just functional programming + templates and some nice syntactic sugar.  And a healthy dose of pure awesome.

Justin Whear:

> If anything, component programming is just functional programming + templates and some nice syntactic sugar.
> And a healthy dose of pure awesome.

What D calls "component programming" is very nice and good, but in D it's almost a joke.

Currently this code inlines nothing (the allocations, the difference and the product):


import std.numeric: dotProduct;
int main() {
    enum N = 50;
    auto a = new int[N];
    auto b = new int[N];
    auto c = new int[N];
    c[] = a[] - b[];
    int result = dotProduct(c, c);
    return result;
}


If you write it in component-style (using doubles here):


import std.math;
import std.algorithm, std.range;

int main() {
    enum N = 50;
    alias T = double;
    auto a = new T[N];
    auto b = new T[N];

    return cast(int)zip(a, b)
           .map!(p => (p[0] - p[1]) ^^ 2)
           .reduce!q{a + b};
}


The situation gets much worse, you see many functions in the binary, that even LDC2 often not able to inline. The GHC Haskell compiler turns similar "components" code in efficient SIMD asm (that uses packed doubles, like double2), it inlines everything, merges the loops, produces a small amount of asm output, and there is no "c" intermediate array. In GHC "component programming" is mature (and Intel is developing an Haskell compiler that is even more optimizing), while in D/dmd/Phobos this stuff is just started. GHC has twenty+ years of head start on this and it shows.

The situation should be improved for D/dmd/Phobos, otherwise such D component programming remains partially a dream, or a toy.

Bye,
bearophile

On Wed, Jul 31, 2013 at 09:16:20PM +0000, Justin Whear wrote:
> On Wed, 31 Jul 2013 12:20:56 +0200, Chris wrote:
> 
> > This is only losely related to D, but I don't fully understand the separation of component programming and OOP
[...]
> A few things:
> 1) The functions used in Walter's example are not methods, they are
> generic free functions.  The "interfaces" they require are not actual
> OOP interfaces, but rather a description of what features the supplied
> type must supply.
> 2) The avoidance of actual objects, interfaces, and methods means that
> the costly indirections of OOP are also avoided.  The compiler is free
> to inline as much of the pipeline as it wishes.
> 3) Component programming simplifies usage requirements, OOP frameworks
> complicate usage requirements (e.g. you must inherit from this class).
> 
> If anything, component programming is just functional programming + templates and some nice syntactic sugar.  And a healthy dose of pure awesome.

Keep in mind, though, that you pay for the avoidance of OO indirections by template bloat. Every combination of types passed to your components will create a new instantiation of that component. In simple cases, this is generally only a handful of copies, so it's not a big deal; but for certain frequently-used components, this can explode to a huge amount of duplicated code. They will all avoid "costly" OO indirections, sure, but you pay for that with larger code size, which means higher rate of CPU cache misses, larger memory footprint of the code, etc..

This makes me wonder if we can somehow have a "happy marriage" of the two approaches. Is it possible to have automatic template instantiation factoring, such that in highly-used templates that generate a lot of copies, can the compiler be made smart enough to figure out that automatically adding indirections to the code to reduce the number of instantiations might be better?

One case I've come across before is containers. For the sake of genericity, we usually use templates to implement containers like, say, a red-black tree. However, most of the code that deals with RB trees don't really care about what type the data is at all; they implement algorithms that operate on the structure of the RB tree, not on the data. Only a small subset of RB tree methods actually need to know what type the data should be (the methods that create a new node and initialize it with data, return the data from a node, etc.). Yet, in a template implementation of RB trees, every single method must be repeatedly instantiated over and over, for every type you put into the container.

Most of these method instantiations may in fact be essentially identical to each other, except perhaps for one or two places where it may use a different node size, or call some comparison function on the data in the nodes.  Ideally, the compiler should be able to know when a method of a templated struct/class is transitively independent of the template parameter, and only emit code for that method once. All other instantiations of that method will simply become aliases of that one instantiation.

This doesn't cover the case where the call chain may pass through methods that don't care about data types but eventually ends at a method that *does* have to care about data types; but this is solvable by factoring the code so that the type-independent code is separated from the type-dependent code, except for one or two runtime parameters (e.g. size of the data type, or a function pointer to the type-dependent code that must be called at the end of, say, a tree traversal).  The compiler may even be able to do this automatically in certain simple cases.


T

-- 
If it breaks, you get to keep both pieces. -- Software disclaimer notice

On 7/31/2013 3:23 PM, bearophile wrote:
> The situation should be improved for D/dmd/Phobos, otherwise such D component
> programming remains partially a dream, or a toy.

Ironically, the component program from the article I wrote:

    void main() {
        stdin.byLine(KeepTerminator.yes)    // 1
        map!(a => a.idup).                  // 2
        array.                              // 3
        sort.                               // 4
        copy(                               // 5
            stdout.lockingTextWriter());    // 6
    }

is 2x faster than the Haskell version:

    import Data.List
    import qualified Data.ByteString.Lazy.Char8 as L main = L.interact $
    L.unlines . sort . L.lines

On 07/31/2013 03:46 PM, Walter Bright wrote:

> is 2x faster

What do you mean exactly? :p

Ali

Walter Bright:

> Ironically, the component program from the article I wrote:
> ...
> is 2x faster than the Haskell version:

Benchmarking code written in two different languages is tricky, there are so many sources of mistakes, even if you know well both languages. But I accept your timing. And I say that it's good :-) We should aim to be better than the Intel Labs Haskell Research Compiler (HRC) :-)

Bye,
bearophile

On 7/31/2013 4:17 PM, bearophile wrote:
> Walter Bright:
>
>> Ironically, the component program from the article I wrote:
>> ...
>> is 2x faster than the Haskell version:
>
> Benchmarking code written in two different languages is tricky, there are so
> many sources of mistakes, even if you know well both languages. But I accept
> your timing. And I say that it's good :-) We should aim to be better than the
> Intel Labs Haskell Research Compiler (HRC) :-)

You are right to be skeptical of cross language benchmarks.

Some data points you might find interesting:

1. I made no attempt to optimize the D version (other than throwing the appropriate compiler switches). It's meant to be the straightforward "naive" implementation.

2. I did not write the Haskell version - Bartosz Milewski did. He admits to not being an expert on Haskell, and there may be faster ways to do it.


I'll also agree with you that the component programming style is new in D, and probably could benefit a great deal from 20 years of concerted effort :-)

I disagree with you that it is a toy, however. Speed is only one measure of utility.

On Thu, 01 Aug 2013 00:23:52 +0200, bearophile wrote:
> 
> The situation should be improved for D/dmd/Phobos, otherwise such D component programming remains partially a dream, or a toy.
> 
> Bye,
> bearophile

I disagree with your "toy" assessment.  I've been using this chaining, component style for a while now and have really enjoyed the clarity it's brought to my code.  I hadn't realized how bug-prone non-trivial loops tend to be until I started writing this way and avoided them entirely. My policy is to aim for clarity and legibility first and to rewrite for performance only if necessary.  Thus far, I don't think I've rewritten anything out of the component programming style, so while probably not optimal, it's been more than good enough.