On 10/31/14 3:07 PM, H. S. Teoh via Digitalmars-d wrote:
> On Fri, Oct 31, 2014 at 12:04:24PM -0700, Walter Bright via Digitalmars-d wrote:
>> On 10/27/2014 12:42 AM, Benjamin Thaut wrote:
>>> I'm planning on doing a pull request for druntime which rewrites
>>> every toString function within druntime to use the new sink
>>> signature. That way druntime would cause a lot less allocations which
>>> end up beeing garbage right away. Are there any objections against
>>> doing so? Any reasons why such a pull request would not get accepted?
>>
>> Why a sink version instead of an Output Range?
>
> To allow toString to be a virtual function, perhaps?
>
> Besides, the sink version basically allows encapsulation of an output
> range -- instead of calling x.toString(outputRange) you just write:
>
> 	x.toString((const(char)[] data) { outputRange.put(data); });

No, please don't do that. It's put(outputRange, data);

-Steve

On 10/31/14 5:01 PM, Walter Bright wrote:
> On 10/31/2014 12:07 PM, H. S. Teoh via Digitalmars-d wrote:
>> On Fri, Oct 31, 2014 at 12:04:24PM -0700, Walter Bright via
>> Digitalmars-d wrote:
>>> On 10/27/2014 12:42 AM, Benjamin Thaut wrote:
>>>> I'm planning on doing a pull request for druntime which rewrites
>>>> every toString function within druntime to use the new sink
>>>> signature. That way druntime would cause a lot less allocations which
>>>> end up beeing garbage right away. Are there any objections against
>>>> doing so? Any reasons why such a pull request would not get accepted?
>>>
>>> Why a sink version instead of an Output Range?
>>
>> To allow toString to be a virtual function, perhaps?
>
> Output ranges can be virtual functions. All an output range is is a type
> with a "put" method.

He said "toString" not "sink". And there are more use cases than a type that implements 'put'.

> What I object to with the sink design is there is no consistency in
> design - we cannot preach ranges as a best practice and then use some
> other methodology.

Keep in mind that saying "toString will take output ranges" means that ALL toString implementers must handle ALL forms of output ranges. It's not an issue with "we don't know what we're doing", it's an issue of "let's not make everyone who wants to spit out a simple string handle 5+ different use cases, and you'd better test for them, because the compiler won't complain until it's used!"

I think toString should be first and foremost SIMPLE. It already was -- return a string. But that forces people to allocate, and we want to avoid that. Using a sink is pretty much just as simple.

> BTW, just to be clear, I applaud fixing druntime to remove unnecessary
> GC allocations, and agree that with proper design most of the
> allocations can go away. It's just that sink and output ranges are both
> designed to solve the same problem in pretty much the same way. The
> difference appears to be little more than tomayto tomahto.

It is a huge difference to say EVERYONE who implements toString will take any templated type that purports to be an output range, vs giving one case to handle.

-Steve

Walter Bright <newshound2@digitalmars.com> wrote:

[snip]

> Have you ever looked at the C openssl.lib? The .h files with it are loaded with metaprogramming done with C macros. Yet I've never heard anyone complain about it.

Those macros are a very common common complaint in my experience.

> C .h files for DLLs are typically stuffed with C macros.

[snip]

> The defense presents openssl as Exhibit A!

Presenting OpenSSL as a case for good interface design is a crime by itself!

Tobi

On Monday, 3 November 2014 at 15:42:57 UTC, Steven Schveighoffer wrote:
> At the moment, you are stuck with most toString calls allocating on the GC every time they are called. I think the virtual call thing should be a pleasant improvement :)

Note that delegates aren't virtual calls, but indirect calls. The former need 2 memory access, the latter none (or 3 vs. 1 if the delegate/object isn't yet in a register).

On 11/3/2014 9:36 AM, Tobias Müller wrote:
> Presenting OpenSSL as a case for good interface design is a crime by
> itself!

Not at all. I presented it as an example of a C library that has a metaprogramming interface, but that interface has not prevented bug fix updates to the shared library itself without requiring recompiling of apps that call it.

All shared C libraries have a metaprogramming interface if they have macros in the .h file.

On 11/3/2014 8:09 AM, Steven Schveighoffer wrote:
> It is a huge difference to say EVERYONE who implements toString will take any
> templated type that purports to be an output range, vs giving one case to handle.

All an output range is is a type with a 'put' method. That's it. You're making it out to be far more complex than it is.

On 11/3/14 4:37 PM, Walter Bright wrote:
> On 11/3/2014 9:36 AM, Tobias Müller wrote:
>> Presenting OpenSSL as a case for good interface design is a crime by
>> itself!
>
> Not at all. I presented it as an example of a C library that has a
> metaprogramming interface, but that interface has not prevented bug fix
> updates to the shared library itself without requiring recompiling of
> apps that call it.
>
> All shared C libraries have a metaprogramming interface if they have
> macros in the .h file.
>

I had a very nasty experience with using a template-based API. I vowed to avoid it wherever possible.

The culprit was std::string -- it changed something internally from one version of libc++ to the next on Linux. So I had to recompile everything, but the whole system I was using was with .so objects.

templates do NOT make good API types IMO.

-Steve

On 11/3/14 4:40 PM, Walter Bright wrote:
> On 11/3/2014 8:09 AM, Steven Schveighoffer wrote:
>> It is a huge difference to say EVERYONE who implements toString will
>> take any
>> templated type that purports to be an output range, vs giving one case
>> to handle.
>
> All an output range is is a type with a 'put' method. That's it. You're
> making it out to be far more complex than it is.
>

Directly from the docs: (http://dlang.org/phobos/std_range.html#isOutputRange)

void myprint(in char[] s) { }
static assert(isOutputRange!(typeof(&myprint), char));

No 'put' in sight, except as a substring of isOutputRange.

I don't think you realize what a beast supporting all output ranges is, or using them (hint: calling r.put for a generic output range is an ERROR).

-Steve

On Monday, 3 November 2014 at 22:33:25 UTC, Steven Schveighoffer wrote:
> On 11/3/14 4:40 PM, Walter Bright wrote:
>> On 11/3/2014 8:09 AM, Steven Schveighoffer wrote:
>>> It is a huge difference to say EVERYONE who implements toString will
>>> take any
>>> templated type that purports to be an output range, vs giving one case
>>> to handle.
>>
>> All an output range is is a type with a 'put' method. That's it. You're
>> making it out to be far more complex than it is.
>>
>
> Directly from the docs: (http://dlang.org/phobos/std_range.html#isOutputRange)
>
> void myprint(in char[] s) { }
> static assert(isOutputRange!(typeof(&myprint), char));
>
> No 'put' in sight, except as a substring of isOutputRange.
>
> I don't think you realize what a beast supporting all output ranges is, or using them (hint: calling r.put for a generic output range is an ERROR).
>
> -Steve

In many cases templates are good because they provide the a way for the programmer to use a library optimized for their particular application.  This is the case for the toString function.  An argument can be made that using templates is dangerous because if they are used incorrectly, the number of template instantiates can blow up.  But this can always be solved by the programmer by changing all their template calls to use the same template parameters.  This allows the template solution to simultaneously support a sink that represents a real function, or a delegate, or whatever the application needs.

I understand that people like having a binary library that instantiates it's own functions that have a static interface and I think there's value to that.  But most of the value is in dynamic libraries that the compiler cannot optimize.  When the compiler can optimize, let it:)

I updated my test code to use a templated sink, here the link:

http://marler.info/dtostring.d


   Method 1: ReturnString
             string toString();
   Method 2: SinkDelegate
             void toString(void delegate(const(char)[]) sink);
   Method 3: SinkTemplate
             void toString(T)(T sink) if(isOutputRange!(T,const(char)[]));
   Method 4: SinkDelegateWithStaticHelperBuffer
             struct SinkStatic { char[64] buffer; void delegate(const(char)[]) sink; }
	     void toString(ref SinkStatic sink);
   Method 5: SinkDelegateWithDynamicHelperBuffer
             struct SinkDynamic { char[] buffer; void delegate(const(char)[]) sink; }
	     void toString(ref SinkDynamic sink);
	     void toString(SinkDynamic sink);


(DMD Compiler on x86) "dmd dtostring.d"
RuntimeString run 1 (loopcount 10000000)
  Method 1     : 76 ms
  Method 2     : 153 ms
  Method 3     : 146 ms
  Method 4     : 157 ms
  Method 5ref  : 165 ms
  Method 5noref: 172 ms
StringWithPrefix run 1 (loopcount 1000000)
  Method 1     : 149 ms
  Method 2     : 22 ms
  Method 3     : 21 ms
  Method 4     : 80 ms
  Method 5ref  : 81 ms
  Method 5noref: 82 ms
ArrayOfStrings run 1 (loopcount 1000000)
  Method 1     : 1 sec
  Method 2     : 81 ms
  Method 3     : 77 ms
  Method 4     : 233 ms
  Method 5ref  : 232 ms
  Method 5noref: 223 ms


(DMD Compiler on x86 with Optimization) "dmd -O dtostring.d"
RuntimeString run 1 (loopcount 10000000)
  Method 1     : 30 ms
  Method 2     : 65 ms
  Method 3     : 55 ms
  Method 4     : 68 ms
  Method 5ref  : 68 ms
  Method 5noref: 67 ms
StringWithPrefix run 1 (loopcount 1000000)
  Method 1     : 158 ms
  Method 2     : 9 ms
  Method 3     : 8 ms
  Method 4     : 63 ms
  Method 5ref  : 64 ms
  Method 5noref: 66 ms
ArrayOfStrings run 1 (loopcount 1000000)
  Method 1     : 1 sec, 292 ms
  Method 2     : 35 ms
  Method 3     : 34 ms
  Method 4     : 193 ms
  Method 5ref  : 198 ms
  Method 5noref: 200 ms

The results aren't suprising.  The template out performs the delegate sink.  In a very big project one might try to limit the number of instantiations of toString by using a specific toString instance that accepts some type common OutputRange wrapper which would make the template version perform the same as the sink delegate version, but for projects that don't need to worry about that, you will get better performance from more compiler optimization.

On 11/3/2014 2:28 PM, Steven Schveighoffer wrote:
> I had a very nasty experience with using a template-based API. I vowed to avoid
> it wherever possible.
>
> The culprit was std::string -- it changed something internally from one version
> of libc++ to the next on Linux. So I had to recompile everything, but the whole
> system I was using was with .so objects.
>
> templates do NOT make good API types IMO.

It seems this is blaming templates for a different problem.

If I have:

  struct S { int x; };

in my C .h file, and I change it to:

  struct S { int x,y; };

Then all my API functions that take S as a value argument will require recompilation of any code that uses it.

Would you conclude that C sux for making APIs? Of course not. You'd say that a stable API should use reference types, not value types.

Having templates or not is irrelevant.