Thread overview
Why is sformat and formattedWrite (appender) allocating GC mem here?
Aug 31, 2019
cc
Aug 31, 2019
Mike Parker
Aug 31, 2019
ag0aep6g
Sep 01, 2019
Nicholas Wilson
Sep 02, 2019
cc
Sep 03, 2019
James Blachly
Sep 03, 2019
ag0aep6g
August 31, 2019
And what, if anything, can I do to avoid it?

	string[] arr = ["abcd", "efgh", "ijkl"];
	char[4096] buf;
	char[] res;
	writeln(GC.stats);
	res = sformat(buf, "%s", arr);
	assert(res.ptr == buf.ptr);
	writeln(res);
	writeln(GC.stats);
	res = sformat(buf, "%s", arr);
	assert(res.ptr == buf.ptr);
	writeln(res);
	writeln(GC.stats);

	// Results:
	Stats(64, 1048512, 80)
	["abcd", "efgh", "ijkl"]
	Stats(272, 1048304, 272)
	["abcd", "efgh", "ijkl"]
	Stats(464, 1048112, 464)

I get similar behavior trying to use formattedWrite with an appender:

	string[] arr = ["abcd", "efgh", "ijkl"];
	auto formatBuffer = appender!(char[])();
	formatBuffer.reserve(4096);
	formatBuffer.clear();
	writeln(GC.stats);
	formatBuffer.formattedWrite!"%s"(arr);
	writeln(formatBuffer.data);
	writeln(GC.stats);
	formatBuffer.clear();
	formatBuffer.formattedWrite!"%s"(arr);
	writeln(formatBuffer.data);
	writeln(GC.stats);

	// Results:
	Stats(12288, 5230592, 4208)
	["abcd", "efgh", "ijkl"]
	Stats(12432, 5230448, 4352)
	["abcd", "efgh", "ijkl"]
	Stats(12576, 5230304, 4496)

Same thing if I use a format string like "%(%s,%)" rather than just "%s".  I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied.  Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer?
August 31, 2019
On Saturday, 31 August 2019 at 12:07:56 UTC, cc wrote:
> And what, if anything, can I do to avoid it?

import core.stdc.stdio : printf;

There are no @nogc versions of the Phobos write* and format functions.

August 31, 2019
On 31.08.19 14:07, cc wrote:
> I'm guessing it's allocating a string first to write the contents of the array and then inserting that string into the buffer I supplied.  Is there no way to have it skip this step and just write each element (plus the joining punctuation, etc) directly into the buffer?

`formatElement` does something like that. It writes the string into a temporary buffer while looking for invalid Unicode. When it finds some, the temporary is discarded and the whole string is formatted differently. When the string is a-ok, the data is copied over to the actual destination.

I'm not sure if that's the best approach, though. The temporary buffer and the string copy are costly.

There is also a closure being allocated for no reason in `sformat` itself. The compiler isn't smart enough to see that it's not really needed.

I've made a pull request to get rid of those allocations:
https://github.com/dlang/phobos/pull/7163
September 01, 2019
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:
> I've made a pull request to get rid of those allocations:
> https://github.com/dlang/phobos/pull/7163

Merged.
September 02, 2019
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:
> I've made a pull request to get rid of those allocations:
> https://github.com/dlang/phobos/pull/7163

Thanks for the responses, very cool seeing these updates happen so fluidly.
September 03, 2019
On 8/31/19 5:12 PM, ag0aep6g wrote:
> I've made a pull request to get rid of those allocations:
> https://github.com/dlang/phobos/pull/7163

Wonderful!
For my own learning, why was a unittest to ensure no GC added to sformat instead of a @nogc annotation?
September 03, 2019
On 03.09.19 16:03, James Blachly wrote:
> For my own learning, why was a unittest to ensure no GC added to sformat instead of a @nogc annotation?

`sformat` uses the GC less now, but it's not @nogc. It can still throw GC-allocated exceptions, e.g. when the arguments don't match the format string.