Jump to page: 1 2
Thread overview
std.conv.to!string(float) Is too heavy
Oct 12, 2021
Hipreme
Oct 12, 2021
IGotD-
Oct 12, 2021
user1234
Oct 12, 2021
Hipreme
Oct 12, 2021
IGotD-
Oct 12, 2021
H. S. Teoh
Oct 12, 2021
Hipreme
Oct 12, 2021
H. S. Teoh
Oct 14, 2021
Hipreme
Oct 14, 2021
H. S. Teoh
Oct 12, 2021
Stefan Koch
Oct 12, 2021
Hipreme
Oct 12, 2021
russhy
Oct 12, 2021
Hipreme
Oct 14, 2021
Ogi
October 12, 2021

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.
The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

October 12, 2021

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

I've seen similar effects, not only with std.conv but with other modules as well. You just use a small part of one module but the compiler imports a lot of code.

It would be very interesting to know what happens underneath and how we can reduce the code bloat.

We should investigate what happens and we can use your example as case study.

October 12, 2021

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.

You compare apples and oranges. Convertion from integer types to string is trivial, converting float types to string is not.

That being said there's some room for improvment. writeln of a float will take the same path as format(%f, v), which contains the code to format float in every possible way, as the specifier is not known at compile time, and finally only a small part of the bloat generated is executed.

This is bit strange, as for writeln, the specifier is known at compile time.

>

The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

October 12, 2021

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.
The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

writeln itself even for plain strings is already quite heavy.
I'd recommend using printf.

I also have a realtively lightweight dtoa function.
it's here:
https://raw.githubusercontent.com/UplinkCoder/fpconv/master/src/fpconv_ctfe.d

October 12, 2021

On Tuesday, 12 October 2021 at 16:05:19 UTC, Stefan Koch wrote:

>

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.
The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

writeln itself even for plain strings is already quite heavy.
I'd recommend using printf.

I also have a realtively lightweight dtoa function.
it's here:
https://raw.githubusercontent.com/UplinkCoder/fpconv/master/src/fpconv_ctfe.d

I'm not defending writeln (specially because in my code it causes 1500KB increase, but I could not reproduce in a simple hello world that increase), but the problem seems that std.conv for floats seems to consistently do that

October 12, 2021

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.
The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

try to build with:

"dflags-ldc": [
    "-linkonce-templates",
],

i'd personally suggest using snprintf, but depending on your usecase that might not be perfect

char[32] tmp = 0;
snprintf(tmp.ptr, tmp.length, "Hello World %f", value);
October 12, 2021

On Tuesday, 12 October 2021 at 17:35:31 UTC, russhy wrote:

>

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.
The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

try to build with:

"dflags-ldc": [
    "-linkonce-templates",
],

i'd personally suggest using snprintf, but depending on your usecase that might not be perfect

char[32] tmp = 0;
snprintf(tmp.ptr, tmp.length, "Hello World %f", value);

snprintf can't be called at compile time, so, it can miss a lot.
That flag didn't seem to do something

October 12, 2021

On Tuesday, 12 October 2021 at 16:04:49 UTC, user1234 wrote:

>

On Tuesday, 12 October 2021 at 14:47:21 UTC, Hipreme wrote:

>

I have wrote a simple hello world using std.conv:

import std.conv:to;
import std.stdio;

export void MAIN(float b)
{
	writeln("Hello World " ~ to!string(b));
}

Building that with dmd app.d -lib creates 500KB lib
With -g, it goes up to 800KB

Using ldc, the build time goes up by 2 times.

if you simply change from float b to int b, the lib goes down from 500KB to 60KB.

You compare apples and oranges. Convertion from integer types to string is trivial, converting float types to string is not.

That being said there's some room for improvment. writeln of a float will take the same path as format(%f, v), which contains the code to format float in every possible way, as the specifier is not known at compile time, and finally only a small part of the bloat generated is executed.

This is bit strange, as for writeln, the specifier is known at compile time.

>

The build times decreases by almost 10x (must check), but I can feel that anyway.

Using std.conv greatly degrades a modularized workflow, by increasing a lot of build time and binary size. I needed implementing my own to! function to get a much better build time for my program.

Lot of people has implemented float to string conversion without requiring 500KB.
I really don't think I need all that 'runtime speed' nor 'ultra precision', it would be great if we had some compiler flag to stop this template bloating thing for we can choose for build speed instead.

Yea, writeln does cause 500KB requirement for floats and structs too (without float members). I guess it is much more template hell than any other thing.

Yea, I forced it for not inlining my number, because, that's the point, most of the time you won't be inlining a number.

I have like, more than 8 modules, some depending on each other, this build time can increase a LOT linking time and the compile time. Usually, the modularized workflow should make things faster and reuse already compiled code. If the stdlib itself weights so much, the modularized workflow will start to feel useless

October 12, 2021

On Tuesday, 12 October 2021 at 18:06:53 UTC, Hipreme wrote:

>

Lot of people has implemented float to string conversion without requiring 500KB.

Floats to string is of course more complicated that integers but 500KB is ridiculous. It should rather be something like 10-20 KB.

It's likely it is a combination of that the optimizer/linker is unable to remove unused functions. Also that imports usually imports more stuff, even if it is just a tiny function from another import.

It's like a box of cables, you just want one cable but when you pull out one cable you get the entire ball of cables.

October 12, 2021
On Tue, Oct 12, 2021 at 07:08:44PM +0000, IGotD- via Digitalmars-d wrote:
> On Tuesday, 12 October 2021 at 18:06:53 UTC, Hipreme wrote:
> > 
> > Lot of people has implemented float to string conversion without requiring 500KB.
> 
> Floats to string is of course more complicated that integers but 500KB is ridiculous. It should rather be something like 10-20 KB.
> 
> It's likely it is a combination of that the optimizer/linker is unable to remove unused functions. Also that imports usually imports more stuff, even if it is just a tiny function from another import.
> 
> It's like a box of cables, you just want one cable but when you pull out one cable you get the entire ball of cables.

It'd be nice to untangle that ball of cables so that less cables get pulled out when you tug at one (so to speak). :-D  We talked about pay-as-you-go modularization of Phobos at least a few times, but so far, that ball of cables persist.


T

-- 
Never ascribe to malice that which is adequately explained by incompetence. -- Napoleon Bonaparte
« First   ‹ Prev
1 2