While up at night with jetlag at DConf, I started toying about solving a small problem. In order to use printf(), the format specifiers in the printf format string have to match the types of the rest of the parameters. This is well known to be brittle and error-prone, especially when refactoring the types of the arguments.

(Of course, this is not a problem with writefln() and friends, but that isn't available in the dmd front end, nor when using betterC. Making printf better would mesh nicely with betterC. Note that many C compilers have extensions to tell you if there's a mismatch, but they won't fix it for you.)

I thought why not use D's metaprogramming to fix it. Some ground rules:

1. No extra overhead
2. Completely self-contained
3. Only %s specifiers are rewritten
4. %% is handled
5. diagnose mismatch between number of specifiers and number of arguments

Here's my solution:

    int i;
    dprintf!"hello %s %s %s %s betty\n"(3, 4.0, &i, "abc".ptr);

gets rewritten to:

    printf("hello %d %g %p %s betty\n", 3, 4.0, &i, "abc".ptr);

The code at the end accomplishes this. Yay!

But what I'd like it to do is to extend it to convert a `string s` argument into `cast(int)s.length, s.ptr` tuple and use the "%.*s" specifier for it.

I completely failed at that. I suspect the language has a deficiency in manipulating expression tuples.

Does anyone see a way to make this work?

Note: In order to minimize template bloat, I refactored most of the work into a regular function, minimizing the size of the template expansions.

------ Das Code ------------
import core.stdc.stdio : printf;

template Seq(A ...) { alias Seq = A; }

int dprintf(string f, A ...)(A args)
{
    enum Fmts = Formats!(A);
    enum string s = formatString(f, Fmts);
    __gshared const(char)* s2 = s.ptr;
    return printf(Seq!(s2, args[0..2], args[2..4]));
}

template Formats(T ...)
{
    static if (T.length == 0)
	enum Formats = [ ];
    else static if (T.length == 1)
	enum Formats = [Spec!(T[0])];
    else
	enum Formats = [Spec!(T[0])] ~ Formats!(T[1 .. T.length]);
}

template Spec(T : byte)    { enum Spec = "%d"; }
template Spec(T : short)   { enum Spec = "%d"; }
template Spec(T : int)     { enum Spec = "%d"; }
template Spec(T : long)    { enum Spec = "%lld"; }

template Spec(T : ubyte)   { enum Spec = "%u"; }
template Spec(T : ushort)  { enum Spec = "%u"; }
template Spec(T : uint)    { enum Spec = "%u"; }
template Spec(T : ulong)   { enum Spec = "%llu"; }

template Spec(T : float)   { enum Spec = "%g"; }
template Spec(T : double)  { enum Spec = "%g"; }
template Spec(T : real)    { enum Spec = "%Lg"; }

template Spec(T : char)    { enum Spec = "%c"; }
template Spec(T : wchar)   { enum Spec = "%c"; }
template Spec(T : dchar)   { enum Spec = "%c"; }

template Spec(T : immutable(char)*)   { enum Spec = "%s"; }
template Spec(T : const(char)*)       { enum Spec = "%s"; }
template Spec(T : T*)                 { enum Spec = "%p"; }

/******************************************
 * Replace %s format specifiers in f with corresponding specifiers in A[].
 * Other format specifiers are left as is.
 * Number of format specifiers must match A.length.
 * Params:
 *	f = printf format string
 *	A = replacement format specifiers
 * Returns:
 *	replacement printf format string
 */
string formatString(string f, string[] A ...)
{
    string r;
    size_t i;
    size_t ai;
    while (i < f.length)
    {
	if (f[i] != '%' || i + 1 == f.length)
	{
	    r ~= f[i];
	    ++i;
	    continue;
	}
	char c = f[i + 1];
	if (c == '%')
	{
	    r ~= "%%";
	    i += 2;
	    continue;
	}
	assert(ai < A.length, "not enough arguments");
	string fmt = A[ai];
	++ai;
	if (c == 's')
	{
	    r ~= fmt;
	    i += 2;
	    continue;
	}
	r ~= '%';
	++i;
	continue;
    }
    assert(ai == A.length, "not enough formats");
    return r;
}
----- End Of Das Code ----------

On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
> While up at night with jetlag at DConf, I started toying about solving a small problem. In order to use printf(), the format specifiers in the printf format string have to match the types of the rest of the parameters. This is well known to be brittle and error-prone, especially when refactoring the types of the arguments.
>
> (Of course, this is not a problem with writefln() and friends, but that isn't available in the dmd front end, nor when using betterC. Making printf better would mesh nicely with betterC. Note that many C compilers have extensions to tell you if there's a mismatch, but they won't fix it for you.)
>
> I thought why not use D's metaprogramming to fix it. Some ground rules:
>
> 1. No extra overhead
> 2. Completely self-contained
> 3. Only %s specifiers are rewritten
> 4. %% is handled
> 5. diagnose mismatch between number of specifiers and number of arguments
>
> Here's my solution:
>
>     int i;
>     dprintf!"hello %s %s %s %s betty\n"(3, 4.0, &i, "abc".ptr);
>
> gets rewritten to:
>
>     printf("hello %d %g %p %s betty\n", 3, 4.0, &i, "abc".ptr);
>
> The code at the end accomplishes this. Yay!
>
> But what I'd like it to do is to extend it to convert a `string s` argument into `cast(int)s.length, s.ptr` tuple and use the "%.*s" specifier for it.
>
> I completely failed at that. I suspect the language has a deficiency in manipulating expression tuples.
>
> Does anyone see a way to make this work?
>
> Note: In order to minimize template bloat, I refactored most of the work into a regular function, minimizing the size of the template expansions.
>
> ------ Das Code ------------
> import core.stdc.stdio : printf;
>
> template Seq(A ...) { alias Seq = A; }
>
> int dprintf(string f, A ...)(A args)
> {
>     enum Fmts = Formats!(A);
>     enum string s = formatString(f, Fmts);
>     __gshared const(char)* s2 = s.ptr;
>     return printf(Seq!(s2, args[0..2], args[2..4]));
> }
>
> template Formats(T ...)
> {
>     static if (T.length == 0)
> 	enum Formats = [ ];
>     else static if (T.length == 1)
> 	enum Formats = [Spec!(T[0])];
>     else
> 	enum Formats = [Spec!(T[0])] ~ Formats!(T[1 .. T.length]);
> }
>
> template Spec(T : byte)    { enum Spec = "%d"; }
> template Spec(T : short)   { enum Spec = "%d"; }
> template Spec(T : int)     { enum Spec = "%d"; }
> template Spec(T : long)    { enum Spec = "%lld"; }
>
> template Spec(T : ubyte)   { enum Spec = "%u"; }
> template Spec(T : ushort)  { enum Spec = "%u"; }
> template Spec(T : uint)    { enum Spec = "%u"; }
> template Spec(T : ulong)   { enum Spec = "%llu"; }
>
> template Spec(T : float)   { enum Spec = "%g"; }
> template Spec(T : double)  { enum Spec = "%g"; }
> template Spec(T : real)    { enum Spec = "%Lg"; }
>
> template Spec(T : char)    { enum Spec = "%c"; }
> template Spec(T : wchar)   { enum Spec = "%c"; }
> template Spec(T : dchar)   { enum Spec = "%c"; }
>
> template Spec(T : immutable(char)*)   { enum Spec = "%s"; }
> template Spec(T : const(char)*)       { enum Spec = "%s"; }
> template Spec(T : T*)                 { enum Spec = "%p"; }
>
> /******************************************
>  * Replace %s format specifiers in f with corresponding specifiers in A[].
>  * Other format specifiers are left as is.
>  * Number of format specifiers must match A.length.
>  * Params:
>  *	f = printf format string
>  *	A = replacement format specifiers
>  * Returns:
>  *	replacement printf format string
>  */
> string formatString(string f, string[] A ...)
> {
>     string r;
>     size_t i;
>     size_t ai;
>     while (i < f.length)
>     {
> 	if (f[i] != '%' || i + 1 == f.length)
> 	{
> 	    r ~= f[i];
> 	    ++i;
> 	    continue;
> 	}
> 	char c = f[i + 1];
> 	if (c == '%')
> 	{
> 	    r ~= "%%";
> 	    i += 2;
> 	    continue;
> 	}
> 	assert(ai < A.length, "not enough arguments");
> 	string fmt = A[ai];
> 	++ai;
> 	if (c == 's')
> 	{
> 	    r ~= fmt;
> 	    i += 2;
> 	    continue;
> 	}
> 	r ~= '%';
> 	++i;
> 	continue;
>     }
>     assert(ai == A.length, "not enough formats");
>     return r;
> }
> ----- End Of Das Code ----------

It uses mixin, so not pretty, but it works...

void main()
{
    int i = 0;
    dprintf!"hello %s %s %s %s betty\n"(3, 4.0, &i, "abc".ptr);

    const msg = "AAA!";
    dprintf!"A dstring '%s'\n"(msg[0 .. 3]);
}

template Seq(A ...) { alias Seq = A; }

int dprintf(string f, A ...)(A args)
{
    import core.stdc.stdio : printf;

    enum Fmts = Formats!(A);
    enum string s = formatString(f, Fmts);
    __gshared const(char)* s2 = s.ptr;
    enum call = function() {
        import std.conv : to;
        string printfCall = "printf(s2";
        foreach(i, T; A)
        {
           static if (is(T : string))
           {
               printfCall ~= ", cast(size_t)args[" ~ i.to!string ~ "].length, args["
                    ~ i.to!string ~ "].ptr";
           }
           else
           {
               printfCall ~= ", args[" ~ i.to!string ~ "]";
           }

        }
        return printfCall ~ ")";
    }();
    //pragma(msg, call); // uncomment to see the final call
    return mixin(call);
}

template Formats(T ...)
{
    static if (T.length == 0)
	    enum Formats = [];
    else static if (T.length == 1)
	    enum Formats = [Spec!(T[0])];
    else
	    enum Formats = [Spec!(T[0])] ~ Formats!(T[1 .. T.length]);
}

template Spec(T : byte)    { enum Spec = "%d"; }
template Spec(T : short)   { enum Spec = "%d"; }
template Spec(T : int)     { enum Spec = "%d"; }
template Spec(T : long)    { enum Spec = "%lld"; }

template Spec(T : ubyte)   { enum Spec = "%u"; }
template Spec(T : ushort)  { enum Spec = "%u"; }
template Spec(T : uint)    { enum Spec = "%u"; }
template Spec(T : ulong)   { enum Spec = "%llu"; }

template Spec(T : float)   { enum Spec = "%g"; }
template Spec(T : double)  { enum Spec = "%g"; }
template Spec(T : real)    { enum Spec = "%Lg"; }

template Spec(T : char)    { enum Spec = "%c"; }
template Spec(T : wchar)   { enum Spec = "%c"; }
template Spec(T : dchar)   { enum Spec = "%c"; }
template Spec(T : string)  { enum Spec = "%.*s"; }

template Spec(T : immutable(char)*)   { enum Spec = "%s"; }
template Spec(T : const(char)*)       { enum Spec = "%s"; }
template Spec(T : T*)                 { enum Spec = "%p"; }

/******************************************
 * Replace %s format specifiers in f with corresponding specifiers in A[].
 * Other format specifiers are left as is.
 * Number of format specifiers must match A.length.
 * Params:
 *	f = printf format string
 *	A = replacement format specifiers
 * Returns:
 *	replacement printf format string
 */
string formatString(string f, string[] A ...)
{
    string r;
    size_t i;
    size_t ai;
    while (i < f.length)
    {
	if (f[i] != '%' || i + 1 == f.length)
	{
	    r ~= f[i];
	    ++i;
	    continue;
	}
	char c = f[i + 1];
	if (c == '%')
	{
	    r ~= "%%";
	    i += 2;
	    continue;
	}
	assert(ai < A.length, "not enough arguments");
	string fmt = A[ai];
	++ai;
	if (c == 's')
	{
	    r ~= fmt;
	    i += 2;
	    continue;
	}
	r ~= '%';
	++i;
	continue;
    }
    assert(ai == A.length, "not enough formats");
    return r;
}

On Thursday, 23 May 2019 at 22:48:33 UTC, Jonathan Marler wrote:
> It uses mixin, so not pretty, but it works...

Similar solution:

--- printf.d	2019-05-24 00:48:44.840543714 +0200
+++ printf_s.d	2019-05-24 00:52:47.829178613 +0200
@@ -1,13 +1,12 @@
 import core.stdc.stdio : printf;

-template Seq(A ...) { alias Seq = A; }
-
 int dprintf(string f, A ...)(A args)
 {
     enum Fmts = Formats!(A);
     enum string s = formatString(f, Fmts);
+    alias args_ = Args!(args);
     __gshared const(char)* s2 = s.ptr;
-    return printf(Seq!(s2, args[0..2], args[2..4]));
+    mixin( q{return printf(s2, } ~ args_ ~ q{);} );
 }

 template Formats(T ...)
@@ -42,6 +41,22 @@
 template Spec(T : const(char)*)       { enum Spec = "%s"; }
 template Spec(T : T*)                 { enum Spec = "%p"; }

+template Spec(T : string)  { enum Spec = "%.*s"; }
+
+template Args(A ...)
+{
+    static if (A.length == 0)
+	enum Args = "";
+    else static if (A.length == 1)
+	enum Args = Arg!(A[0]);
+    else
+	enum Args = Arg!(A[0]) ~ ", " ~ Args!(A[1 .. A.length]);
+}
+
+template Arg(alias string arg) { enum Arg = "cast(int)"~arg.stringof~".length,"~arg.stringof~".ptr"; }
+
+template Arg(alias arg) { enum Arg = arg.stringof; }
+
 /******************************************
  * Replace %s format specifiers in f with corresponding specifiers in A[].
  * Other format specifiers are left as is.

On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
> [snip]
>
> I completely failed at that. I suspect the language has a deficiency in manipulating expression tuples.
>
> 

What a coincidence, I had this exact problem today as well. It seems currently the only way to do this is either with mixins, or using tuple.

Assuming you already have all of the arguments in a tuple:

    auto args = tuple(...);

And args[x] is a string, you can do this:

    auto args_prime = tuple(args[0..x], args[x].length, args[x].ptr, args[x..$]);

You then need to do some template magic to expand all such arguments... Using mixin is probably better.

> template Spec(T : byte)    { enum Spec = "%d"; }
> template Spec(T : short)   { enum Spec = "%d"; }
> template Spec(T : int)     { enum Spec = "%d"; }
> template Spec(T : long)    { enum Spec = "%lld"; }
>
> template Spec(T : ubyte)   { enum Spec = "%u"; }
> template Spec(T : ushort)  { enum Spec = "%u"; }
> template Spec(T : uint)    { enum Spec = "%u"; }
> template Spec(T : ulong)   { enum Spec = "%llu"; }
>
> template Spec(T : float)   { enum Spec = "%g"; }
> template Spec(T : double)  { enum Spec = "%g"; }
> template Spec(T : real)    { enum Spec = "%Lg"; }
>
> template Spec(T : char)    { enum Spec = "%c"; }
> template Spec(T : wchar)   { enum Spec = "%c"; }
> template Spec(T : dchar)   { enum Spec = "%c"; }
> template Spec(T : string)  { enum Spec = "%.*s"; }
>
> template Spec(T : immutable(char)*)   { enum Spec = "%s"; }
> template Spec(T : const(char)*)       { enum Spec = "%s"; }
> template Spec(T : T*)                 { enum Spec = "%p"; }

this can all be simplified:

static foreach(k,v: ["byte":"%d", "short":"%d", ...])
    mixin(`template Spec(T : `~k~`) { enum Spec = "`~v~`"; }`);

The string mixin is not necessary but easier than an aliasSeq.

On 23.05.19 21:33, Walter Bright wrote:
> But what I'd like it to do is to extend it to convert a `string s` argument into `cast(int)s.length, s.ptr` tuple and use the "%.*s" specifier for it.
[...]
> Does anyone see a way to make this work?

I don't know if this satisfies the "no extra overhead" rule. Maybe when `arrlen` and `arrptr` are inlined?

int dprintf(string f, A ...)(A args)
{
    enum Fmts = Formats!(A);
    enum string s = formatString(f, Fmts);
    __gshared const(char)* s2 = s.ptr;
    import std.meta: staticMap;
    return printf(Seq!(s2, staticMap!(arg, args)));
}

template arg(alias a)
{
    static if (is(typeof(a) == string))
        alias arg = Seq!(arrlen!a, arrptr!a);
    else alias arg = a;
}

auto arrlen(alias a)() { return a.length; }
auto arrptr(alias a)() { return a.ptr; }

template Spec(T : string) { enum Spec = "%.*s"; }

void main()
{
    int i;
    dprintf!"hello %s %s %s %s betty %s\n"(3, 4.0, &i,
        "abc".ptr, "foobar");
}

// ... rest of the code unchanged ...

On 5/23/19 6:58 PM, Yuxuan Shui wrote:
> On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
>> [snip]
>>
>> I completely failed at that. I suspect the language has a deficiency in manipulating expression tuples.
>>
>>
> 
> What a coincidence, I had this exact problem today as well. It seems currently the only way to do this is either with mixins, or using tuple.
> 
> Assuming you already have all of the arguments in a tuple:
> 
>      auto args = tuple(...);
> 
> And args[x] is a string, you can do this:
> 
>      auto args_prime = tuple(args[0..x], args[x].length, args[x].ptr, args[x..$]);
> 
> You then need to do some template magic to expand all such arguments... Using mixin is probably better.

Did you try .expand with the tuple?

On Friday, 24 May 2019 at 00:41:31 UTC, Andrei Alexandrescu wrote:
> On 5/23/19 6:58 PM, Yuxuan Shui wrote:
>> On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
>>> [snip]
>>>
>>> I completely failed at that. I suspect the language has a deficiency in manipulating expression tuples.
>>>
>>>
>> 
>> What a coincidence, I had this exact problem today as well. It seems currently the only way to do this is either with mixins, or using tuple.
>> 
>> Assuming you already have all of the arguments in a tuple:
>> 
>>      auto args = tuple(...);
>> 
>> And args[x] is a string, you can do this:
>> 
>>      auto args_prime = tuple(args[0..x], args[x].length, args[x].ptr, args[x..$]);
>> 
>> You then need to do some template magic to expand all such arguments... Using mixin is probably better.
>
> Did you try .expand with the tuple?

It's 1 character shorter to just write someTuple[0..$] :)

On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
> string formatString(string f, string[] A ...)
> {
>     string r;
>     size_t i;
>     size_t ai;
>     while (i < f.length)
>     {
> 	if (f[i] != '%' || i + 1 == f.length)
> 	{
> 	    r ~= f[i];


Are you sure this works for betterC? It's been a while for me, but I think it won't, the string appends will stop it.

Good job at getting unit tests and final switch in!

> ----- End Of Das Code ----------

On Friday, 24 May 2019 at 04:49:22 UTC, bpr wrote:
> On Thursday, 23 May 2019 at 19:33:15 UTC, Walter Bright wrote:
>> string formatString(string f, string[] A ...)
>> {
>>     string r;
>>     size_t i;
>>     size_t ai;
>>     while (i < f.length)
>>     {
>> 	if (f[i] != '%' || i + 1 == f.length)
>> 	{
>> 	    r ~= f[i];
>
>
> Are you sure this works for betterC? It's been a while for me, but I think it won't, the string appends will stop it.
>
> Good job at getting unit tests and final switch in!
>
>> ----- End Of Das Code ----------

Indeed it doesn't work with -betterC flag. Easily testable on run.dlang.io

This probably would work if CTFE was supported when compiling with betterC.