Thread overview
Detecting ElementType of OutputRange
Feb 26, 2022
Vijay Nayar
Feb 26, 2022
Stanislav Blinov
Feb 26, 2022
Vijay Nayar
Feb 26, 2022
Stanislav Blinov
Feb 26, 2022
Vijay Nayar
February 26, 2022

I was working on a project where it dealt with output ranges, but these ranges would ultimately sink into a source that would be inefficient if every single .put(T) call was made one at a time.

Naturally, I could make a custom OutputRange for just this resource, but I also got the idea that I could make a generalized BufferedOutputRange that would save individual .put(T) calls into memory until a threshold is reached, and then make one bulk call to the output stream it wraps with a single .put(T[]) call.

While working on the template for this buffering OutputRange, I originally used ElementType on the output range I was given, hoping to detect what type of .put(T) is permitted. However, I found out that ElementType only works for input ranges as you can see here: https://github.com/dlang/phobos/blob/6bf43144dbe956cfc16c00f0bff7a264fa62408e/std/range/primitives.d#L1265

Trying to find a workaround, I ultimately created this, and my question is, is using such a template a good idea or a terrible idea? Is it safe to assume that ranges should have a put method that may take arrays that I can detect? Should I give up on the idea of detecting the OutputRange type, and instead require the programmer to explicitly declare the output type for fear of them using a range that doesn't take arrays?

Here is the source of what I was thinking of, let me know your thoughts:

import std.range;
import std.traits;

// A specialization of std.range.ElementType which also considers output ranges.
template ElementType(R)
if  (is(typeof(R.put) == function))  // Avoid conflicts with std.range.ElementType.
{
  // Static foreach generates code, it is not a true loop.
  static foreach (t; __traits(getOverloads, R, "put")) {
    pragma(msg, "Found put method, params=", Parameters!(t).length);
    // Because all code gets generated, we use a 'done' alias to
    // tell us when to stop.
    static if (!is(done)) {
      // Attempts to save Parameters!(t) into a variable fail, so it is repeated.
      static if (Parameters!(t).length == 1 && is(Parameters!(t)[0] T : T[])) {
        pragma(msg, "put for array found");
        // Setting the name of the template replaces calls
        // to ElementType!(...) with T.
        alias ElementType = T;
        alias done = bool;
      } else static if (Parameters!(t).length == 1 && is(Parameters!(t)[0] T)) {
        pragma(msg, "put for single found");
        alias ElementType = T;
        alias done = bool;
      }
    }
  }
  static if (!is(done)) {
    alias ElementType = void;
  }
}

unittest {
  // Works for simple 1-element puts for structs.
  struct Ham0(T) {
    void put(T d) {}
  }
  assert(is(ElementType!(Ham0!float) == float));

  // Works for classes too, which have array-based puts.
  class Ham1(T) {
    void put(T[] d) {}
  }
  assert(is(ElementType!(Ham1!float) == float));

  // Distracting functions are ignored, and if single & array
  // puts are supported, the element type is still correct.
  struct Ham2(T) {
    void put() {}
    void put(float f, T[] d) {}
    void put(T[] d) {}
    void put(T d) {}
  }
  assert(is(ElementType!(Ham2!int) == int));
}
February 26, 2022

https://dlang.org/phobos/std_range_primitives.html#isOutputRange

February 26, 2022

On Saturday, 26 February 2022 at 11:44:35 UTC, Stanislav Blinov wrote:

>

https://dlang.org/phobos/std_range_primitives.html#isOutputRange

This method requires the caller to explicitly declare the output range element type, which I was hoping to have to avoid, if it can be detected using reasonable assumptions.

February 26, 2022

On Saturday, 26 February 2022 at 12:26:21 UTC, Vijay Nayar wrote:

>

On Saturday, 26 February 2022 at 11:44:35 UTC, Stanislav Blinov wrote:

>

https://dlang.org/phobos/std_range_primitives.html#isOutputRange

This method requires the caller to explicitly declare the output range element type, which I was hoping to have to avoid, if it can be detected using reasonable assumptions.

Considering that put is quite typically implemented as a template, I don't think that would be possible in general.

The question is, do you really need that? Your BufferedOutputRange can test the underlying range using isOutputRange in its own implementation of put, where the type of element is known, i.e. to test whether it can bulk-write a slice (or a range of) elements or has to make per-element calls to put.

February 26, 2022

On Saturday, 26 February 2022 at 12:39:51 UTC, Stanislav Blinov wrote:

>

Considering that put is quite typically implemented as a template, I don't think that would be possible in general.

That is what I found as well, for example, the implementation of put from Appender and RefAppender, thus the algorithm fails to detect the put methods.

>

The question is, do you really need that? Your BufferedOutputRange can test the underlying range using isOutputRange in its own implementation of put, where the type of element is known, i.e. to test whether it can bulk-write a slice (or a range of) elements or has to make per-element calls to put.

This is exactly what I was doing, having the user pass in the element type themselves, and then the BufferedOutputRange check for both isOutputRange!(ORangeT, ElemT) and is(typeof(ORangeT.init.put([ ElemT.init ]))). In short, isOutputRange has many ways to be satisfied, (https://dlang.org/phobos/std_range_primitives.html#.put), but I want to restrict things further to only output streams that can do a bulk put.

Thanks for you advice. While automatic detection would be a nice convenience, it's not a deal breaker and it's also not unreasonable to expect the caller to know the element type they are inserting either.