After reading most of the tour.dlang.org website, I was completely surprised & confused encountering 'KeepTerminator', a 'Flag' used by the File.byLine function. With no examples denoting how to use it.

Most confusing was the way the documentation (website & in-editor) used;
1. Yes.keepTerminator
2. KeepTerminator.yes
3. Flag!"keepTerminator".yes
& Don't get me started on the autocomplete trying to get me to use KeepTerminator.Flag.yes (VSCode & code-d)

Now some documentation stated these flags are used to improve readability, when compared to simple booleans, but all these different notations seem super confusing, especially since I dont understand why Flag requires a !"Name" part.
And I haven't read anything about it in the language tour, nor have I found any discussion on the topic elsewhere.
I did find, for some reason, a second 'KeepTerminator' documentation page . . . which is from std.string, which understandably did not help clear things up.

Even further confused by the in-editor documentation, is this function a template?:
"
byLine(Terminator = char, Char = char) (KeepTerminator keepTerminator = No,keepTerminator, Terminator terminator = '\\n')
"

I'm sorry, I may be asking for too much, but I've got so many questions and found few answers.
1. Did I miss some tutorial?
2. Why does flag require a name (!"Name")?
3. Is byLine a template, if so, how would one use it differently?

On Saturday, 19 December 2020 at 23:16:00 UTC, Rekel wrote:
> After reading most of the tour.dlang.org website, I was completely surprised & confused encountering 'KeepTerminator', a 'Flag' used by the File.byLine function. With no examples denoting how to use it.
>
> Most confusing was the way the documentation (website & in-editor) used;
> 1. Yes.keepTerminator
> 2. KeepTerminator.yes
> 3. Flag!"keepTerminator".yes
> & Don't get me started on the autocomplete trying to get me to use KeepTerminator.Flag.yes (VSCode & code-d)

Flag is a templated enum, documented here:

http://phobos.dpldocs.info/std.typecons.Flag.html

It has two members, Flag!"name".yes and Flag!"name".no. As explained in the documentation linked above, Yes and No are structs that provide (slightly) less verbose aliases for these enum members. The documentation for Yes and No is here:

http://phobos.dpldocs.info/std.typecons.Yes.html
http://phobos.dpldocs.info/std.typecons.No.html

> Even further confused by the in-editor documentation, is this function a template?:
> "
> byLine(Terminator = char, Char = char) (KeepTerminator keepTerminator = No,keepTerminator, Terminator terminator = '\\n')
> "

Yes, it is a template. If you look at the section on "parameters" in the documentation

http://phobos.dpldocs.info/std.stdio.File.byLine.1.html#parameters

...you can see that the template parameter Char is the character type used for the lines. So if you wanted to have lines of wchar instead of char, you could use `.byLine!(wchar, wchar)`.

On Saturday, 19 December 2020 at 23:16:00 UTC, Rekel wrote:

>
> Most confusing was the way the documentation (website & in-editor) used;
> 1. Yes.keepTerminator
> 2. KeepTerminator.yes
> 3. Flag!"keepTerminator".yes

Your confusion arises from the fact that KeepTerminator is combining multiple distinct D features to achieve its goal.

As Paul said, std.typecons.Flag is a templated enum of type bool. It's declared to take a string as its template parameter, and it has two fields: yes, and no. Minus the documentation:

template Flag(string name) {
    ///
    enum Flag : bool
    {
        no = false,
        yes = true
    }
}

The template parameter serves to make Flag!"foo" a distinct type from Flag!"bar".

Now, the goal of Flag is to make the purpose of a boolean function parameter more clear. Sometimes, we can misremember what a boolean parameter indicates. Does true mean do this extra thing or does it mean don't do this extra thing? Flag removes the doubt.

However, having to write Flag!"keepTerminator".yes all the time is more annoying that simply writing true, so functions that use Flag usually have a corresponding alias defined in the module scope to give it a less annoying syntax:

alias KeepTerminator = Flag!"keepTerminator";

Because of this, you can write KeepTerminator.yes and KeepTerminator.no.

std.typecons also has two structs: Yes and No. Both are implemented with a bit of template magic. D supports a feature called "Forwarding" in structs and classes and implements it via a special template called opDispatch:

https://dlang.org/spec/operatoroverloading.html#dispatch

The idea is that if you call a member function on a struct or class, and the class does not have a member function of that name, then the compiler will look for an opDispatch template implementation in that class or struct. If it finds one, it will call the template with the name of the missing function.

There are a number of use cases for this, if you look at the examples in the documentation of opDispatch, you'll find that there are two examples of implementing it as a function template and one that looks like this:

struct D
{
    template opDispatch(string s)
    {
        enum int opDispatch = 8;
    }
}

This impelemtation is an enum template that's essentially creating an enum with a single member, also called a "manifest constant" (a compile-time constant):

https://dlang.org/spec/enum.html#manifest_constants

This is an eponymous template, which means it can be accessed directly as opDispatch without using dot notation on the template name (opDispatch.opDispatch). So given `d` of type `D`, when the compiler sees `d.foo`, it looks for `foo` in the `D` struct. It doesn't find it, but it does find `opDisptach`, so it instantiated `d.opDispatch!"foo"` which, in this case, produces the number `8` as a compile-time constant.

Both the Yes and No structs use this technique:

struct Yes
{
    template opDispatch(string name)
    {
        enum opDispatch = Flag!name.yes;
    }
}

So when you call Yes.keepTerminator, you're getting Flag!"keepTerminator".yes as a result.

The point behind the structs is so that people who make use of Flag in their function parameter lists don't need to actually define an alias:

"The structs Yes and No are provided as shorthand for Flag!"Name".yes and Flag!"Name".no and are preferred for brevity and readability. These convenience structs mean it is usually unnecessary and counterproductive to create an alias of a Flag as a way of avoiding typing out the full type while specifying the affirmative or negative options."

So when implementing your function with a Flag!"foo", you can skip the alias and users can call the function with Yes.foo and No.foo.

However, I believe that the aliases for KeepTerminator in std.string (for splitLines and lineSplitter) and std.stdio (for byLine and byLineCopy) predate the implementation of the Yes and No structs in std.typecons, but were kept around so as not to break any code.

So, to summarize:

> 1. Yes.keepTerminator

This is because of Yes is a struct with an opDispatch template that "forwards" to Flag!"keepTerminator".yes. This is the preferred syntax and will work with any Flag parameter.

> 2. KeepTerminator.yes

This is because KeepTerminator is an alias to Flag!"keepTerminator". This syntax will only work on any given Flag parameter if the function implementer defines the alias.

> 3. Flag!"keepTerminator".yes

This is because Flag is a templated enum that takes a string parameter and has two members: yes and no. This always works, because it's the root feature for which the above two syntaxes were implemented as conveniences.

> & Don't get me started on the autocomplete trying to get me to use KeepTerminator.Flag.yes (VSCode & code-d)

code-d uses DScanner to implement autocompletion. It can get confused in certain instances when compile-time features are involved. If there isn't an issue on this yet, you should report it:

https://github.com/dlang-community/D-Scanner/issues

On 12/19/20 4:40 PM, Mike Parker wrote:

>> 1. Yes.keepTerminator
>
> This is because of Yes is a struct with an opDispatch template that
> "forwards" to Flag!"keepTerminator".yes. This is the preferred syntax
> and will work with any Flag parameter.

I use Flag a lot but I am always bugged by how ugly the !"foo" part is especially compared to Yes.foo. A section I had removed from my DConf presentation asked whether we could add opDispatch to templates as well. That would allow us to say Flag.foo.

I don't know how we could fit it in the syntax but it could be something like this:

template Flag() {
  auto opDispatch(string s)() {
    alias opDispatch = FlagImpl!s;
  }
}

Another thought that came to me to solve the same issue was to allow string template parameters without needing to write the double quotes:

// Re-purposing the 'static' keyword for fun. :)
template Flag(static string s) {
  // ...
}

So we could either write Flag!"foo" or Flag!foo. Similar to how opDispatch would convert unknown symbols to strings. Perhaps like this?

template Flag(opDispatch s) {
  // ...
}

Ali

Thanks for all the help! This makes it make a lot more sense now, I'm surprised it's not part of the dlang tour.

> The template parameter serves to make Flag!"foo" a distinct type from Flag!"bar".
Does this mean other flag yes's will not be accepted?

> https://dlang.org/spec/operatoroverloading.html#dispatch
Also regarding the other examples given, why does Phobos use templating so heavily, in situation in which I am totally confused as to why generics would be necessary.
I seem to be totally confused as to how this template system works. It was introduced as a kind of generic, like in Java, but things like 'string s' seem to me like parameters.
For example;
```
class C
{
    void opDispatch(string s)(int i)
    {
        writefln("C.opDispatch('%s', %s)", s, i);
    }
}
```
I'm pretty sure I'm confusing something, though I don't see the point of using this instead of something like 'opDispatch(string s, int i)`?
I also came across a similar thing in the File.byLine documentation.
(https://dlang.org/library-prerelease/std/stdio/file.by_line.html)

On Sunday, 20 December 2020 at 14:07:56 UTC, Rekel wrote:

>> The template parameter serves to make Flag!"foo" a distinct type from Flag!"bar".
> Does this mean other flag yes's will not be accepted?

Yes.

>
>> https://dlang.org/spec/operatoroverloading.html#dispatch
> Also regarding the other examples given, why does Phobos use templating so heavily, in situation in which I am totally confused as to why generics would be necessary.
> I seem to be totally confused as to how this template system works. It was introduced as a kind of generic, like in Java, but things like 'string s' seem to me like parameters.

Java's generics are good for what they do, bolted onto an existing language in a non-breaking way as they were, but they are a pale shadow of a real metaprogramming system. D's template metaprogramming is not anything like Java generics. It's more akin to C++ templates---much more powerful than what Java offers you.

If Java's generics are all you know, then breaking through your confusion and answering your questions is going to require more than a forum post. You'll want to read up and get some hands on.

I have an introductory post on the D blog which is the first in a series (that I'll have finally have time to continue in the new year). It goes over the very, very basics:

https://dlang.org/blog/2020/07/31/the-abcs-of-templates-in-d/

After you read that introduction, you should look into Ali's coverage of templates in 'Programming in D':

http://ddili.org/ders/d.en/

When you're ready for more, Phillippe Sigaud's tutorial is excellent (though several old, it's still mostly relevant):

https://github.com/PhilippeSigaud/D-templates-tutorial

On 12/20/20 9:07 AM, Rekel wrote:
> Does this mean other flag yes's will not be accepted?

The long and short of it is that Flag is supposed to make you NAME what your flag is for.

The idea is to prevent things like:

byLine(true);

Reading the code, what does "true" mean? You have to look up the documentation to see what it is. And then you might have things like:

foo(true, true, false, true);

What do all those mean?

Instead, you have to name what the flag is for:

byLine(KeepTerminator.yes);

The Yes/No structs and Flag structs are there to help you name your flag, and have some awkward D way to do this.

This will likely all go away in the future with named parameters (deprecating Flag is one of the reasons named parameters were proposed/accepted).

I personally avoid Flag in my code because I think it's awkward and cumbersome.

-Steve

On Sunday, 20 December 2020 at 15:04:29 UTC, Steven Schveighoffer wrote:
> On 12/20/20 9:07 AM, Rekel wrote:
>> Does this mean other flag yes's will not be accepted?
>
> The long and short of it is that Flag is supposed to make you NAME what your flag is for.
>
> The idea is to prevent things like:
>
> byLine(true);
>
> Reading the code, what does "true" mean? You have to look up the documentation to see what it is. And then you might have things like:
>
> foo(true, true, false, true);
>
> What do all those mean?
>
> Instead, you have to name what the flag is for:
>
> byLine(KeepTerminator.yes);
>
> The Yes/No structs and Flag structs are there to help you name your flag, and have some awkward D way to do this.
>
> This will likely all go away in the future with named parameters (deprecating Flag is one of the reasons named parameters were proposed/accepted).
>
> I personally avoid Flag in my code because I think it's awkward and cumbersome.
>
> -Steve

With named parameters, do you refer to python-esque function calls?
That makes a lot more sense to me personally, although I'm only just learning D, as I likewise wouldnt use `Number!"Wheels".4` for a function call.

By the way, where can I see Flag is (/ will be?) deprecated? It doesn't show in the library reference, however I may be looking in the wrong place.

On Sunday, 20 December 2020 at 15:18:44 UTC, Rekel wrote:

>
> By the way, where can I see Flag is (/ will be?) deprecated? It doesn't show in the library reference, however I may be looking in the wrong place.

It hasn't been yet.