Thread overview
Please help me understand this function signature: std.stdio.File.byLine
Jul 15, 2021
Scotpip
Jul 15, 2021
Tejas
Jul 15, 2021
Scotpip
Jul 15, 2021
H. S. Teoh
Jul 15, 2021
Scotpip
Jul 15, 2021
H. S. Teoh
Jul 15, 2021
Scotpip
Jul 15, 2021
jfondren
July 15, 2021

Hi folks

Settling down to write my first app but have come to a grinding halt. This is my first system-level language so I'm afraid I'll be asking some naïve questions.

All I'm trying to do is read a text file with Windows line endings into an array for line-by-line processing.

The relevant function appears to be std.stdio.File.byLine.

The default isn't breaking the lines properly, so I have to pass in the line ending. But the signature has me baffled:

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator = No.keepTerminator,
  Terminator terminator = '\x0a'
)
if (isScalarType!Terminator);

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator,
  Terminator terminator
)
if (is(immutable(ElementEncodingType!Terminator) == immutable(Char)));

To specify the line ending, it appears to be asking for a type "Terminator" which I can't find in the library. Doing the obvious doesn't work:

// Err: cannot deduce function from argument types
myFile.byLine(`\r\n`);

Also, how would you specify the Char type as a parameter?

The examples are unenlightening as they only show the default case, so any help would be much appreciated. Any wider tips on how to read these cryptic signatures would be a bonus!

July 15, 2021

On Thursday, 15 July 2021 at 18:08:45 UTC, Scotpip wrote:

>

Hi folks

Settling down to write my first app but have come to a grinding halt. This is my first system-level language so I'm afraid I'll be asking some naïve questions.

All I'm trying to do is read a text file with Windows line endings into an array for line-by-line processing.

The relevant function appears to be std.stdio.File.byLine.

The default isn't breaking the lines properly, so I have to pass in the line ending. But the signature has me baffled:

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator = No.keepTerminator,
  Terminator terminator = '\x0a'
)
if (isScalarType!Terminator);

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator,
  Terminator terminator
)
if (is(immutable(ElementEncodingType!Terminator) == immutable(Char)));

To specify the line ending, it appears to be asking for a type "Terminator" which I can't find in the library. Doing the obvious doesn't work:

// Err: cannot deduce function from argument types
myFile.byLine(`\r\n`);

Also, how would you specify the Char type as a parameter?

The examples are unenlightening as they only show the default case, so any help would be much appreciated. Any wider tips on how to read these cryptic signatures would be a bonus!

Sorry I'm on mobile right now so can't help much, but if you're a beginner, please read the book "programming in D" by Ali Cehreli.

http://ddili.org/ders/d.en/

If you just want to learn about files for now, visit this link, it contains a chapter of his book(regrettably it doesn't cover your exact usecase but maybe readln might work out for you)

http://ddili.org/ders/d.en/files.html

July 15, 2021

On Thursday, 15 July 2021 at 18:36:30 UTC, Tejas wrote:

>

Sorry I'm on mobile right now so can't help much, but if you're a beginner, please read the book "programming in D" by Ali Cehreli.

http://ddili.org/ders/d.en/

If you just want to learn about files for now, visit this link, it contains a chapter of his book(regrettably it doesn't cover your exact usecase but maybe readln might work out for you)

http://ddili.org/ders/d.en/files.html

Thanks - I'm aware of the book - it's what gave me the confidence to have a go at D - but as you say it doesn't cover this use-case. I also have the old Alexandrescu book but it doesn't cover this either.

I'm new to systems work, but I'm not new to programming - I wrote my first programme on punch cards... I must have learned a dozen languages over the years, but this signature is the most cryptic I've ever encountered. Maybe if you're a C++ maven it makes sense, but it's pretty opaque to anyone else. When the examples are thin it really does make it hard for people trying to work their way into the community. So specific help on the issue of understanding this signature and reading in a Windows text file would still be very much appreciated.

July 15, 2021
On Thu, Jul 15, 2021 at 06:08:45PM +0000, Scotpip via Digitalmars-d-learn wrote: [...]
> ```
> auto byLine(Terminator, Char) (
>   KeepTerminator keepTerminator = No.keepTerminator,
>   Terminator terminator = '\x0a'
> )
> if (isScalarType!Terminator);
> 
> auto byLine(Terminator, Char) (
>   KeepTerminator keepTerminator,
>   Terminator terminator
> )
> if (is(immutable(ElementEncodingType!Terminator) == immutable(Char)));
> ```
> 
> To specify the line ending, it appears to be asking for a type "Terminator" which I can't find in the library. Doing the obvious doesn't work:
> 
> ```
> // Err: cannot deduce function from argument types
> myFile.byLine(`\r\n`);
> ```
> Also, how would you specify the Char type as a parameter?

First, notice that there are two sets of parentheses in the above quoted declarations. The first set are compile-time parameters (template parameters), while the second are runtime parameters.

`Terminator` and `Char` are listed as compile-time parameters, meaning that they are types specified by the caller, so there is no definition for them -- the caller defines what they will be. (More on this later.)

Secondly, note the order of parameters in the second set of parentheses: `keepTerminator` first, then `terminator`. This is why your example above doesn't compile: you're trying to specify a terminator where the function expects a KeepTerminator parameter.

Now, a fully-specified invocation of byLine would specify both compile-time and runtime parameters, e.g.:

	auto r = File(...)
		.byLine!(string,char)(Yes.KeepTerminator, "\r\n");

In this case, Terminator == string, Char == char.

But generally, the D compiler is pretty good at inferring types for you automatically, so in this case, since the runtime parameters already adequately imply what the compile-time arguments will be, so you could just leave them out and write simply:

	auto r = File(...)
		.byLine(Yes.KeepTerminator, "\r\n");


> The examples are unenlightening as they only show the default case, so any help would be much appreciated. Any wider tips on how to read these cryptic signatures would be a bonus!

Please file a bug against the documentation about this.  Examples should cover not only the default case, but should also illustrate how to use non-default values.  This is a shortcoming in the documentation.


T

-- 
MS Windows: 64-bit rehash of 32-bit extensions and a graphical shell for a 16-bit patch to an 8-bit operating system originally coded for a 4-bit microprocessor, written by a 2-bit company that can't stand 1-bit of competition.
July 15, 2021

On Thursday, 15 July 2021 at 18:08:45 UTC, Scotpip wrote:

>

The relevant function appears to be std.stdio.File.byLine.

The default isn't breaking the lines properly, so I have to pass in the line ending. But the signature has me baffled:

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator = No.keepTerminator,
  Terminator terminator = '\x0a'
)
if (isScalarType!Terminator);

auto byLine(Terminator, Char) (
  KeepTerminator keepTerminator,
  Terminator terminator
)
if (is(immutable(ElementEncodingType!Terminator) == immutable(Char)));

To specify the line ending, it appears to be asking for a type "Terminator" which I can't find in the library.

Terminator appears a few times in the signature. The first appearance, in byLine(Terminator, ...)(...), is a template parameter. So Terminator is whatever you want that otherwise works.

>

Doing the obvious doesn't work:

// Err: cannot deduce function from argument types
myFile.byLine(`\r\n`);

The first parameter is the keepTerminator flag. That wysiwyg string is four bytes long and includes the literal slashes.
https://dlang.org/spec/lex.html#wysiwyg

$ rdmd --eval '`\r\n`.representation.writeln'
[92, 114, 92, 110]

Here's a simple dos2unix:

import std.stdio, std.typecons;

void main() {
    foreach (line; stdin.byLine!(string, char)(No.keepTerminator, "\r\n"))
        writeln(line);
}
>

Also, how would you specify the Char type as a parameter?

The !(string, char) isn't necessary in the last example, but that's how you'd provide it.

Usage:

$ echo -ne "hi\r\nthere\r\n" | dmd -run dosin|od -c
0000000   h   i  \n   t   h   e   r   e  \n
0000011
>

The examples are unenlightening as they only show the default case, so any help would be much appreciated. Any wider tips on how to read these cryptic signatures would be a bonus!

July 15, 2021
Outstanding answers, folks. This is a great community!

If I ever manage to get on top of this cussed language, I hope to reciprocate by posting some material aimed at newbies coming from areas like line-of-business with scripting languages, which is where I've done most of my coding. This is a different world.

Even though this is a learner's forum, it's clear that most of the people taking up D have a strong background in gnarly systems languages like C++. The on-ramp for people like me is pretty steep.

I'm not whinging - given the lack of corporate backing it's pretty amazing how mature this language has become - and I'm very aware of the huge and selfless investment the developers have made. But it's definitely more intimidating to tackle D than one of the more trendy languages like Go or Julia where there are plentiful learning resources.

For starters, it's now clear to me that a strong understanding of Templates is essential to make much headway - that's why I got stuck here. They are dealt with towards the back of the books, but you really can't understand much of the library code without them. That's the first gap I'm going to fill.

So thanks again - without this kind of support it would be pretty much impossible for newbies like me to get up to speed.
July 15, 2021
On Thu, Jul 15, 2021 at 08:24:57PM +0000, Scotpip via Digitalmars-d-learn wrote: [...]
> For starters, it's now clear to me that a strong understanding of Templates is essential to make much headway - that's why I got stuck here. They are dealt with towards the back of the books, but you really can't understand much of the library code without them. That's the first gap I'm going to fill.
[...]

My personal favorite approach to D template functions is to not think of them as templates in the first place, but rather as functions with *compile-time* parameters (in addition to the usual runtime parameters). I.e., parameters that get baked in at compile-time. Since the parameters are processed at compile-time, they can do a more than the usual runtime parameters can, such as receiving types and aliases and other such things that only exist at compile-time.  The function can then act on these arguments at compile-time to adapt itself to the task thus specified, without incurring the runtime overhead of a runtime check.


T

-- 
There are two ways to write error-free programs; only the third one works.
July 15, 2021

On Thursday, 15 July 2021 at 21:23:58 UTC, H. S. Teoh wrote:

>

My personal favorite approach to D template functions is to not think of them as templates in the first place, but rather as functions with compile-time parameters (in addition to the usual runtime parameters).

Thanks - that does seem like a helpful analogy.

D's generics look pretty powerful to people like me who have only encountered them in Java and C#. I'm not writing libraries so I probably don't need to get too deep into the weeds, but they are clearly one of the standout features and well worth digging into!