There is a discussion about D vs Go going on in several threads(yey for multithreading!), and one thread is about an article by Gary Willoughby that claims that Go is not suitable for sophisticated programmers(http://forum.dlang.org/thread/mev7ll$mqr$1@digitalmars.com). What's interesting about this one is the reddit comments, which turned into an argument between simple languages that average programmers can use and complex languages that only the top 1% of intelligent programmers can use, but they can extract more out of them.

But the thing is - the world of the top programmers is not really separate from that of average programmers. Professional development teams can have a few top programmers and many average one, all be working on the same project. Open source projects can have top programmers working on the core while allowing average programmers to contribute some simple features. Top programmers can write libraries that can be used by average programmers.

To allow these things, top programmers and average programmers should be able to work on the same language. Of course, any language that average programmers can master should be easy for a top programmer to master - but the thing is, we also want the top programmer to be able to bring more out of the language, without limiting them by it's over-simplicity. This will also benefit the average programmers, since they also improve the quality of the libraries and modules they are using.

This idea is nothing new, and was mentioned in the main(=biggest) current D vs Go thread(http://forum.dlang.org/thread/mdtago$em9$1@digitalmars.com?page=3#post-jeuhtlocousxtezoaqqh:40forum.dlang.org). What I want to talk about here is the seams. The hurdles that in practice make this duality harder.

Let's compare it to another duality that D(and many other languages, mainly modern systems languages) promotes - the duality between high-level and low-level. Between write-code-fast and write-fast-code.

The transition between high-level and low-level code in D consists by a change of the things uses - which language constructs, which idioms, which functions. But there aren't any visible seams. You don't need to use FFI or to dynamically load a library file written in another language or anything like that - you simply write the high-level parts like you would write high-level code and the low-level parts like you would write low-level code, and they just work together.

The duality between high-level D and low-level D is seamless. The duality between simple D and complex D - not so much.

The seams here exist mainly in understanding how to use complex code from simple code. Let's take std.algorithm(.*) for example. The algorithms' implementations there are complex and use advanced D features, but using them is very simple. Provided, of course, that you know how to use them(and no - not everything that you know becomes simple. I know how to solve regular differential equations, but it's still very complex to do so).

The problem, as Andrei Alexandrescu pointed out(http://forum.dlang.org/thread/mdtago$em9$1@digitalmars.com?page=6#post-mduv1i:242169:241:40digitalmars.com), is learning how to use them. Ideally you'd want to be able to look at a function's signature and learn from that how to use it. It's name and return type should tell you what it does and it's argument names and types should tell you what to send to it. The documentation only there for a more through description and to warn you about pitfalls and edge cases.

But when it comes to heavily templated functions - understanding the signature is HARD. It's hard enough for the top programmers that can handle the complex D features - it's much harder for the average programmers that could have easily used these functions if they could just understand the documentation.

Compare it, for example, to Jave. Even if a library doesn't contain a single documentation comment, the auto-generated javadoc that contains just the class tree and method signatures is usually enough to get an idea of what's going where. In D, unless the author has provided some actual examples, you are going to have a hard time trying to sort out these complex templated signatures...

That's quite an hurdle to go though when wanting to use complex code from simple code(or even from other complex code). That's the ugly seam I'm talking about.

Now, if you are working on a big project(be it commercial or open-source), you can find lot's of examples how to use these complex functions, and that's probably how you'd tackle the problem. When you are using some library you usually don't have that luxury - but these libraries usually have the generated ddoc at their website. Of course - that generated ddoc is full with complex templated signatures, so that's not very helpful...

So, what can be done? Maybe the ddoc generator, instead of writing the whole signature as-is, can emit a more human-readable version of it?

Let's look at the example Andrei mentioned - startsWith. Let's take a look at the first overloaded signature:

uint startsWith(alias pred = "a == b", Range, Needles...)(Range doesThisStart, Needles withOneOfThese) if (isInputRange!Range && Needles.length > 1 && is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[0])) : bool) && is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[1..$])) : uint));

Let's break it down and see what the user needs in order to use the function:

`uint` - the return type. Needed.
`startsWith` - the function name. Needed.
`(alias pred = "a == b",` - a template argument that the user might want to supply - Needed.
`Range, Needles...)` - template arguments that should usually be inferable. The function won't work without them, but since the user doesn't actually supply them - I'll mark them as not needed.
`(Range doesThisStart, Needles withOneOfThese)` - the function's arguments. Needed.

The rest are constraints that check the template arguments. They aren't needed when you try to use the function - though they might be helpful at figuring out why the compiler yells at you when you use it wrong.

So, if we take only the needed parts, we get this signature:

uint startsWith(alias pred = "a == b")(Range doesThisStart, Needles withOneOfThese);

Well, doesn't this look much easier to grasp? Of course, it omits some very critical information. It doesn't tell you what are `Range` and `Needles` - you can look for these types in the docs and find nothing. It also doesn't tell you that `Needles` is variadic.

Well - what's stopping us from adding this information *below* the signature? What if ddoc would generate something like this:

uint startsWith(alias pred = "a == b")(Range doesThisStart, Needles... withOneOfThese);
  where:
    Range is an inferred template argument
    Needles is a variadic inferred template argument
    isInputRange!Range
    Needles.length > 1
    is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[0])) : bool)
    is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[1..$])) : uint)

We've broken the signature into the parts required to use the function and the parts required to FULLY understand the previous parts. The motivation is that the second group of parts is also important, so it needs to be there, but it creates a lot of unneeded noise so it shouldn't be a direct part of the signature(at least not in the doc). It's similar to the docs of other types used in the signature - it's important to have these docs somewhere accessible, but you don't want to add them in the middle of the signature because it'll make it unreadable.



This idea, of course, is not a finally cooked proposal yet. We need a way to tell ddoc which template arguments are supposed to be inferred(can this always be done automatically?) and the last two entries in my example are not super-trivial to grok(I can rewrite them by-hand to make them super-simple - but can ddoc do it automatically? and how?). The point of this thread is to start a discussion about making ddoc generate documentations that are more... well... human readable.

On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
> ...snip...

So tl;dr; make the template constraints in ddoc less prominent?

The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)

On 3/26/15 3:45 PM, Alex Parrill wrote:
> On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
>> ...snip...
>
> So tl;dr; make the template constraints in ddoc less prominent?
>
> The "new library reference preview" under Resources seems to already
> have this (example:
> http://dlang.org/library/std/algorithm/searching/starts_with.html)

Yes, it's so unprominent, I can't find it :)

I don't think this was intentional...

-Steve

On Thu, Mar 26, 2015 at 07:32:51PM +0000, Idan Arye via Digitalmars-d wrote: [...]
> uint startsWith(alias pred = "a == b")(Range doesThisStart, Needles...
> withOneOfThese);
>   where:
>     Range is an inferred template argument
>     Needles is a variadic inferred template argument
>     isInputRange!Range
>     Needles.length > 1
>     is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[0])) : bool)
>     is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[1..$])) : uint)
> 
> We've broken the signature into the parts required to use the function and the parts required to FULLY understand the previous parts. The motivation is that the second group of parts is also important, so it needs to be there, but it creates a lot of unneeded noise so it shouldn't be a direct part of the signature(at least not in the doc). It's similar to the docs of other types used in the signature - it's important to have these docs somewhere accessible, but you don't want to add them in the middle of the signature because it'll make it unreadable.

This is not a new idea:

	https://issues.dlang.org/show_bug.cgi?id=13676

The question is, who's gonna implement it?


T

-- 
If you want to solve a problem, you need to address its root cause, not just its symptoms. Otherwise it's like treating cancer with Tylenol...

On Thursday, 26 March 2015 at 19:45:19 UTC, Alex Parrill wrote:
> On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
>> ...snip...
>
> So tl;dr; make the template constraints in ddoc less prominent?
>
> The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)

This will not solve the readability problem:

- what is a range?
- what is a needle?
- what is a predicate?

Phobos is reinventing the OOP encapsulation without providing the minimal tools. IMHO, something similar to "interface" is mandatory to define constraints for "range" and the relationship between "needle" and "range". In another topic, someone proposed the keyword "constraint" - or - extend the concept of "interface" to other kind of types than classes and this will allow some compile-time interface implementation. Therefore, when I write:

void foo(IInputRange range) { ... }, range can be a class implementing IInputRange or any other type which supports the same calls described by IInputRange.

Another approach is to provide enough overloads for most used scenarios, like testing for prefixes in a string with case sensivity or not.

On Thursday, 26 March 2015 at 21:36:56 UTC, rumbu wrote:
> On Thursday, 26 March 2015 at 19:45:19 UTC, Alex Parrill wrote:
>> On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
>>> ...snip...
>>
>> So tl;dr; make the template constraints in ddoc less prominent?
>>
>> The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)
>
> This will not solve the readability problem:
>
> - what is a range?
> - what is a needle?
> - what is a predicate?

Wouldnt that just be "Further documentation" for advanced programming concepts?
Only simple easy to understand examples are needed to begin with.

It should flow like a tree.

On Thu, Mar 26, 2015 at 11:05:28PM +0000, Israel via Digitalmars-d wrote:
> On Thursday, 26 March 2015 at 21:36:56 UTC, rumbu wrote:
> >On Thursday, 26 March 2015 at 19:45:19 UTC, Alex Parrill wrote:
> >>On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
> >>>...snip...
> >>
> >>So tl;dr; make the template constraints in ddoc less prominent?
> >>
> >>The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)
> >
> >This will not solve the readability problem:
> >
> >- what is a range?
> >- what is a needle?
> >- what is a predicate?
> 
> Wouldnt that just be "Further documentation" for advanced programming
> concepts?
> Only simple easy to understand examples are needed to begin with.
> 
> It should flow like a tree.

There has already been a recent push to add hyperlinks in Phobos docs whenever "input range", "output range", etc., are mentioned, to the definitions of these terms. Well, currently they point to the isInputRange, isOutputRange, etc., templates in std.range.primitives, which may not be the best introduction to the concept of ranges, but at least it will tell you which methods must be supported by these ranges.

Perhaps there should be a glossary page on dlang.org where these terms can link to, instead.


T

-- 
The richest man is not he who has the most, but he who needs the least.

On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:

> The problem, as Andrei Alexandrescu pointed out(http://forum.dlang.org/thread/mdtago$em9$1@digitalmars.com?page=6#post-mduv1i:242169:241:40digitalmars.com), is learning how to use them. Ideally you'd want to be able to look at a function's signature and learn from that how to use it. It's name and return type should tell you what it does and it's argument names and types should tell you what to send to it. The documentation only there for a more through description and to warn you about pitfalls and edge cases.
>
> But when it comes to heavily templated functions - understanding the signature is HARD. It's hard enough for the top programmers that can handle the complex D features - it's much harder for the average programmers that could have easily used these functions if they could just understand the documentation.
I think the documentation should simply contain the unittests - they show quite well how to call the template, from minimal cases to the complex ones.
Ok, they tend to show a lot of edge-cases, but even the very simplest usages of a function should be unit-tested, so at least those have to be part of the documentation.

On Thursday, 26 March 2015 at 21:36:56 UTC, rumbu wrote:
> On Thursday, 26 March 2015 at 19:45:19 UTC, Alex Parrill wrote:
>> On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
>>> ...snip...
>>
>> So tl;dr; make the template constraints in ddoc less prominent?
>>
>> The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)
>
> This will not solve the readability problem:
>
> - what is a range?
> - what is a needle?
> - what is a predicate?
>
> Phobos is reinventing the OOP encapsulation without providing the minimal tools. IMHO, something similar to "interface" is mandatory to define constraints for "range" and the relationship between "needle" and "range".

Never heard that complain about python's protocols which have no
language support whatsoever.

On Friday, 27 March 2015 at 13:07:30 UTC, Panke wrote:
> On Thursday, 26 March 2015 at 21:36:56 UTC, rumbu wrote:
>> On Thursday, 26 March 2015 at 19:45:19 UTC, Alex Parrill wrote:
>>> On Thursday, 26 March 2015 at 19:32:53 UTC, Idan Arye wrote:
>>>> ...snip...
>>>
>>> So tl;dr; make the template constraints in ddoc less prominent?
>>>
>>> The "new library reference preview" under Resources seems to already have this (example: http://dlang.org/library/std/algorithm/searching/starts_with.html)
>>
>> This will not solve the readability problem:
>>
>> - what is a range?
>> - what is a needle?
>> - what is a predicate?
>>
>> Phobos is reinventing the OOP encapsulation without providing the minimal tools. IMHO, something similar to "interface" is mandatory to define constraints for "range" and the relationship between "needle" and "range".
>
> Never heard that complain about python's protocols which have no
> language support whatsoever.

I've never used Python, so take this as a live experiment:

I googled for "startswith python":

str.startswith(prefix[, start[, end]])
Return True if string starts with the prefix, otherwise return False. prefix can also be a tuple of prefixes to look for. With optional start, test string beginning at that position. With optional end, stop comparing string at that position.

In 3 seconds, I understood the basic usage, without ANY knowledge about Python. Even if I don't know Python, the brackets told me that "start" and "end" are optional. str, prefix, start, end - comprehensible simple words. The only thing remaining is the "tuple" thing. But I can use this function without knowing what a "tuple" is.

Looked also in the source code to find out that startsWith is locale sensitive, something ignored in phobos.

https://hg.python.org/cpython/file/4ebe1ede981e/Objects/stringobject.c#l2903

Now compare with this:

uint startsWith(alias pred = "a == b", Range, Needles...)(Range doesThisStart, Needles withOneOfThese) if (isInputRange!Range && Needles.length > 1 && is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[0])) : bool) && is(typeof(.startsWith!pred(doesThisStart, withOneOfThese[1..$])) : uint));
bool startsWith(alias pred = "a == b", R1, R2)(R1 doesThisStart, R2 withThis) if (isInputRange!R1 && isInputRange!R2 && is(typeof(binaryFun!pred(doesThisStart.front, withThis.front)) : bool));
bool startsWith(alias pred = "a == b", R, E)(R doesThisStart, E withThis) if (isInputRange!R && is(typeof(binaryFun!pred(doesThisStart.front, withThis)) : bool));
*Checks whether the given input range starts with (one of) the given needle(s).*

I'm not a native English speaker, but a range can start with a needle? Where is the haystack? :)

Anyway, in the Python built-in lib I didn't find any levenshtheinDistance, boyermooreFinder or schschchcscshshscscshshscscssscsshcswarzSort.