On Thursday, 9 January 2014 at 20:40:30 UTC, H. S. Teoh wrote:
> On Thu, Jan 09, 2014 at 09:19:40PM +0100, Jacob Carlborg wrote:
>> On 2014-01-09 17:35, Marco Leise wrote:
>> 
>> >I Phobos should follow OpenGL in this regard and use a
>> >prefix like `etc` for useful but not finalized modules, so
>> >early adapters can try out new modules compare them with any
>> >existing API in Phobos where applicable (e.g. streams,
>> >json, ...) and report any issues. I have a feeling that right
>> >now most modules are tested by 2 people prior to the merge,
>> >because they spent a life in obscurity.
>> 
>> That has been suggested before and the counter argument is that
>> people will start using and complain when it's changed, even if it's
>> in an experimental. Someone here said that the javax. packages
>> originally was experimental packages to they continued to live in
>> the javax namespace to avoid breaking changes.
> [...]
>
> Maybe instead of calling it 'etc' we should outright call it
> 'experimental'. If you have code like:
>
> 	import experimental.myawesomemodule;
> 	...
>
> I doubt you'd object very much when you have to rename it to:
>
> 	import std.myawesomemodule;
> 	...
>
> since the word 'experimental' staring you in the face every time you
> open up the file will be a constant nagging reminder that you're
> depending on something unstable, giving you motivation to want to move
> it to something stable as soon as you can.
>
>
> T

I was of the opinion that phobos needed an experimental section for getting real world testing of proposed modules but these days I think we should just stick things up on dub (including modules proposed for inclusion in phobos).

On Thu, Jan 09, 2014 at 08:53:12PM +0000, Craig Dillabaugh wrote: [...]
> Thats the thing.  In most cases the correct way to do something in D, does end up being rather nice.  However, its often a bit of a challenge finding the that correct way!
> 
> When I had my troubles I expected to find the library solutions in std.string (remember I rarely use D's string processing utilities). It never really occurred to me that I might want to check std.array for the function I wanted. So what it std.array is imported when I import std.string, as a programmer I still had no idea 'split()' was there!
> 
> At the very least the documentation for std.string should say something along the lines of:
> 
> "The libraries std.unicode and std.array also include a number of functions that operate on strings, so if what you are looking for isn't here, try looking there."

Yeah, any public imports should be mentioned somewhere in the docs, otherwise it's just random invisible magic as far as the end-user is concerned ("Hmm, I imported std.string in one module, and array.front works, but in this other module, array.front doesn't work! Why? Who knows.");

Please submit a pull request to add that to the docs.


T

-- 
People walk. Computers run.

On Thursday, 9 January 2014 at 20:53:13 UTC, Craig Dillabaugh wrote:
> Thats the thing.  In most cases the correct way to do something in D, does end up being rather nice.  However, its often a bit of a challenge finding the that correct way!

Yeah, and indeed it is a bit weird that so many of the functions moved from std.string to std.array. (Yet are still specialized on strings... I think they have to be in the same module just to be in the same overload set though.)

> "The libraries std.unicode and std.array also include a number of functions that operate on strings, so if what you are looking for isn't here, try looking there."

Aye, I think the documentation could use a few higher level overviews that bring the modules together.

On 10 January 2014 03:40, John Colvin <john.loughran.colvin@gmail.com>wrote:

> On Thursday, 9 January 2014 at 17:39:00 UTC, Adam D. Ruppe wrote:
>
>> On Thursday, 9 January 2014 at 14:08:02 UTC, Manu wrote:
>>
>>>  string y = find(retro("Hello"), 'H');
>>>
>>
>> import std.string;
>> auto idx = lastIndexOf("Hello", 'H');
>>
>> Wow, that's unbelievable difficult. D sucks.
>>
>
> How on earth did I miss that...
>

I have to wonder the same thing.
It's just not anything like anything I've ever called it before I guess.
I guess I started with find, and then it refers you to retro if you want to
reverse find, and of course, by this time I'm nowhere near std.string
anymore. Hard to find something if you're not even looking in the same file
:/

On Thu, Jan 09, 2014 at 11:28:07PM +0000, Brad Anderson wrote:
> On Thursday, 9 January 2014 at 20:40:33 UTC, H. S. Teoh wrote:
> >On Thu, Jan 09, 2014 at 06:25:33PM +0000, Brad Anderson wrote:
> >>On Thursday, 9 January 2014 at 14:08:02 UTC, Manu wrote:
> >[...]
> >>>On a side note, am I the only one that finds std.algorithm/std.range/etc for string processing really obtuse?  I can rarely understand the error messages, so say it's better than STL is optimistic.
> >>
> >>I absolutely hate the "does not match any template declaration" error. It's extremely unhelpful for figuring out what you need to do and anytime I try to do something fun with ranges I can expect to see it a dozen times.
> >
> >Yeah, that error drives me up the wall too. I often get screenfuls of errors, dumping 25 or so overloads of some obscure Phobos internal function (like toImpl) as though an end-user would understand any of it.  You have to parse all the sig constraints (and boy some of them are obscure), *understand* what they mean (which requires understanding how Phobos works internally), and *then* try to figure out, by elimination, which is the one that you intended to match, and why your code failed to match it.
> >
> >I'm almost tempted to say that using sig constraints to differentiate between template overloads is a bad idea. Instead, consider this alternative implementation of toImpl:
> >
> >	template toImpl(S,T)
> >		// N.B.: no sig constraints here
> >	{
> >		static if (... /* sig constraint conditions for overload #1 */)
> >		{
> >			S toImpl(T t)
> >			{
> >				// implementation here
> >			}
> >		}
> >		else static if (... /* sig constraint conditions for overload #2
> >*/)
> >		{
> >			S toImpl(T t)
> >			{
> >				// implementation here
> >			}
> >		}
> >		...
> >		else // N.B.: user-readable error message
> >		{
> >			static assert(0, "Unable to convert " ~
> >				T.stringof ~ " to " ~ S.stringof);
> >		}
> >	}
> >
> >By putting all overloads inside a single template, we can give a useful default message when no overloads match.
> >
> 
> Interesting and there is a lot of flexibility there. It does make the functions a lot more verbose though for something that is really the compiler's job (clearly describing errors).

The way I see it, is that any sig constraints should go on the outer template, and should define the *logical* scope of all overloads encompassed therein. E.g., if you have a set of overloads for sqrt, say, then the outer template would have a sig constraint that matches any number-like type. Within the template, each individual overload would handle various concrete types, and the static assert at the end is essentially saying "in theory your arguments should match *something* in this template, but currently your particular combination of types isn't implemented by any overload".

Or, put another way, the outer template represents the "logical" meta-function that does some given task (e.g., sqrt computes the square root of *something*), whereas the inner overloads provide the actual set of available implementations that implement that meta-function (computes the square root of an int, computes the square root of a float, etc.). My hypothesis is that you get the wall-of-template-errors problem when there's a logical meta-function (or a small number of them) that, for implementational reasons, consists of a large set of overloads. By treating the logical meta-function as an actual entity (the outer template), we can give a unified error message of failure to implement the meta-function for the requested types, rather than many error messages for each of the many overloads, most of which are irrelevant to the user.


> >Alternatively, maybe sig constraints can have an additional string parameter that specifies a message that explains why that particular overload was rejected. These messages are not displayed if at least one overload matches; only if no overload matches, they will be displayed (so that the user can at least see why each of the overloads didn't match).
> >
> 
> Each constraint would have a string? I think that would help for some of the more obscure constraints that aren't wrapped up in an eponymous template helper but I don't think it'd help with the problem generally because the problem is identifying which exact constraint failed.

True.


> Example:
> 
>     void main()
>     {
>       import std.algorithm, std.range;
>       struct A { }
>       auto a = recurrence!"n"(0).take(5).find(A());
>     }
> 
> This is the error message you get:
> 
> ---
> /d14/f101.d(5): Error: template std.algorithm.find does not match
> any function template declaration. Candidates are:
> /opt/compilers/dmd2/include/std/algorithm.d(3650):
> std.algorithm.find(alias pred = "a == b", R, E)(R haystack, E
> needle) if (isInputRange!R &&
> is(typeof(binaryFun!pred(haystack.front, needle)) : bool))
> /opt/compilers/dmd2/include/std/algorithm.d(3713):
> std.algorithm.find(alias pred = "a == b", R1, R2)(R1 haystack, R2
> needle) if (isForwardRange!R1 && isForwardRange!R2 &&
> is(typeof(binaryFun!pred(haystack.front, needle.front)) : bool) &&
> !isRandomAccessRange!R1)
> /opt/compilers/dmd2/include/std/algorithm.d(3749):
> std.algorithm.find(alias pred = "a == b", R1, R2)(R1 haystack, R2
> needle) if (isRandomAccessRange!R1 && isBidirectionalRange!R2 &&
> is(typeof(binaryFun!pred(haystack.front, needle.front)) : bool))
> /opt/compilers/dmd2/include/std/algorithm.d(3821):
> std.algorithm.find(alias pred = "a == b", R1, R2)(R1 haystack, R2
> needle) if (isRandomAccessRange!R1 && isForwardRange!R2 &&
> !isBidirectionalRange!R2 && is(typeof(binaryFun!pred(haystack.front,
> needle.front)) : bool))
> /opt/compilers/dmd2/include/std/algorithm.d(4053):
> std.algorithm.find(alias pred = "a == b", Range, Ranges...)(Range
> haystack, Ranges needles) if (Ranges.length > 1 &&
> is(typeof(startsWith!pred(haystack, needles))))
> ---
> 
> Where do you even begin with that flood of information? To fix it all you really want to see is which constraint you didn't satisfy. An error message like this would help greatly:
> 
> ---
> /d539/f571.d(5): Error: template std.algorithm.find call fails all
> constraints. Candidates are:
> /opt/compilers/dmd2/include/std/algorithm.d:
>   (3650) find(alias pred = "a == b", R, E)(R haystack, E needle):
>               isInputRange!R
>            && is(typeof(binaryFun!pred(haystack.front, needle)) :
> bool) <- FAILS
>   (3713) find(alias pred = "a == b", R1, R2)(R1 haystack, R2
> needle):
>               isForwardRange!R1
>            && isForwardRange!R2 <- FAILS
>            && is(typeof(binaryFun!pred(haystack.front,
> needle.front)) : bool)
>            && !isRandomAccessRange!R1
>   (3749) find(alias pred = "a == b", R1, R2)(R1 haystack, R2
> needle):
>               isRandomAccessRange!R1 <- FAILS
>            && isBidirectionalRange!R2
>            && is(typeof(binaryFun!pred(haystack.front,
> needle.front)) : bool)
>   (3821) find(alias pred = "a == b", R1, R2)(R1 haystack, R2 needle)
>               isRandomAccessRange!R1 <- FAILS
>            && isForwardRange!R2
>            && !isBidirectionalRange!R2
>            && is(typeof(binaryFun!pred(haystack.front,
> needle.front)) : bool)
>   (4053) find(alias pred = "a == b", Range, Ranges...)(Range
> haystack, Ranges needles)
>               Ranges.length > 1 <-- FAILS
>            && is(typeof(startsWith!pred(haystack, needles)))

But still, this will dump out a whole bunch of overloads that aren't necessarily interesting to the user. I mean, if I want to search for an int in an int[], but accidentally passed a string instead of an int, then I'm really only interested in seeing how the overload that handles int[] searching failed to match my string argument; I don't care about why the sig constraints failed for the overload that handles linked lists, for example.

Perhaps a better solution lies in distinguishing the logical scope of the function, vs. requirements on its argument types within that scope. For example, the find() overload that searches T[] for some T, has T[] as its scope, whereas within this scope, it imposes certain requirements on the needle U (e.g., U must be comparable with an element of T). This suggests that it should be implemented like this:

	auto find(R,S)(R haystack, S needle)
		if (is(R == T[], T)) // <-- defines the scope of this function
	{
		static if (isComparable(S, ElementType!R)) // <-- Defines type requirements within this function's scope
		{
			// implementation here
		}
		else static if (isComparable(ElementType!S, ElementType!R)) // <-- ditto
		{
			// implementation here
		}
		else
			static assert(0, "Don't know how to search for "
				~ S.stringof ~ " in " ~ R.stringof);
	}

Then when you try to search for a string in an int[], for example, it will first match this overload of find(), then fail the static if conditions because the needle you passed in doesn't match the type requirements for searching an int[].

Note that I've grouped at least two of the current find() overloads under a single overload above -- because they are just two implementations for handling two cases within the same scope: searching an array. The fact that array-searching is implemented by two distinct algorithms is irrelevant to the user, and so it makes sense to "hide" them inside a single function's body.

So to summarize:
(1) use sig constraints to define the scope of an overload; and
(2) use static if inside the function body (or template body) to enforce
type requirements within that scope.

This solves the problem of needing the compiler to somehow read your mind and figure out exactly which of the 56 overloads of find() you intended to match but failed to.


T

-- 
The only difference between male factor and malefactor is just a little emptiness inside.

On 10 January 2014 04:00, Adam D. Ruppe <destructionator@gmail.com> wrote:

> On Thursday, 9 January 2014 at 17:54:05 UTC, Dicebot wrote:
>
>> It is not the same thing as sample with byGrapheme though.
>>
>
> Right, but it works for ascii (and others) and shows std.string isn't as weak as being said in this thread.
>

So is it 'correct'? The docs don't really say what it does. Is 'index' in
bytes, in codepoints, or in graphemes? Looks like bytes, but then it talks
about std.utf.UTFException, so maybe codepoints?
Being correct is constantly being thrown around as the 'value' in why
everything's so fucking hard... if this function isn't 'correct', then we
have a disparity.

I also don't think it excuses any of my other points. There shouldn't be
4-5+ modules where you have to look whenever you want to find string
related stuff.
In this case, my explicit example is just the straw that broke the camels
back. My experience still stands; every time I try to do any serious string
work, I waste far more time than I care to, and I HATE doing it. Makes me
feel dirty and I don't enjoy my programming time (which I ususally do
enjoy). In my experience, if you're not enjoying programming, something
went wrong.

The D docs are pretty terrible, they don't do much to help you find what
you're looking for.
You have a massive block of function names at the top of the page, you have
to carefully scan through one by one, hoping that it's named something
obvious that will stand out to you, and in the event it doesn't have a
helper function, you need to work out the proper sequence of
algorithm/range/whatever operations to do what you want (and then repeat
the process finding the small parts you need across a bunch of modules).

Blah! </endrant>

On 10 January 2014 06:19, Jacob Carlborg <doob@me.com> wrote:

> On 2014-01-09 17:35, Marco Leise wrote:
>
>  I Phobos should follow OpenGL in this regard and use a
>> prefix like `etc` for useful but not finalized modules, so
>> early adapters can try out new modules compare them with any
>> existing API in Phobos where applicable (e.g. streams,
>> json, ...) and report any issues. I have a feeling that right
>> now most modules are tested by 2 people prior to the merge,
>> because they spent a life in obscurity.
>>
>
> That has been suggested before and the counter argument is that people will start using and complain when it's changed, even if it's in an experimental.


I've heard that, and I think that's a lame argument. Would people rather break peoples code *who deliberately chose to use a beta feature, and accept the contract while doing so (that it would later be moved to 'std' proper)*, or consistently produce features that have very little proven foundation in practical application? It takes year(/s) before enough people can have had a crack at a new API in enough scenarios to reveal where it went right, and where it went wrong.

In the case of std.simd, I'm not ever going to consider presenting it for inclusion until such a time I'm absolutely happy with it (although in this case, it's also just not finished ;), and since it's not readily available, that really just relies on my using it in enough of my own projects that I manage to satisfy myself... it makes no sense.

Someone here said that the javax. packages originally was experimental
> packages to they continued to live in the javax namespace to avoid breaking changes.
>
> --
> /Jacob Carlborg
>

On Thursday, 9 January 2014 at 14:25:20 UTC, Benjamin Thaut wrote:
> I feel exactly the same.  C# has way more utility functions that are named in a way that actually helps you understand what they do.

Interesting, I've had the opposite experience. I keep trying to perform range operations and C# doesn't have them. Slicing is of course ever more desired.

That isn't to say C# is bad, but

    if(string.IsNullOrEmpty(str))

vs

    if(str.empty)

keeps throwing me off.

On 10 January 2014 06:34, Jacob Carlborg <doob@me.com> wrote:

> On 2014-01-09 15:07, Manu wrote:
>
>> This works fine:
>>    string x = find("Hello", 'H');
>>
>> This doesn't:
>>    string y = find(retro("Hello"), 'H');
>>    > Error: cannot implicitly convert expression (find(retro("Hello"),
>> 'H')) of type Result!() to string
>>
>> Is that wrong? That seems to be how the docs suggest it should be used.
>>
>
> As other as said, the problem is that "find" returns a range, which is not implicitly convertible to "string". The main reason is to avoid temporary allocations when chaining algorithms.
>
> If it was the other way around you would probably be complaining it wasn't efficient enough ;)
>

Then there's probably a fundamental problem somewhere, and it should be
re-thought at a lower level.
Perhaps even something super simple like a can't-go-wrong naming
convention, that makes it REALLY plain when string related function are
dealing with bytes, codepoints, or graphemes?
It would seem to be that a lot of the confusion and complexity surrounding
strings is because it tries to be 'correct' (and varying levels of correct
in different circumstances), but there are no clear relationships between
different functions that deal with these different versions of
'correct'-ness.


 On a side note, am I the only one that finds std.algorithm/std.range/etc
>> for string processing really obtuse?
>> I can rarely understand the error messages, so say it's better than STL
>> is optimistic.
>> Using std.algorithm and std.range to do string manipulation feels really
>> lame to me.
>> I hate looking through the docs of 3-4 modules to understand the
>> complete set of useful string operations (std.string, std.uni,
>> std.algorithm, std.range... at least).
>>
>
> You forgot std.array ;)


I did! And there are probably others too.
You can't do anything without std.typecons either. Although not directly
related, it's always seems to be there alongside.

 I also find the names of the generic algorithms are often unrelated to
>> the name of the string operation.
>> My feeling is, everyone is always on about how cool D is at string, but
>> other than 'char[]', and the builtin slice operator, I feel really
>> unproductive whenever I do any heavy string manipulation in D.
>>
>
> You have built-in appending, concatenation, using strings in switch statements and so on.


Correct, those things are good. That is where 'D is awesome at strings' ends though, in my opinion.

 I also hate that I need to import at least 4-5 modules to do anything
>> useful with strings... I feel my program bloating and cringe with every gigantic import that sources exactly one symbol.
>>
>
> I agree with you. I have built up a small library through out the years that basically allows me to only import a single module to do most string operations I need.
>

I suspect your effort is not uncommon. Is this not clear evidence of a critical problem?

You probably don't like it but you could have a look at Tango as well. It
> contains two useful modules (for this case). One for handling arbitrary array operators and one for string operations.
>
> tango.core.Array
> tango.text.Util
>
> https://github.com/SiegeLord/Tango-D2 http://siegelord.github.io/Tango-D2/


Yeah... I want less libraries, not more :/

On Fri, Jan 10, 2014 at 10:56:27AM +1000, Manu wrote:
[...]
> The D docs are pretty terrible, they don't do much to help you find
> what you're looking for.
> You have a massive block of function names at the top of the page,

Yeah, that blob of links is useless unless you already knew what you were looking for (kinda defeats the purpose).

The hand-classified table of functions in std.algorithm and std.range is more useful, IMO. At least it lets you use divide-and-conquer to zoom down to your area of interest, whereas the order of links in the blob of links has no relation whatsoever to the functionality provided.

The order of docs for each symbol also follows the order in the source code, which may not necessarily follow a logical order. This makes browsing the docs difficult -- one minute it's describing find() overloads, next minute it's talking about set unions, then after that it's back to findAfter(), then it jumps to remove(), etc.. Try finding what you want when the docs are 50 pages of this random jumping around. All the more this makes a hand-classified table of symbols indispensable.


> you have to carefully scan through one by one, hoping that it's named something obvious that will stand out to you, and in the event it doesn't have a helper function, you need to work out the proper sequence of algorithm/range/whatever operations to do what you want (and then repeat the process finding the small parts you need across a bunch of modules).
[...]

I will say, though, that taking the time to learn where things are in std.algorithm, std.range, std.string, and std.array helps immensely in knowing where to look for things in the future. This doesn't excuse the poor state of the docs, of course, nor the non-intuitive placement of some of the functions in Phobos, but you're likely to feel far less frustrated if you took the time to familiarize yourself with where things are. :)

I usually don't have too much trouble finding what I need when it comes to string manipulation. But then again, when I fail to find something within 15 seconds of looking at the obvious places, I just import std.regex and proceed to crush the proverbial ant with the proverbial nuclear warhead. :-P


T

-- 
It's bad luck to be superstitious. -- YHL