Branch: refs/heads/stable
  Home:   https://github.com/D-Programming-Language/phobos
  Commit: f5843a3cfe286271d28ddda9cb47a6d690d75538
      https://github.com/D-Programming-Language/phobos/commit/f5843a3cfe286271d28ddda9cb47a6d690d75538
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/traits.d

  Log Message:
  -----------
  add isStringLike to std.traits

- select only structs/static array that are implicitly convertible to a string

- also add internal peelStringLike helper


  Commit: f8af54dce783832a8424aa3b5bd86f51d324cbf1
      https://github.com/D-Programming-Language/phobos/commit/f8af54dce783832a8424aa3b5bd86f51d324cbf1
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/algorithm/comparison.d

  Log Message:
  -----------
  fix levenshteinDistance for string-like types


  Commit: 994ef503b2b19f0a25b1706f34b591ca663880f5
      https://github.com/D-Programming-Language/phobos/commit/994ef503b2b19f0a25b1706f34b591ca663880f5
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/path.d

  Log Message:
  -----------
  std.path fixes for string-like types


  Commit: 34097e85953d346a86d5f0e7be38aca0890a3862
      https://github.com/D-Programming-Language/phobos/commit/34097e85953d346a86d5f0e7be38aca0890a3862
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/string.d

  Log Message:
  -----------
  std.string fixes for string-like types

- consolidate tests
- add a few missing overloads


  Commit: 6e58799ca3499a5081b80e2793ef87f2b831b4c1
      https://github.com/D-Programming-Language/phobos/commit/6e58799ca3499a5081b80e2793ef87f2b831b4c1
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/uni.d

  Log Message:
  -----------
  std.uni fixes for string-like types


  Commit: 3a8a9ea7939cf2e5c4a6452597681975ad3fb866
      https://github.com/D-Programming-Language/phobos/commit/3a8a9ea7939cf2e5c4a6452597681975ad3fb866
  Author: Martin Nowak <code@dawg.eu>
  Date:   2015-10-25 (Sun, 25 Oct 2015)

  Changed paths:
    M std/file.d

  Log Message:
  -----------
  std.file fixes for string-like types

- fixes Issue 15027


  Commit: 384410aa188f4300b80d7f6dbf2145ecc8c03e92
      https://github.com/D-Programming-Language/phobos/commit/384410aa188f4300b80d7f6dbf2145ecc8c03e92
  Author: Dmitry Olshansky <dmitry.olsh@gmail.com>
  Date:   2015-10-26 (Mon, 26 Oct 2015)

  Changed paths:
    M std/algorithm/comparison.d
    M std/file.d
    M std/path.d
    M std/string.d
    M std/traits.d
    M std/uni.d

  Log Message:
  -----------
  Merge pull request #3770 from MartinNowak/string_like

fix Issue 15027 – rangified functions no longer work with string-like types


Compare: https://github.com/D-Programming-Language/phobos/compare/325715024df8...384410aa188f

Hi folks, isStringLike is a fine idea but there are a number of issues I noticed while looking at the PR and the related pieces it works with.

1. isAggregate is not the best choice of names for something that's a struct, class, union, or interface. In C++ aggregates are a completely different notion (http://stackoverflow.com/questions/4178175/what-are-aggregates-and-pods-and-how-why-are-they-special) so essentially we're defining our own terminology. Not the best thing to do. Closest term I can think of is isUserDefinedType, but that would include enumerated types.

The larger question is why this peculiar notion is needed strongly enough to get its own name.

2. I take issue with the current definition of isStringLike, which is:

 (isAggregateType!T || isStaticArray!T) && is(StringTypeOf!T);

So... a string is unlike a string, which I find unfitting. Furthermore, the entire notion of isStringLike and StringTypeOf is dedicated to work around what seems to be a bug in the typesystem (alias this fails to implement real subtyping). It seems that is the right place to focus work on.

A worse problem is that isStringLike fails to capture the more frequent "string-like" structure: (a) lazy ranges of characters and (b) reference counted strings. These are crucial pieces. So we're not making progress toward actually defining what we think a string-like data structure is.

I think a string-like structure is:

(a) an array of characters or a subtype thereof

(b) some other forward range with ElementType some character type, and potentially ElementEncodingType a different character type as well.

That's pretty much it. Algorithms dealing with string-like data should be able to work with this definition, or temporarily convert to arrays of characters.

More importantly, while looking at https://github.com/D-Programming-Language/phobos/pull/3770/ and related work I can't stop noticing a growth of layers of complexity that seems poorly scalable, instead of an increase in generic simplicity and clarity of concepts. That does not bode well and we need to curb it lest we end up with a standard library we ourselves can't make sense of.

I do understand there are issues to fix and we need to move forward. But tactical solutions cannot substitute a lack of strategy.

Please let us all stop and draw a little breath, and figure out how to replace the monumental work in PR 3770 with simplifying abstractions. Once something like isStringLike becomes part of Phobos, we're stuck with it for eternity - there's no way we can fix it without breaking code.

I don't have the bandwidth to scrutinize all Phobos changes and at the same time work on refcounted safe collections (my focus). But please keep me informed of all public name additions to Phobos, and also discuss them in the forum. If we keep on introducing new notions at this pace, we'll drown Phobos in its own attempts at being rigorous.


Thanks,

Andrei

_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On 26-Oct-2015 18:15, Andrei Alexandrescu via phobos wrote:
> Hi folks, isStringLike is a fine idea but there are a number of issues I noticed while looking at the PR and the related pieces it works with.
>
[snip]
> I think a string-like structure is:
>
> (a) an array of characters or a subtype thereof
>
> (b) some other forward range with ElementType some character type, and potentially ElementEncodingType a different character type as well.

Sadly the main reason for isStringLike is
(c) alias this to something that has a or b defined above.

>
> That's pretty much it. Algorithms dealing with string-like data should be able to work with this definition, or temporarily convert to arrays of characters.
>
> More importantly, while looking at https://github.com/D-Programming-Language/phobos/pull/3770/ and related work I can't stop noticing a growth of layers of complexity that seems poorly scalable, instead of an increase in generic simplicity and clarity of concepts. That does not bode well and we need to curb it lest we end up with a standard library we ourselves can't make sense of.
>
> I do understand there are issues to fix and we need to move forward. But tactical solutions cannot substitute a lack of strategy.
>
> Please let us all stop and draw a little breath, and figure out how to replace the monumental work in PR 3770 with simplifying abstractions. Once something like isStringLike becomes part of Phobos, we're stuck with it for eternity - there's no way we can fix it without breaking code.
>

Release is unavoidable. Let us ship it then figure out the optimal string like definition. The whole auto-decoding mess together with alias this mess is important to solve but must not prevent us from releasing on time.

>
>
> Thanks,
>
> Andrei
>
> _______________________________________________
> phobos mailing list
> phobos@puremagic.com
> http://lists.puremagic.com/mailman/listinfo/phobos


-- 
Dmitry Olshansky

On 10/26/15 12:08 PM, Dmitry Olshansky via phobos wrote:
> On 26-Oct-2015 18:15, Andrei Alexandrescu via phobos wrote:
>> Hi folks, isStringLike is a fine idea but there are a number of issues
>> I noticed while looking at the PR and the related pieces it works with.
>>
> [snip]
>> I think a string-like structure is:
>>
>> (a) an array of characters or a subtype thereof
>>
>> (b) some other forward range with ElementType some character type, and
>> potentially ElementEncodingType a different character type as well.
>
> Sadly the main reason for isStringLike is
> (c) alias this to something that has a or b defined above.

Nonono, (a) includes that case ("... or a subtype thereof"). The point of "alias this" is it makes subtyping happen. Any failure to do so is a failure of "alias this" and needs fixing.

> Release is unavoidable. Let us ship it then figure out the optimal
> string like definition. The whole auto-decoding mess together with alias
> this mess is important to solve but must not prevent us from releasing
> on time.

If we release now, we're stuck with isStringLike forever. Any chance of at least making it package?


Andrei
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On 26-Oct-2015 19:20, Andrei Alexandrescu wrote:
> On 10/26/15 12:08 PM, Dmitry Olshansky via phobos wrote:
>> On 26-Oct-2015 18:15, Andrei Alexandrescu via phobos wrote:
>>> Hi folks, isStringLike is a fine idea but there are a number of issues I noticed while looking at the PR and the related pieces it works with.
>>>
>> [snip]
>>> I think a string-like structure is:
>>>
>>> (a) an array of characters or a subtype thereof
>>>
>>> (b) some other forward range with ElementType some character type, and potentially ElementEncodingType a different character type as well.
>>
>> Sadly the main reason for isStringLike is
>> (c) alias this to something that has a or b defined above.
>
> Nonono, (a) includes that case ("... or a subtype thereof"). The point of "alias this" is it makes subtyping happen. Any failure to do so is a failure of "alias this" and needs fixing.
>
Agreed. That is my opinion as well, however luck of any attention from
you and Walter led to the point where we
need (a) and (b) to work right now and we hacked it (somewhat of
speaking for the community here, correct me if I'm wrong).
>> Release is unavoidable. Let us ship it then figure out the optimal string like definition. The whole auto-decoding mess together with alias this mess is important to solve but must not prevent us from releasing on time.
>
> If we release now, we're stuck with isStringLike forever. Any chance of at least making it package?
>
I believed it was private/package hack. Anyhow  we can redefine how isStringLike exactly works as long as semantics stay sane.

-- 
Dmitry Olshansky

On 10/26/15 12:27 PM, Dmitry Olshansky wrote:
> Agreed. That is my opinion as well, however luck of any attention from
> you and Walter led to the point where we
> need (a) and (b) to work right now and we hacked it (somewhat of
> speaking for the community here, correct me if I'm wrong).

I agree we need to make progress but that's a bit of a tautology and requires judgment on what is progress etc.

FWIW this is one of those cases in which short-term progress may hurt long-term progress. Again: tactical prowess, no matter how impressive, should not be a substitute for strategy.

>>> Release is unavoidable. Let us ship it then figure out the optimal
>>> string like definition. The whole auto-decoding mess together with alias
>>> this mess is important to solve but must not prevent us from releasing
>>> on time.
>>
>> If we release now, we're stuck with isStringLike forever. Any chance
>> of at least making it package?
>>
> I believed it was private/package hack. Anyhow  we can redefine how
> isStringLike exactly works as long as semantics stay sane.

I see isStringLike is public and documented. At the very least we need to make it undocumented.


Andrei
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On 26 Oct 2015, at 18:35, Andrei Alexandrescu via phobos wrote:
>>>> Release is unavoidable. Let us ship it then figure out the optimal
>>>> string like definition. The whole auto-decoding mess together with alias
>>>> this mess is important to solve but must not prevent us from releasing
>>>> on time.
>>>
>>> If we release now, we're stuck with isStringLike forever. Any chance
>>> of at least making it package?
>>>
>> I believed it was private/package hack. Anyhow  we can redefine how
>> isStringLike exactly works as long as semantics stay sane.
>
> I see isStringLike is public and documented. At the very least we need to make it undocumented.

I've not really been following this discussion, so please disregard the comment if it does not make sense, but: Wouldn't "isStringLike" appear in template constraints? If so, then not documenting it is bound to create even more confusion.

 — David
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On 10/26/2015 9:27 AM, Dmitry Olshansky via phobos wrote:
> Agreed. That is my opinion as well, however luck of any attention from you and Walter led to the point where we
> need (a) and (b) to work right now and we hacked it (somewhat of speaking for the community here, correct me if I'm wrong).

My recommendation was to remove the alias this in DirEntry that tried to turn it into a string. A DirEntry is not a wrapper for a string, in particular, it does not support range primitives, which is why the string functions failed when changed to have InputRange parameters. Adding range primitives to DirEntry would cause other weird problems, because DirEntry is fundamentally not a range (a range 'consumes' its data via popFront() which makes no sense for DirEntry).

Yes, that breaks:

    DirEntry direntry;
    remove(direntry);

which must be rewritten as:

    DirEntry direntry;
    remove(direntry.name);

and we're not supposed to break existing code. But the alternative, which is redoing every template that accepts an InputRange, seems an awfully heavy burden. Even if we do this change pervasively in Phobos, how does that help anyone else who uses InputRanges?

Tl,dr: the alias this in DirEntry is an abusive bad practice, and should never have happened.
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On Monday, 26 October 2015 at 21:27:37 UTC, Walter Bright wrote:
> and we're not supposed to break existing code. But the alternative, which is redoing every template that accepts an InputRange, seems an awfully heavy burden. Even if we do this change pervasively in Phobos, how does that help anyone else who uses InputRanges?
>
> Tl,dr: the alias this in DirEntry is an abusive bad practice, and should never have happened.

I agree that for DirEntry it may have been dubious practice, but it is a problem in general.

Since nobody else is making the case for the elephant in the room, I'll make the case for changing IFTI to consider AliasThis. Please excuse the informal and clumsy presentation.

Let's use isDir as our function template and DirEntry as the offending subtype:

---
@property bool isDir(R)(R name) if (isInputRange!R && isSomeChar!(ElementEncodingType!R));

foreach(f; dirEntries("/", "*", SpanMode.shallow, false))
    if(isDir(f)) // Error
        ...
---

Errors with: "Error: template std.file.isDir cannot deduce function from argument types !()(DirEntry)"

IFTI is in effect when calling isDir: isDir!DirEntry is attempted, which doesn't match the template constraints.

However, explicitly instantiating isDir to receive the supertype works:

---
foreach(f; dirEntries("/", "*", SpanMode.shallow, false))
    if(isDir!string(f)) // OK
        ...
---

This gives us the same behaviour we had before templatization: the argument is implicitly converted to the supertype at the call-site.

An IFTI rule supporting this doesn't have to be complicated AFAICS: if there are no matches for exact type T, recursively try supertypes of T until first match. If no matches, error out. If more than one match at one level of AliasThis (hypothetical multiple AliasThis), error out.

Interestingly, a side effect would be that it gives more importance to template constraints. I don't know if that's a good or a bad thing.
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos

On Monday, 26 October 2015 at 21:27:37 UTC, Walter Bright wrote:
> My recommendation was to remove the alias this in DirEntry that tried to turn it into a string. A DirEntry is not a wrapper for a string, in particular, it does not support range primitives, which is why the string functions failed when changed to have InputRange parameters. Adding range primitives to DirEntry would cause other weird problems, because DirEntry is fundamentally not a range (a range 'consumes' its data via popFront() which makes no sense for DirEntry).
>
> Yes, that breaks:
>
>     DirEntry direntry;
>     remove(direntry);
>
> which must be rewritten as:
>
>     DirEntry direntry;
>     remove(direntry.name);
>
> and we're not supposed to break existing code. But the alternative, which is redoing every template that accepts an InputRange, seems an awfully heavy burden.

DirEntry is not the only source of problems. For example, the rangification also broke ae.utils.funopt, which also uses structs with an alias this'd string.

http://blog.thecybershadow.net/2014/08/05/ae-utils-funopt/
_______________________________________________
phobos mailing list
phobos@puremagic.com
http://lists.puremagic.com/mailman/listinfo/phobos