On Tuesday, 7 January 2020 at 01:10:08 UTC, Jonathan M Davis wrote:
> Are you talking about putting them all in a single module or all in a single package?

Sorry, for being unclear here. I thought of a package.

> Now, personally, I don't think that anything regarding file formats should have been in the standard library in the first place.

Thinking about this whole stuff, I noticed, that there are two different points of view, which should be separated: The idealist view and the pragmatic view. IMHO both are important.

So when I got you right, from an idealists view, you'd say these file formats should be removed from phobos, but from a pragmatists view this looks much more difficult.

I think, I share this point of view. But I'd like to get rid of them anyway.

> For the most part, I don't see any point in removing any of these modules, since that would break existing code,

Well, every module, that is kept inside Phobos produces (lots of) maintainance work. From my perspective, we are missing resources here. So I prefere a controlled breaking of code (with deprecation and all) instead of having the code rosting and breaking uncontrolled sooner or later.

I came up with this issue, when I looked at my own comment on issue 17709 [1]: I found the reason for this issue and I think I could fix that in a reasonable amout of time. But is it worth it doing so if that module might be removed or replaced in the near future? Wouldn't it be much better to use that time to fix a bug at a more important place?

But on the other side: How does such a comment look like to someone how is using std.xml and found that issue, cause he stumbled over the same problem? Wouldn't it be better to remove std.xml completely in the first place?

[1] https://issues.dlang.org/show_bug.cgi?id=17709

And than, there is something more to be thought of: The (public) perception of Phobos. If it contains parts, that are more or less broken, useless or outdated, this will be the parts where people will look at, when they judge. It won't help much to have other parts, that work well.

> BTW, base64 isn't really a file format. It's an encoding.

Really? So why isn't it in std.encoding? :-) I know, that base64 is somewhat different, maybe it's in the gray area... Or look at it the other way round: Isn't zipping also just encoding and unzipping decoding?

On 08/01/2020 1:44 AM, berni44 wrote:
>> BTW, base64 isn't really a file format. It's an encoding.
> 
> Really? So why isn't it in std.encoding? :-) I know, that base64 is somewhat different, maybe it's in the gray area... Or look at it the other way round: Isn't zipping also just encoding and unzipping decoding?

Perhaps, just perhaps std.encoding should be a package too ;)



A zip file is a container, an encoding is not a container, it is a representation (just with different assumptions i.e. the base).

On Tuesday, 7 January 2020 at 05:55:24 UTC, Steven Schveighoffer wrote:
> I definitely think we should have a repository system that separates "blessed" and maintained projects from all the ones that are added by random people.

The whole thing feels a little bit like an onion: The core is the core language, than the runtime, then Phobos and then, there is a skin missing (the one you describe) and then code.dlang.org and maybe next to whole internet.

The closer to the core, the more reliable, but less flexible. So in Phobos there should only be essential stuff, like "writeln".

On Tue, Jan 07, 2020 at 12:17:57PM +0000, berni44 via Digitalmars-d wrote:
> On Tuesday, 7 January 2020 at 00:10:08 UTC, H. S. Teoh wrote:
> > After so many decades of academia and industry alike trumpeting code reuse, I'm starting to become skeptical that perhaps King Code Reuse has invisible clothes.
[...]
> Years ago I came to the same conclusion but in a completely different area: I was programming generators for sudoku puzzles and it's variants. This looks like being ideal for OOP, because all those variations are quit similar, but it proved not to be.

Yeah, OOP is another of those things that IMNSHO has been overhyped far beyond its actual scope of usefulness. It *does* have its uses, that's undeniable, but only to certain classes (har har) of programming problems. In writing GUI components, for example, it seems to work really well.  Trying to shoehorn *every* programming problem into an OO model, however, to me is a code smell.

(Ironically, modern OO design advice is to avoid inheritance and embrace composition, which to me is a sign that polymorphism via inheritance, one of the foundational features of OOP, isn't as widely applicable as it's hyped to be.)


> The puzzles, the program produced, where not really good. And the reason was, that every variation needs it's own optimization for good results and that was almost impossible with all those code sharing. Eventually I rewrote everything with a single program for every variation. That worked much better, although it was code repetition over and over again.

Mind you, I'm not against code reuse via refactoring per se. Too much code repetition is also a code smell... but refactoring and generalizing to the point of obsession is a sign that it has gone off the deep end.

Sometimes, it's just not worth the trouble, and the technical debt, to factor out the common code of two (or more) similar pieces of code. They may be similar but differ in complex ways that cannot be easily captured via a simple abstraction, and if you try to forcefully push the refactoring through anyway, you usually end up with a completely disproportionately more complex architecture that suffers from obvious symptoms of premature (or over-) generalization and over-engineering. The result usually is an order of magnitude more difficult to maintain, disproportionately harder to understand, and contains disproportionately more LOC.  In such cases, the best solution may actually be copy-n-paste + modify. The code may well turn out to be simpler and more maintainable that way!


T

-- 
Why do conspiracy theories always come from the same people??

On Tuesday, January 7, 2020 5:44:40 AM MST berni44 via Digitalmars-d wrote:
> On Tuesday, 7 January 2020 at 01:10:08 UTC, Jonathan M Davis
>
> wrote:
> > Now, personally, I don't think that anything regarding file formats should have been in the standard library in the first place.
>
> Thinking about this whole stuff, I noticed, that there are two different points of view, which should be separated: The idealist view and the pragmatic view. IMHO both are important.
>
> So when I got you right, from an idealists view, you'd say these file formats should be removed from phobos, but from a pragmatists view this looks much more difficult.
>
> I think, I share this point of view. But I'd like to get rid of them anyway.

std.xml is probably the only one from your list that I'd argue should be seriously considered for being removed from Phobos and moved into undead sooner rather than later. I don't know quite what state std.json is in, so maybe it should have the same done to it, though it's had some work done on it recently to try to improve it. I see no reason to rip out stuff like std.csv or std.zip at this point though. They work and are useful. They also aren't fundamentally broken in the way that std.xml is AFAIK.

> > For the most part, I don't see any point in removing any of these modules, since that would break existing code,
>
> Well, every module, that is kept inside Phobos produces (lots of) maintainance work. From my perspective, we are missing resources here. So I prefere a controlled breaking of code (with deprecation and all) instead of having the code rosting and breaking uncontrolled sooner or later.
>
> I came up with this issue, when I looked at my own comment on issue 17709 [1]: I found the reason for this issue and I think I could fix that in a reasonable amout of time. But is it worth it doing so if that module might be removed or replaced in the near future? Wouldn't it be much better to use that time to fix a bug at a more important place?
>
> But on the other side: How does such a comment look like to someone how is using std.xml and found that issue, cause he stumbled over the same problem? Wouldn't it be better to remove std.xml completely in the first place?
>
> [1] https://issues.dlang.org/show_bug.cgi?id=17709

std.xml is broken. We've agreed for years now that it should go. It's just that there has been no agreement on removing it without having a replacement, which is why it's still there. I don't think that it's worth your time to work on it. However, out of the list of modules that you provided, std.json is the only other one where I recall any real discussion about replacing or removing. Certainly, spending time fixing a bug in something like std.base64 or std.zip is not a waste of time.

> > BTW, base64 isn't really a file format. It's an encoding.
>
> Really? So why isn't it in std.encoding? :-) I know, that base64 is somewhat different, maybe it's in the gray area... Or look at it the other way round: Isn't zipping also just encoding and unzipping decoding?

Encoding involves taking information and converting it into another format which contains the same information in a different manner. File formats may use encodings for some of the information that they contain, but an encoding has to do with how information is encoded. e.g. Unicode code points are encoded with UTF-8, UTF-16, or UTF-32. All three encodings contain exactly the same information, but the way that that information is encoded differs. And while UTF-8 may be used inside a file, it is in no way tied to files. It's just a way that Unicode character information is encoded.

base64 is a binary encoding, whereas std.encoding deals with character encodings. zip is a file / container format. It uses different compression algorithms to encode binary information internally, but zip itself is a container format, not an encoding. It does far more than encode a string of data in a different way like an encoding does.

std.encoding is also a bit of an oddball. It's an older module that probably needs to be revamped / redesigned. It has some level of support for various character encodings - including UTF-8, UTF-16, and UTF-32 - but we have std.utf for UTF handling, and std.utf is what gets used by Phobos for handling UTF encodings. std.encoding is class-based and has some range support, but it isn't really range-based aside from improvements that have been made to it over time. It does get some occasional tweaks, but largely, it's an older module with an older design that doesn't necessarily fit all that well into the rest of Phobos. I don't know what should be done with it though. Some of what it does is stuff that we really should have, but it probably needs to be redesigned. However, somebody would have to step up to do that.

- Jonathan M Davis

I'd be good with thinking about moving std.xml to dub. But not to unDead, as there is no replacement.

On Tuesday, January 7, 2020 7:20:05 PM MST Walter Bright via Digitalmars-d wrote:
> I'd be good with thinking about moving std.xml to dub. But not to unDead, as there is no replacement.

Why? I thought that the whole point of undead was for stuff from Phobos that we wanted to get rid of but still wanted to be available for older code would go to undead. I don't see how whether there is a replacement in Phobos is relevant. e.g. std.stream and friends were put into undead, and we don't really have a replacement for those. You can solve the problems that those modules try to solve in a different way using what's in Phobos, but we don't really have a stream solution in Phobos right now. What would be better about putting std.xml in its own project instead of undead?

And honestly, from what I've seen of std.xml, I think that anyone using it should just plain seriously reconsider. I don't know how buggy it actually is, but it clearly was doing some stuff wrong such that it wouldn't be able to properly handle at least some XML files (though at the moment, I don't recall exactly what I found in there).

And we do have multiple xml solutions available on code.dlang.org now. So, while we may not have a "blessed," default solution for people to switch to, there are better options available than std.xml.

Also, I think that putting std.xml in its own project makes it too likely that someone else would use it in a new project, whereas if it's in undead, it's unlikely to be used by new projects, just older projects that were already using it. We've considered std.xml to be a poor solution for years now (we've even had a warning in its documentation that it was going to go away because it isn't up to our current standards), and I don't think that we should be encouraging its use. Putting it in undead makes it available without presenting it as a solution that people should be using. It's also where we've already been putting all of the modules that we decided to get rid of from Phobos, and we haven't really used the criteria of needing a replacement previously as far as whether it goes in undead or elsewhere. That's just been part of the question of whether we would remove it, and in some cases, we have removed stuff without really having a replacement in Phobos (most notably std.stream and friends).

- Jonathan M Davis

On 1/7/2020 10:51 PM, Jonathan M Davis wrote:
> [...]

You raise some good points. I am not an XML user, so I don't have an informed opinion about this.

On Wednesday, 8 January 2020 at 08:26:33 UTC, Walter Bright wrote:
> On 1/7/2020 10:51 PM, Jonathan M Davis wrote:
>> [...]
>
> You raise some good points. I am not an XML user, so I don't have an informed opinion about this.

Filed a PR to add std.xml to undeaD: https://github.com/dlang/undeaD/pull/37

OK?

On 11/01/2020 3:11 AM, berni44 wrote:
> On Wednesday, 8 January 2020 at 08:26:33 UTC, Walter Bright wrote:
>> On 1/7/2020 10:51 PM, Jonathan M Davis wrote:
>>> [...]
>>
>> You raise some good points. I am not an XML user, so I don't have an informed opinion about this.
> 
> Filed a PR to add std.xml to undeaD: https://github.com/dlang/undeaD/pull/37
> 
> OK?

This should have a vote and then yes we can deprecate it.