Porting std.typecons to Phobos 3

While JMD continues his work on Ranges V3 he mentioned that it would help him out if we could get started porting std.typecons to Phobos 3. Therefore, I'd like to open this module up to the community to see what everyone thinks of the plan we've developed.

The first thing I'd like to consider is renaming to phobos.sys.types. It is difficult to name std.typecons because it is a grab-bag of type-related tools. We can continue with typecons if desired, however, I would note that "cons" is not a standard abbreviation for "constructor", that'd be "ctor".

Second, JMD proposed that we re-organize this module into a package due to it's grab-bag nature. While I want to stress that we won't be doing this with every module Walter has asked that we evaluate modules for reasonable splits to make the files smaller and easier to read. Given the nature of std.typecons we think that this module is an excellent candidate. My proposal is to break this module up by the categories in the document with the exception of Nullable which will get it's own file module due to it's size.

Speaking of Nullable, after a rather embarrassing situation with a Phobos 2 PR on Nullable JMD and I agreed that it should be renamed as it does not actually behave like a nullable but instead behaves as an "Optional" type. We considered Optional, Maybe, and settled on Option as that appears to the more popular naming choice in other languages and is almost as short as Maybe. If you disagree I'd love to hear your case.

The last question is how we want to deal with existing bugs in V2 as we port the code to V3. My view is that it would be prudent to fix any bugs that have been opened as we go in both V2 and V3. Currently I've found a number of bugs against Nullable and Tuple, the two types I'd like to start with.

• First, some of these bugs are likely already fixed and just didn't get cleared by the bot for whatever reason, but I don't have the domain knowledge to know which, if any, already have been. If somebody could flag those I'll get them closed.
• Second, because some of these fixes will involve breaking changes, we need to decide which bugs we want to fix in both V2 and V3 and which we want to fix only in V3. If you could reply with "Both" or "V3" for fixes that'd be most helpful.
• Lastly, I found these via simple search through the bug list, if they don't apply, or if I am missing any, please let me know.

Nullable Bug List:

9637 - std.typecons.nullable and nullableRef functions (It looks to me like these exist, do the existing methods satisfy the issue?)
12323 - std.typecons.Nullable of a fixed-size array
13092 - std.algorithm.cartesianProduct of Nullable Tuple of strings array too
17615 - Deprecate using reference types for Nullable
21420 - Nullable.get default parameter is not lazy
22279 - alias this in Nullable(T, T nullValue) causes Error in writeln
24270 - Pointer to Nullable of recursive struct
24233 - T[].canFind(Nullable!T()) returns true when it used to return false

Tuple Bug List:

8494 - Return value for Tuple.opAssign
9593 - Optional type-free printing of a std.typecons.Tuple
12743 - tuple printing of alias names
13104 - std.typecons.tupleOp
15740 - Indexing a std.typecons.Tuple with names doesn't work in ctfe
15905 - Tuple Op Assignment Overload incorrect?
18426 - Conversion from Tuple with named fields to Tuple without named fields should work during compilation
20031 - Tuple.toString does not work with mutable toString
20228 - simple toString for Tuple!()s
20966 - std.typecons.Tuple cannot handle non-copyable types
21011 - Variant and tuples by index
21401 - Writelning a tuple ignores member's toString.
21429 - Cannot sort large tuple arrays at compile time
21888 - sorting medium sized tuples at compile time not possible
24138 - [REG 2.103] MemberFunctionsTuple does not see final functions
24285 - Cannot swap a std.typecons.Tuple (Open PR for this Issue)

• https://github.com/WalterBright/documents/blob/master/sumtype.md

• https://github.com/tgehr/DIPs/blob/tuple-syntax/DIPs/DIP1xxx-tg.md

I couldn't find the proposed DIP for nullable, so let's see what other language did:

• Rust: https://doc.rust-lang.org/rust-by-example/std/option.html

  • not builtin, but they have pattern matching, so it's as bad to use

• Zig: built in and very nice to use: https://ziglang.org/documentation/master/#Optionals

• C3: same, built in, very nice to use: https://c3-lang.org/language-common/optionals-essential/

• Swift: builtin(hack), very nice to use: https://developer.apple.com/documentation/swift/optional

• odin: builtin, in the sense you have nothing to import, https://odin-lang.org/docs/overview/#maybet

I could go on and mention the other popular high level language, C#, Java, Kotlin, they all agreed to promote them as essential language feature

The question to be asked is, there is demand for better Tuple, Sumtype and Optionals, why not take the opportunity to give better, ie: not requiring importing bunch of templates

We want great build time and great error messages

"The US government wants developers to stop using C and C++"

https://www.theregister.com/2024/11/08/the_us_government_wants_developers/

Only the compiler can help, the difference between a library and a core concept of a language, the plan is to make D attractive to that crowd, and not make them discover better C/C++ in Rust or others

On Saturday, 9 November 2024 at 05:44:37 UTC, ryuukk_ wrote:

They are not built-ins in C#, it's a library type: System.Option<T>. A cursory search shows that Java is also a library type, specifically java.util.Optional<T>. Please do a cursory web search before making false assertions.

This is letting perfect be the enemy of good. As it stands today, there are no approved DIP's for either of these types. Much less merged implementations. And until an implementation is merged into DMD, Phobos cannot count on the existence of something.

Standard libraries are necessarily highly conservative things. Currently, we don't even allow some things that have been implemented, such as DIP1000, because they are too complex/buggy/etc. It is reckless to take a dependency on something that hasn't even been specified, much less implemented. If these get implemented in the language prior to the release of Phobos 3, we can reevaluate, but even then we may chose not to use it if the implementation is too buggy or incomplete.

On Saturday, 9 November 2024 at 04:34:20 UTC, Adam Wilson wrote:

To reiterate my position from the planning forums; v2 tuple and nullable are some of the worse code in phoboes.

Tuple belong in std.meta right next to aliasSeq, because it could be just a seq in a struct

struct Tuple(T...){
  T expand; alias expand this;
}
auto tuple(T...)(T t)=>Tuple!T(t);
auto pair(T,S)(T t,S s)=>Tuple!(T,S)(t,s);

(really go try the syntax that falls out of this; it is just this simple)

Fundamentally, phoboes v2 works the same way, possibly its the only way it even could work in d expect it hides it under 10000 lines of code.

Likewise nullable but its less exterme(you have to write the op overloads) but it should still be a simple struct with a bool attached, I think there should be two for the failsafe vs fail dangerous usecases.

On Saturday, 9 November 2024 at 05:44:37 UTC, ryuukk_ wrote:

Thanks for links.
In D there are many options as well :)

https://code.dlang.org/packages/optional
https://code.dlang.org/packages/d-option
https://code.dlang.org/packages/lib-option
https://code.dlang.org/packages/optibrev
https://code.dlang.org/packages/option

and expected
https://code.dlang.org/packages/expected
https://code.dlang.org/packages/expectations

On Saturday, 9 November 2024 at 13:47:22 UTC, monkyyy wrote:

struct Tuple(T...){
  T expand; alias expand this;
}
auto tuple(T...)(T t)=>Tuple!T(t);
auto pair(T,S)(T t,S s)=>Tuple!(T,S)(t,s);

I chatted with Timon a bit on Discord last night and he is in a position to get language level tuples into the compiler before V3 ships. He has the implementation mostly done so all he really needs to do is write up the DIP and get it approved.

One solution proposed was to add Tuple to V3 with a deprecation warning and no docs to unblock range work, and then remove it when language tuples land.

I am intrigued by the concept of a fail-unsafe Option type, but I'm not sure what the use case would be. When would you want to explicitly do the unsafe thing? And could you not just achieve the same result by putting ! in front of the value presence check?

On Sunday, 10 November 2024 at 01:16:40 UTC, Adam Wilson wrote:

I assume you got the terminology backwards, because computer science picked up the term failsafe to just mean a mechanism that detects edge cases.

Given a mechanism that detects an edge case, do you explode? If no, failsafe, if yes fail dangerous(or as the wokes call it "fail secure")

Given a sane range api with the ability to implement indexOf, if you fail to find the index you should return a maybe!typeof(key) assuming maybe is the premissive one, and option is the restrictive one; current countUntil lets you do math with the -1 and maybe sometimes that just works, if you threw an exception in some code path youd need a seperate nogc version, you'd break some versions of code that just work etc.