On 1/7/19 5:10 PM, H. S. Teoh wrote:
> On Mon, Jan 07, 2019 at 04:54:15PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
>> On 1/7/19 4:41 PM, H. S. Teoh wrote:
>>> On Mon, Jan 07, 2019 at 01:25:17PM -0800, Manu via Digitalmars-d wrote:
>>> [...]
>>>> Perhaps AliasSeq should live somewhere different?  I'm feeling
>>>> like a lean/trimmed-down core.meta might want to exist next to
>>>> core.traits though; it seems reasonable.
>>>
>>> I'm afraid this would set the wrong precedent -- since there's
>>> core.traits for std.traits and core.meta for std.meta, why not also
>>> have core.typecons, core.range, and then it's all gonna go downhill
>>> from there, and before you know it there's gonna be core.stdio and
>>> core.format... *shudder*
>>
>> std.internal.traits contains pieces of std.meta -- a quick look shows
>> it has AliasSeq (but under the name TypeTuple),
> 
> What, wut...?  `TypeTuple` still exists?! I thought we had gone through
> a somewhat painful deprecation cycle just to kill it off. Or is that
> cycle not done yet...?

It's an internal definition, so it can be anything it wants it to be. It could be Tuple or Foobar.

> 
> 
>> allSatisfy, anySatisfy, Filter, staticMap. We might as well just move
>> the whole thing there.
> 
> I dunno, IMO allSatisfy, anySatisfy, and esp. Filter and staticMap are
> all heavy-weight templates (not in terms of code complexity, but in the
> sheer amount of templates that will get instantiated when you use them,
> AKA compiler slowdown fuel).  I'm not so sure they should go into
> druntime.

Again, they are already there, and for a reason.

>> The fear of moving other pieces is real, but only if you look
>> superficially.  traits and meta are really part of the language, I
>> can't imagine using D without it (and neither can any of the people
>> who put pieces of those modules into druntime).
> 
> Actually, I find myself redefining AliasSeq locally with a shorter name
> all the time.  Scarily enough, doing that is shorter than typing `import
> std.traits : AliasSeq;`.

The point is to have it universally recognized construct. When your code's documentation says `TList` or whatever, then someone has to go figure that out. If it says `AliasSeq`, it's known what it is.

>> I don't want to see anything more complex and interdependent get
>> sucked in.  I know the goal for Manu right now is emplace, which does
>> feel like a language feature. But as has been said many times here,
>> std.traits is a no-brainer, and std.meta is really a building block
>> that std.traits uses.
> [...]
> 
> Wait, so you're saying we should move the *entire* std.meta and
> std.traits to druntime...?

At least std.meta. std.traits can possibly be split into critical language-enabling traits and the less important ones, but I don't know off the top of my head which ones are those. But I would say std.meta is composed only of the former variety.

-Steve

On Mon, Jan 7, 2019 at 1:42 PM H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Mon, Jan 07, 2019 at 01:25:17PM -0800, Manu via Digitalmars-d wrote: [...]
> > Okay, so this is a challenging effort, since phobos is such a tangled rats nets of chaos...
>
> It's already gotten better over the years in some ways (though not others -- unfortunately I'm afraid std.traits might be one of the places where things have probably gotten more tangled).  It certainly hasn't lived up to the promise of that old Phobos Philosophy page that once existed but has since been removed, of Phobos being a collection of lightweight, self-contained, mostly-orthogonal, reusable components.  It has become quite the opposite, where the dependency graph of Phobos modules is approaching pretty close to being a complete graph.  (And yes, there are some pretty deep-seated cyclic dependencies that thus far nobody has been able to truly unravel in any satisfactory way.)
>
>
> > But attempting to move some traits immediately calls into question std.meta.  I think we can all agree that Alias and AliasSeq should be in druntime along with core traits... but where should it live? Should there be core.meta as well? It's kinda like core.traits, in that it doesn't include runtime code, it doesn't increase the payload of druntime.lib for end-users..
>
> Shouldn't all of core.traits be like that?  I'd hardly expect any runtime component to be associated with something called 'traits'.
>
>
> > Perhaps AliasSeq should live somewhere different?
> > I'm feeling like a lean/trimmed-down core.meta might want to exist
> > next to core.traits though; it seems reasonable.
>
> I'm afraid this would set the wrong precedent -- since there's core.traits for std.traits and core.meta for std.meta, why not also have core.typecons, core.range, and then it's all gonna go downhill from there, and before you know it there's gonna be core.stdio and core.format... *shudder*

Well, I think you're kinda catastrophising here... I said very clearly
"The only way forward is to take each hurdle one at a time", and I
mean that as literally as possible.
One at a time...

The question is: core.meta... yeah?
No precedents are being set.. we're not opening flood gates, it's a
singular question.

> > ...yes, this process will go on and on. The only way forward is to take each hurdle one at a time... and ideally, in attempting this effort, we can de-tangle a lot of cruft during the process.
>
> I'm tempted to say we should put everything in core.traits for now. And just the absolute bare minimum it takes to meet whatever druntime needs, and nothing more.

Are you saying we should put *everything* in core.traits? That is, put AliasSeq in core.traits? It's objectively NOT a 'traits'...

On Mon, Jan 7, 2019 at 2:10 PM H. S. Teoh via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Mon, Jan 07, 2019 at 04:54:15PM -0500, Steven Schveighoffer via Digitalmars-d wrote:
> > On 1/7/19 4:41 PM, H. S. Teoh wrote:
> > > On Mon, Jan 07, 2019 at 01:25:17PM -0800, Manu via Digitalmars-d wrote: [...]
> > > > Perhaps AliasSeq should live somewhere different?  I'm feeling like a lean/trimmed-down core.meta might want to exist next to core.traits though; it seems reasonable.
> > >
> > > I'm afraid this would set the wrong precedent -- since there's core.traits for std.traits and core.meta for std.meta, why not also have core.typecons, core.range, and then it's all gonna go downhill from there, and before you know it there's gonna be core.stdio and core.format... *shudder*
> >
> > std.internal.traits contains pieces of std.meta -- a quick look shows it has AliasSeq (but under the name TypeTuple),
>
> What, wut...?  `TypeTuple` still exists?! I thought we had gone through a somewhat painful deprecation cycle just to kill it off. Or is that cycle not done yet...?
>
>
> > allSatisfy, anySatisfy, Filter, staticMap. We might as well just move the whole thing there.
>
> I dunno, IMO allSatisfy, anySatisfy, and esp. Filter and staticMap are all heavy-weight templates (not in terms of code complexity, but in the sheer amount of templates that will get instantiated when you use them, AKA compiler slowdown fuel).

Ummm, well, yes and no. staticMap is DEFINITELY like you say, one of
the worst, and I really think it needs a language solution.
C++'s `...` expression is exactly staticMap, and I wonder if we need
an expression like that in-language to take load off staticMap,
because it's perhaps one of the slowest parts f the language.
We can translate almost anything to CTFE, *except* code that invokes
staticMap, so I really reckon it needs a language tool.

This is an aside, if we wanna talk about staticMap, we should start a new thread.

> I'm not so sure they should go into druntime.

Then you're kinda making a case that NONE of it should go in druntime,
because they're such common building blocks.
Also, you've already lost the game, because they're already in
druntime (in core.internal).

> > The fear of moving other pieces is real, but only if you look superficially.  traits and meta are really part of the language, I can't imagine using D without it (and neither can any of the people who put pieces of those modules into druntime).
>
> Actually, I find myself redefining AliasSeq locally with a shorter name all the time.  Scarily enough, doing that is shorter than typing `import std.traits : AliasSeq;`.

I've been known to do that too... I could do that for core.traits aswell, but that seems pretty lame.

> > I don't want to see anything more complex and interdependent get sucked in.  I know the goal for Manu right now is emplace, which does feel like a language feature. But as has been said many times here, std.traits is a no-brainer, and std.meta is really a building block that std.traits uses.
> [...]
>
> Wait, so you're saying we should move the *entire* std.meta and std.traits to druntime...?

I'd like to be more selective than that; at very least, audit every single symbol coming across. But some part of me thinks this may actually be a very reasonable proposal...

On Monday, January 7, 2019 3:10:02 PM MST H. S. Teoh via Digitalmars-d wrote:
> On Mon, Jan 07, 2019 at 04:54:15PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
> > On 1/7/19 4:41 PM, H. S. Teoh wrote:
> > > On Mon, Jan 07, 2019 at 01:25:17PM -0800, Manu via Digitalmars-d
> > > wrote:
> > > [...]
> > >
> > > > Perhaps AliasSeq should live somewhere different?  I'm feeling like a lean/trimmed-down core.meta might want to exist next to core.traits though; it seems reasonable.
> > >
> > > I'm afraid this would set the wrong precedent -- since there's core.traits for std.traits and core.meta for std.meta, why not also have core.typecons, core.range, and then it's all gonna go downhill from there, and before you know it there's gonna be core.stdio and core.format... *shudder*
> >
> > std.internal.traits contains pieces of std.meta -- a quick look shows it has AliasSeq (but under the name TypeTuple),
>
> What, wut...?  `TypeTuple` still exists?! I thought we had gone through a somewhat painful deprecation cycle just to kill it off. Or is that cycle not done yet...?

We're talking about druntime internals here. TypeTuple was one of the things that got copied into druntime as internal. It was later renamed to AliasSeq in Phobos, but the copied stuff in druntime didn't necessarily get changed, since it was all internal and had nothing to do with the Phobos stuff it originated from.

> > allSatisfy, anySatisfy, Filter, staticMap. We might as well just move the whole thing there.
>
> I dunno, IMO allSatisfy, anySatisfy, and esp. Filter and staticMap are all heavy-weight templates (not in terms of code complexity, but in the sheer amount of templates that will get instantiated when you use them, AKA compiler slowdown fuel).  I'm not so sure they should go into druntime.

Historically, we've tried to only put stuff in druntime that needs to be in druntime for druntime to do what it does. That does sometimes get stretched (e.g. we put all of the OS bindings in druntime rather than just the ones that druntime needs), but Phobos is the standard library, not druntime. druntime is the runtime. If something needs to be there so that the runtime can do its thing, then we put it in druntime, but not much else should be there. Certainly, anything that is there needs to be stuff that's actually core. So, in that respect, it's pretty questionable to move all of std.traits and std.meta into druntime wholesale.

What we had for a while was traits just being copied into druntime where they were needed, resulting in duplicate implementations in various places (especially for TypeTuple). Later, they were largely consolidated into core.internal.traits, but they were still completely separate from Phobos. More recently, the traits in core.internal.traits have had their std.traits implementations turned into simple wrappers that use the core.traits symbols (they're not aliased, because that doesn't work well with the documentation, so you get an extra layer of template with every trait whose implementation is in druntime).

So, if Manu needs a trait for something in druntime, the normal thing to do at this point would be to just add it to core.internal.traits and then potentially make Phobos wrap the druntime symbol. We don't actually need to move anything wholesale, nor do we need something like core.traits which is intended to make the traits publicly available from druntime.

Another issue here if you actually look at core.traits and std.meta is that there are actually several templates which use other pieces of Phobos. e.g. isInputRange gets used in std.meta.aliasSeqOf as does std.array.array, and std.traits.packageName uses startsWith. So, while many of the templates from std.traits and std.meta could be moved, actually trying to move them all wouldn't actually work without reimplementing yet other pieces of Phobos.

Personally, I think that the only real benefit of having something like core.traits over having core.internal.traits is that you can move the documentation for those symbols to their druntime implementations and make the Phobos implementations actual aliases with just a link to the druntime documentation instead of needing a thin wrapper template. Anyone wanting to avoid Phobos can still use those traits from std.traits and std.meta just fine, since it wouldn't involve linking against Phobos, just importing the module (even some of the traits that we can't move would work just fine, because they'd just be pulling in other Phobos symbols that don't involve linking). But it does get pretty weird to have some of the traits in core and some in std with no real obvious distinction, since it's based on what druntime needs. So, in that respect, keeping them internal is cleaner.

Either way, I think that it's quite clear that we can't move everything.

- Jonathan M Davis

On Saturday, 5 January 2019 at 21:12:54 UTC, Manu wrote:

> We should move them to core.traits, and that should be their official home. It really just makes sense. Uncontroversial low-level traits don't belong in phobos.

I'm not really doing much with D anymore, so I apologize for interrupting the conversation, but I had an idea I wanted to offer just in case it might help.

This was something I tried to tackle about 6 months ago, called it "UtiliD" (https://forum.dlang.org/post/wgkbamnlraustaycbbya@forum.dlang.org).  I ultimately failed due to the tangle of Phobos, and because my life priorities changed I gave up and deleted my repository (oops!).

Anyway, my suggestion is to create a new library separate from druntime and phobos that has no dependencies whatsoever (no libc, no libstdc++, no OS dependencies, no druntime dependency, etc.).  I mean it; **no dependencies**.  Not even object.d.  The only thing it should require is a D compiler.

That library can then be imported by druntime, phobos, betterC builds, or even the compiler itself. It will take strict enforcement of the "no dependency" rule and good judgment to keep the scope from ballooning, but it may be a good place for things like `traits`, `meta` and others.

Hope I'm not just making noise.

Mike

On Tuesday, 8 January 2019 at 01:44:08 UTC, Mike Franklin wrote:

> Anyway, my suggestion is to create a new library separate from druntime and phobos that has no dependencies whatsoever (no libc, no libstdc++, no OS dependencies, no druntime dependency, etc.).  I mean it; **no dependencies**.  Not even object.d.  The only thing it should require is a D compiler.
>
> That library can then be imported by druntime, phobos, betterC builds, or even the compiler itself. It will take strict enforcement of the "no dependency" rule and good judgment to keep the scope from ballooning, but it may be a good place for things like `traits`, `meta` and others.

I spent some time trying to think through some of the issues with druntime, and came up with this:

Right now, druntime is somewhat of a monolith trying to be too many things.
  * utilities (traits, string utilities, type conversion utilities, etc...)
  * compiler lowerings
  * C standard library bindings
  * C++ standard library bindings
  * C standard library bindings
  * Operating system bindings
  * OS abstractions (thread, fibers, context switching, etc...)
  * Compiler lowerings
  * DWARF implementation
  * TLS implementation
  * GC
  * (probably more)

So, I suggest something like this:
----------------------------------
* core.util - a.k.a utiliD - Just utility implementations written in D (e.g `std.traits`, `std.meta`, etc. No dependencies whatsoever. No operating system or platform abstractions. No high-level language features(e.g. exceptions)
    * public imports: (none)
    * private imports: (none)

* core.stdc - C standard library bindings - libc functions verbatim; no convenience or utility implementations
    * public imports: (none)
    * private imports: core.util

* core.stdcpp - C++ standard library bindings - libstdc++ data structures verbatim; no convenience or utility implementations
    * public imports: (none)
    * private imports: core.util

* sys - OS/Platform bindings - operating system implementations verbatim; no convenience or utility implementations
    * public imports: (none)
    * private imports: core.util

* core.pal - Platform/OS abstractions - threads, fibers, context switching, etc.
    * public imports: (none)
    * private imports: core.util, sys, core.libc

* core.d - compiler support (compiler lowerings, runtime initialization, TLS implementation, DWARF implementation, GC, etc...)
    * public imports : core.util
    * private imports : core.pal

* druntime - Just a top-level package containing public imports, aliases, and compiler support. No other implementations
    * public imports: core.pal, core.d
    * private imports: core.util

* std - phobos
    * public imports: (none)
    * private imports: druntime

There are likely other suitable ways to organize it, but that's just what I could come up with after thinking through it a little.

I would prefer if each of those were in their own repository and even move some of them to Deimos or dub, but that would probably irritate a lot of people.  I'd also prefer to have each of those in their own packages, but D is probably too deep in technical debt for that.  (See also https://issues.dlang.org/show_bug.cgi?id=11666)

So, to make it more palatable, I suggest:
-----------------------------------------
  * `core.util` gets own repository so it can be independently added to other repositories as a self-contained/freestanding dependency

  * `core.stdc`, `core.stdcpp`, `sys`, `core.pal`, and `core.d` all go into the druntime monolith like it is today.

  * phobos remains much like it is today.

In the context of the discussion at hand, `std.traits`, `std.meta`, and other utilities can be moved to `core.util`.  `core.util` can then be added as a dependency to dmd, druntime, and phobos.  The rest will probably have to wait for D3 :/

Mike

On Saturday, 5 January 2019 at 21:31:38 UTC, kinke wrote:
> On Saturday, 5 January 2019 at 21:12:54 UTC, Manu wrote:
>> We should move them to core.traits, and that should be their official home. It really just makes sense. Uncontroversial low-level traits don't belong in phobos.
>
> I fully agree.

YES, YES!! Agree!!!

On Monday, 7 January 2019 at 21:54:15 UTC, Steven Schveighoffer wrote:
> std.internal.traits contains pieces of std.meta -- a quick look shows it has AliasSeq (but under the name TypeTuple),

That's fine internally, but std.traits still uses *Tuple for ten public templates:
https://github.com/dlang/phobos/pull/6227

If we move any of those to core.traits, please can we finally fix the names.

> traits and meta are really part of the language,

Some in std.traits are tightly coupled to the language, e.g. isInteger. Some are utility templates, e.g. ConstOf, CopyConstness. I think only the former should be public in druntime. Select and select aren't even traits, they should have been in std.meta.

Except perhaps AliasSeq, (Alias, Instantiate) all of std.meta seems to be utility templates rather than language feature wrappers.

On 1/8/19 1:59 PM, Nick Treleaven wrote:
> On Monday, 7 January 2019 at 21:54:15 UTC, Steven Schveighoffer wrote:
>> std.internal.traits contains pieces of std.meta -- a quick look shows it has AliasSeq (but under the name TypeTuple),
> 
> That's fine internally, but std.traits still uses *Tuple for ten public templates:
> https://github.com/dlang/phobos/pull/6227
> 
> If we move any of those to core.traits, please can we finally fix the names.

You'd have to convince Andrei, as he seemingly nixed the PR.

>> traits and meta are really part of the language,
> 
> Some in std.traits are tightly coupled to the language, e.g. isInteger. Some are utility templates, e.g. ConstOf, CopyConstness. I think only the former should be public in druntime. Select and select aren't even traits, they should have been in std.meta.

What I mean is that you reach for things like what is available in std.traits quite often when doing template constraints or type manipulation. I'd consider ConstOf and CopyConstness to be in that group.

> Except perhaps AliasSeq, (Alias, Instantiate) all of std.meta seems to be utility templates rather than language feature wrappers.

What I mean is that I consider them part of D language, not like a library feature that is optional.

But in any case, much of std.traits relies on std.meta. We would at least need the parts in std.meta that are used to build std.traits.

-Steve

On Tuesday, 8 January 2019 at 01:44:08 UTC, Mike Franklin wrote:
> Anyway, my suggestion is to create a new library separate from druntime and phobos that has no dependencies whatsoever (no libc, no libstdc++, no OS dependencies, no druntime dependency, etc.).  I mean it; **no dependencies**.  Not even object.d.  The only thing it should require is a D compiler.
>
> That library can then be imported by druntime, phobos, betterC builds, or even the compiler itself. It will take strict enforcement of the "no dependency" rule and good judgment to keep the scope from ballooning, but it may be a good place for things like `traits`, `meta` and others.

I also feel the need for at least 1 another base library. My focus is on the fundamental compiler support functions, like initializing/comparing/copying arrays and general associative arrays support, as they are fundamental to the language and their compilers (not talking about TypeInfos, ModuleInfos, Object etc.).
I think we need such a base library in order to improve -betterC and its available language features. The important thing would be to try to reduce the external dependencies of that lib to an absolute minimum, similar to rust's core library (just 5 symbols: mem{cpy,cmp,set} + rust_begin_panic + rust_eh_personality), although we'll probably need some some primitives, e.g., malloc/realloc/free. If that's possible, using D for bare-metal targets without a C library (e.g., a future WebAssembly version with direct access to GC, or your own OS kernel/firmware) would probably become awesome, as you'd only need to implement maybe a dozen of symbols.