I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives.

Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible?

It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them.

Should they be moved?

On Thursday, 23 May 2019 at 06:16:08 UTC, Manu wrote:
> I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives.
>
> Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible?

I think Andrei said it best here:  https://forum.dlang.org/post/q7j4sl$17pe$1@digitalmars.com

---
> Oh, and druntime must go.
> 
> The whole distinction between the runtime library and the standard library is clowny and has more to do with poorly ported tradition from other languages, than with anything else.
> 
> We need one standard library that is entirely pay-as-you-go (e.g. empty main() means no library file is ever needed for linking) and that offers an opt-in continuum.
---

So, the divide between druntime and phobos shouldn't even exist.  But what does a united druntime/Phobos look like?  How do we organize it logically, and modularly with clear boundaries, so it doesn't turn into a tangled mess and users only pay for what they use?  I don't know, but I'm trying to figure that out.

Andrei and I have a difference of opinion on how to achieve that which is apparent in this conversation:  https://github.com/dlang/dmd/pull/9814  I'm trying to engage with him via e-mail and other PRs (e.g. https://github.com/JinShil/utiliD/pull/7) to incrementally work something out and get a clearer vision of what the future should look like.

There are also some prerequisites that need to be implemented first before some of this stuff will work right (e.g. converting some runtime hooks to templates).  That's also being worked on.

> It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them.
>
> Should they be moved?

For now, yeah, perhaps they should just be moved to `core.range` or something like that until a better vision of what a united, pay-as-you-go druntime/phobos is worked out.

I'm mentoring a GSoC project this year and we could actually benefit from having some of that range stuff in druntime.

And this isn't unique to the range stuff.  There's quite a bit in Phobos (std.traits, std.conv, std.meta, etc...) that really needs to be available to all D programs including druntime and the compiler itself.

Mike

On Wed, May 22, 2019 at 11:45 PM Mike Franklin via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
>
> On Thursday, 23 May 2019 at 06:16:08 UTC, Manu wrote:
> > I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives.
> >
> > Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible?
>
> I think Andrei said it best here: https://forum.dlang.org/post/q7j4sl$17pe$1@digitalmars.com

Right... but we're a decade away from that vision, we should do something for the time being.


> > It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them.
> >
> > Should they be moved?
>
> For now, yeah, perhaps they should just be moved to `core.range` or something like that until a better vision of what a united, pay-as-you-go druntime/phobos is worked out.
>
> I'm mentoring a GSoC project this year and we could actually benefit from having some of that range stuff in druntime.
>
> And this isn't unique to the range stuff.  There's quite a bit in Phobos (std.traits, std.conv, std.meta, etc...) that really needs to be available to all D programs including druntime and the compiler itself.

Indeed. Although only subsets of each of those files for my money, and they feel improperly organised to me.

So, std.range.primitives -> core.range ?

On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:

> So, std.range.primitives -> core.range ?

Take your best guess, submit a PR and see what the reviewers say.  If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.

Mike

On Thursday, 23 May 2019 at 07:26:34 UTC, Mike Franklin wrote:
> On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:
>
>> So, std.range.primitives -> core.range ?
>
> Take your best guess, submit a PR and see what the reviewers say.
>  If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.

A more conservative approach would be to put them in `core.internal`.  `core.internal.traits` is already there.  My understanding is that anything in `core.internal` is not intended for public consumption, so we could start there without committing to a public interface that we'd have to maintain.  It can always be exposed publicly later when the future becomes more clear.

Mike

On Thursday, 23 May 2019 at 07:40:20 UTC, Mike Franklin wrote:
> On Thursday, 23 May 2019 at 07:26:34 UTC, Mike Franklin wrote:
>> On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:
>>
>>> So, std.range.primitives -> core.range ?
>>
>> Take your best guess, submit a PR and see what the reviewers say.
>>  If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.
>
> A more conservative approach would be to put them in `core.internal`.  `core.internal.traits` is already there.  My understanding is that anything in `core.internal` is not intended for public consumption, so we could start there without committing to a public interface that we'd have to maintain.  It can always be exposed publicly later when the future becomes more clear.
>
> Mike

Yes, let's go with core.internal.range.

Motivation: when we can the package name, we have the unique opportunity of getting rid of some historic range missteps like auto-decoding for strings or using a special save() constructor for ForwardRanges to support class ranges.

On Thursday, 23 May 2019 at 06:41:04 UTC, Mike Franklin wrote:

>> Oh, and druntime must go.
>> 
>> The whole distinction between the runtime library and the standard library is clowny and has more to do with poorly ported tradition from other languages, than with anything else.
>> 
>> We need one standard library that is entirely pay-as-you-go (e.g. empty main() means no library file is ever needed for linking) and that offers an opt-in continuum.
> ---
>
> So, the divide between druntime and phobos shouldn't even exist.  But what does a united druntime/Phobos look like?  How do we organize it logically, and modularly with clear boundaries, so it doesn't turn into a tangled mess and users only pay for what they use?  I don't know, but I'm trying to figure that out.
>

We actually do need a runtime library. It should contain code for compiler generated runtime function calls. Nothing else. Users should have no need to import it or even have access to it.

The rest should be moved. User callable code might go to phobos. Files that have only bindings and definitions might be in phobos or in utiliD. Then there is code and imports that are not for users but used internally in the library. I am not sure where they should be but utiliD might be a good place.

What is left in runtime should not depend on phobos or utiliD. Some libc calls are always needed but they shoud be used as little as possible.

On Thursday, 23 May 2019 at 09:46:14 UTC, Timo Sintonen wrote:

> We actually do need a runtime library. It should contain code for compiler generated runtime function calls. Nothing else.

I agree. But consider that that runtime library is also written in D, and can have some complex constructs.  That is why I want to build something like utiliD, so it can be imported by that runtime library and used to make the runtime library's implementation more idiomatic D.  It can also be used by applications that aren't using any runtime features (e.g. some bare-metal platforms, and consuming D libraries from other languages such as C).

> Users should have no need to import it or even have access to it.

This is where I disagree because I'm trying to make the language opt-in and pay-as-you-go.  What I'm envisioning is a "Language as a Library".  If one wants classes they `import classes;`.  If one wants exceptions, they `import exceptions;` etc.  The modules one imports contain the compiler lowerings and other constructs to support the feature.  The compiler will emit an error if it can't find something that's not imported, including the compiler hooks, (just like it does today if one tries to use a symbol that isn't imported)  Users opt-in to a feature by importing the feature.  Note that this behavior is predicated on work being done to move all runtime lowerings into the semantic phase of the compiler instead of the IR stage like it is today.  That work is currently underway.

Walter and Andrei don't appear to like that idea.  What they appear to propose is to make *all* runtime constructs templates.  The runtime library is automatically imported, but because it's all templates, there's nothing to link.  Users opt-in to a feature by simply using it.  Either the instantiate one of the runtime templates explicitly, or by using the feature, the compiler lowers an expression to a runtime template implicitly.

I'm skeptical of that.  For example, do we need to declare `class Object() {}` instead of `class Object {}` in the runtime template library?  And think how that would need to scale all the way down the call stack if each runtime implementation, and constructs it uses, and each construct that those constructs use all need to be templated.  And what would that do to compile-times? Each module would have to re-compile all the runtime stuff they utilize.  And what about the potential for template bloat?

Mike

On Thursday, 23 May 2019 at 11:00:13 UTC, Mike Franklin wrote:
> [snip]
>
> This is where I disagree because I'm trying to make the language opt-in and pay-as-you-go.  What I'm envisioning is a "Language as a Library".  If one wants classes they `import classes;`.  If one wants exceptions, they `import exceptions;` etc.  The modules one imports contain the compiler lowerings and other constructs to support the feature.  The compiler will emit an error if it can't find something that's not imported, including the compiler hooks, (just like it does today if one tries to use a symbol that isn't imported)  Users opt-in to a feature by importing the feature.  Note that this behavior is predicated on work being done to move all runtime lowerings into the semantic phase of the compiler instead of the IR stage like it is today.  That work is currently underway.
>

I like the sound of this, though I think it would be helpful to have some sense of the size of the impact (on compile-times, run-times, and memory, at least) when those features are used (and not just when they are not used).

On Thursday, 23 May 2019 at 11:00:13 UTC, Mike Franklin wrote:
> On Thursday, 23 May 2019 at 09:46:14 UTC, Timo Sintonen wrote:
>
> Walter and Andrei don't appear to like that idea.  What they appear to propose is to make *all* runtime constructs templates.  The runtime library is automatically imported, but because it's all templates, there's nothing to link.  Users opt-in to a feature by simply using it.  Either the instantiate one of the runtime templates explicitly, or by using the feature, the compiler lowers an expression to a runtime template implicitly.

I would also favour this approach.

> I'm skeptical of that.  For example, do we need to declare `class Object() {}` instead of `class Object {}` in the runtime template library?  And think how that would need to scale all the way down the call stack if each runtime implementation, and constructs it uses, and each construct that those constructs use all need to be templated.  And what would that do to compile-times? Each module would have to re-compile all the runtime stuff they utilize.  And what about the potential for template bloat?

These are good questions.

I've never understood complaints about template bloat however. If the linker does its job, does it matter that individual object files are larger?