I've been thinking for a while to add a module `arsd.core` upon which the others can depend. This would have some interop definitions and implementation of the pieces necessary to make a shared event loop.

I also wrote most of a new text layout class over this weekend. My original intention was to embed it in simpledisplay.d since it was going to be tightly coupled to it and only useful with its draw text functions anyway.

However, after writing most of it, turns out it isn't actually coupled to simpledisplay at all, except for one font-related interface, which doesn't even have to work in terms of pixels! This means it could potentially be used by terminal.d as well and be the base of both character grid / tui and gui applications. The terminal.d ScrollbackBuffer *might* be able to use it (though since that thing already works as-is, idk if i would actually change it) and some of the terminal.d LineGetter could potentially be merged with a minigui LineEdit.

The benefits aren't exactly groundbreaking, but there's some potential here.

On the other hand, this would break the build for anyone who updates the individual files, since it'd change the import web.

I'd also have to shuffle some basic definitions like Point and Rectangle out of color.d and into core.d, which could affect static imports.

Before:
   terminal.d = stand-alone
   cgi.d = stand-alone
   http2.d = stand-alone
   color.d = stand-alone
   simpledisplay.d = needs color.d

etc. etc.

After, if we make all the changes:

   terminal.d = needs core.d, textlayouter.d
   cgi.d = needs core.d
   http2.d = needs core.d
   color.d = stand-alone
   simpledisplay.d = needs color.d, core.d, textlayouter.d

Anyone who downloads the individual files - which I've been telling people they can do for many years now - will have to get more of them and update their build process.

Moreover, the core event loop stuff might require you to actually run it in your thread, but I think I can do that right without breaking interop with outside systems anyway... and public imports can fixup reshuffled module members, so the main breakage here is for users who download the files.



So my question to those of you who use my libs... does this sound worth it to you? Of course, you can always just not update the file again, and stay happy with the version you have, but it seems likely that something will come up eventually that forces you to resync with me, so even though this breakage might not affect you today, I ask you to reply anyway since it might affect you tomorrow.

I expect I stress out too much over this, but I do try to take my promises of long-term support seriously and this is something that's worked without breakage for about 7 years now (which is where I added color.d and renamed the imports) so while it is small, I still don't want to break things without a real benefit.

PS if you do use my libs and wanna show off the project, please do! I'm always curious if people actually find these things useful.

BTW I am also tempted to change the policy entirely and say you should clone the repo and use dmd -i. Then I could reorganize stuff without making it a breaking change anymore (Though I wouldn't do it often, once I change the policy, I can do it without waiting for my annual batch of breaking changes.)

I'd be curious about user's thoughts on this too, even though I'm less likely to actually do it. (I ask about the core.d and textlayouter.d since I'm already leaning toward that direction.)

On Monday, 19 December 2022 at 13:27:37 UTC, Adam D Ruppe wrote:

> Before:
>    terminal.d = stand-alone
>    cgi.d = stand-alone
>    http2.d = stand-alone
>    color.d = stand-alone
>    simpledisplay.d = needs color.d
>
> etc. etc.
>
> After, if we make all the changes:
>
>    terminal.d = needs core.d, textlayouter.d
>    cgi.d = needs core.d
>    http2.d = needs core.d
>    color.d = stand-alone
>    simpledisplay.d = needs color.d, core.d, textlayouter.d
>
Go for it!
The stand-alone policy may have been useful, but now I tend to use several of the files anyway, so the dependency doesn't matter (to me).

On Mon, Dec 19, 2022 at 8:31 AM Adam D Ruppe via Digitalmars-d wrote:
> Anyone who downloads the individual files - which I've been telling people they can do for many years now - will have to get more of them and update their build process.
>

My need is for the single file; but whatever you choose to do, hopefully it can be done in a way that will not be too disruptive to users of individual parts of your offerings like me.

> PS if you do use my libs and wanna show off the project, please do! I'm always curious if people actually find these things useful.

I have been happy to have cgi.d available as a standalone file, that I
use for a simple web sqlite search widget for generated document
collections which are available in a few formats (html, epub, pdf via
latex).
 arsd/cgi.d is the cgi behind, eg:

searches: anticipate OR legend https://sisudoc.org/spine_search?sf=anticipate%20OR%20legend&rt=idx&ec=on&url=on&sml=1000&smo=0 https://sisudoc.org/spine_search?sf=anticipate%20OR%20legend&rt=txt&ec=on&url=on&sml=1000&smo=0

searches: linux OR windows https://sisudoc.org/spine_search?sf=linux OR windows&rt=idx&ec=on&url=on&se=on&sml=1000&smo=0 https://sisudoc.org/spine_search?sf=linux OR windows&rt=txt&ec=on&url=on&se=on&sml=1000&smo=0

which searches a small test collection of commons permissible licensed
books/documents:
https://sisudoc.org/spine/authors.html
https://sisudoc.org/spine/topics.html

(The origins of this interest go back to what was claimed to be one of
the worlds first 200 websites, (on international commercial law)
available from 1993, in early days for html).
I have not shared the D code (repo currently only contains earlier
ruby version).

I consider myself an outlier. I wish to be able to work with the tools
of my choice, and have become happy to explore on my own.
The two most notable "tools" here would be: D (dlang) and things Nix
(including NixOS).

https://dlang.org/
https://forum.dlang.org/group/general

https://nixos.org/
https://discourse.nixos.org/

There appears to be not much overlap of interest (in each other)
between the projects.
https://github.com/lionello/dub2nix
although I have been building with static downloaded locally git
tracked dependencies off the file system.

(I wish there were more in each community that used (even depended on)
the other).

> So my question to those of you who use my libs... does this sound worth it to you?

Not sure how the whole shebang is used by others and what it will mean
to your project as a whole.
Would it just mean my having to download an additional file?

Thank you for all your offerings. No doubt there are many others like me who benefit from the multitude of different parts of it.

On Monday, 19 December 2022 at 13:27:37 UTC, Adam D Ruppe wrote:

> So my question to those of you who use my libs... does this sound worth it to you?

You are best able to judge the benefits, but from my perspective, the cost of downloading an extra file is minor. You could always have a script that copies all the files for each use case into its own directory, or to create a .zip for each of them if there's an issue.

> I expect I stress out too much over this, but I do try to take my promises of long-term support seriously and this is something that's worked without breakage for about 7 years now (which is where I added color.d and renamed the imports) so while it is small, I still don't want to break things without a real benefit.

I personally think avoiding breakage is very important. That's why I typically create new projects rather than making breaking changes to existing projects. What you're proposing barely qualifies as breakage.

> PS if you do use my libs and wanna show off the project, please do! I'm always curious if people actually find these things useful.

I'm not someone has lots of users of my projects. However, I have one that has actually been used by other people - I've even come across people recommending it in HN comments. The entire thing is this 25-line file:

https://github.com/bachmeil/tiddlyd/blob/main/tiddly.d

You made changes to the handling of data inside PUT requests in cgi.d so it would work.

Your libraries have been helpful to me. I learned basic web development by reading your book and then doing related things with your libraries. Not that web development is part of my job or anything. It was just nice to learn. From time to time I'll add a web interface on localhost to a D utility I've written. That sort of thing.

On Monday, 19 December 2022 at 16:03:07 UTC, Ralph Amissah wrote:
> Would it just mean my having to download an additional file?

For cgi.d yes, it would just mean downloading core.d too and compiling them together.

So if you currently do

dmd yourfile.d cgi.d

you'd do

dmd yourfile.d cgi.d core.d

(or whatever is necessary with your build system to add the other file)

And always download both files together if you update.


I was considering changing the policy to require you to download the whole repo and use dmd -i to build it (dmd -i will automatically include any referenced modules as-needed, while leaving the others out, so you still get the benefits of separation, it isn't like all 80 files are included in your build every time, while I'd be able to factor things out more often), but that is an even bigger change and I don't think I want to go that far.

But adding just the one or maybe two files to a build is much closer to a balanced decision.


I know this isn't big in the grand scheme of things, but it is something I've resisted doing for years. Even when I do add inter-module dependencies, I'd be careful to make it opt-in somehow. For example, if you use the cgi.dispatcher() function, then it depends on dom.d too.... but if you never call that, you don't need dom.d at all.

But for what I'm thinking about now, I'm going to use it in many functions so that approach of "you don't pay for it if you don't call it" isn't going to work.

So I gotta decide to either do it or not at least for the one core.d file.

> Thank you for all your offerings. No doubt there are many others like me who benefit from the multitude of different parts of it.

No, thank you for answering! I've also long been curious how many users it actually has. I sometimes get emails from people asking questions, and a couple times been surprised to see some of my code pop up in projects I've never heard of before, but I still have very little actual knowledge as to how many people actually use it so it is cool to hear from people.

On Monday, 19 December 2022 at 13:27:37 UTC, Adam D Ruppe wrote:
>
> PS if you do use my libs and wanna show off the project, please do! I'm always curious if people actually find these things useful.

I'm a user of arsd.dom and a few others. It will be a very minimal annoyance if arsd breaks compat more. Besides, it can be non-breaking when using dub subpackages, using sourceFiles.

On Mon, Dec 19, 2022 at 01:27:37PM +0000, Adam D Ruppe via Digitalmars-d wrote: [...]
> Before:
>    terminal.d = stand-alone
>    cgi.d = stand-alone
>    http2.d = stand-alone
>    color.d = stand-alone
>    simpledisplay.d = needs color.d
> 
> etc. etc.
> 
> After, if we make all the changes:
> 
>    terminal.d = needs core.d, textlayouter.d
>    cgi.d = needs core.d
>    http2.d = needs core.d
>    color.d = stand-alone
>    simpledisplay.d = needs color.d, core.d, textlayouter.d
[...]

This will probably break a few of my projects that use arsd.terminal. But since the fix is so easy, and I use git submodules to pull in arsd (meaning I will get all the files anyway), this isn't a big deal. I'll just fix a couple of build scripts and that's all.

On that note, though, it would be nice if D had some kind of feature where only actual dependencies end up in the executable, i.e., end-to-end pay-as-you-go.  But that's just a dream... We've been trying to make druntime/Phobos pay-as-you-go for years, and still haven't come close to the ideal state yet.


T

-- 
Prosperity breeds contempt, and poverty breeds consent. -- Suck.com

On Monday, 19 December 2022 at 20:18:54 UTC, H. S. Teoh wrote:
> On that note, though, it would be nice if D had some kind of feature where only actual dependencies end up in the executable, i.e., end-to-end pay-as-you-go.

It does, and it has for over five years now! This is the only reason why this change is even on the table at all.

This is what `dmd -i` does. Only things actually imported are compiled at all. Then you keep your imports local in templates which means they now happen on function-level granularity.

So for example, if you do

---
import arsd.dom;

void main() {
   auto document = new Document("<html></html");
}
---

And then `dmd -i yourapp.d`, it will automatically pull in arsd.dom to the build... but not the rest of the arsd modules.

Now change it to:

---
import arsd.dom;

void main() {
   auto document = new Document();
   document.parseGarbage("<html></html");
}
---

And the same `dmd -i yourapp.d` will now automatically pull in arsd.dom and arsd.characterencodings because parseGarbage uses that to convert things to utf8.

Or try:

---
import arsd.dom;

void main() {
   auto document = Document.fromUrl("http://dlang.org/");
}
---

And again, the same `dmd -i yourapp.d` will pull arsd.dom, arsd.characterencodings, and arsd.http2 automatically, again because it knows you called fromUrl which imports arsd.http2.

It took a little bit of effort for me to write the code in a way to do all this correctly, but the compiler works with it.

The linker can also further strip unreferenced things from the executable afterward, and if you're careful about your import web, this can be very effective (and it sometimes can be too even if you aren't careful, but any module constructors being present have a habit of defeating the linker's stripping since it can't know if it is actually necessary or not).

> We've been trying to make druntime/Phobos pay-as-you-go for years, and still haven't come close to the ideal state yet.

That's because they wrote things very differently than I do. arsd explicitly avoids dependency webs - which is why adding this proposed `core.d` module would be a breaking change, since I need to make an exception to that policy (and like i said in a previous message, I did consider suspending that policy entirely, but I don't think I'm going to do that. One advantage of more modules is, when combined with `dmd -i`, you can get pretty granular on what is compiled in. Just this doesn't always actually bring you a win.)

Yes, please, go for it!

I only use two of the impacted modules anyway, and fixing my build scripts will be trivial.

My habit is to clone the whole git repo already, so that's a no-op.