On Wednesday, 27 April 2022 at 14:37:35 UTC, H. S. Teoh wrote:
>> Dub is more like maven from java world. I.e. it uses declarative approach to define a project and how to build it.
>
> Why can't a plugin DSL be declarative too?

It can, I was just thinking that dub is more like maven in java world, and for some reason was thinking that you'd extend config syntax to plugins (which is wrong).

It would be worth to check if existing language can be employed instead of custom dsl. It should lessen the need of maintenance then. Perhaps interpreter for D, or javascript?

>> Code based config build systems in D are more like gradle, since gradle dsl is just plain groovy with some ast macros.
> [...]
>
> IMO, it's better to have a declarative language for writing plugins, so that you do not have to recompile dub to install a plugin.

That's where shared library jumps in, to avoid dub recompilation.
The idea is that dub once seeing that a project requires plugin x, will download it from official repo, and load it and let it run at right events. This though, means shared lib use should be without hiccups in D.

If shared lib hiccups are not fixed or avoided in dub, then following can be done for a plugin based system:
1. Complete recompile of dub for each project combination of plugins, in order to run them.
2. Some form of microservices where each plugin is a standalone process.
3. Use of a interpreted language, declarative or imperative (your proposal).

Personally I'd prefer plugins as shared libs, and a well defined api for dub core through which plugins could interact and perform their job, while for configuration being done through existing sdl and json formats.

It would still be interesting to see what you can come up with custom DSL :)

On Thu, Apr 28, 2022 at 10:06:37PM +0000, Alexandru Ermicioi via Digitalmars-d wrote:
> On Wednesday, 27 April 2022 at 14:37:35 UTC, H. S. Teoh wrote:
> > > Dub is more like maven from java world. I.e. it uses declarative approach to define a project and how to build it.
> > 
> > Why can't a plugin DSL be declarative too?
> 
> It can, I was just thinking that dub is more like maven in java world, and for some reason was thinking that you'd extend config syntax to plugins (which is wrong).

Config syntax could be extended in such a way that plugins use a different subtree in the config.


> It would be worth to check if existing language can be employed instead of custom dsl. It should lessen the need of maintenance then. Perhaps interpreter for D, or javascript?

I don't like the idea of an embedded interpreter. If you really need code-based plugins, might as well just use dynamic libraries, or build the core primitives into dub and expose some kind of external declarative API for it.


[...]
> > IMO, it's better to have a declarative language for writing plugins, so that you do not have to recompile dub to install a plugin.
> 
> That's where shared library jumps in, to avoid dub recompilation. The idea is that dub once seeing that a project requires plugin x, will download it from official repo, and load it and let it run at right events.  This though, means shared lib use should be without hiccups in D.

I don't like the idea of using binary plugins for several reasons.

First, do we *really* need code-based plugins?  If the user really, absolutely needs a programmatic way to express what he wants (and I have my doubts about this), why can't we expose, say, some hooks that he can run shell scripts (or full-fledged D programs if he insists) from?

Second, we will have to precompile and distribute plugins per platform, which adds maintenance burden on our already thin manpower.

Third, we will need some kind of cryptographic signature mechanism for ensuring the authenticity of plugins (we don't want dub to be the vehicle for a security exploit by, e.g., a MITM attack that replaces plugin code with malicious code).

Fourth, ongoing maintenance will be much harder than a declarative DSL for plugins because of Turing completeness.  It will also be harder to tell what exactly a plugin is doing. Anyone could write a dub plugin that does who-knows-what when you run it. A declarative plugin is much easier to review and understand/maintain.


> If shared lib hiccups are not fixed or avoided in dub, then following
> can be done for a plugin based system:
> 1. Complete recompile of dub for each project combination of plugins,
> in order to run them.

Or use a DSL and avoid recompilation / cross-platform / security issues altogether. ;-)


> 2. Some form of microservices where each plugin is a standalone process.

I think this is total overkill. At least for the initial plan of making dub support non-D builds.


> 3. Use of a interpreted language, declarative or imperative (your
> proposal).
[...]

Actually, my proposal is far less ambitious than an interpreted language. :-D  The main idea is to refactor dub into some generic primitives for dependency resolution / building, and use config directives to customize these primitives such that they do the right thing for non-D builds.

I may be a little too optimistic here, though. :-D


T

-- 
My program has no bugs! Only undocumented features...

On 30/04/2022 12:52 AM, Tobias Pankrath wrote:
> Do you do this for every build or per compiler? `dub describe` doesn't work that well if you use multiple compilers.

How can ``dub describe`` be improved for multiple compilers?

On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole wrote:
>
> On 30/04/2022 12:52 AM, Tobias Pankrath wrote:
>> Do you do this for every build or per compiler? `dub describe` doesn't work that well if you use multiple compilers.
>
> How can ``dub describe`` be improved for multiple compilers?

`dub describe` specifically should provide paths to the actual artifacts
instead of the symlinked locations, which are the same for all
build configurations.

While integrating it with nix, I have wished for:

1. Better control over where dub places and searches for stuff, although `HOME=. dub ..` works.

2. A way to just download a projects and all it's dependencies and place it somewhere.

3. That dub describe points to the actual build artifacts instead of symlinked locations
used by differnet locations.

If you look at the dub integration code of meson, 3/4 of it is finding stuff that dub is supposed to have already built after it parsed the output of `dub describe`.

On 30/04/2022 4:50 AM, Tobias Pankrath wrote:
> On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole wrote:
>>
>> On 30/04/2022 12:52 AM, Tobias Pankrath wrote:
>>> Do you do this for every build or per compiler? `dub describe` doesn't work that well if you use multiple compilers.
>>
>> How can ``dub describe`` be improved for multiple compilers?
> 
> `dub describe` specifically should provide paths to the actual artifacts
> instead of the symlinked locations, which are the same for all
> build configurations.
> 
> While integrating it with nix, I have wished for:
> 
> 1. Better control over where dub places and searches for stuff, although `HOME=. dub ..` works.
> 
> 2. A way to just download a projects and all it's dependencies and place it somewhere.

$ dub fetch vibelog --cache=local
$ dub build vibelog --cache=local

That'll do it, but you shouldn't need to build to do all the dependencies...

https://github.com/dlang/dub/issues/1930

Okay so:

$ dub describe vibelog --cache=local

Will also do the job.

> 3. That dub describe points to the actual build artifacts instead of symlinked locations
> used by differnet locations.

targetPath	string	The destination path of the output binary - this setting does not support platform suffixes

That needs to support platform suffixes for what you are wanting.
Describe would just follow suit.

On Windows the artifacts are copied not symlinked. So the paths are correct (even if not desirable).

On Friday, 29 April 2022 at 17:02:55 UTC, rikki cattermole wrote:
> On 30/04/2022 4:50 AM, Tobias Pankrath wrote:
>> On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole wrote:
>>>
>>> On 30/04/2022 12:52 AM, Tobias Pankrath wrote:
>>>> Do you do this for every build or per compiler? `dub describe` doesn't work that well if you use multiple compilers.
>>>
>>> How can ``dub describe`` be improved for multiple compilers?
>> 
>> `dub describe` specifically should provide paths to the actual artifacts
>> instead of the symlinked locations, which are the same for all
>> build configurations.
>> 
>> While integrating it with nix, I have wished for:
>> 
>> 1. Better control over where dub places and searches for stuff, although `HOME=. dub ..` works.
>> 
>> 2. A way to just download a projects and all it's dependencies and place it somewhere.
>
> $ dub fetch vibelog --cache=local
> $ dub build vibelog --cache=local

That doesn't really work, you need `HOME=.`.

> That'll do it, but you shouldn't need to build to do all the dependencies...
>
> https://github.com/dlang/dub/issues/1930

I've started a PR for that some years ago, but gave up https://github.com/dlang/dub/pull/1898

> Okay so:
>
> $ dub describe vibelog --cache=local
>
> Will also do the job.
>
>> 3. That dub describe points to the actual build artifacts instead of symlinked locations
>> used by differnet locations.
>
> targetPath	string	The destination path of the output binary - this setting does not support platform suffixes
>
> That needs to support platform suffixes for what you are wanting.
> Describe would just follow suit.
>
> On Windows the artifacts are copied not symlinked. So the paths are correct (even if not desirable).

That doesn't change the issue that `dub describe --compiler=ldc` says that a file it at <path/to/file>, but then you do `dub build --compiler=dmd` and suddenly <path/to/file> is something different.