On 16/10/2020 12:00 PM, norm wrote:
> My apologies I stand corrected, I thought git had the interface to read just all the metadata without pulling all the data.

When I last looked at it a few years ago, my statement was true.

After more git versions being released, it may no longer be true for a lot of use cases.

Either way, you still need the archive as well (which is a separate feature that is also optional, or was at one point). So you may end up with needing to clone it fully anyway.

I don't see a problem with checking out a repository to get the info + archive unless it is a big repo. So that would end up being our fallback anyway if you want non-supported providers.

On 2020-10-15 00:21, norm wrote:

> Git can clone the metadata without checking out HEAD if that is all dub requires, which sounds more like what the dub registry requires.

With the DMD release 2.094.0, Dub now seems to support arbitrary git repositories [1].

[1] https://dlang.org/changelog/2.094.0.html#git-paths

-- 
/Jacob Carlborg

On Thursday, 15 October 2020 at 23:26:03 UTC, rikki cattermole wrote:
> On 16/10/2020 12:00 PM, norm wrote:
>> My apologies I stand corrected, I thought git had the interface to read just all the metadata without pulling all the data.
>
> When I last looked at it a few years ago, my statement was true.
>
> After more git versions being released, it may no longer be true for a lot of use cases.
>
> Either way, you still need the archive as well (which is a separate feature that is also optional, or was at one point). So you may end up with needing to clone it fully anyway.
>
> I don't see a problem with checking out a repository to get the info + archive unless it is a big repo. So that would end up being our fallback anyway if you want non-supported providers.

It's not the same as an API that just gets you the tags, but partial cloning and sparse checkouts are supported by recent git versions (could also throw in shallow clones for repos with a lot of history).  Maybe that's good enough.

https://www.git-scm.com/docs/partial-clone
https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/

Here's a quick test with vibe.d (full clone is 24.84MB).  Most projects have under a few hundred KB of history.

$ git clone --filter=tree:0 --no-checkout https://github.com/vibe-d/vibe.d
Cloning into 'vibe.d'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 6232 (delta 0), reused 9 (delta 0), pack-reused 6220
Receiving objects: 100% (6232/6232), 1.56 MiB | 1.43 MiB/s, done.
Resolving deltas: 100% (64/64), done.
$ cd vibe.d/
$ git tag | head
0.7.20-rc.1
test
test2
v0.7.0
v0.7.0-beta.3
v0.7.1
v0.7.10
v0.7.11
v0.7.12
v0.7.13
$ git log | head
commit b5afadbe53849db5f3beacbbac1f2ec028f24770
Merge: a567566 61bff81
Author: Sönke Ludwig <sludwig@rejectedsoftware.com>
Date:   Thu Oct 15 12:44:53 2020 +0200

    Merge pull request #2483 from tchaloupka/httpclient_leak

    Fix refcounted FD leak in HTTPClient

commit 61bff81cfbf97843fdd1a47390bf5f20fba5b876
$