Jump to page: 1 2
Thread overview
D for microservices: ldc, rdmd, dub now available on Alpine x86_64
Nov 05
aliak
Nov 06
sarn
Nov 13
user
November 05
Hi all,
Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices.

At BPF Korea, we're working on a blockchain written in D, and wanted to be able to easily test and distribute our node using Alpine images, but there was no package for it yet.

However, thanks to the work of many contributors before (Joakim Noah, yshui, Petar Kirov/zombinedev, and many others), most of the porting was already done and it was just a matter of fixing a few small issues and and creating the package definitions.

A package for `dub` (v1.18.0), `dtools` (ddemangle & rdmd), and `ldc` (v1.18.0) are now available in the `testing` repository of Alpine Linux edge. As `testing` is not enabled by default, you will need to specify the repository (or add it to your `/etc/apk/repositories`) when installing the packages.
For example:
```
apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub
```
This command needs to originate from an `alpine:edge` image as it links with a recent libc and LLVM.
If you just want the compiler, you still need to provide `ldc` & `ldc-static` for things to work out of the box.

More complete example of how we use it to build a D program, using multi-stage builds:
```
# Build from source
FROM alpine:edge AS Builder
ARG DUB_OPTIONS
RUN apk --no-cache add build-base git <other dependencies>
RUN apk --no-cache add -X http://dl-cdn.alpinelinux.org/alpine/edge/testing ldc ldc-static dtools-rdmd dub
ADD . /root/myproject/
WORKDIR /root/myproject/
# Note: This will redownload your dependencies every time, which doesn't play well with docker
# We use submodules for dependencies, hence we have `--skip-registry=all`
RUN dub build --compiler=ldc2 ${DUB_OPTIONS}

# Runner
FROM alpine:edge
COPY --from=Builder /root/project/executable /usr/bin/executable
RUN apk --no-cache add libexecinfo <runtime dependencies>
WORKDIR /root/
ENTRYPOINT [ "/usr/bin/executable" ]
```
`DUB_OPTIONS` can be used to select a build, for example enabling coverage in a CI pipeline.

What's next ?
1) There is a pending PR (https://github.com/alpinelinux/aports/pull/12006) to have GDC working on all architectures alpine supports, not just x86_64.
2) Adding a package for gdmd
3) Rebuild packages based on GDC, so that all architectures are supported.
4) Move the packages to community so they are available out of the box. It would be great for it to happen by the end of the month, as the next alpine release would be around end of December according to their schedule, but that depends on how long PR take to be reviewed.
5) A DMD package for x86 and x86_64 shouldn't be hard to make either
November 05
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
> Hi all,
> Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices.
>
> [...]

This is great! Much thanks to all for all the work towards this!
November 05
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
> [..]

That's great news! Thanks a lot for your hard work!

> # Note: This will redownload your dependencies every time, which doesn't play well with docker

I have been meaning to add a docker and CI friendly command to dub that fetches all dependencies without building them.
In the Node.js world they have the `npm ci` [0] command.

[0]: https://docs.npmjs.com/cli/ci.html
November 05
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
> Hi all,
> Recently there have been inquiries about support for D on Alpine Linux, a distribution mostly used in combination with Docker to create lightweight container images for microservices.
>
> At BPF Korea, we're working on a blockchain written in D, and wanted to be able to easily test and distribute our node using Alpine images, but there was no package for it yet.

I am confused... Why do you need LDC, DMD, dub, etc on Alpine? Can't you build it anywhere and just put the final artifact (or set of artifacts) inside an Alpine-based container?
November 05
On Tue, Nov 5, 2019 at 12:05 PM Dejan Lekic via Digitalmars-d-announce <digitalmars-d-announce@puremagic.com> wrote:
>
> On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
> > Hi all,
> > Recently there have been inquiries about support for D on
> > Alpine Linux, a distribution mostly used in combination with
> > Docker to create lightweight container images for microservices.
> >
> > At BPF Korea, we're working on a blockchain written in D, and wanted to be able to easily test and distribute our node using Alpine images, but there was no package for it yet.
>
> I am confused... Why do you need LDC, DMD, dub, etc on Alpine? Can't you build it anywhere and just put the final artifact (or set of artifacts) inside an Alpine-based container?

Generally no, because Apline use musl libc instead of glibc, so there are some issues with that
November 05
On Tuesday, 5 November 2019 at 11:49:20 UTC, Daniel Kozak wrote:

> Generally no, because Apline use musl libc instead of glibc, so there are some issues with that

The correct way is to use static linking and putting only the binary in a Docker image, i.e. "from scratch" [1] ;). But using Alpine and musl will help with building the binary.

[1] https://hub.docker.com/_/scratch
November 05
Am 05.11.2019 um 10:48 schrieb Petar Kirov [ZombineDev]:
> On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
>> [..]
> 
> That's great news! Thanks a lot for your hard work!
> 
>> # Note: This will redownload your dependencies every time, which doesn't play well with docker
> 
> I have been meaning to add a docker and CI friendly command to dub that fetches all dependencies without building them.
> In the Node.js world they have the `npm ci` [0] command.
> 
> [0]: https://docs.npmjs.com/cli/ci.html

Sounds like "dub upgrade --missing-only" will do that, as it implicitly downloads all dependencies in addition to filling the (non-existent in this case) gaps.
November 05
On Tuesday, 5 November 2019 at 02:16:28 UTC, Mathias Lang wrote:
>
> At BPF Korea, we're working on a blockchain written in D

Hello,

Sorry if this has been said already: would you consider being listed in https://dlang.org/orgs-using-d.html?

Thanks
November 06
On Tuesday, 5 November 2019 at 12:20:04 UTC, Jacob Carlborg wrote:
> On Tuesday, 5 November 2019 at 11:49:20 UTC, Daniel Kozak wrote:
>
>> Generally no, because Apline use musl libc instead of glibc, so there are some issues with that
>
> The correct way is to use static linking and putting only the binary in a Docker image, i.e. "from scratch" [1] ;). But using Alpine and musl will help with building the binary.
>
> [1] https://hub.docker.com/_/scratch

And the neat way to do that is with a multi-stage build: one Dockerfile, with an Alpine container building the binary, then copying to a FROM scratch container:

https://docs.docker.com/develop/develop-images/multistage-build/

The musl build is practically necessary because glibc has effectively given up standalone static binary support.  So, thanks BPF Korea :)
November 06
On 2019-11-06 02:02, sarn wrote:

> And the neat way to do that is with a multi-stage build: one Dockerfile, with an Alpine container building the binary, then copying to a FROM scratch container

I've used the "smith" tool as well [1]. It has some additonal help with dynamically linked code. It will use "ldd", recursively, to track dependencies and automatically add those. It will also do a couple of other things, like making the filesystem read only.

> The musl build is practically necessary because glibc has effectively given up standalone static binary support.  So, thanks BPF Korea :)

Yeah, it helps. Although it still possible to statically link with glibc, although you might get some warnings.

[1] https://github.com/oracle/smith

-- 
/Jacob Carlborg
« First   ‹ Prev
1 2