Issue eBPF kernel programs with ldc?

I have been trying to get eBPF kernel programs running with D on the LDC compiler. The object file generated by the LDC compiler fails when attempting to run it with the loader/user program because of an undefined reference to a BPF function, however the object file generated with the clang compiler works as expected.

Here is a simple hello world example:

//hello_world.d

@system:
@nogc:
extern(C):

@(ldc.attributes.section("license")) __gshared immutable(char)[3] LICENSE = "GPL";

import core.stdc.stdlib;
import core.stdc.stdarg;
import ldc.attributes;

long bpf_trace_printk(const(char)* fmt, uint fmt_size, ...);
@(ldc.attributes.section("xdp_md")) struct xdp_md;

@(ldc.attributes.section("xdp")) int xdp_prog(xdp_md* ctx)
{
    bpf_trace_printk("Hello, eBPF!\n", 14);
    return 0;
}

Here is the terminal compilation and outputs:

$ ldc2 --O2 --betterC -c -g --nogc --march=bpf hello_world.d -of hello_world_d.o
$ llvm-objdump -h hello_world_d.o

hello_world_d.o:        file format elf64-bpf

Sections:
Idx Name                Size     VMA              Type
  0                     00000000 0000000000000000
  1 .strtab             000000f6 0000000000000000
  2 .text               00000000 0000000000000000 TEXT
  3 xdp                 00000030 0000000000000000 TEXT
  4 .relxdp             00000020 0000000000000000
  5 license             00000003 0000000000000000 DATA
  6 .rodata.str1.1      0000000e 0000000000000000 DATA
  7 .debug_loc          00000023 0000000000000000 DEBUG
  8 .debug_abbrev       000000a6 0000000000000000 DEBUG
  9 .debug_info         000000d4 0000000000000000 DEBUG
 10 .rel.debug_info     00000180 0000000000000000
 11 .debug_str          000000f3 0000000000000000 DEBUG
 12 .debug_pubnames     00000083 0000000000000000 DEBUG
 13 .rel.debug_pubnames 00000010 0000000000000000
 14 .debug_pubtypes     0000005d 0000000000000000 DEBUG
 15 .rel.debug_pubtypes 00000010 0000000000000000
 16 .BTF                00000146 0000000000000000
 17 .rel.BTF            00000010 0000000000000000
 18 .BTF.ext            00000060 0000000000000000
 19 .rel.BTF.ext        00000030 0000000000000000
 20 .eh_frame           00000030 0000000000000000 DATA
 21 .rel.eh_frame       00000010 0000000000000000
 22 .debug_line         00000046 0000000000000000 DEBUG
 23 .rel.debug_line     00000010 0000000000000000
 24 .symtab             00000120 0000000000000000

$ bpftool btf dump file hello_world_d.o
[1] PTR 'hello_world.xdp_md*' type_id=0
[2] FUNC_PROTO '(anon)' ret_type_id=3 vlen=0
[3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[4] FUNC 'xdp_prog' type_id=2 linkage=global
[5] ARRAY '(anon)' type_id=1 index_type_id=6 nr_elems=3
[6] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[7] VAR 'LICENSE' type_id=5, linkage=global
[8] DATASEC 'license' size=0 vlen=1
        type_id=7 offset=0 size=3 (VAR 'LICENSE')

The loading/user program written in D works for kernel programs written in C, but does not work for D kernel programs and gives the following output:

$ ldc2 --betterC -L-lbpf -L-lelf -L-lz -of=loader_d loader_d.d
$ sudo ./loader_d
libbpf: elf: skipping unrecognized data section(20) .eh_frame
libbpf: elf: skipping relo section(21) .rel.eh_frame for section(20) .eh_frame
libbpf: failed to find BTF for extern 'bpf_trace_printk': -2
ERROR: opening BPF object file failed

Attempting to run with the bpftool, gives the same response. The D loading program will be appended at the end for completeness.

This is the equivalent C kernel program:

//hello_world.c

#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

char LICENSE[] SEC("license") = "GPL";

SEC("xdp")
int xdp_prog(struct xdp_md *ctx) {
    bpf_trace_printk("Hello, eBPF!\n", 14);
    return XDP_PASS;
}

Then the compilation and object dump:

$ clang -O2 -g -target bpf -c hello_world.c -o hello_world.o
$ llvm-objdump -h hello_world.o

hello_world.o:  file format elf64-bpf

Sections:
Idx Name                   Size     VMA              Type
  0                        00000000 0000000000000000
  1 .strtab                000000fa 0000000000000000
  2 .text                  00000000 0000000000000000 TEXT
  3 xdp                    00000030 0000000000000000 TEXT
  4 .relxdp                00000010 0000000000000000
  5 license                00000004 0000000000000000 DATA
  6 .rodata.str1.1         0000000e 0000000000000000 DATA
  7 .debug_abbrev          000000f5 0000000000000000 DEBUG
  8 .debug_info            00000110 0000000000000000 DEBUG
  9 .rel.debug_info        00000040 0000000000000000
 10 .debug_str_offsets     00000070 0000000000000000 DEBUG
 11 .rel.debug_str_offsets 000001a0 0000000000000000
 12 .debug_str             00000132 0000000000000000 DEBUG
 13 .debug_addr            00000020 0000000000000000 DEBUG
 14 .rel.debug_addr        00000030 0000000000000000
 15 .BTF                   00000259 0000000000000000
 16 .rel.BTF               00000010 0000000000000000
 17 .BTF.ext               00000060 0000000000000000
 18 .rel.BTF.ext           00000030 0000000000000000
 19 .debug_frame           00000028 0000000000000000 DEBUG
 20 .rel.debug_frame       00000020 0000000000000000
 21 .debug_line            000000a7 0000000000000000 DEBUG
 22 .rel.debug_line        00000090 0000000000000000
 23 .debug_line_str        00000086 0000000000000000 DEBUG
 24 .llvm_addrsig          00000002 0000000000000000
 25 .symtab                00000138 0000000000000000

$ bpftool btf dump file hello_world.o
[1] PTR '(anon)' type_id=2
[2] STRUCT 'xdp_md' size=24 vlen=6
        'data' type_id=3 bits_offset=0
        'data_end' type_id=3 bits_offset=32
        'data_meta' type_id=3 bits_offset=64
        'ingress_ifindex' type_id=3 bits_offset=96
        'rx_queue_index' type_id=3 bits_offset=128
        'egress_ifindex' type_id=3 bits_offset=160
[3] TYPEDEF '__u32' type_id=4
[4] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[5] FUNC_PROTO '(anon)' ret_type_id=6 vlen=1
        'ctx' type_id=1
[6] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[7] FUNC 'xdp_prog' type_id=5 linkage=global
[8] INT 'char' size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
[9] ARRAY '(anon)' type_id=8 index_type_id=10 nr_elems=4
[10] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[11] VAR 'LICENSE' type_id=9, linkage=global
[12] ARRAY '(anon)' type_id=8 index_type_id=10 nr_elems=14
[13] DATASEC 'license' size=0 vlen=1
        type_id=11 offset=0 size=4 (VAR 'LICENSE')

The C program executes correctly with the loading program. It does seem to have much more detail than the D object code, perhaps because things are correctly exported. I tried looking into the GDC compiler but, did not find a bpf export capability, and the --fno-exceptions flag did not remove the .eh_frame from the obejct file which may be responsible for the warnings in the D output.

Compilers:

$ ldc2 --version
LDC - the LLVM D compiler (1.40.0-git-eef3229):
  based on DMD v2.110.0 and LLVM 18.1.3
  built with LDC - the LLVM D compiler (1.39.0)
  Default target: x86_64-pc-linux-gnu
  Host CPU: znver4
  http://dlang.org - http://wiki.dlang.org/LDC

$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Thanks in advance, sorry that this was a long one.

The D loader code is given below.

import core.stdc.stdio;
import core.stdc.stdlib;


extern(C):

struct bpf_object;
struct bpf_program;
struct bpf_link;
struct bpf_object_open_opts;


bpf_object* bpf_object__open_file(const char *path, const bpf_object_open_opts* opts);
int libbpf_get_error(const bpf_object*);
bpf_program* bpf_object__find_program_by_name(const bpf_object *obj, const char *name);
int bpf_object__load(bpf_object* obj);
int if_nametoindex(const char*);
int bpf_link__destroy(bpf_link*);
void bpf_object__close(bpf_object*);
bpf_link* bpf_program__attach_xdp(const bpf_program* prog, int ifindex);
int bpf_link__fd(const bpf_link* link);


void read_trace_pipe(int n)
{
    static char[256] buf;
    buf[] = '\0';
    auto trace_file = fopen("/sys/kernel/debug/tracing/trace_pipe", "r");
    if(trace_file == null)
    {
        fprintf(stderr, "trace pipe could not be opened!\n");
        return;
    }
    for(int i = 0; i < n; ++i)
    {
        fprintf(stderr, "print for %d\n", i);
        auto nobs = fread(buf.ptr, typeof(buf[0]).sizeof, buf.length - 1, trace_file);
        if(nobs > 0)
        {
            fprintf(stdout, "%s", buf.ptr);
        }
    }
    fprintf(stdout, "\n");
    return;
}


int main()
{
    bpf_link* link = null;
    bpf_program* prog;
    bpf_object* obj;
    const char* ifname = "lo";
    const char* filename = "hello_world_d.o";

    int link_fd;
    int ifindex;

	obj = bpf_object__open_file(filename, null);
	if (libbpf_get_error(obj)) {
		fprintf(stderr, "ERROR: opening BPF object file failed\n");
		return 0;
	}

	prog = bpf_object__find_program_by_name(obj, "xdp_prog");
	if (!prog) {
		fprintf(stderr, "ERROR: finding a prog in obj file failed\n");
		goto cleanup;
	}

	/* load BPF program */
	if (bpf_object__load(obj)) {
		fprintf(stderr, "ERROR: loading BPF object file failed\n");
		goto cleanup;
	}

	// Get the index of the network interface
    ifindex = if_nametoindex(ifname);
    if (ifindex == 0) {
        fprintf(stderr, "Failed to get interface index");
        bpf_object__close(obj);
        return 1;
    }

    // Attach the eBPF program to the network interface using XDP
    link = bpf_program__attach_xdp(prog, ifindex);
    if (!link) {
        fprintf(stderr, "Failed to attach eBPF program");
        bpf_object__close(obj);
        return 1;
    }

    // Retrieve the file descriptor for the link
    link_fd = bpf_link__fd(link);
    if (link_fd < 0) {
        fprintf(stderr, "Failed to retrieve BPF link file descriptor");
        bpf_link__destroy(link); // Clean up the link
        bpf_object__close(obj);
        return 1;
    }

    read_trace_pipe(5);

    goto cleanup;

    cleanup:

        bpf_link__destroy(link);
        bpf_object__close(obj);
        return 0;
}

On Sunday, 5 January 2025 at 14:27:57 UTC, Richard (Rikki) Andrew Cattermole wrote:

static long (* const bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6;

I have updated the kernel code to:

@system:
@nogc:
extern(C):

@(ldc.attributes.section("license")) __gshared immutable(char)[3] LICENSE = "GPL";

import core.stdc.stdlib;
import core.stdc.stdarg;
import ldc.attributes;


// Define a type alias for the function signature
alias BpfTracePrintk = long function(const(char)* fmt, uint fmt_size, ...);

// Declare and initialize the function pointer
__gshared static BpfTracePrintk bpf_trace_printk = cast(BpfTracePrintk) 6;


@(ldc.attributes.section("xdp_md"))
struct xdp_md {
	uint data;
	uint data_end;
	uint data_meta;
	uint ingress_ifindex;
	uint rx_queue_index;
	uint egress_ifindex;
}

@(ldc.attributes.section("xdp")) int xdp_prog(xdp_md* ctx)
{
    bpf_trace_printk("Hello, eBPF!\n", 14);
    return 0;
}

And getting a different error:

$ ldc2 --O2 --betterC -c -g --nogc --fno-moduleinfo --march=bpf hello_world.d -of hello_world_d.o
$ ldc2 --betterC -L-lbpf -L-lelf -L-lz -of=loader_d loader_d.d
$ sudo ./loader_d

libbpf: elf: skipping unrecognized data section(21) .eh_frame
libbpf: elf: skipping relo section(22) .rel.eh_frame for section(21) .eh_frame
libbpf: BTF loading error: -22
libbpf: -- BEGIN BTF LOAD LOG ---
magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 312
str_off: 312
str_len: 374
btf_total_size: 710
[1] PTR hello_world.xdp_md* type_id=2 Invalid name

-- END BTF LOAD LOG --
libbpf: Error loading .BTF into kernel: -22. BTF is optional, ignoring.
libbpf: prog 'xdp_prog': BPF program load failed: Invalid argument
libbpf: prog 'xdp_prog': -- BEGIN PROG LOAD LOG --
unknown opcode 8d
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: prog 'xdp_prog': failed to load: -22
libbpf: failed to load object 'hello_world_d.o'
ERROR: loading BPF object file failed

I updated xdp_md when it complained about it, it looks like it might be having trouble with the type name being prefixed with the module name hello_world?

On Monday, 6 January 2025 at 07:26:02 UTC, Richard (Rikki) Andrew Cattermole wrote:

I don't actually think it matters what I do with the struct it still gives the same error until I remove the argument completely. So if I just use struct xdp_md as in the C code it outputs the same error, and if I use void * as the argument, it outputs the same error. When I remove the argument completely I get the error:

PTR extern (C) long function(const(char)* fmt, uint fmt_size, ...) @nogc @system type_id=8 Invalid name

I use a cast like this cast(BpfTracePrintk)6 for the value because D will not allow a cast to void* in this case.

But all of this may not be that relevant, it looks like from the output, these variables remain as D symbols and are not being translated with the --march=bpf switch to instructions the kernel can interpret.