Jump to page: 1 2
Thread overview
A 'lazy' compilation mode to make writting multiplatform code easier without having to clutter the project
Feb 22, 2023
ryuukk_
Feb 22, 2023
ryuukk_
Feb 22, 2023
jmh530
Feb 22, 2023
ryuukk_
Feb 22, 2023
ryuukk_
Feb 22, 2023
ryuukk_
Feb 22, 2023
ryuukk_
Feb 22, 2023
ryuukk_
Feb 23, 2023
ryuukk_
February 22, 2023

Hello,

My project is kinda special, i am targeting all 3 major desktop OS's as well as the web with WebAssembly

So my code is full of version(WASM) / version(Windows), i wish i didn't need to do that because it clutters the code for no real benefits, makes things harder to follow/read as i constantly need to readjust to the indentation level and need to process version() despite my function named properly

It's kinda repeating the C/C++ mistake with unreadable/unfollowable #ifdef everywhere

I try to structure my code so that i put platform specific code is in their own module when it's too big

But for 3-4 simple functions it gets annoying real quick

When i separate platform specific code into their own modules, i have then to do a lot of maintenance to then include the proper modules when i compile for the specific target, or when i refactor my code, i need to make sure my dub.json is in sync

This is kinda annoying..

So what if the compiler would follow the code path and only compile what is used from the module, so there is no longer a need to be overly noisy with bunches of version code blocks and extra indentation everywhere

One benefit of that mode would be faster builds when importing modules from phobos when one need just few functions from it 

Because right now it is very ugly, and my dub.json gets quite large that it becomes painful to read/edit (yeah, i'm also not a fan of json, it's too noisy/verbose)

{
    "name": "game",
    "targetType": "executable",
    "targetName": "game",
    "targetPath": "../../bin",
    "workingDirectory": "../../bin",
    "sourcePaths": [ "src" ],
    "importPaths": [ "src" ],

    "stringImportPaths": [
        "../../bin/"
    ],

    "buildTypes": {
        "prod": {
            "buildOptions": [ "releaseMode" ],
            "versions": [ "PROD" ],
            "dflags": [
                "-preview=rvaluerefparam",
            ],
            "dflags-ldc": [
                "-O3",
            ],
            "dflags-windows-ldc": [
                "-O3",
            ],
        },
        "prod-debug": {
            "buildOptions": [ "debugMode", "debugInfo" ],
            "versions": [ "PROD" ]
        },
    },

    "configurations": [
        {
            "name": "desktop",
            "mainSourceFile": "src/app.d",
            "versions": [ "GAME", "DESKTOP", "MONGOOSE", "GL_30", "GLFW_33", "FT_211", "BindFT_Static"],
            "sourcePaths": [
                "../../better_d/rt",
                "../../better_d/network",
                "../../better_d/mongoose",
                "../../better_d/glfw",

                "../dawn",
                "../kshared/",
            ],
            "importPaths": [
                "../../better_d/",
                "../../better_d/rt",
                "../",
            ],
            "libs-windows": [
                "ws2_32",
                "Winmm",
                "USER32",
                "GDI32",
                "ADVAPI32",
                "CRYPT32",
                "Shell32"
            ],
            "libs-linux": ["freetype", "z", "ssl", "crypto", "x11"],
            "sourceFiles-windows": [
                "../../better_d/glfw/bin/windows/glfw3.lib",
                "../../better_d/freetype/bin/windows/freetype.lib",

                "../../better_d/mongoose/bin/windows/libcrypto.lib",
                "../../better_d/mongoose/bin/windows/libssl.lib",
                "../../better_d/mongoose/bin/windows/mongoose.lib",

                "../../better_d/zlib/bin/windows/zlib.lib",
            ],
            "sourceFiles-linux": [
                "../../better_d/glfw/bin/linux/libglfw3.a",
                "../../better_d/mongoose/bin/linux/mongoose.a",
            ],
            "dflags": [
                "-preview=rvaluerefparam",
                "-betterC"
            ],
            "dflags-dmd": [
                "-preview=bitfields",
            ],
	        "lflags-windows-dmd": [ "/OPT:REF"],

            "lflags-windows": [
                "/NODEFAULTLIB:libcmt.lib"
            ],
        },

        {
            "name": "dll",
            "versions": [ "GAME", "DESKTOP", "MONGOOSE", "GL_30", "GLFW_33", "FT_211", "BindFT_Static"],
            "targetType": "dynamicLibrary",
            "targetName": "gamed",
            "sourcePaths": [
                "../../better_d/rt",
                "../../better_d/network",
                "../../better_d/mongoose",
                "../../better_d/glfw",

                "../dawn",
                "../kshared/",
            ],
            "importPaths": [
                "../../better_d/",
                "../../better_d/rt",
                "../",
            ],
            "libs-windows": [
                "ws2_32",
                "Winmm",
                "USER32",
                "GDI32",
                "ADVAPI32",
                "CRYPT32",
                "Shell32"
            ],
            "libs-linux": ["freetype", "z", "ssl", "crypto", "x11"],
            "sourceFiles-windows": [
                "../../better_d/glfw/bin/windows/glfw3.lib",
                "../../better_d/freetype/bin/windows/freetype.lib",

                "../../better_d/mongoose/bin/windows/libcrypto.lib",
                "../../better_d/mongoose/bin/windows/libssl.lib",
                "../../better_d/mongoose/bin/windows/mongoose.lib",

                "../../better_d/zlib/bin/windows/zlib.lib",
            ],
            "sourceFiles-linux": [
                "../../better_d/glfw/bin/linux/libglfw3.a",
                "../../better_d/mongoose/bin/linux/mongoose.a",
            ],
            "dflags": [
                "-preview=rvaluerefparam",
                "-betterC"
            ],
            "dflags-dmd": [
                "-preview=bitfields",
            ],
	        "lflags-windows-dmd": [ "/OPT:REF" ],

            "lflags-windows": [
                "/NODEFAULTLIB:libcmt.lib"
            ],
        },
        {
            "name": "wasm",
            "targetName": "game",
            "mainSourceFile": "src/app.d",
            "versions": [ "GAME", "WASM"],

            "sourcePaths": [
                "../../better_d/rt",
                "../dawn",
                "../kshared/"
            ],

            "importPaths": [
                "../../better_d",
                "../",
            ],

            "dflags-ldc": [
                "--fvisibility=hidden",
                "-linkonce-templates",
                "-preview=rvaluerefparam",
                "-O3",
                "-flto=thin",
            ],
            "lflags-ldc": [
                "-z","stack-size=1048576",
                "--stack-first",

                "-allow-undefined"
            ],
            "sourceFiles": [
                "../../better_d/zlib/bin/wasm32/zlib.a",
            ]
        }
    ]
}

Example of module:

module rt.event;

version(Windows)
{
    enum OK = true;
    // windows specific imports/aliases
}
else version(Posix)
{
    enum OK = true;
    // posix specific imports/aliases
}
else version(WASM)
{
    enum OK = false;
}

static if(OK)
{
    // common code

    version(Windows)
    {
        // windows specific common code
    }
    else version(Posix)
    {
        // posix specific common code
    }
}

What if:

module rt.event;

// import windows specific
import windows = ..;

// import posix specific
import posix = ..;

void poll()
{
    version (Windows)
        poll_windows();
    else version(Posix)
        poll_posix();
}


void poll_posix()
{
    posix.___();
}
void poll_windows()
{
    windows.___();
}

Compiler should know that if i am on windows, when i call poll() it'll only ever call poll_windows(), why fetch import posix = ..;and compile poll_posix()?

version is nice when i want to separate code path within a function, but everything outside doesn't make much sense as it clutters code

If a lazy compilation mode is not possible, then what about something that would allow something like that:

module rt.event;

version(WASM)
{
    return; // no event in WASM target
}

(..)

or

module rt.event; // exclude: WASM

(..)
February 22, 2023

Odin for example has a feature to make platform specific modules easier to deal with, it's quite useful

//+build windows
//+private
package sync

import "core:time"
import win32 "core:sys/windows"

https://github.com/odin-lang/Odin/blob/master/core/sync/primitives_windows.odin#L1

February 22, 2023

On Wednesday, 22 February 2023 at 14:23:09 UTC, ryuukk_ wrote:

>

Hello,

My project is kinda special, i am targeting all 3 major desktop OS's as well as the web with WebAssembly

[snip]

Compiler should know that if i am on windows, when i call poll() it'll only ever call poll_windows(), why fetch import posix = ..;and compile poll_posix()?

[snip]

If poll_posix is made private, is the compiler smart enough to optimize it away on Windows?

February 23, 2023
Alternative approach:

```d
module rt.event;

version(Windows)
	public import ... : poll;
else version(...)
	public import ... : poll;
else
	static assert(0, "I don't know how to handle event loop here");
```

```d
module ...;
version(Windows):

void poll() {

}
```
February 22, 2023
On Wednesday, 22 February 2023 at 14:39:49 UTC, Richard (Rikki) Andrew Cattermole wrote:
> Alternative approach:
>
> ```d
> module rt.event;
>
> version(Windows)
> 	public import ... : poll;
> else version(...)
> 	public import ... : poll;
> else
> 	static assert(0, "I don't know how to handle event loop here");
> ```
>
> ```d
> module ...;
> version(Windows):
>
> void poll() {
>
> }
> ```

static assert is the problem here, it won't compile on wasm target
February 22, 2023
On Wednesday, 22 February 2023 at 14:51:41 UTC, ryuukk_ wrote:

here my actual module:

```D
module rt.event;

version(Windows)
{
    enum OK = true;
    import core.sys.windows.winsock2;
    import core.stdc.errno;
    import core.stdc.string;

    alias WSAPOLLFD = pollfd;
    alias PWSAPOLLFD = pollfd*;
    alias LPWSAPOLLFD = pollfd*;
    struct pollfd
    {
        SOCKET fd;
        short events;
        short revents;
    }

    alias poll = WSAPoll;
    extern(Windows) nothrow @nogc int WSAPoll(LPWSAPOLLFD fdArray, ulong fds, int timeout);

    enum short POLLRDNORM = 0x0100;
    enum short POLLRDBAND = 0x0200;
    enum short POLLIN = (POLLRDNORM | POLLRDBAND);
    enum short POLLPRI = 0x0400;

    enum short POLLWRNORM = 0x0010;
    enum short POLLOUT = (POLLWRNORM);
    enum short POLLWRBAND = 0x0020;

    enum short POLLERR = 0x0001;
    enum short POLLHUP = 0x0002;
    enum short POLLNVAL = 0x0004;

    alias socket_t = size_t;
}

version(Posix)
{
    enum OK = true;
    import core.stdc.string;
    import core.sys.posix.poll;
    import core.stdc.errno;
    alias socket_t = int;
    enum INVALID_SOCKET = -1;
}

version(Linux)
{
    import core.sys.linux.epoll;

    struct EPollSelector(int MAX)
    {
        epoll_event[MAX] fds;
        size_t _count;
	    int epollFD_;
	    int fdsCount_ = 0;

        void create()
        {
            epollFD_ = epoll_create(1);
        }

        Selector selector()
        {
            return Selector.create_it!(EPollSelector)(&this);
        }

        bool insert_fd(socket_t fd, FDEvent ev = FDEvent.READ)
        {
            auto c = _count;
            auto len = fds.length;
            if (c >= len)
                return false;

            epoll_event it;
            it.data.fd = fd;

            if (ev == FDEvent.READ)
                it.events = EPOLLIN;
            else if (ev == FDEvent.WRITE)
                it.events = EPOLLOUT;

            int result = epoll_ctl(epollFD_, EPOLL_CTL_ADD, fd, &it);

            // fds[_count++] = it;

            return result;
        }

        bool update_fd(socket_t fd, FDEvent ev)
        {
            epoll_event it;
            it.data.fd = fd;

            if (ev == FDEvent.READ)
                it.events = EPOLLIN;
            else if (ev == FDEvent.WRITE)
                it.events = EPOLLOUT;

            int result = epoll_ctl(epollFD_, EPOLL_CTL_MOD, fd, &it);
            return result == 0;
        }

        bool remove_fd(socket_t fd)
        {
            int result = epoll_ctl(epollFD_, EPOLL_CTL_DEL, fd, null);
            return result == 0;
        }

        WaitStatus wait(int timeout)
        {
            fdsCount_ = epoll_wait(epollFD_, fds.ptr, _count, timeout);
            if (fdsCount_ < 0)
            {
                switch (errno)
                {
                case EINTR:
                    return WaitStatus.NONE;
                default:
                    return WaitStatus.ERROR;
                }
            }

            if (fdsCount_ == 0)
                return WaitStatus.NONE;
            else
                return WaitStatus.OK;
        }

        FDResult get_status(epoll_event* ev)
        {
            auto fd = ev.data.fd;
            // import rt.dbg;
            // LINFO("r: {} {} {}", _count, p.events, p.revents);

		    if (ev.events & EPOLLERR)
                return FDResult(fd, FDStatus.ERROR);

		    if ((ev.events & EPOLLIN) || (ev.events & EPOLLHUP))
                return FDResult(fd, FDStatus.READ);

		    if (ev.events & EPOLLOUT)
                return FDResult(fd, FDStatus.WRITE);

            return FDResult(fd, FDStatus.NONE);
        }

        int opApply(scope int delegate(FDResult) dg)
        {
            int result;
            for (int i = 0; i < _count; i++)
            {
                auto ptr = &fds[i];
                auto res = get_status(ptr);
                if ((result = dg(res)) != 0)
                    break;
            }
            return result;
        }
    }

}


static if (OK)
{
    struct PollSelector(int MAX)
    {
        pollfd[MAX] fds;
        size_t _count;

        void create()
        {
            _count = 0;
            foreach(ref it; fds)
            {
                it.fd = INVALID_SOCKET;
                it.events = 0;
                it.revents = 0;
            }
        }

        Selector selector()
        {
            return Selector.create_it!(PollSelector)(&this);
        }

        bool insert_fd(socket_t fd, FDEvent ev = FDEvent.READ)
        {
            auto c = _count;
            auto len = fds.length;
            if (c >= len)
                return false;

            pollfd it;
            it.fd = fd;
            if (ev == FDEvent.READ)
                it.events = POLLIN;
            else if (ev == FDEvent.WRITE)
                it.events = POLLOUT;
            it.revents = 0;
            fds[_count++] = it;

            return true;
        }

        bool update_fd(socket_t fd, FDEvent ev)
        {
            foreach(ref it; fds)
            {
                if (it.fd == fd)
                {
                    if (ev == FDEvent.READ)
                        it.events = POLLIN;
                    else if (ev == FDEvent.WRITE)
                        it.events = POLLOUT;
                    it.revents = 0;
                    return true;
                }
            }
            return false;
        }

        bool remove_fd(socket_t fd)
        {
            int index = -1;
            foreach(i, ref it; fds)
            {
                if (it.fd == fd)
                {
                    index = cast(int) i;
                    it.fd = INVALID_SOCKET;
                    it.events = 0;
                    it.revents = 0;
                    break;
                }
            }

            if (index >= 0)
            {
                memmove(fds.ptr + index, // dest
                        fds.ptr + index + 1, // src
                        (_count - index) * pollfd.sizeof); // num bytes
            }

            return index >= 0;
        }

        WaitStatus wait(int timeout)
        {
            int activity = poll(fds.ptr, _count, timeout);
            if (activity < 0)
            {
                switch (last_error())
                {
                case EAGAIN:
                case EINTR:
                    return WaitStatus.OK;
                default:
                    return WaitStatus.ERROR;
                }
            }

            if (activity == 0)
                return WaitStatus.NONE;
            else
                return WaitStatus.OK;
        }

        FDResult get_status(pollfd* p)
        {
            auto fd = p.fd;

            import  rt.dbg;

            // LINFO("r: {} {} {}", _count, p.events, p.revents);

            if (fd >= 0)
            {
                if (p.revents & POLLERR)
                    return FDResult(fd, FDStatus.ERROR);

                if (p.revents & POLLIN)
                    return FDResult(fd, FDStatus.READ);

                if (p.revents & POLLOUT)
                    return FDResult(fd, FDStatus.WRITE);
            }

            return FDResult(fd, FDStatus.NONE);
        }

        int opApply(scope int delegate(FDResult) dg)
        {
            int result;
            for (int i = 0; i < _count; i++)
            {
                auto ptr = &fds[i];
                auto res = get_status(ptr);
                if ((result = dg(res)) != 0)
                    break;
            }
            return result;
        }
    }

    struct Selector
    {
        struct VTable
        {
            bool function(const void* self, socket_t fd, FDEvent ev) insert_fd;
            bool function(const void* self, socket_t fd, FDEvent ev) update_fd;
            bool function(const void* self, socket_t fd) remove_fd;
            WaitStatus function(const void* self, int timeout) wait;
            FDResult function(const void* self, pollfd* p) get_status;
        }
        void* ptr;
        VTable* vt;
        static Selector create_it(T)(void* pointer)
        {
            static struct gen(T)
            {
                __gshared static VTable vtable = {
                    insert_fd: &insert_fd_impl,
                    update_fd: &update_fd_impl,
                    remove_fd: &remove_fd_impl,
                    wait: &wait_impl,
                    get_status: &get_status_impl,
                };

                static bool insert_fd_impl(const void* ptr, socket_t fd, FDEvent ev)
                {
                    auto self = cast(T*) ptr;
                    return self.insert_fd(fd, ev);
                }
                static bool update_fd_impl(const void* ptr, socket_t fd, FDEvent ev)
                {
                    auto self = cast(T*) ptr;
                    return self.update_fd(fd, ev);
                }
                static bool remove_fd_impl(const void* ptr, socket_t fd)
                {
                    auto self = cast(T*) ptr;
                    return self.remove_fd(fd);
                }
                static WaitStatus wait_impl(const void* ptr, int timeout)
                {
                    auto self = cast(T*) ptr;
                    return self.wait(timeout);
                }

                static FDResult get_status_impl(const void* ptr, pollfd* p)
                {
                    auto self = cast(T*) ptr;
                    return self.get_status(p);
                }

            }
            assert(pointer);
            return Selector(pointer, &gen!(T).vtable);
        }
    }

    enum WaitStatus
    {
        OK,
        NONE,
        ERROR
    }

    enum FDEvent : bool
    {
        READ,
        WRITE
    }

    enum FDStatus : ubyte
    {
        NONE,
        READ,
        WRITE,
        ERROR,
        STOP
    }

    struct FDResult
    {
        socket_t fd = INVALID_SOCKET;
        FDStatus status;
    }


    package:

    int last_error()
    {
        version (Windows)
            return WSAGetLastError();
        else version (Posix)
            return errno;
        else
            static assert(0);
    }
}
```
February 23, 2023
Why wouldn't this work?

```d
else version(WASM) {
	void poll() {
		pragma(inline, true);
		// or assert(0);
		// or noreturn ext.
	}
} else static assert(0);
```
February 22, 2023
On Wednesday, 22 February 2023 at 14:56:54 UTC, Richard (Rikki) Andrew Cattermole wrote:
> Why wouldn't this work?
>
> ```d
> else version(WASM) {
> 	void poll() {
> 		pragma(inline, true);
> 		// or assert(0);
> 		// or noreturn ext.
> 	}
> } else static assert(0);
> ```

That would work, but i'd then need to apply the same for every functions and stuff, but at that point i'd rather have the extra indentation and skip that

A lazy mode or a way to exclude a module for a target easily are really the only solutions that i'm looking forward, the simplified example provided was to show a usecase
February 22, 2023

On Wednesday, 22 February 2023 at 14:39:41 UTC, jmh530 wrote:

>

On Wednesday, 22 February 2023 at 14:23:09 UTC, ryuukk_ wrote:

>

Hello,

My project is kinda special, i am targeting all 3 major desktop OS's as well as the web with WebAssembly

[snip]

Compiler should know that if i am on windows, when i call poll() it'll only ever call poll_windows(), why fetch import posix = ..;and compile poll_posix()?

[snip]

If poll_posix is made private, is the compiler smart enough to optimize it away on Windows?

It does, but not if inside i call functions from a posix specific import, i get: Error: undefined identifier even thought it's never called

February 22, 2023

Adam in the IRC chat mentioned using dmd -i, i'll experiment with it and report back, after reading the doc it looks like that's what i need?

https://dlang.org/dmd-windows.html#switch-I

« First   ‹ Prev
1 2