Is there a way to only link against it when it's needed automatically? I was hoping dmd would do that automatically.
You are explicitly linking to GtkD.lib and win32.lib, that's what's causing the increase in size.