Good to have a thread on it, thank you for taking this step Walter !
Although, it looks to me that the numbering scheme and the git tags are not related to the build, but rather related to the release.
So you might consider a release master, not just build.
This will involve doing (or delegating to the proper person) the changelog, define a release cycle, which will include pre-release (rc / sanity check), release (communication / building / test / upload ...), and possibly post-release (be sure the security fixes are being backported to the supported versions, ie) actions.
IMO that would be very beneficial for D to have someone holding that role.