The way I see it, the notion of having one build with strippable unittests is a nice idea but technically challenging. It's also low impact - today concurrent CPU is cheap so running two concurrent unrelated builds can be made as fast as one....
The simple effective step toward improvement is to uniformize the format of assertion errors in unittests and to make it easy with tooling to create unittest and non-unittest builds that are gated by the unittests succeeding.