Problem is that the lug nuts/stubs job is to hold the wheel on,not support the entire rig.
Every time you hit a bump you are sending the impact into the side of the lug studs,this can cause the lug studs or the spacer itself to fail.
You will also find cheap spacers are not anodized. Raw aluminum will rot over time weakening the part which can lead to failure.