That's not exactly true. We could very much have nice things AND raiding if Blizzard was willing to cut back on the development of eleventybillion boss raid tiers and use some of those resources for almost anything else, but most certainly if they used it for dungeon content.
Raiding at 4 lvls has destroyed the game for no other reason than there is no type of content other than raiding that isn't utterly untuneable because of a 50 ilvl gap in players. That 50 ilvl gap is only necessitated by the fact that the developers know that virtually no one would raid if they could get decent gear by playing other parts of the game.
So I'd say the reason we can't have nice things is not so much because of raiders and they're whining about having to do other things but more because the developers already know from past experience what raid participation looks like when there are alternatives available and they don't want to make alternatives, they want to make raids.