There are actions which, when practiced on mass scale, would yield terrible results.
Thus, to practice them on small scale is also bad, since every action also has potential to multiply itself and turn from small scale to large scale.
For example, if parents spank a child, it sends message to that child how doing violence because of small disagreements is okay, which later in life increases amount of violence child commits.
Thus, any deviation from Categorical Imperative on small scale can easily result in deviation on large scale.
When society doesnt follow Categorical Imperative, what happens is a contradiction.
Society which follows Categorical Imperative is a society without contradiction.
But if it ababdons Categorical Imperative, amounts of contradictions increase.
First, contradictions between individuals rise.
Contradictions in survival, property, power, and contradictions in activities make it impossible for society to remain comfortable to live in.
For example, if some person can kill another and get away with it, such society becomes terrible to live in.
If person can rape others or steal from others, such person becomes in contradiction with others. They cannot live in peace.
If any type of body violation becomes common, what always results is the justification of even more body violations, due to consistency.
Therefore, society which deviates from Categorical Imperative is a society which practices negative and false moral consistencies, such as "if that person can do bad things, so can I.".
For example, a society where lies are widespread, people are going to justify their lies by saying how other people would lie in that situation too.
But society built on lies cannot build truth, which results in harm.
Thus, abandoning Categorical Imperative will likely result in decay of society, especially due to negative justification such as "two wrongs make a right", "if they can, so can I", "everyone else would do this bad thing too"...