Judging the Effects of Policy

Can we know if a policy has succeeded? Some have argued that we never can, but most economists and policymakers at least act as if we can - whether through economic or statistical analysis, through simple observation, or in some other way. However, confirmation bias and ideology may also blind us. The US stimulus package is a case in point. Those who advocated for it contend that it succeeded, at least to ward off a worse crisis - and say that it should have been bigger, but at least it was something. Those who argued against it contend that it failed. How can we know?

Many who contend that it failed observe that the advocates of the policy cited a report by the administration's Council of Economic Advisers which predicted 9% unemployment without the proposed stimulus, and not more than 8% with it; whereas with the stimulus package in place unemployment actually exceeded 9%. Defenders of the stimulus program have replied that without the stimulus it would have been even worse. How can we know?

It is not possible (nor likely to be morally justifiable) to perform laboratory tests on social systems. Some economists argue that one can only use logical theory and deduction to make broad observations and predictions about economic behavior - that we cannot use models or statistics to predict precise outcomes or prove or disprove theory. However, it makes logical sense that if there is a situation in which a choice is to be made about whether to enact a policy, especially one introduced to avert a major crisis and which is expected to have a dramatic effect curtailing that crisis, and most else remains the same (something approximating "ceteris paribus"), observers should have some idea whether the policy was successful. If a prediction is made about the differing outcomes with and without that policy, we should look closely at the actual outcome and conclude whether or not the policy in fact achieved its intended results.

In 2009 many economists and policymakers argued that a massive "stimulus" package, a ramping up of government spending, was needed to avert economic crisis and turn the economy around. In 1917 before taking power Lenin made a similar prediction about his policy. The policy of swift nationalization of banks and major industry was argued by its supporters to be critical for avoiding crisis.

Lenin described what he saw as "the chief and principal measure of combating, of averting, catastrophe and famine." He said that it was well known, but "these measures are not being adopted only because, exclusively because, their realisation would affect the fabulous profits of a handful of landowners and capitalists." Lenin argued that the answer was state control of the economy:

This measure is control, supervision, accounting, regulation by the state, introduction of a proper distribution of labour-power in the production and distribution of goods, husbanding of the people’s forces, the elimination of all wasteful effort, economy of effort. Control, supervision and accounting are the prime requisites for combating catastrophe and famine.

Did it work? It is clear that it did not. Hardly anyone disagrees with this analysis. The economy slid into severe famine and ruin. Markets disappeared, as indeed Lenin intended, but nothing of use replaced them. The state found it near-impossible to even feed the people of the cities, let alone construct a production and distribution system that would alleviate the hardship the people already faced.

Of course, civil war replaced the war with Germany, and this did not help the government in its tasks. Other factors ensured there was nothing like a true "ceteris paribus" situation. Yet even Lenin understood that he had made a mistake. His belief in Marx's economic and social framework convinced him that the problem was that the policies were introduced too soon and too rapidly, but he did concede that his policies were to blame for the failure to avoid crisis, and that in fact his policies had exacerbated the crisis Russia had been facing when he took power.

Why? Because our previous economic policy, if we cannot say counted on (in the situation then prevailing we did little counting in general), then to a certain degree assumed—we may say uncalculatingly assumed—that there would be a direct transition from the old Russian economy to state production and distribution on communist lines.
...
Our Mistake

...we made the mistake of deciding to go over directly to communist production and distribution. We thought that under the surplus-food appropriation system the peasants would provide us with the required quantity of grain, which we could distribute among the factories and thus achieve communist production and distribution.

I cannot say that we pictured this plan as definitely and as clearly as that; but we acted approximately on those lines. That, unfortunately, is a fact. I say unfortunately, because brief experience convinced us that that line was wrong

What did Lenin do when he realized his mistake? He "retreated." He reversed the bulk of the policies which he saw had produced the negative results. "In substance, our New Economic Policy signifies that, having sustained severe defeat on this point, we have started a strategical retreat."

When restrictions on trade were lifted, there was an immediate blossoming of market activity, which resulted in an end to shortages and an improvement in living standards of all the people. Writer Mikhail Bulgakov described the change that NEP brought. "On Kuznetskii Most [a main street in Moscow], the painted faces of toy figures made by artel [co-operative] craftsmen smile. In the former Shanks store, ladies’ hats, stockings, boots, and furs gaze out at the clouds..... There is a confectioners shop at every step." Most importantly, the food shortages and famine were left in the past. "The luxurious displays at the gastronomes are startling. Mounds of crates with canned goods, black caviar, salmon, smoked fish, oranges." They were affordable enough that even the poorest Russians were made far better off.

This was obvious to all, regardless of ideology, and is not debated today by historians. It was clear that it was the wrong policy - whether it was because it was the wrong time to enact the policy or whether the policy would always produce these results*, at least all agree that Lenin's assertion that only his polices of nationalization and a swift advance toward a fully socialized economy could avert crisis was wrong and worsened, rather than alleviating, the hardships facing Russia when at that time.

I contend that the same method can be used to judge any major policy enacted to avert crisis. If the policymaker asserts that it will fend off a crisis, and the potential crisis is predicted to be of a certain magnitude, and the alternative path is anticipated and described, the policy should be judged accordingly. If the outcome is equal or greater than the expected level of the crisis path, the policy should be assumed to have failed unless a very good explanation is given to explain this discrepancy; this is simple common sense, and use of plain observation. Few citizens subjected to policy would disagree, and even given the complexities of society and economy, the ordinary citizen is right in this case.

--

* Trotsky, although not at first, also conceded the policies of that period were a failure, and although like Lenin he contended that they were simply enacted too soon, his descriptions of why they failed suggest otherwise. In 1924, Trotsky defended a return to use of markets, and economic independence of enterprises, explaining that, "With the liquidation of the market and of the credit system each factory resembled a telephone whose wires had been cut." In other words, there was no information being channeled from or to each factory or enterprise. Information that normally travels via the market through the medium of prices and profit and loss calculations had vanished, and the factories did not know how to produce efficiently, leading to chaos in production, shortages, and massive waste.