Why data should be our guiding light on public policy

Assuming their model is accurate, that’s a significant reduction — getting HIV rates among drug injectors nearly zero — and it’s all thanks to access to lots of data and lots of computing power. Recently, another group of researchers in Europe developed a computer model that found a strong correlation between web censorship and high violence rates while times of social unrest — a timely finding given the current state of world affairs.

Last week, I explained how Xerox is working to help Los Angeles and other cities get a better view of their traffic so they can try to make life more efficient and less congested for citizens, during simultaneously reducing pollution and optimizing budgetary resources. To achieve these goals, Xerox and other companies in this space are gathering data from everywhere — cars, mass-transit systems, traffic sensors, cell phones, weather databases — and developing complex machine learning models to determine how everything is connected.

However, as anyone who reads GigaOM regularly probably knows, decision-makers don’t need in-house supercomputers or data scientists on staff to inform their policies with data. All they actually need is an internet connection. Data sets are available everywhere you look, including at data marketplaces just as Factual and Infochimps, at Data.gov, and even increasingly on news sites just as the Guardian. Thanks to cloud computing, the resources necessary to analyze this data are cheap and plentiful.

The state of the world right now

Look at the state of the world right now. Droughts, deficits, civil wars, obesity epidemics. A skeptic would argue that the old methods of public policy decision-making, driven largely by political and economic concerns, haven’t worked out too then. Why not give data a chance to take the lead? In the wake of the great recession, smart businesses truly have.

Derrick has been a innovation journalist since 2003 and has been covering cloud computing, big data and other emerging IT trends for GigaOM since 2009. He has written the words "cloud" and "Hadoop" possibly more than any other person on the planet. Derrick lives in Las Vegas and has a...