Big data generally leads to more intelligence, which leads to greater efficiencies. But when it comes to energy, big data threatens to reverse everything the modern smart grid stands for, according to researchers who spent the last several years studying big data energy consumption around the world.

The relationship between big data processing and the smart grid is a tightly interdependent, double-edged sword. On one hand, the smart grid needs big data centers to digest all the information that will help companies develop “greener” and more efficient operations, using things like advanced transmission sensors to real-time weather modeling.

But on the other hand, the very process of churning that information is a black hole of superfluous energy burn.

Data centers consume around 3 percent of the world’s total electricity, producing 200 million metric tons of carbon dioxide. And it’s not going down as power demands only grow; data centers in the U.S. are projected to consume 140 billion kilowatts per year by 2020, resulting in $13 billion per year in electricity bills.

Interdependency between smart grid and big data enterprise


Extensive research exists on achieving greater energy efficiency in big data and smart grids respectively, but very few researchers have zoomed out to focus on both.

“It’s always hard to look at the big picture. This is a mammoth system, and experts tend to get laser-focused on individual components,” said Zakia Asad, researcher. “We saw an opportunity to take a step back and come up with a move-forward plan for alignment and implementation.”

After dissecting the data cycle and conducting surveys with organizations around the world, Asad and her team brought theory to practice by proposing a comprehensive, six-plane strategy to help the big data enterprise boost its energy profile.

Here are some key takeaways of the six planes:

  1. An efficient server: Dynamic voltage and frequency scaling, server consolidation, job scheduling, and efficient hardware characteristics (such as NAND flash solid-state drives) are all ways to save energy
  2. Virtualization techniques: Virtualization is accepted as the most popular power management approach used by data centers, but only 30 percent of production servers deploy it.
  3. Networking techniques: Network devices are responsible for nearly a third of the total energy consumption in a data center, and links are highly under-utilized. Methods such as software-defined networking-based traffic engineering (e.g., OpenFlow) can help.
  4. Energy-savvy IT frameworks: Reduce idle IT equipment time by shortening the communication phase of distributed applications. The popular programming framework Hadoop HDFS has started integrating solar energy into its cluster, and there are other ways to optimize frameworks with high-level virtual machine shifting, mitigation of congestion incidents, etc.
  5. Environmental business model approaches: Lack of proper pricing models can lead to inefficiency. Businesses should gear up for energy-aware practices by measuring and reporting energy consumption metrics, and consider charging customers for power uses (Microsoft currently does this).
  6. Greener energy portfolio management: An intelligent demand-response strategy is crucial for any big data enterprise as it expands globally. It can also consider offering special incentives to customers for energy savings.

a six-plane approach to improving big data energy profile


Each plane has its own detailed ecosystem, but Asad is quick to note that the tactics complement one another, and it’s important to treat the approach holistically. This can be overwhelming for IT, data, and storage-based companies, so the researchers recommend implementing the steps on a per-job basis in accordance with highly specific workloads and applications.

“What surprised me most was the complex interplay between different parties and agendas,” added Asad. “We didn’t realize the human and social factors that are pushing and at the same time resisting the green revolution. The common goal is to get smarter and greener, but the big data enterprise is a business, and those companies have sub-incentives to appeal to customers wanting the highest performance at the cheapest rates.”

While the main challenge moving forward will be funding, the researchers hope the next handshake deal will be between the smart grid and the big data enterprise.

Learn more about big data and the smart grid on IEEE Xplore.