How the cloud fits into the big data technology stack

What is the technology stack that makes big data go? And how does it work with cloud computing? Here’s how three successful companies – SoftLayer, Cloudant and Rosetta Stone – work at different layers of the big data technology stack, TechRepublic reports.

The foundation of a big data processing cluster is made of machines. Like relational data clusters, these machines usually have plenty of memory, CPU and storage. However, big data machines don’t have to be scaled up – they can be scaled out by adding more machines. The ability to scale out makes them a good match for cloud computing.

SoftLayer is a hardware IaaS provider – it does not deal with NoSQL directly but does deliver the clusters required to run them. Nathan Day, chief scientist at Softlayer, said they can “deliver a cluster of servers for things like the NoSQL solutions, so with things like Riak and Mongo, a customer can come and say ‘I want my own cluster of NoSQL servers. I want three of them in Amsterdam. I want three of them in Singapore.'”

These machines are often physical rather than virtual because bare metal makes performance less painful – with the unfortunate side effect of making the bill more painful. Day commented on bare metal versus virtual machines.

“We did a comparison test between deploying Mongo on bare metal and doing a deployment on a cloud…It’s very consistent on bare metal, as you’d expect, because you’re single tenant running on your hardware – it behaved very predictably. In a public virtual machine cloud, where you can’t control aspects of storage and even CPU and RAM access, the results varied wildly”.

Read more