The Internet produces huge amounts of data every day. Around 2.5 quintillion bytes of data has been produced till now, so much that 90% of the data in the world today has been created just in the last two years alone. Sources of data include Social Media, Digital Pictures and Videos, Transaction Records and Cell Phone GPS signals. In 2013, the total amount of data in the world was estimated to be 4.4 zettabytes. This number is expected to rise steeply to 44 zettabytes by 2020.
Let us see some statistics of data on the Internet.
It processes 2.5 billion pieces of content and 500+ terabytes of data every day. It includes 2.7 billion ‘Like’ actions and 300 million photos per day, scanning roughly 105 terabytes every half an hour.
Some statistics of YouTube include:
1. Around one hour of video is uploaded every second
2. Over 4 billion videos are viewed Per day
3. Over 3 billion hours of video Are watched each month.
4. In 2011 alone, YouTube had more than one Trillion views(140 viewsfor every personal on Earth)
5. Around 30% off YouTube traffic is from the US where is the remaining 70% is from the rest of the world.
Google processes around 3.5 billion request requests per day. It stores over 10 exabytes (10 billion GB) of data.
An estimated 1000 petabyte of data stored across more than 1,400,000 Amazon servers.
With these statistics and estimates, it is clear that data generation is on the rise exponentially every day. It is a challenge to maintain these huge amounts of data and moreover, analyze it.
The solution for managing these enormous quantities of data is through Big data.
Big data is characterized by the following features of data:
Volume (amount of data)
Velocity (rate at which data flows)
Variety (types of data)
Veracity (authenticity of data)
Big data is managed by the Hadoop software. It consists of the HDFS(Hadoop Distributed File System) which manages structured and unstructured data. An example of this is the Infosphere Big insights developed by the IBM. Big Data can store both structured and unstructured data).
The Processing and Analysis of Data plays a crucial role in Businesses for determining Customer Trends or getting Feedback (Social Media).
Data is the new Oil in the market.