The First question that strikes in our mind about Big Data Technology is, what is Big Data and why it is called so?
Be patient, as today you’re going to learn everything you need to know about Big Data and why we have been using this and in today’s date why Big data is so important.
Big Data is different from the traditional data processing software. In Big Data the data sets are very large and complex. The concept of Big Data was introduced by industry analyst Doug Laney in the early 2000s. The Main Concept of Big Data Based on 3(Three) “V”s- Volume, Variety and Velocity. Let us explains what are the Volume, Variety and Velocity.
Volume: As the volume term explains everything that it is pointing about the memory space. The companies and organizations collect data from many sources including social media(Facebook, Twitter, etc.), business transactions and information from sensor or machine-to-machine data. Before the introduction of Big Data Technology it was very problematic job to handle all the data but new technologies (for example Hadoop) have eased the burden.
Variety: The variety describes all kinds of data where structured, unstructured, numeric, text documents, email, video, geospatial, audio, stock ticker data and financial transactions and other traditional databases.
Velocity: The Velocity is the frequency of incoming data which needs to be processed. In other words we can explain Velocity in case of Big Data is, Speed of Creation or Change. For example, the speed of transfer of data for status updates of Facebook or Whatsapp, text messages or Credit and Debit card swipes being sent on a single telecom carrier per minute of each day should give you a good idea about velocity.
Use of Big Data
Big Data can be a very useful tool in analyzing the incredibly complex stock market moves and aid in making global financial decisions.
For example, intelligent and extensive analysis of the big data available on Google Trends can aid in forecasting the stock market. Though this is not a fool-proof method, it definitely is an advancement in the field.
What are the Benefits to use of Big Data Technology?
- Low cost storage to store data that was discarded earlier.
- Powerful multi-core processors.
- Low latency possible by distributed computing: Compute clusters and grids connected via high-speed networks.
- Virtualization: Partition, Aggregate, isolate resources in any size and dynamically change.
- Affordable storage and computing with minimal man power.
- Better understanding of task distribution (MapReduce), computing architecture (Hadoop),
- Advanced analytical techniques (Machine learning).
- Managed Big Data Platforms: Cloud service providers, such as Amazon Web Services provide Elastic MapReduce, Simple Storage Service (S3) and HBase – column oriented database. Google’ BigQuery and Prediction API.
- Open-source software: OpenStack, PostGresSQL 10. March 12, 2012: Obama announced $200M for Big Data research. Distributed via NSF, NIH, DOE, DoD, DARPA, and USGS (Geological Survey).