Big Data

Machine learning / By

What is Big Data?

Big Data is large volume of data, that can be categorized based on relevance and that can be further subjected to analysis to best come up with a decision

 

BIG DATA-> ANALYSIS->DECISION

Characteristics of Big Data

  1. Volume
  2. Variety
  3. Velocity
  4. Value
  5. Complexity

Volume means how much data is available. Variety means how diverse the data is. Velocity is how fast this data is being generated. Value, how valuable is the generated data. Complexity, how does the data fit in the overall Big Data structure.

How Big Data is stored?

  1. Hyperscale computing environment
  2. Scaled out NAS
  3. Object storage

Hyperscale computing environment is used by large companies like Facebook, Google, etc. They are large servers which run analysts tool like Hadoop, NoSQL and Cassandra.

Scaled out NAS is used by Small businesses which is file access storage sharing unit that deals with lot of files.

Object storage is a tree like algorithm that indexes files to find the exact location of the file.

Where is the data being collected?

  1. Medical Advances : To improve healthcare.
  2. Government Aid : It gains data from people, social life to improve public transport, grow economy.
  3. Globalization and E-Business : Gives competitive advantage for businesses.

All this helps in personalisation of data that is part of computational intelligence. It is opening doors to neural networks and deep learning to make the decisions just like how humans do.

What are the application areas?

  1. Industry : for increasing production, reducing cost, detecting faults
  2. Logistics : Autonomous robots and drones, Autonomous vehicles, Sensor technology
  3. Medical : Personalized treatments, lifestyle applications, population profiling
  4. Energy: Micro grids
  5. Farming: Smart greenhouses, IOT powered agriculture
  6. Building : Urban planning, Drone delivery
  7. Home: Smart Home

Issues in Big Data and their Solutions

ISSUE SOLUTION
Risk to data privacy. Clarity to who owns the data.
Data Discrimination : AI make decisions like some human decisions based on prejudice about socio-economic background, race and gender bias that has been associated with stereo types for years. We need to make sure the decisions are made on facts not beliefs.
E-Policy changes without notifying end user. Every E-Policy changes must be notified to the end user.
Lack of Cyber-security. Ensure Cyber-security.
Cloud service provider giving personalised services to businesses only. Cloud service provider giving personalised services to normal customers also.
Absence of Data diversity. Presence of Data diversiy.