Big Data Architecture

Summary:

Big Data is a specialization in analytics where extremely large sets of data are analyzed computationally to understand patterns, trends, and associations. Big Data also looks into how all these patterns relate to humans their behaviors and interactions. The problems often associated with Big Data are very complex and difficult to analyze and solve. The sheer volume of the data and the information can make it immensely challenging to get business insights; however if analyzed there are many benefits.

Analysis:

Data can be “stored, acquired, processed, and analyzed” and so can Big Data. Each and every source that produces data has different attributes and when the data is compiled and analyzed many more factors come need to be considered; some of these attributes are: “frequency, volume, velocity, type, and veracity.” It is important to consider all these factors when choosing a Big Data architecture to ensure that all your needs are met and to minimize chances of errors and malfunctions.

One of the primary steps in ensuring that the right architecture is being implemented and used is by identifying the business problem. Business problems can be categorized by the industry that Big Data is being used in. Some of the industries are Utilities, Telecommunications, Marketing, Customer Service, Retail, and Healthcare. Each industry can use big Data to further their profits by making certain processes more efficient and productive. Some examples IBM provide are: Utility companies using big data to predict power consumption; Telecommunications companies to analyze customer analytics with web, social, and transtraction data; Marketing and retailers can use social media and web data to get insights on their customer base; and with many other industries the use of the data can prove to be very valuable. Regardless of the problem, by categorizing the problems by type can make it easier to understand the characteristics.

There are a few main characteristics to consider when choosing a Big Data Architecture. The analysis type, processing methodology, data frequency and size, data type, content format, data source, data consumers, and hardware. The analysis type affects several other decisions about products, tools, hardware, data sources, and expected data frequencies; some types of analysis include fraud detections and trend analysis’. Processing methodologies can have a major impact on the big data architecture chosen and implemented as it is the technique applied to handle the data and with different techniques and technique combinations there will be different architectural needs. The frequency and size at which the data is captured/received and processed is also important to consider. Social Media can be considered as On Demand while weather can be considered as real time or continuous. The data type and content format and source are all very important to consider especially when receiving the data. Not only is the the source important to consider but also the consumers of the processed data: Business processes, Business users, Enterprise applications, Individual people in various business roles, Part of the process flows, and other data repositories or enterprise applications.

All of these characteristics are important factors in determining an architecture type for a Big data solution; especially since Big Data handles so much data every factor can have a major impact.

Sources

https://www.ibm.com/developerworks/analytics/library/bd-archpatterns1/bd-archpatterns1-pdf.pdf

Images

http://bigdata.inf.upv.es/wp-content/uploads/2016/05/bg.jpg