SUBSCRIBE NOW
IN THIS ISSUE
PIPELINE RESOURCES

Big Data and Machine Learning

By: Amit Thombre

Big data is used to describe data which is so much larger and complex, that it is difficult to process using on hand database management tools or traditional processing applications. It refers to large, diverse, complex, longitudinal, and distributed data sets generated from instruments, sensors, Internet transactions, email, video, click streams, and all other digital sources available today and in the future. Through advances in communication technology, people and things are becoming increasingly interconnected. For example today millions of telephone calls are made every second from mobile to mobile or from mobile to landline. The data on the numbers called and the time taken for each call is stored for billing purposes and for further analysis. This forms a case of Big Data. This data can reach up to terabytes within a day or days depending on the operator.

There are three widely-adopted characteristics to define Big Data: volume, variety and velocity. Today, the volume of data being generated is staggering. Twitter alone generates more than seven terabytes of data every day; Facebook 10TB and some enterprises generate terabytes of data every hour of every day of the year.


With the explosion of sensors and smart devices, as well as social collaborations technologies, data in an enterprise displays ever-increasing variety. This data has simultaneously become complex; it includes not only traditional data, but also raw, semi-structured and unstructured data from web pages, web log files, search indexes, social media forums, email, documents, sensor data from active and passive systems, and more.

This volume and variety of data is generated faster than the blink of an eye, which is the velocity metric. For example the social media data or the mobile call data is generated continuously. Gaining an edge over your competition can mean identifying a trend, problem or opportunity only seconds or even microseconds before someone else. Based on this velocity, data needs to be analyzed in real-time to leverage its full advantage.

Why Machine Learning?

Every business is interested in knowing how it will perform in the future. The ability to predict the parameters which affect a business in the future—predictive analytics—is becoming increasingly important. This is where machine learning comes into picture. By making accurate predictions, companies can improve planning, head off potentially large expenses, improve services, and compete more effectively.

Human minds are limited by their ability to analyze a handful of variables at one time. Machine learning, however, can “think” about hundreds of variables and use them to find patterns and deep insights into the data. It ultimately can assist management in making important decisions that affect the way business is carried out. In an increasingly competitive climate, machine learning could soon become a differentiator that separates the winners from the losers.

Where is Machine Learning used?

One popular application for Machine Learning (ML) is speech emotion recognition, which aims to automatically identify the emotional or physical state of a human being from his or her voice. It has found increasing use in security, learning, medicine, entertainment, etc. Abnormal emotion (e.g. stress and nervousness) detection in audio surveillance can help detect a lie or identify a suspicious person. Web-based e-learning has prompted more interactive functions between computers and human users. The automatic recognition of emotions in speech can also be useful in clinical studies, psychosis monitoring and diagnosis.



FEATURED SPOTLIGHT

Latest Updates

Click to Discover>




Subscribe to our YouTube Channel