Gartner Augmented Analytics

The business world has declared Data as the new Oil, and nearly everyone agrees that data analytics is good for business. If data analysis is done properly, which isn’t easy, then data can drastically increase revenue, streamline processes and identify anomalies.  

The problem with data analysis, at least when you are trying to do it properly, is it’s difficult. Data on its own is useless, Data + Business Understanding = Value! 

Business Problem Identified – but Why? 

For example, the data has revealed that the company profits have decreased 10% in the last quarter. What it hasn’t done is tell me why they have decreased?  

What are the key factors that have contributed to the profits decreasing? To understand this you’d need to go deeper; How many sales? Was the value of the sales lower but volume of sales higher? How much expenditure did the company have in the last quarter? It’s possible that you can’t obtain all this information from one system, you may need data from a variety of sources.  

Once you have that data, what it may not reveal is business context, such as a new strategic vision, new product launch, a product removed from sale, new website, or a key member of staff leaving? 

When you combine data with business context, we can start to find out what connects what, which area directly impacts another. As we start to reveal this insight, it starts to become actionable insights, which can help the company, in this example, identify why profits have decreased, and therefore what action needs to be taken to stop the decline in profits.  

To be able to do all this it sounds difficult, complex and honestly expensive. You may need to hire individuals to perform these steps for your business, often known as Data Scientists or Data Engineers.  

Augmented Analytics  + OAC 

However, there is a new wave emerging in 2019, Augmented Analytics. The ability to utilise machine learning and natural language processing to automate data preparation and enable insight to be delivered to business users. The outcome is to present clear results and provide access to sophisticated tools so business users can make day-to-day decisions with confidence. Users can go beyond opinion and bias to get real insight and act on data quickly and accurately. 

Recently I’ve been preparing a presentation on ADW and OAC for the Ireland OUG. For the presentation I used a synthetic dataset to present what could be achieved through ADW and OAC. Within minutes of loading the data into OAC, I could see overall description of the fraud field as well as revealing key attributes which have influencing factors on the fraud column. This insight would prove invaluable as I went on to build supervised machine learning models to predict fraud on future transactions.  

What does accurate actually mean?

When we predict the class we can then measure how accurate it has predicted the classification. There are several methods to do this:

Confusion Matrix

For a binary classification problem the table has 2 rows and 2 columns. Across the top is the observed class labels and down the side are the predicted class labels. Each cell contains the number of predictions made by the classifier that fall into that cell.

		Observed
		Survived	Deceased
Predicted	Survived	10	2
	Deceased	4	15

Accuracy Paradox

Sometimes it may be desirable to select a model with a lower accuracy because it has a greater predictive power on the problem.

For example, in a problem where there is a large class imbalance, a model can predict the value of the majority class for all predictions and achieve a high classification accuracy, the problem is that this model is not useful in the problem domain. For example, Fraud occurs in less than 3% of transactions, so we could just say all transactions are ok, if you looked at the overall accuracy, then it would say our prediction was 97% accurate, however look under the hood, in the fraud class we would be 100% incorrect. 

Precision

Precision is the number of True Positives divided by the number of True Positives and False Positives. Put another way, it is the number of positive predictions divided by the total number of positive class values predicted. It is also called the Positive Predictive Value (PPV).

Precision can be thought of as a measure of a classifiers exactness. A low precision can also indicate a large number of False Positives.

Recall

Recall is the number of True Positives divided by the number of True Positives and the number of False Negatives. Put another way it is the number of positive predictions divided by the number of positive class values in the test data. It is also called Sensitivity or the True Positive Rate.

Recall can be thought of as a measure of a classifiers completeness. A low recall indicates many False Negatives.

What is AUC - ROC Curve?

AUC (Area under the Curve) - ROC (Receiver Operating Characteristics) curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability. It tells how much model is capable of distinguishing between classes. Higher the AUC, better the model is at predicting 0s as 0s and 1s as 1s. By analogy, Higher the AUC, better the model is at distinguishing between those who survived and those who didn’t.

The ROC curve is plotted with True Positive Rate (TPR) against the False Positive Rate (FPR) where TPR is on y-axis and FPR is on the x-axis.

An excellent model has AUC near to the 1 which means it has good measure of separability. A poor model has AUC near to the 0 which means it has worst measure of separability. In fact it means it is reciprocating the result. It is predicting 0s as 1s and 1s as 0s. And when AUC is 0.5, it means model has no class separation capacity whatsoever.

Accuracy Built In

Thankfully with the Oracle Advanced Analytics they have us covered. When applying machine learning models to data, they provide an option where you can monitor the accuracy, precision and recall of your models. 

Accuracy can be shown through the model performance. Here it enables us to see the True-False Matrix (0 and 1 in this case) and the accuracy within each class. 

 

Its possible to go one step further and investigate the AUC through the model as well:

If in my case you've trained a number of models, then don't panic, you can make an informed model selection a the click of a button, by comparing the model accuracy's and AUC all in 2 clicks. 

 

R vs Python vs SQL

In this exercise, I decided to test R against Python 

Classification problem, these problems are about assigning an object to a group. In our case we have 2 classes, those who survived (0) and those who didn’t (1). Classification algorithms look to assign a class based on a probability. 

On the Kaggle website there is a starter problem, Titanic (link), the objective of the challenge is using a training data set can you predict who would survive and who would not survive the Titanic. Kaggle provides a training data set and a test data set. Once you've made your predictions you upload the test data set to Kaggle to obtain a test score. 

So I thought, I wonder which would perform the best, R, Python or Oracle Advanced Analytics. To make the test fair I decided to implement an out of the box decision tree in all 3 of languages:

R Code

The R code took approximately 25 mins to code up, with a little cleaning of the data through to producing the file for submission. 

The final prediction score was 76.55

Python Code

In a similar style using Jupyter notebook I did the same task, prediction with a decision tree, specifically the Sklearn package. In total from reading the file in to producing the submission file it took approximately 20 mins to complete. 

The final prediction score was 71.29

Finally Oracle Advanced Analytics

I used the Oracle prebuilt VM to enable me to have Advanced analytics, loaded the data into a database table and created a OAA model, again using the decision tree approach. Implementation time from loading the data to file production was approximately 5 mins. 

The prediction score was 76.55

Based on this, it would seem to suggest without any major tuning, OAA will work as efficiently as the open source counter parts. In addition it was quicker and some would say easier. In my next blog we'll explore what accuracy is and how to investigate it with OAA.

Journey to the cloud

So today we'll discuss the migration to the cloud. A journey that on the surface or at least the sales pitch, sounds picture perfect. A swift migration to cloud and all your problems are solved. Well maybe not....

So our migration started like most others, in our case a on-premise Exadata, working well but not offering the strategic future that we wanted. Cloud on the other hand, offers the Exadata, the software as a service as well as the posserbility of bursting CPU's and/ or storage when needed rather than always having them.

In our instance we were not only moving Exadata to cloud but also from OBIEE on premise to the Oracle cloud BICs software as a service. Our first problem was initially we were on 11.7 of OBIEE, a version that you can't just migrate straight to cloud. So after many conversations, we upgraded, and rightly or wrongly went to the newest version avaiable for us to download, OBIEE 12c. This was out preparation work for the cloud.

Now time to get our hands on that shiny new cloud.... well sort of....
Firstly if your migrating data to the cloud, you'll need to decide how your going to transfer the data. In our case we didn't have a VPN installed, so the corporate network stopped us accessing the oracle cloud directly. Next our data wasn't big enough to qualify for the Oracle data service. As all good problem solvers we managd to use the tools at our disposal to get the end result.

We transfered the source data (transactional data) to the on-premise X4, total size 90GB. We then encrypted and used the HCC in low-query Archieve mode to compress the data down to an impressive 9GB. Now we're cooking on gas as they say, the file was now small enough to use the internet (with appropriate security in place) to upload the data file to the cloud. Repeat a number of times and eventually you manage to get the data to the cloud - not ideal but at least in our control.

Once the dat was there, the next problem was moving the RPD and catalog from OBIEE to BICs. Now having done the prep work, geniunly we thought this part would be easy. Oh how I was wrong....

Oracle often maintain a cloud first deelopment structure, see this link for details, however this wasn't strictly true in BICs. It turned out that the OBIEE on-premise version was ahead of the BICs version. Unfortunatly there isn't a way to "downgrade" a OBIEE VM, once you've upgraded your stuck. The RPD there is a was to downgrade. There is a good guide to this here.

Anyway on the 5th June the BICs envorinment was finally upgraded, enabling catalog from on-premise to loud to happen, in our case causing a 4 week delay to the project.

All in all we've managed to migrate to cloud, but it wasn't that easy journey you've heard about. Now we're here, well there are still some snags, but more details of that in my next post.

NLP – Just another acronym?

Data within a business is ever evolving and is being generated as we speak, as we tweet, as we send emails and in various other activities. Approximately 80% of corporate information is available in textual data formats¹

• Structured: highly organised datasets which is easily searchable such as a database/Excel sheet
• Unstructured: data which doesn’t have a pre-defined model and doesn’t fit nicely into a database. Examples include PDF’s, emails, phone conversations, Tweets

Natural Language Processing helps us understand text and have the ability to produce insights from text data. It is a branch of data science that consists of systematic processes for analysing, understanding, and deriving information from the text data in a smart and efficient manner.

Text Pre-processing

Unstructured Data is typically very messy or “noisy” so it needs to be cleaned. Good practice is to follow the below where appropriate:

• Make all text the same case (upper or lowercase)
• Remove numbers
• Remove punctuation
• Remove noise (words not required such as “re” & “fw” in an email context)

The end aim is to be left with only data you’re interested in and removing what you don’t.

Normalization

Normalization is the process of making similar words the same (normalizing) and some useful examples are:

Stemming:The removal of suffixes (“ing”, “ly”, “es”, “s” etc) from a word. 

Removing stop words: removal of common words in the English language: “the”, “is”, “at” etc.). These form the structure of a sentence but not necessarily the context.

Feature Engineering

Term Frequency: 

If a word is identified as appearing many times, it can act as a way of identifying common themes in a body of text. 

Wordcloud:

A visual representation of common words or themes appearing in a body of text.

Text Classification: a technique used to classify a word/sentence into a specified group. This could be applied to something like an Email Spam Filter, as emails fit nicely into one of two groups, either spam or not spam. Models can be trained using training/test data to define what spam is/isn’t. 

The above are just some examples of what could be used in Natural Language Processing but this is by no means an extensive list. 

1. UR-RAHMAN, N. and HARDING, J.A., 2012. Textual data mining for industrial knowledge management and text classification: A business oriented approach. Expert Systems with Applications, 39 (5), pp. 4729 – 4739