AbstractWestern Power Plc is a mid-rangeenergy supply company providing gas and electricity to about an estimatedmillion households.The company has faced consumer churn, effecting the profitsof the business.The project aims to produce predictive models of customerattrition for single-product (electricity or gas) customers cancelling and fordual product (electricity and gas) customers cancelling one product or both.Thisproject provides theoretical analysis using scenario based data andpredictions.
Suggesting and highlighting the importance of the data modellingtolls chosen for Western Power PLC. IntroductionOver the last decade the energy market has facedcontinues changes across the country. Until recent changes in policy andownerships, the energy industry concerns were not on retention of customers dueto the monopolies in power, gas and water supplier.
OFGEM the governmentregulator for the electricity and downstream natural gas markets in GreatBritain in a study in December 2017 confirmed there was an estimated 15%increase in the number of consumers switching energy suppliers. With thechanges in the market company’s like Western Power plc, a mid-range energysupply company face increases in competition from the bigger service providersand entry firms. LiteratureReview Prior studiessurrounding customer attrition consistently defined it as a measure ofconsumers within the company leaving or choosing alternative providers. Churnrate can be formulated as : Customer Churn Rate = (Customers beginning ofperiod – Current customers end of period ) / Customers beginning of period Robert C.
Blattberg (2008), highlights the two major types of customer churn as “voluntaryand involuntary. Voluntary leads back to customer satisfaction and competition involuntarybeing more the choice of the company to terminate contract. Data miningtechniques including cluster analysis and decision trees can be used to analyseand predict churn. Hung,Yen, and Wang (2006) whilst studying customerchurn found neural networks gave more reliable results when compared topredictive models like for example decision trees.
However decision trees andlinear regression have been found to be the most common form of predictivemodelling in consumer engagement and retention studies. Further studies foundsolutions to the resolution of customer churn and preventing it. The neuralnetwork model was more effective in identifying the behaviours of consumerswithin the market.
Wei and Chiu (2002) highlighted the importanceof consumer data and used this to estimate the customer attrition. The chosen methodology within the assignment is Cross-industrystandard process for data mining. Crisp DM is defined by co- founder TomKhabaza in CRISP-DM Overview, October2016 as cross-industry standard process for data mining.
It is ahierarchical process model, set out in phases. These phases are broken downbelow: 1. Business Understanding 2. Data Understanding 3.
Data Preparation 4. Modelling 5. Evaluation 6. Deployment Industry review Beingwithin a commodity based industry, Western PLC has to deal with variety ofpolicy changes and agreements whilst remaning competitively attractive. Theindustry pricing is hugely based on supply and demand. When the demand of theproduct increases (usually during the colder periods of the year) the more thecompany is assumed to supply at a higher price.
This leads to a continueschanges in the supply and demand of the commodity. As earlier mentioned theincrease in customer churn is predominantly due to the changes in regulationsand an increase in smaller suppliers within the market. Amongst the big sixWestern Plc would be within this category, attracting consumers seeking a”better deal” Many customer are using price comparison services online to makethe switch. Hypothesis:IN order for us to get the predictive model which best suitsthe companys needs we need to under stand what sor of churn ther company isfacing. Based on the literature review and the industry background the biggest causeis usually income or region. However there can be a variety of test hypothesiswhich can be use din the evaluation and testing phase of the model . Exmaplesinclude : Does income impact the churn? o Regional differences in churn rate?o Whatis causing the consumers to leave?o Whatarea of the business is leading to churn?o Isit a particular type of customer leaving?o Whichcustomers are most at risk of cancelling?o Isthe increase in churn seasonal? CriticalSuccess Factor TheCritical Success Factor (CSF), in this occasion will be a solution to reduce orreverse this trend in consumer attrion. This can be measured against the valueswithin the last three years.
Being that the company is in the early stages ofpredictive analytics this will be used in future to continue improving theanalytics. CustomerSegmentation Within theenergy industry segmentation of the consumer is often difficult. This is due tothe variety of consumer bases available.
If the predictive model was to bebased on gender for example this might not be reflective of the core reasonssurrounding the churn. Historically the industry has used demographic customer segmentationleading on to income based and usage/ needs-based. It is also to be noted and later investigatedwhether or not cyclical changes in churn are experienced . Project Outline For the project I choose ot use crisp dm as my main processto structure the project. Each phase outlines specific deliverables anddocumentation requirements, which will aid in the data mining process forWestern PLC’s Businessunderstanding Businessunderstanding is the core of crisp dm. The phase aims to gain insight into whatthe company’s goals are and how the data mining process can be used withinthese.
It has four main tasks defined: 1. Identifythe Business Goals 2. SituationAssessment 3. DataMining goals 4. ProjectPlan Within the Western PLC project, the business goalshave been set out as producing or suggestion a predictive model for reducingcustomer attrition for single-product (electricity or gas) customers cancellingand for dual product (electricity and gas) customers cancelling one product orboth.
Based on the baseline numbers and data from the last 3 years, this shouldreduce by 10% within a 6 month period. This should fundamentally improve thesuppliers overall revenue and increase in margin. In order to meet the business goals, test andsample data will be used for the decision tree model. Figure 2: Funnel of Businessobjectives, highlights the key areas and data which will be used for modelling.Figure 2:Funnel of Business objectivesDataunderstanding and preparation The dataunderstanding phase of Crisp DM aims to gather, describe, explore and verifydata quality. Western Plc mentioned 5 databases which hold differentinformation.
Figure 3 provides a brief example of the information which wouldbe stored in each data set. There willbe key similarities in the Customer, Billing, and Maintenance databases. Thiscould be an advantage in this instance as it will allow for a comparison inregression modelling especially if we are investigating any linearrelationships between the assumed variables.
If there is churn showing withinthe maintenance area of the business in comparison to the marketing area thedirectors are left with a choice of which area to target. Figure3: Databases Data Sets There are over 30variables in the data sets however the key areas for the project arehighlighted below: Key areas of interestfor Western PLC would be : · Customers – o Consumers demographic location: Clustermodelling can be used to see if there is a better provider within a specificregion or area . This will allow the marketing department to target this areaof business. This could also depend on how rural consumers live and the servicebeing provided. This is also found in the marketing database.o Age: The Age distribution of the data will allowfor cluster modelling.
It was previously assumed older age groups tend to bemore loyal to utility providers. Comparison sites have over the last fewyears changed target market and aprovided ser vices including door to doocomparisons. · Billing o Contract length : Consumers with a longerrunning contract are more likely to stay with Western PLC. o Salary – Full time part time employement,catergorically splitting the incomes can also give us a straight forwardpredictive model. o Payment plan – within the billing database thepayment plan and average billing amount can provide insight into churn. o Usage: Demand and supply analysis proveswith the increase demand for the service the supply will reduce increasing theprice. At this point Western PLC churning consumers can be profiled. Sample WesternPLC data will be sampled to estimated 20% of the consumers at 20,000 recordsselected at random.
This will provide 20,000 Consumers, including churnedbetween January 2015 – 2018. A general summary of the data can be produced in Rwith the coding Summary (mydata) code. Over the last 3 years we will find a mean average and immediately seethe highest churn occurred in Year 2 (quarter 2) Data Preparation The selection of the sample data isthe first step to data gathering and preparation. It allows us to find a correlationbetween consumers who have churned.
This will give us a rough guide of where tostart with the data mining process. Variables evident in the data for Westernpower plc are included within the demographic structure of the consumer baseand salary/ household incomes. Data preparation can be summaraise in figure 4. Figure 4:Data Preparation tasks. Datapreparation takes the biggest percentage of time during the project process. Despiteit taking the longest time it often referred back to during the modelling phaseas it impacts the results and the overall analytics achieved. Data preparationphase covers all most important phases to construct the final data set thatwill be fed into the models from the original selected data set. With WesternPLC we have selected, 20,000 sample data set including churned consumers.
Once the selected data set is identified thiswill need to be cleaned. Cleaning of the data refers back to a key element ofdata mining, the quality of the data. Data quality Data quality investigates thecore foundations of the data used within our model. This is considered one ofthe roots to good analytics and data mining. To ensure the data used is fit forpurpose, in this process we verify the following elements of the data set: · Completeness – Verifying if the majority of the data includes one or morevalues. This is critical for our regional model as the address and locationfilter needs to be completed.
· Uniqueness – ensuring when measured with the other data there is only one ofits kind. This can be an issue when consumers are based in the sameflat/apartment complex. It is important to differentiate consumers.
· Timeliness – Ensuring we have up to date contact details for the marketingteam in Western plc to offer new products and keep consumers. · Validity and Accuracy – human error- · Consistency –patterns Data profilingis also included in the preparation phase. This involves the validity, accuracyand completeness element of data quality. However we then have to decide whatto do if the data is in accurate or missing. Missing data can have significantimpact on predictive modelling and decision tree modelling in software’s likefor example R. Some researchers suggest removal of null values however this isdependent on the size and the amount of data that is indeed null.
We will alsoneed to ensure all duplicates are removed from the data set and how to dealwith the consumers who have moved to new address’s during the 3 year phase. At the endof this phase we should be confident in the data we are testing and using inthe model. Normalisations of the data should be completed and data constructionmay be relevant to in the case of Western Plc deciding if we are going toverify the model based on region, area code, city, town, post code etc. Onceall the data is consistent and arranged accordingly in the different databaseswe can now integrate the data. When youhave the same data across different databases, the opportunity is ripe forerrors and duplicates.
The first step toward successful integration is seeingwhere the data is and then combining that data in a way that’s consistent. Hereit can be extremely worthwhile to invest in proven data quality and accuracytools to help coordinate and sync information across databasesDataexploration stage will be key for us determining relationships between pairs orsmall number of attributes. In the project this will show a direct correlationbetween the area/ postcode to the income within the area. This analysis maydirectly address your data mining goals. They may also contribute to or refinethe data description and quality reports, and feed into the transformation andother data preparation steps needed for further analysis DataModelling In the projectI will be using classification and regression model.
The modelling phase Modelling is the part of the Cross-Industry Standard Processfor Data Mining (CRISP-DM) process model that most data miners like best. Yourdata is already in good shape, and now you can search for useful patterns inyour data. Classification Categorical decisiontree builds classification or regression models in the form of a treestructure.Ina classification problem, you typically have historical data (labelledexamples) and unlabelled examples. Each labelled example consists of multiplepredictor attributes and one target attribute (dependent variable). The valueof the target attribute is a class label. The unlabelled examples consist ofthe predictor attributes only.
The goal of classification is to construct amodel using the historical data that accurately predicts the label (class) ofthe unlabelled examples. RegressionNumerical–we take the categorical area vs the payment plans? Regression modelling aims to estimate and identify a pattern between twovariables. IN this instance my dependent variable is the amount of consumerswho churned.
As mentioned above, regression analysis estimates therelationship between two or more variables. This will be through, investigatingthe relationship between Churn and the different variables. The strongestlinear regression variable will be further investigates through theclassification predictive model. Regression creates predictive models. Thedifference between regression and classification is that regression deals withnumerical/continuous target attributes, whereas classification deals withdiscrete/categorical target attributes. In other words, if the target attributecontains continuous (floating-point) values, a regression technique isrequired.
If the target attribute contains categorical (string or discreteinteger) values, a classification technique is called for.The most common formof regression is linear regression, in which a line that best fits the data iscalculated, that is, the line that minimizes the average distance of all thepoints from the line. Results We findthat the increase in churcn is being caused by the regional difference inincome. The payment plans are met by a few of long term consumers who are inthe more affluent rural areas. With options form competirots consumers in thelower income post codes are faced with the easy transition to a lower pricedservice provide.d The desire results from t hemodelling are identified inFigure 5. This is the aimed results form the test data once the modell has beentested . CustomerChurn Rate = (Customers beginning of period – Currentcustomers end of period ) / Customers beginning of period ) Customer Churn Rate = (15, 000-5000)/15000 = 66% Modelling Results = (10,000-8000)/10000 = 20% Figure 5 :Training data versus test data.
The model is flexible enough to be amended andadjusted to suit the marketing departemnts tareeget area. This can be filteredinto different departments investigating the different variables mentioned inour data exploration elemnt of the project. This is also to match the companysgrowth plan and to ensure it allows for the numbers to increase . Themodels should provide WesternPower Plc its customers, the testing period starts. It normallytakes up to a few months. The model is fine-tuned according to the results.
Such custom-built models have a solid advantage compared to automaticallygenerated models – they stay very flexible and can be developed according toeach company’s growing demands. Evaluation:WesternPower Plc Key concerns and objectiveslaid in the reduciton of churn not just the discorvery of where the retentionis low. In order for the business objectives to be met the model can be used toidentify which variable is causeing either the single or the double churnconsumers to leav. A keyobjective is to determine if there is some important business issue that hasnot been sufficiently consideredFortelecommunication industry, most of the studies have proposed customer churnprediction by using data mining techniques as mentioned in previous section.However, these methodologies have some disadvantages. Heuristic-based approachand analytical methods are inconvenient for complicated problems, and also itis hard to collect pure data for statistical methods. Moreover, a mathematicalmodel is required for simulation, correlated variables are unsuitable fordecision trees, and a clear confidential data set is needed for neural networks(Lee et al.
, 2009). neuralnetworks gave more reliable results Conclusioncampaignswill run for 6 months, and that both campaign response rate and churn rateswill be monitored regularly during this test period.Depending on the results ofthe assessment and the process review, you now decide how to proceed.
Do youfinish this project and move on to deployment, initiate further iterations, orset up new data mining projects? You should also take stock of your remainingresources and budget as this may influence your decisions. List of possible actions Youmight have a data problemManypractitioners that concern themselves with churn focus on a handful of “usualsuspect” reasons for churn: a customer progressed through on-boarding butfailed to fully realize value from the product, the sponsor who purchased theproduct and evangelized it to the rest of the organization left, etc. Somesavvy practitioners look a bit more broadly than that.But thereare myriad things that influence churn that aren’t being recognized, andremedied, by companies who maintain a narrow focus on the most obviousinfluences and the narrow data set that describes them. Did your company putout an ad campaign that angered customers? Perhaps your organization made the news… in a negative way.
To truly understand the reasons for churn, you mustwiden your lens to incorporate all of the data from across the enterprise.It’simportant to keep in mind that your customers’ interaction with your brand isfrequently bigger and broader than what you can find in your CRM or ERP. Totruly understand the reasons for churn, you must widen your lens to incorporateall of the data from across the enterprise.
Data from different departments —HR, Ops, Finance, Marketing, etc. — even public data, may hold the keys to yourchurn problem References R. A.Soeini, K. V. Rodpysh.
2012. Applying Data Mining to Insurance Customer ChurnManagement. IACSIT Hong Kong Conference. 30: 82–92. 3 J.
Hadden. 2008. ACustomer Profiling Methodology for Churn Prediction.
Cranfield University. 4L. Bin, S. Peiji, L. Juan. 2007. Customer Churn Prediction Based on theDecision Tree in Personal Handyphone System Service. International ConferenceService Systems and Service Management.
5 Y. Zhang, M. Berry. 2011.Behavior-Based Telecommunication Churn Prediction with Neural Network Approach.International Symposium on Computer Sciance and Society. pp. 307–310.
6 C.Kirui, L. Hong, W. Cheruiyot, H. Kirui. 2013. Predicting Customer Churn inMobile Telephony Industry Using Probabilistic Classifiers in Data Mining.International Journal of Computer Science.
10(2): 165–172. 7 V. Effendy.2014. Handling Imbalanced Data in Customer Churn Prediction Using CombinedSampling and Weighted Random Forest. Information and Communication Technology(ICoICT), 2014 2nd International Conference.
pp. 325–330. http://www.sciencedirect.com/science/article/pii/S0957417410014089 Customer RelationshipManagement: The Foundation of Contemporary Marketing .
..By Roger J. Baran, Robert J. Galka Handbook of Marketing andFinanceedited by Shankar Ganesan Fundamentals of MachineLearning for Predictive Data Analytics: Algorithms ..
.ByJohn D. Kelleher, Brian Mac Namee, Aoife D’Arcy https://www.linkedin.com/pulse/3-things-customer-churn-surprise-most-people-radhika-subramanianAbbott,D., Applied Predictive Analytics, Wiley, 2004 Brown, M., Data Mining forDummies, Wiley, 2014. Helberg, C, Data Mining with Confidence, SPSS 2002, 2ndedition.
Linoff, G.S. &Berry M. J.
, Data Mining Techniques: For Marketing,Sales, and Customer Relationship Management, Wiley 2011, (3rd edition) McCue,C., Data Mining and Predictive Analysis: Intelligence Gathering and CrimeAnalysis, Elsevier Butterworth-Heinemann, 2007. Siegel, E., PredictiveAnalytics, Wiley, 2014. Taylor, J., Decision Management Systems: A practicalGuide to using Business Rules and Predictive Analytics, IBM Press 2012. Brownch 1&2: chatty intro, gives a realistic picture Abbott ch 1: a bit deeper,good intro, from a statistical background Linoff & Berry ch 1&2:Standard text, a lot more detail Helberg ch 1: Historical perspective,statistical background, short, rare Siegel ch 1&2: Good light intro, whatinformed business person should know..
. Taylor ch 1&2: Different angle,based on decision management BociJ,P., Greasley, A. & Hickie, S., Business Information Systems, 4th edition,Prentice Hall, 2008.
Especially sections on CSF analysis, pp 324-325 &522-523. UoB classmark: 658.403 8 BOC Campbell, D., Stonehouse, G. &Houston, B.
, Business Strategy: An Introduction, 1999. UoB classmark: 658.4012CAM, Aldrich Good introduction in early chapters.
Ference, T. & Thurman,P. W., MBA Fundamentals: Strategy, Kaplan, 2009. UoB classmark 658.401 2 FERGrant, Robert M., Contemporary Strategy Analysis, 9th edition, Wiley, 2016.
CRISP-DMOverview Tom Khabaza, October 2016 https://studentcentral.brighton.ac.uk/bbcswebdav/pid-3043130-dt-content-rid-5645643_1/courses/MM701_2017/CRISP-DM%20Overview%2001%20161014.pdf SEMMA Applied in Industry – Slides https://studentcentral.brighton.ac.uk/bbcswebdav/pid-3051212-dt-content-rid-5665024_1/courses/MM701_2017/SEMMA%20Applied%20in%20Industry.pdfQand A email: [email protected] http://www.independent.co.uk/news/business/news/energy-supplier-switch-customers-2017-record-numbers-trade-association-a8151646.html