“Unprecedented” is a term with which we’ve all become quite familiar over the last few months. COVID-19 has changed our society and our economy quite drastically. In predictive analytics “unprecedented” has far reaching implications – simply put it’s difficult to build models when we do not have data that reflects similar trends to what we will expect moving forward.

So, it’s very unlikely that the models that you have deployed – whether it be in originations, account management, IFRS9 or collections are working as expected. In 2 previous blogs here and here we covered how data is changing during the COVID-19 period.

What to do?

The first thing to do would be to conduct a model HealthCheck. Principa’s Analytical ICU will assess the health of your models and triage them into four categories. At the very least you will need to realign your models, but the likelihood is that you may need to fine-tune or rebuild your models completely.Rebuilding your models

If you decide you need to rebuild your models, then you have a challenge at hand.  The diagram below represents a scorecard build time-line. The observation and outcome period are the periods from which we would extract data for the scorecard build. What is evident is that the observation period and outcome period do not coincide with the COVID-19 crisis. This means that the scorecards that you would rebuild may not be suitable for the current economic climate and you are back at square one.There is a solution, though, and that is to adopt machine learning models and to approach the scorecard build with a “short-outcome/strict-performance” methodology.

Short-outcome/ strict-performance

A short-outcome/ strict-performance approach will involve sampling the data from a shorter period of time and use a very strict performance definition (e.g. Any missed payment = ”Bad”).  This approach will allow you to sample from a period that is more representative of the current environment. The time lines below illustrate that a scorecard built in August/September could utilise April/May observations (i.e. the beginning of the COVID-19 period).One of the reasons one uses a long observation period is to take in the full annual cycle. As we are looking at catering for the COVID-19 “cycle” the shorter term is more appropriate. The common good/bad definition (for example “ever 3+” = “Bad”) is better, as it allows for the separation of truly good payers from truly bad payers. The stricter definition means that you’ll pick up technical arrears and a few lazy payers in your bad definition. This may weaken the models slightly, but the gains from being able to model for the COVID-19 period should outweigh the losses from the lesser performance definition. Another challenge with the short-outcome models will be the population size. The modelling approach will be different with the smaller population and we may use coarser classing.

Model longevity and machine learning

Once you deploy these “COVID-19” models, the models should be monitored. Redevelopment will likely need to happen sooner than traditional models. It is therefore suggested that Principa’s Quick-Step machine learning models would be most appropriate here allowing you to leverage off new COVID-19 data every quarter and reducing the cost of a sorecard-build.In our next blog we will be covering Principa’s Quick-Step Machine Learning and why this is a great solution to switch models in-and-out during the COVID-19 recovery period.

To find out how Principa can help you with an Analytics ICU or the building of short-outcome/ strict-performance credit models, contact us on info@principa.co.za.