Motivation of Risk Prediction

Customers who do not repay the distributed loan on time represent an enormous risk for a bank: First, because they create an unintended gap in the bank’s planning, and second, because the collection of the repayment amount additionally causes additional time and cost for the bank.

On the other hand, (interest rates for) loans are an important revenue stream for banks. If a person’s loan is rejected, even though they would have met the repayment deadlines, revenue is lost, as well as potential upselling opportunities.

Banks are therefore highly interested in a risk prediction model that accurately predicts the risk of future customers. This is where supervised learning models come into play.

Data Overview

n = 1,000 observations of bank customers

• credit_risk: is the customer a good or bad credit risk?
• age: age in years
• amount: amount asked by applicant
• credit_history: past credit history of applicant at this bank
• duration: duration of the credit in months
• employment_duration: present employment since
• foreign_worker: is applicant foreign worker?
• housing: type of apartment rented, owned, for free / no payment
• installment_rate: installment rate in percentage of disposable income
• job: current job information
• number_credits: number of existing credits at this bank
• other_debtors: other debtors/guarantors present?
• other_installment_plans: other installment plans the applicant is paying
• people_liable: number of people being liable to provide maintenance
• personal_status_sex: combination of sex and personal status of applicant
• present_residence: present residence since
• property: properties that applicant has
• purpose: reason customer is applying for a loan
• savings: savings accounts/bonds at this bank
• status: status/balance of checking account at this bank
• telephone: is there any telephone registered for this customer?

Preprocessing

We first load the data from the rchallenge package (you may need to install it first) and get a brief overview.

# install.packages("rchallenge")
library("rchallenge")
data("german")
skimr::skim(german)

Split Data in Training and Test Data

Your task is to split the german dataset into 70 % training data and 30 % test data by randomly sampling rows. Later, we will use the training data to learn an ML model and use the test data to assess its performance.

# Sample ids for training and test split
set.seed(100L)
train_ids = sample(row.names(german), 0.7*nrow(...))
test_ids = setdiff(..., train_ids)

# Create two datasets based on ids
train_set = german[...,]
test_set = german[...,]

Create a Classification Task

Install and load the mlr3verse package which is a collection of multiple add-on packages in the mlr3 universe (if you fail installing mlr3verse, try to install and load only the mlr3 and mlr3learners packages). Then, create a classification task using the training data as an input and credit_risk as the target variable (with the class label good as the positive class). By defining an mlr3 task, we conceptualize the ML problem we want to solve (here we face a classification task). As we have a classification task here, make sure you properly specify the class that should be used as the positive class (i.e., the class label for which we would like to predict probabilities – here good if you are interested in predicting a probability for the creditworthiness of customers).

library(mlr3verse)
task = as_task_classif(x = ..., target = ..., ... = "good")

Train a Model on the Training Dataset

The created Task contains the data we want to work with. Now that we conceptualized the ML task (i.e., classification) in a Task object, it is time to train our first supervised learning method. We start with a simple classifier: a logistic regression model. During this course, you will, of course, also gain experience with more complex models.

Fit a logistic regression model to the german_credit training task.

logreg = lrn("classif.log_reg")
logreg$train(...)

Inspect the Model

Have a look at the coefficients by using summary(). Name at least two features that have a significant effect on the outcome.

summary(yourmodel$model)

Predict on the Test Dataset

Use the trained model to predict on the hold-out/test dataset.

pred = yourmodel$predict_newdata(...)

Evaluation

What is the classification error on the test data (200 observations)?

pred_logreg$score()

Predicting probabilities instead of labels

Similarly, we can assess the performance of our model using the AUC. However, this requires predicted probabilities instead of predicted labels. Evaluate the model using the AUC. To do so, retrain the model with a learner that returns probabilities.