Transfer learning for financial decisions

Banks and lenders live and die by how well they judge risk. Every new applicant, every existing customer, every change in the market is a fresh question — how likely is this to go wrong? — and the answer has to be both accurate and continuous, evolving with the customer and the institution. Machine learning has become central to answering it, but it carries a hidden cost: every time the question shifts, the model that answers it has to be retrained from scratch, a process that is slow, computationally expensive, and throws away everything the institution already learned. This paper asks whether a model can instead carry knowledge forward — reusing what it learned on one financial problem to learn the next one faster.

The problem

Artificial neural networks learn to solve new problems through a process that is, in the authors’ words, computationally intense and time-consuming. They are also among the best-performing classifiers of financial data: financial institutions have tested them against statistically sound methods and found them one of the most robust ways to classify financial data at scale. The trouble is that they normally start from nothing each time. A model trained to score German credit applicants knows nothing about Australian credit applicants, even though both tasks are, at heart, the same kind of judgement.

The obvious fix is transfer learning — taking the knowledge a model acquired on a source task and reusing it on a target task. But there is a catch that has limited transfer learning in practice. Most transfer methods assume the source and target tasks are related; when they are not, transfer can backfire, a failure mode known as negative transfer that actively damages the model’s ability to learn. In a real institution the valuable knowledge is scattered across departments and even across institutions, and those tasks are often not obviously similar. The open question this paper tackles is precisely that one: how do you transfer knowledge across different, sometimes unrelated, financial tasks — and reap a benefit instead of paying a penalty?

Figure 1. Transfer learning: reuse knowledge instead of starting from scratch.

What we did

The approach borrows its central intuition from an unexpected place: behavioural genetics, the study of how traits are inherited across individuals in a population. The paper proposes a novel selective-breeding technique that extends the behavioural-genetics transfer-learning framework of Kohli, Magoulas and Thomas (2013). The idea is to treat each neural network as an individual in a population and let genetic algorithms — artificial intelligence’s search heuristic that mirrors natural selection — evolve that population toward networks that are good not only at the current task but at transferring what they know to new ones.

Two behavioural-genetics ideas do the work. The first is twin studies: each population is split between monozygotic twins (genetically identical — in network terms, identical hyper-parameters) and dizygotic twins (fraternal — 50% shared hyper-parameters), all with random initial weights. Mixing identical and fraternal twins keeps enough variation in the population to learn multiple unrelated tasks. The second is the environment: a filter is applied to each individual’s training data to simulate socio-economic status — drawn from research showing children with fewer learning resources perform worse — by randomly removing between 0% and 40% of the training examples for each network.

The genuinely new ingredient is the selective-breeding step. Rather than breeding only the top performers — which would produce a population obsessed with optimal accuracy on one task and useless at transfer — the method deliberately mates the best performers (selected for accuracy) with medium-level performers (selected to keep the population flexible). That synergy between optimal results and transfer flexibility is what alleviates negative transfer. Standard genetic-algorithm machinery handles the rest: single-point crossover and a 0.1% mutation rate. As a useful by-product, the framework also assesses task similarity, measured by the mean difference between the weight spaces of the best networks trained on each task — the smaller the difference, the more related the tasks.

The experiments used three real financial datasets from the UC Irvine Machine Learning Repository, two of which are industry standards for benchmarking classifiers on financial data. The Australian set (Statlog Australian Credit Approval) holds 690 credit-card applications with 14 mixed attributes; the German set (Statlog German Credit Data) holds 1,000 credit records with 24 numerical attributes; and the Banknote set holds 1,372 cases of genuine-versus-forged banknotes extracted from real images. All three are binary classification tasks, and all were normalised to a common feature dimensionality and split 60/20/20 into training, validation and testing.

Figure 2. Selective breeding of neural networks, inspired by behavioural genetics.

What we found

The experiments evolved populations of 1,200 neural networks, each trained for 1,000 epochs, over 20 generations, using overall misclassification error on the test set as the fitness criterion. For each of the three financial tasks the method produced a population optimised for that task as the source, and each optimised population was then trained and tested on all three tasks as targets.

Two results stand out. First, the approach achieved positive transfer across the diverse financial tasks — knowledge built up on one problem carried over to help the others, with no abrupt collapse in performance when moving from a source task to a different target. On the test data this translated to roughly 87% accuracy on Australian credit, around 75% on German credit, and about 98% on Banknote authentication (classification errors near 13%, 25% and 2% respectively), holding steady regardless of which task was used as the source. Second, and most directly, the paper compares this against a population of 1,200 randomly initialised networks — the same models without the evolved, transferred knowledge. The random baseline made far more mistakes, with test errors of 37.5% on Australian, 28.7% on German, and 3.01% on Banknote. In other words, the transfer approach was considerably more accurate than starting cold, and the gap was largest on the hardest task.

The authors are careful and explicit about what this is not. Compared with specialist methods in the literature, this is not the best-performing approach on any single dataset — and that is by design. Those specialist methods were each built to solve one of these problems in isolation; this framework instead solves all three together and shares knowledge between the solutions, prioritising clean positive transfer over squeezing out the last point of accuracy on a single task. The paper presents its results as credible numerical evidence for the new approach while naming clear room to improve — more sophisticated breeding techniques, and building ensembles from the evolved populations — as future work.

Figure 3. What transfer bought on the financial tasks.

Why it matters

The practical promise is faster, knowledge-reusing models for financial risk and decisioning. A lender does not face one frozen problem; it faces a stream of related-but-different questions across products, customer segments and time, and the knowledge to answer them is scattered across departments and institutions. A framework that can carry learning from one such task to the next — without the usual penalty when the tasks are not obviously alike — points toward decisioning systems that adapt quickly instead of being rebuilt from scratch, and that make use of an institution’s accumulated knowledge rather than discarding it.

There is a broader idea here too, one that runs through stm.ai’s FinTech and market-intelligence work: AI that is transferable and evidence-linked rather than brittle and single-purpose. The same selective-breeding machinery that also measures how related two tasks are is, in effect, a model that knows something about when its knowledge should and should not be reused — a useful kind of self-awareness in any high-stakes decision system. It should be read for what it is: a 2015 method paper, demonstrated on three public benchmark datasets, presenting promising evidence rather than a deployed product. But the direction it points — models that accumulate and responsibly transfer knowledge across the many related decisions a financial institution actually makes — is exactly the kind of practical, well-grounded AI that financial decisioning needs.

C. Stamate, G.D. Magoulas, M.S.C. Thomas — “Transfer learning approach for financial applications”, arXiv:1509.02807 (2015). Read the paper.