For this edition of ‘Into the Business,’ Estimator travelled to Amsterdam to interview Xinzheng Huang, member of the market risk model validation team at ING.
The market risk model validation team is responsible for validating a wide range of models:
Value-at-Risk models for interest rate, foreign exchange, equity, commodity and credit products, such as forwards, futures, swaps and options.
Default and credit rating migration model for bonds and credit default swaps.
Counterparty credit risk and initial margin models for financial derivatives
Pricing, hedging and prepayment models for mortgages
Valuation and replication models for savings/current accounts
Other models for operational risk, business risk, liquidity risk, etc.
Introduce yourself. Who are you? What did you study?
I am originally from China. I did my BSC in Statistics in China, and in 2003 I came to Delft to study Applied Mathematics. I finished my PhD in 2009, at the height of the financial crisis, and then I started working at Fortis Bank Nederland, which was just bought by the Dutch state and later merged with the Dutch part of ABN AMRO. In 2011 I moved to Rabobank, where I worked for 2 years. And now I have been working for ING, doing market risk model validation, for the past 6 years. So I basically passed all the big banks in the Netherlands!
So what makes ING different than the other banks?
Well, I mainly focus on the content of my job, so to me, there isn’t a big difference between the banks, I always focus on the models, either on model development or validation. ING is a nice place to work being very international!
The team is very international you say, where do all these people come from?
We have a team with about 20 people, coming from all over the world like Belgium, China, Colombia, Hungary, Italy, Russia, Slovakia, Turkey…
Can you tell us a bit more about what you do? What is model validation?
So let me describe what I do, starting at the basics. Banks are exposed to three types of risk: credit risk, market risk, and operational risk. To cover the potential losses, coming from these risks, banks are required to hold capital. The capital ratio is probably one of the most important numbers regulators or investors will look at, to assess how healthy a bank is. To calculate the required capital, a bank will develop models. And in big banks, there are typically two types of teams, teams developing these models, and teams validating these models.
In essence, models are only simplistic representations of reality, and they will never be perfect. You always have risk associated with the models, being a model risk. And model risk can be very important to a bank.
Let me give you two examples:
The Gaussian copula model, which was widely used to price collateralized debt obligations, is blamed to be the formula that “killed the Wall Street” and “devastated the global economy”.
In the London Whale case where JP Morgan lost 6.2 billion dollars, the bank’s risk evaluation formula underestimated risk by half because of a spreadsheet error.
So, you see, model risk is very important, and model validation plays an important role to mitigate that risk. Model validation has three key objectives. 1) We need to make sure the model is reliable, compliant with policies and regulations, and that it is valid for the intended use. 2) We need to find and fully understand the limitations and weaknesses of the model. 3) We need to contribute to the ongoing model development by providing recommendations for improvement.
You say you studied applied mathematics, is it similar to econometrics?
What I studied was mostly probability theory and stochastics, and I still use my knowledge daily, and I also use a lot of econometrics knowledge. The two are very much connected! Time series analysis is very important for my job for example, and there are quite some econometricians in my team, as well as mathematicians and statisticians.
What is an important skill to have when you work in a model validation team?
You need to have good theoretical knowledge of statistics, probability theory, and you also need very good programming skills. In our team, you have the opportunity to learn and evaluate a wide range of models, on a variety of topics. A (validation) project takes around 3-4 months, and then you move on to another project, so you need to understand the topic and the model inside out in a short time period, and that can be very intensive. But this is also very fun as you learn a lot about the financial world!
Is there anything that surprised you when you started working at ING? Or when you started working in general?
When you are a student, you study a lot of complex models and theory. However, what I see is that banks typically use rather simple models. If you use a GARCH(1,1) model in a bank, it is already considered as a complex model, whereas for a student it might seem rather basic. But this is because banks deal with other problems. For example, you need to model over 50 currencies. Although it might be very easy to model one currency very well, to model all 50 at the same time is way more difficult due to the dependence among the currencies and the explosion of the number of parameters. The type of problems you solve in real life can be different from what you study at university.
Do you think it’s a shame that banks use such simple models, or do you think it’s necessary?
Personally, I like simple models, if they work of course. If you introduce more parameters, making the model more complex, you will always improve the fit (R squared) clearly. However, you will need to be able to understand the complex models. It’s important to strike a balance between usability and complexity.
Do you often have a case, a model, where you think a lot can be done to improve that model?
There will always be something that can be done better. For me this is also the fun part of model validation, you can really see you contribute something to the model. It makes my work more fulfilling.
So you discussed a fun aspect of model validation, but can you give an example of something you do not like about model validation?
Let’s say you have an opinion about a model, and how it can be improved. But the model developers might not agree. because it may lead to a significant amount of work for the model developers, and it can have consequences to the business. You need to convince them that implementing our recommendation improves the model and is necessary. So, besides a lot of technical skills, we also need strong persuasive skills. Sometimes it can be a big challenge to convince them! And I’m still working on improving my persuasiveness.
How does a project for you look like?
We start our validation by asking for relevant model documentation. Model developers send documents for us to study. Many developers are here in the Netherlands so we can meet them to discuss some aspects of the model more efficiently, but for some projects, we must rely on conference calls and emails as there are developers in eg., UK, US, Italy, Spain, Poland, Australia.
We spend typically 3-4 months on a project, analyzing the assumptions underlying the model, checking the correctness of the implementation, assessing the model performance, etc. We work in pairs, so you always have someone to talk to, especially when you are not sure about something. This is also a good way of working for juniors to learn from more senior colleagues.
The outcome of our validation is a report, where we issue some recommendations to improve the model. This report is discussed and agreed with the model developers and then approved at a committee.
How does a typical day look like for you, or a model validator?
Well, most of the time you just sit at your desk and work on your project, eg, writing the validation report, performing data analysis, etc. From time to time you talk to your co-validator on your project. Some days you have meetings with the developers to discuss problems or solutions, other days you have meetings with the team.
Each quarter we organize a make-it-happen week, we leave our daily work aside and work on innovation and automation ideas to improve our way of working. For example, one colleague built a python tool that everyone in the team can easily retrieve a large amount of market data with just a few clicks on an internal webpage.
Does it ever happen that you do not approve a model?
It certainly can happen and it did happen a few times in the past. In this case, we need to resolve any dispute with the model developers to agree that the models are not adequate. Subsequently, the model must be redeveloped.
Do you always have a new project ready to start?
Yes, in reality, we have a shortage of resources, there are too many projects that need to be done! We are constantly looking for people to join our team! Right now we are actively searching for people with knowledge in financial econometrics. People are hired from abroad because we cannot find enough people here in the Netherlands. We have opportunities for full-time positions, work-along internships, and also MSc thesis internship (if you are interested, check out ING’s career website, or connect on LinkedIn).
Which language would you suggest?
Right now python is what we mostly use, together with Matlab or R. While we do get codes from the developers, when possible we have to write codes ourselves. There are also standard tests we have to perform, and those are done in Python.
Do you have any advice for people who are reading the article or want to work in ING?
Study hard of course, and program more! Programming is so important in this field, and it is becoming important in all fields.