bunq, a Dutch digital banking platform, has started describing itself as “the first AI-powered bank in Europe.” However, like most firms that claim to leverage AI, there is more hype than actual value being delivered to consumers. During the past few years, artificial intelligence has become a hot buzzword, but very few people actually understand how AI can be effectively applied to various scenarios or applications.
There are certain use-cases for which AI is just not ready yet, as we will see after reviewing feedback from a Fintech industry professional.
Jason Mikula, a Fintech industry professional, reveals that he put bunq‘s AI chatbot, Finn, to the test. First, he couldn’t help but notice the name, Finn, which is the same as the one Chase used for its “ill-fated” digital-only effort several years back.
Putting that aside, Jason noted via LinkedIn that Bunq helpfully “suggests kinds of questions you can ask, including examples like ‘What’s the Indian place I ate at last week?’, ‘How much did I spend on groceries this month?’, and ‘How do I order a Credit card?'”
Jason says that he “tested variants of all of these, and the results were not impressive.”
He said that he also tried “a couple variations asking how much he had spent on groceries (a category) and tried asking about a specific merchant (Albert Heijn), but Finn’s responses to every variation of this question were incorrect, undercounting the number of transactions and amount spent.”
Curiously, when he asked “how much he had spent at Albert Heijn, the generated text response didn’t even match a list of transactions the app showed below.”
Asking “how do I order a credit card” DID produce a response “with a list of steps to take in the app to do so — but one would still need to manually DO the seven listed steps. Not sure how this is superior to having a fixed FAQ?”
He also reportedly tried asking “What was the name of the burger place I recently ate at?”
He revealed that Finn flagged a transaction “at Shake Shack in late December – not bad! – though there was also a more recent transaction at Burger Bar that it did NOT flag.”
Lastly, he tried asking some higher-level, “more general questions, like “How can I save more money?”, which Finn seemed to freeze when trying to answer, not even forming a complete sentence in response. When I asked “How can I invest my money?”, Finn responded with a list of incoming deposit transactions that had nothing to do with investing.”
To be fair to bunq, they do “label the feature as being in “beta,” though [he is] not sure your average user would know or understand what that means.”
When it comes to relying on AI-powered features, like Finn, it “is imperative that they have near-perfect accuracy.”
After all, Jason noted that if you have to “double check every analysis or piece of advice from your bank chatbot, it’s probably not really saving you very much time.”
bunq’s management noted:
“It’s amazing to see that you’ve been using Finn and doing an extensive review for it. User feedback is paramount to us, but we also want to iterate that Finn is still learning in the beta phase and we continue to welcome our user to share their experience so it can help improving the evolution of Finn. Share your ideas on together.bunq.com or you can simply tap the thumbs up/thumbs down button to rate your experience using Finn and stay tuned for more exciting stuff from us.”