Skip to main content

The importance of outcome monitoring

08/07/2024

The importance of outcome monitoring

I thought that it would be worthwhile drawing attention to the recent Financial Conduct Authority (FCA) update on the multi-firm review in the insurance sector around outcome monitoring. The firms they assessed were the larger players, but the review recognised the proportionate approach for smaller firms. This may help us frame the July 2024 ‘Board pack’ requirements, recognising that the FCA took a snapshot as at December 2023 of the selected firms board reporting at the time. This is not uncommon, including requesting a series of Board minutes in the lead up to a key milestone.

The end of July 2024 deadlines for 'open' and 'closed' products is rapidly approaching. Consumer Duty Services has been supporting a number of firms with Duty diagnostics and preparing their Board packs.

FCA expectations

The FCA has stated in the review that 'we' need:

“clearly articulated tolerances in place for all key metrics, with formal governance established to ensure that they were set at an appropriate level and scrutinised [at least] annually”.

In the debt management space, this is likely to be relevant to where a customer is no longer suitable for a commercial or free-to-consumer DMP. Fair Value assessments and Product Assurance Frameworks need to have evolved over the last 12 months, which should include trend analysis and any interventions over this period. This may be particularly relevant around the outcomes of financial promotions relative to the original objectives of marketing campaigns and the profile of customers receiving debt advice.

Having embedded key KPIs and KRIs around FG22/5 over a period of time is important, this allows trend analysis on the most important (e.g. attrition, payment performance, engagement rates, levels of vulnerability at the time of onboarding).

Firms must regularly monitor the outcomes retail customers receive to identify whether they are meeting their obligations under the Consumer Duty.

With the general election behind us, the CONC 8 review is now likely to commence in earnest. Firms working with Consumer Duty Servicesneed to anticipate what the FCA approach will be before and after 31 July 2024, with some firms pre-selected to provide their Board packs in August 2024.

The recent Consumer Support RFI responses will be analysed by the FCA and further firms selected for Phase 2 activities.

 

FCA 'deep dive'

“In December 2023 we requested the most recent board and/or committee reporting from 20 larger insurance firms. These included general insurers, life insurers, insurance intermediaries and regulated third-party outsourcers which service insurers.
“We asked firms to show how they monitor, assess, and test the outcomes customers are receiving, along with actions firms had taken after identifying poor outcomes.”

PRIN 2A.9 covers the monitoring of consumer outcomes. Fair value assessments are a key aspect of this along with regular reviews of Product Assurance Frameworks. These need to be a core component of the annual Board pack taking account of key events over the last 12 months, including new policy rules (e.g. PS23/5, PS24/2 and PS23/3).

The recurring issue in debt management and DMP administration is non-engagers that are likely to meet the following criteria – “any group of retail customers is experiencing different outcomes compared to another group of retail customers of the same product”. This is where educational campaigns can be useful around the consequences of non-engagement.


Recordkeeping

Adequate recordkeeping under SYSC 3 and 9 feature strongly in the FCA report. SYSC 9 featured heavily in the previous 2 thematic reviews of the debt management sector ahead of the CONC 8 review and review of the FCA vulnerability guidance (FG21/1).

From prior experience with thematic reviews, this can be painful if customer journeys can’t be readily pulled together in chronological sequence on short notice request. I have been through several thematic reviews with real ‘deep dives’. This can be most acute if the FCA determine a case to be ‘vulnerable’ and they go through all communications and CRM notes/flags from the point that vulnerability is ‘identified’. This is relevant to the transformation planning onto new CRM platforms and continuous improvement programmes.

An early conclusion was that few firms were able to provide clear evidence of where the monitoring of outcomes had directly led to proactive action being taken to improve these outcomes, where necessary. QA frameworks with effective feedback loops are one of the key tools to evidence this. The FCA has focused on niche aspects of processes (e.g. claims handling, vehicle write off settlements) where poor outcomes are more common, including cases being submitted to FOS for adjudication.

Poor practices observed

“While most insurers showed they were monitoring the timeliness of claims and number/percentage of declined claims, we saw little monitoring of whether overall good outcomes were being achieved in relation to claim settlements. We only saw a few examples where firms demonstrated claim settlements being comprehensively monitored, tested, assessed, and understood. This included data such as: analysis of overall settlement values, benchmarking of claim settlements, claims complaints, root cause analysis of declined or withdrawn claims, and internal deep-dive reviews (such as internal audits).
“This aligns with the findings of our review into insurers' valuation of vehicles which found most firms did not collect basic data on motor total loss claims, such as the number, scale, and reasons for increases to initial settlement offers.”

MI

Some firms were able to demonstrate clear thinking about the categories of data they required, such as: service level agreement (SLA) compliance, product performance data, second- and third-line reviews, customer feedback, internal quality assurance, complaints analysis, process completion, product governance findings, employee surveys and deep dive analysis. Firms that had demonstrated such thinking were able to provide more meaningful and comprehensive insights on customer outcomes.

Frameworks developed to test customer outcomes

Some firms increased emphasis on testing customer outcomes by better understanding customer journeys. The FCA saw examples of clear, risk-based approaches to testing customer journeys (including those for different customer types) which encompass the overall processes and impact on a customer’s outcome.

"This was sometimes developed as a new initiative, or sometimes adapted from existing Quality Assurance frameworks. In the good examples we saw, the analysis from such testing was clearly and directly linked to the four Duty outcomes. We saw evidence that a clear testing framework led to good identification of poor outcomes, and such an approach allowed root causes to be identified and rectified."

Vulnerability monitoring

The FCA would expect monitoring to identify whether distinct groups of customers, such as customers with characteristics of vulnerability, get worse outcomes than other customers for the same product (e.g. a DMP).

We have generated a vulnerability survey that we created from the FCA survey that closed in May 2024. Firms can look at some of the priority gaps from this and look at how these are taken to the Board.

Extended training is available as required to plug staff knowledge and confidence gaps.

PayPlan has taken a lead in vulnerability monitoring

Kevin Still is an BSI ISO 22458 associate consultant. Inclusive design is a key factor in establishing the right customer engagement frameworks. We are also pleased to support innovative collaborations in this space like the pilot between PayPlan and MorganAsh and their MARS platform.

 

PayPlan were one of the first financial services to become ISO 22458 accredited and then renew their accreditation.

Andrew Gething , Managing Director of MorganAsh, added:

“Since its launch, MARS has enabled businesses to identify and assess vulnerability in an objective and consistent way. Good data is a cornerstone of managing consumer vulnerability and absolutely critical in meeting the reporting requirements of regulators. Having greater intelligence enables us to deliver a far better and more tailored service to clients, something PayPlan and all our users will absolutely benefit from.”

Products & Services

"We saw firms taking assurance that completion of, or lack of material findings from product reviews or fair value assessments, automatically indicated good outcomes were being achieved. In line with the FCA’s Product Governance Thematic Review (this review is ongoing, an update was issued to relevant manufacturers on 26 February 2024), firms should be wary in taking undue comfort solely from a review having been completed. Overall, firms should ensure that their monitoring enables them to identify any risk that they are not acting to deliver good outcomes for retail customers.”

Price & value

“Some firms also made little or no mention of certain other aspects which determine the price a customer receives, such as commission or charges, or the operational costs in running the product, or key aspects of value, such as the use of overall product benefits. Firms will need to consider the data they require to monitor effectively under this heading; insurers, insurance intermediaries and outsourced service providers will need to consider this in relation to their insurance products.
“Where we did see a wider suite of MI being used to assess this outcome, it sometimes lacked granularity; we did not see evidence of whether value is achieved for different customer types: for example, different age ranges or for customers with characteristics of vulnerability.”

Consumer understanding

“We consistently saw evidence that firms were reviewing existing communications to assess whether they met the requirements of the Duty. We saw some firms that had gone to considerable efforts to redesign their ‘communication review’ processes to assist with ensuring the consumer understanding outcome was delivered.”

More developed approaches were able to show a wide range of sources, including tailored customer surveys and using focus groups. Such insights were considered alongside outcomes testing work, and the FCA saw evidence from firms making changes to processes, communications, or staff training in response to such insights.

In February 2024, Consumer Duty Services sponsored a collaborative 'intelligibility' event involving Amplified Global™, StepChange Debt Charity and DEMSA. This looked at the importance of delivering effective communications that are intelligible to consumers in your target audience. Poor literacy and numeracy skills, including 'English not first language', have been focal points for the FCA in podcasts and webinars through 2023 into 2024. Link to event

 

Last week Plain Numbers launched their The Plain Numbers in Practice report.

 

Consumer support

Some of this will no doubt reinforce some of the findings from the recent Consumer Support survey that closed on 5 June 2024 that many firms were involved with in the debt sector.

“We continue to see substandard service levels across insurance sectors and therefore encourage firms to ensure the customer support targets and SLAs they aspire to are appropriate to ensure their customers are supported.”

On the positive side:

“We saw a firm extensively map all key customer journeys and monitor various service data across them, including timeliness of each engagement and end-to-end transactions. The firm complemented this data with QA findings, customer feedback and complaints, and identified points where service improvements could be made to enhance the customer support offered. The firm also assessed elements of the customer journey for different types of customers to identify specific barriers, in particular for customers with characteristics of vulnerability.”

The FCA were critical on the insurance review around firms relying on Trustpilot type feedback.

Conclusion

Trade bodies like the DEBT MANAGERS STANDARDS ASSOCIATION LIMITED have been communicating with their members around the intensity of the FCA engagement and the demands that this places on firms.

This is also true in the debt resolution sector, including debt buyers and DCAs. By way of example, a number of firms will have completed the FCA Consumer Duty data request by 2 July 2024, only to receive the quarterly cost-of-living data request on 5 July 2024. Others have already been told that they will need to supply their Board pack in August 2024. Phase 2 picks will take place following several of the data requests and surveys in May and June 2024 (e.g. Consumer Support RFI).

There is some regulatory acknowledgement of these demands:

“We acknowledge that you may be receiving a number of requests at this time, but these data requests are separate requests with different purposes. We would emphasise the importance of this request in tracking the impacts of the cost-of-living crisis for consumer credit products in a consistent way across firms. Where possible we will be using data already provided.”

FCA Consumer Duty webinar - One year on - 31/7/2024

Sheldon Mills

The FCA has scheduled a Consumer Duty webinar for 31 July 2024. Sheldon Mills has already published an update on LinkedIn. He outlines the principles behind the Consumer Duty and highlights the important role data plays in testing whether firms are achieving good outcomes for customers.

The webinar will focus on the impact the Duty has had in its first year, including examples of good practice and areas for improvement. They will look at priorities for the year ahead.

Sheldon Mills and Graeme Reynolds of the FCA will be joined by Abby Thomas , Chief Executive and Chief Ombudsman, of the Financial Ombudsman Service .

It will be interesting to hear what the Ombudsman has to say with the impact of various FCA interventions likely to stimulate more complaints in different regulated sectors.

FCA - Consumer Duty: 1 year on

References

https://www.fca.org.uk/publications/multi-firm-reviews/insurance-multi-firm-review-outcomes-monitoring-under-consumer-duty