In the world of data and metrics, the integrity of measurement methods can be easily compromised by various fudge factors, especially when dealing with third-party vendors. These subtle manipulations or adjustments, whether intentional or inadvertent, can significantly skew the results and paint an inaccurate picture of vendor performance. For instance, selecting specific data ranges that favor desired outcomes, omitting outliers that don’t fit the narrative, or tweaking algorithms to produce more favorable numbers can all distort the truth. These fudge factors can creep into reports, creating misleading statistics that impact decision-making and strategy. It’s crucial for organizations to recognize and mitigate these influences to ensure their data-driven insights remain reliable and authentic.
Fair and Objective Challenges
When recovery managers strategize their vendor use, dealing with various levels of agencies and types of accounts, it may be difficult to maintain fair and objective measurement comparisons among vendors. The diverse nature of these relationships means that each vendor might be handling different volumes, account complexities, and recovery scenarios, making it challenging to apply a one-size-fits-all measurement approach. This complexity can lead to discrepancies in performance metrics, as some vendors may excel in high-volume, low-complexity accounts, while others might be more effective with low-volume, high-complexity cases. Additionally, the methods and criteria used to evaluate vendor performance can vary widely, further complicating efforts to ensure a level playing field. Recovery managers must therefore employ a nuanced and flexible approach to performance measurement, ensuring that comparisons account for these variations and truly reflect each vendor’s capabilities and contributions.
Good data is not always available for measuring the performance of third-party agencies, posing a significant challenge for organizations striving for accurate vendor assessment. So many people want to start by looking at a historical perspective, but I like to say, that can be hysterical. Relying too heavily on historical data can often lead to misguided conclusions and strategies. Historical data, while useful, is frequently laden with context-specific nuances that may no longer apply to the current environment. Changes in market conditions, technological advancements, and evolving consumer behaviors can render past performance less relevant to present challenges. Consider the changing marketplace over the last five years. Many collection and recovery managers are using 2019 as a better indicator of performance comparison than current years. It’s essential to learn from the past, but equally important to approach data with a forward-thinking mindset. Balancing historical perspectives with an understanding of how data can be skewed is important to fairly and objectively evaluate vendor performance.
Benchmarking Without A Level Playing Field
All users of collection agencies do not have a level playing field, so the first thought on measurement is to be careful of when and how you compare to others. Many people ask for benchmarks of performance, so they can compare, particularly in a changing environment, but that information can be fraught with difficulties as you really have no idea of what makes up the numbers. Benchmarks can provide a useful reference point, but without understanding the underlying data, their value is limited. For example, an agency reporting high recovery rates might be working with less complex accounts, or they might exclude certain challenging cases from their metrics. Similarly, differences in reporting standards and data collection methods can lead to inconsistencies that skew benchmark data. In a dynamic environment, where market conditions and consumer behavior are constantly evolving, the relevance and accuracy of benchmarks can quickly become outdated. This underscores the importance of digging deeper into the context behind the numbers rather than taking them at face value.
In an ideal world, a recovery manager has access to detailed and expansive reporting of collections and recovery on accounts placed for collection. But, even when a system provides strong measurement capabilities, there are a variety of Fudge Factors that can come into play with the strategic use of a variety of collection vendors.
The Fudge Factors
Generally, I like to think as Fudge as a good thing, but just like there are a variety of types of fudge in the candy store, there are a variety of Fudges that affect the numbers and the ability to fairly and accurately measure vendor performance. Some of the typical Fudge Factors that affect vendor measurement include:
- Placement Date Fudge
- Time Fudge
- Good Job Fudge
- Backlog Fudge
- Recall/Bankruptcy Fudge
- Changing the Risk Pool Fudge
- System Fudge
Placement Date Fudge Factor – One of the easiest ways to confuse the numbers is in the method of assigned or placing accounts with the agencies. The placement methods make a big difference in the statistics. Consistent date of placements is important for measurements to be accurate. Daily placements, when volume permits, allow for a more constant flow of work to the agency. The next most consistent flow would be weekly placements. Credit grantor consistency helps keep statistics more predictable. When placements are sporadic and/or inconsistent, it is more difficult for agencies to consistently staff at maximum levels. Statistics can also be misleading when credit grantors transmit placements on different days with different agencies.
Some credit grantors state they are unable to place daily even though the volume would warrant it, but that they place “as close to weekly” as they can manage, knowing that sometimes the work doesn’t leave the office in a timely manner. This automatically impacts recovery comparisons. Credit grantors that place accounts monthly must hold to the same date of placement, preferably the first of each month, for all agencies to avoid artificial inflation of the collection dollars placed. For example, if placements occur at the end of each month, the uncollected dollars for the first month are artificially inflated and the agencies would not have a recovery opportunity for the first month placement statistics.
Time Fudge Factor – Time is a critical factor. Age of paper has proven to be a significant factor in recoveries. A credit grantor’s priorities establish the time factor, and those priorities are subject to frequent change. Internal management change, internal pressures change, agencies change, results change, etc. The only constant is that change will take place. The best method to be pro-active with change would be with a consistent ongoing plan, maintaining cognizance that the credit grantor’s actions involving the age of the accounts placed affect the recoveries.
Good Job Fudge – An interesting fudge factor that may come into play is the effect of an agency that does a terrific job, or a perceived terrific job, with one type of account. For example, when an agency excels in skip tracing, the credit grantor sends more skip accounts to the agency. Therefore, the “good job” tends to result in more accounts that are harder to collect, i.e., skips, being placed with the agency. This type of action has also been seen by region, when an agency is very good in a “tough” region. Don’t jeopardize an agency’s results or comparisons by placing an unusual number of hard to collect, skip or other such accounts with the “good” agency while sending the routine regular accounts to other agencies, without knowing and applying the information during agency comparisons. Or, if you want to do that for the expected increased dollars collected, just be careful not to directly compare that agency’s results with others that are getting different types of accounts.
Backlog Fudge Factor – Often, credit grantors will have a backlog of accounts due to internal champion/challenger projects, or other types of internal system or manpower problems. Credit grantors should always advise the agencies of pending backlog placements to enable the agencies to prepare, if possible, for the placement peak. Sometimes the accounts are placed over a period of time; however, at times they are placed in a one-month timeframe. Backlog placements may affect the statistical trend as well as the collector’s attitude and response time to the credit grantor’s portfolio. Backlog placement may also lead to a variance in the statistics, which must be noted by the credit grantor as performance comparisons and yearly projections are conducted. It is important to know and maintain documentation when the actions of the credit grantor are affecting the recoveries.
If feasible, backlog placements would better serve both the credit grantor and the agency if the accounts were placed in a separate special project where results and collection statistics were not combined with the regular work.
Recall/Bankruptcy Fudge – Some credit grantors do not count recalled, placed in error, or other such accounts in the placement number monthly summary, yet some credit grantors do count the accounts. What about dispute accounts? What about bankruptcy accounts? What about accounts paid at the time of placement? What about fraud accounts? What about the rare unusual account for $50,000 when the normal average balance is $1,500? Sometimes it is not as easy to be fair and comparable as it would seem. Each agency should know and follow the same action as their credit grantors, which would furnish an accurate and fair result comparison.
Changing the Risk Pool Fudge – When risk management decides to “dig deeper into the risk pool” information is not always relayed to the collection and recovery departments, so accounts that are likely to be more difficult to collect can be added to the same tracking pool of accounts that are historically sent to the agencies. This can look like the agency is not performing, when in fact, it’s not the case. Sometimes an onsite audit or evaluation of work effort is important when looking at the numbers with changing conditions.
System Fudge – Creditors should be very careful when system changes or enhancements are being conducted. Oftentimes system changes, which are not expected to affect the credit grantor’s agency system process, does just that, the system fudge occurs. The credit grantor’s agency management team must be alert to supposedly “transparent” system changes. In addition, when system “gremlin” problems do occur, immediate notification should be given to the agencies, plus the problem, dates involved, etc., should be documented and considered when agency comparisons are conducted.
Offsetting Measurements Challenges
To offset some of the challenges of measurement and fudge factors, more than one measurement method should be used, combined with work activity reviews and a variety of other activities and strategies. Since this is a blog and not a book – more thoughts on various types of measurement methods for different situations can be addressed in a subsequent blog. The most comparable measurement is collections batchtracked to their placement period. But when bringing a new agency onboard, there will be questions as to how to compare against older agencies. There is both a learning curve and a time delay. A recovery diagonal allows the first month the agency had the accounts to be compared against a comparable selection of older agencies. Also, watch carefully, as some agencies are slower out the gate, but catch up in month 2 and 3. The most cost-effective measurement comparison is net dollars. By utilizing the net measurements whereby the agency fees are reduced from the recoveries prior to comparison, the credit grantor obtains a truer cost/recovery picture. To get more detailed information, and updates on this topic and other vendor management, collections and compliance topics – sign up for our twice monthly newsletter.
Closing Thoughts
Navigating the complexities of measuring third-party vendor performance requires a multifaceted approach. It’s not enough to rely solely on data and metrics; a deeper understanding of the context and nuances behind the numbers is essential. Historical data, while informative, must be balanced with real-time insights and forward-looking strategies to remain relevant and effective.
Recovery managers must be proactive in identifying and mitigating fudge factors, ensuring that performance evaluations are fair and accurate. This involves using multiple measurement methods to cross-verify data, maintaining transparency in all reporting processes, and fostering open communication with vendors to gain a clearer picture of their capabilities and challenges.
Furthermore, integrating qualitative assessments such as on-site reviews and direct discussions with vendors can provide valuable context that data alone cannot capture. This holistic approach not only improves the accuracy of performance assessments but also strengthens relationships with vendors, fostering collaboration and mutual understanding.
As market conditions and consumer behaviors continue to change, the ability to adapt and stay vigilant against potential distortions in data will be crucial. By combining robust data analysis with strategic intuition and thorough oversight, organizations can navigate these challenges and achieve superior recovery results. In the end, the goal is to ensure that performance metrics truly reflect the value and effectiveness of third-party vendors, supporting informed decision-making and driving continuous improvement in recovery operations.
Author: Judy Hammond
Judy Hammond is founder and President of Resource Management Services, Inc. The corporation was founded in 1986 and specializes in auditing and consulting, serving the collection and recovery industry. As President of Resource Management Services, Inc., she has more than 35 years of experience with an emphasis on operational reviews for compliance and operational effectiveness of collection operations, both for creditors’ internal collection and recovery operations as well as collection agencies and attorneys. She has worked with top banks and financial institutions, utilities, credit unions and telcoms, (and their vendors) and has conducted many Best Practices projects. She is author of various industry publications: “Comprehensive Agency/Attorney Usage Study,” “Comprehensive Agency/ Attorney Usage Study II” and “Collect More From Collection Agencies”. Her work with creditors who were looking to sell debt for the first time, and subsequent Buyer/Seller research was the foundation for the second corporation, The Debt Marketplace, Inc. She worked with Dennis Hammond as co-founders of the Debt Buyers’ Association, (now RMAi), building the foundations for industry standards, as well as the original code of ethics. She developed and produced two industry conferences, Collection and Recovery Solutions and Debt Connection Symposium & Expo, from their inception in 2002 and 2006, respectively, to 2022. Prior to starting her own company, she worked with two large collection agencies.
Sign Up for the Twice Monthly Newsletter
Just enter your email address at the top orange bar at:
Collection Compliance Experts – “The Power of Expertise: Oversight Perfected”
It’s that easy! Twice a month – we provide blog updates and Resources for the Collection and Industry Professional.
Your email is just for this newsletter. We never sell your information. No fee. Opt-out at any time.
