Everywhere you look, people are talking about the rising delinquencies. Having been around the collection industry for a while, I anticipate the next step will be hiring additional vendors – whether it’s collection agencies, or collection attorneys or repossession forwarders or agents. Due diligence is time-consuming and can be draining – particularly when you spend efforts to evaluate a prospective vendor and then decide against that company. Bringing on a new vendor is a big investment in time and energy, as well as cost. It doesn’t feel productive to say no, I don’t necessarily think that is a good fit, particularly if you are in need to make a decision right away. It’s much easier to just believe the hype and move forward with the checklist.
I can give you a list of reasons people have told me that they don’t bother with checking references:
- “No one would give me a bad reference, if they provided a reference they must be good.”
- “They work for a lot of large companies – they must be ok.”
- “I already know they are doing work for some of my peers.”
- “I have a robust due diligence that includes license checks, security reviews, compliance checks, and onsite reviews – I don’t see the need to spend more time on reference checking.”
Be Careful of the Marketing Hype
As you begin your due diligence process, don’t get caught believing everything you hear from the marketing teams. The sales pitches can be glossy and relatable, and tell a wonderful story – and it’s easy to think you don’t need this step – but there are way too many instances where skipping this important step causes real-life problems. For example, one company hired a collection vendor that boated a robust compliance framework. However, the company had a history of non-compliance with regulatory requirements that would have been brought to light with reference checks. Another company boasted the ability to add staff quickly to provide backup call center support, however, the reference checks on past clients revealed high turnover and inadequate training.
It’s a Piece of the Puzzle
Choosing an agency can be an extremely detailed and comprehensive process. One piece of the puzzle can be supplied by checking the agency’s references. It shouldn’t be the only selection criteria, but can help the credit professional determine where the marketing fluff and the actual facts meet.
Some people assume that agencies will always provide reference names of people who will say nice things about them. Not always true! Many agencies do not bother to check with their clients before providing their names as a reference. Some don’t check whether or not the client is satisfied with their service. At other times, names of references stay on proposal templates for years after the reference client has discontinued using the agency. One of my favorite reference checking story was when I called a reference who said “I can’t believe they gave you my name and number. We dropped them 4 years ago and had to sue them to get our accounts back.”
Bias, Consistency and Interpretation
Agency reference checking can be a very eye‑opening experience. For maximum benefit from the references and the process, the client needs to be aware of many things. First, the reference checker must understand the potential rater errors and biases which need to be avoided. Then, the reference checker must develop a consistent format for reference checking. The format should facilitate fair evaluation of responses from a variety of people and company backgrounds. Finally, the reference checker must be able to interpret and understand the results by matching the reference’s responses to their own company’s circumstances and then determine the value to their organization.
Rater Errors and Biases
The person you call to provide a reference for a company may unconsciously provide an inaccurate reference due to personal bias or lack of knowledge. The credit professional can avoid these problems by being aware of the potential rater errors and by quantifying the process as much as possible.
A review of common rater errors and biases will help the reference checker in the evaluation. Rater errors can include: comparing apples and oranges, valuing comments from less knowledgeable references, length of service bias, halo effect, pitchfork effect, central tendency and loose and tight raters.
Comparing apples and oranges can occur when the agency provides a reference that does not place similar accounts to your company’s accounts. At times, agencies will list very large companies as their references. Some people will assume that if a large company uses the agency, or if many Fortune 500 companies use the agency, they must be okay. This is just not always the case. There are many different reasons companies choose and use agencies. Often, many people will not bother to call and check with the reference if the reference company is well known. They value the idea of the large client using the agency more than checking to see if the agency is even doing a good job for that client. References which are comparable to your type of work, or in your industry, are more appropriate than simply large company references, particularly if the large company reference given is unlike your company. For example, if your company is a large bank, then a large bank reference is appropriate and most comparable. In contrast, if your company is a local cable TV company, then other small businesses or utilities in the area may be your best reference. Certainly balance ranges make a significant difference. If your company places very large balance accounts, look closely at the large balance references. If your company places accounts which are generally $50 to $100, you need to look at similar small balance references for the best inference of how well an agency may work your accounts. Also compare the volume of accounts. If the reference company places ten accounts per month, but your volume is closer to 1,000, understand the difference between the two of you as clients to the agency. Would your volume inundate the agency? In contrast, if you place 10 accounts per month and the volume the reference places is 10,000 accounts per month, will your accounts be a significant part of the agency’s business?
Valuing comments from less knowledgeable references is another potential rater error. Often times when we ask for a person’s opinion we don’t ask their background or why we should be valuing their opinion. Sometimes it’s not easy to determine how knowledgeable the reference is on the subject, either. After all, I don’t suggest you call and ask someone to provide a reference and then say, “Why should I listen to you anyway…” But, you can determine how experienced the reference is with agencies, how they evaluate and rank the agencies they are using and whether their experience is relevant to your situation.
Length of service bias comes into view when an agency is assumed to be performing adequately because of the length of time the relationship has existed. Both the client and the agency may have developed a false sense of security and not respond effectively to new information, new technology and changes in the accounts themselves.
The halo effect is one of the most common rater errors. It is often seen in the performance evaluations of individuals. Generally, the reviewer gives a favorable review to all areas of performance based on good efforts in only a few areas.
The pitchfork effect is the opposite of the halo effect, although not as well known. In this case, the rater dislikes a certain agency or its people, and all the ratings are influenced by that.
Central tendency is a condition where the rater avoids all extremes of high and low ratings. In this case, the rater will rank almost everyone average.
Loose raters generally like to avoid conflict. They prefer not to point out areas of weak performance. The loose rater does a disservice to both himself and the agency.
The tight rater believes that no one should ever get a “10.” This person sets high standards that are generally unobtainable. Seldom are ratings of “outstanding” or “excellent” used. Attempting to compare responses from a loose rater and a tight rater makes the job of evaluating agency references especially difficult.
Developing A Consistent and Fair Reference Checking Format
Start your process by developing a standard format for checking references and evaluating the answers, including a scoring system that will assign value to references most closely aligned to your organization’s needs.
One of the first steps is to determine what questions will be asked of the reference. Asking questions can be more difficult than it appears at first. Not only what subjects should be covered, but how they are approached is important. The ideal reference grid will ask questions so that the answers fit into a planned response pattern. This is not always easy but can be done with a little concentration and effort – prior to the first phone call. For example, instead of open-ended questions where the answers vary widely, ask the reference company to rate the agency above average, average or below average. Or, use a one‑to‑five rating, or a one‑to‑ten scale. For example, instead of “how well does the agency collect for you?” try questions such as “would you rate the agency average, above average or below average in the area of dollars collected for your company? This format can be used for the majority of your questions. This concentrated effort in asking the questions in a comparable format will make the next step of evaluating the responses easier. Imagine trying to weigh the various responses of OK, average, good, fair, all right, about the same, and assign some relative points. By giving the reference a framework for responding, you make your evaluation easier. With a response of three areas, you can assign points for above average, average and below average and allow the reference to make the judgement knowing your criteria. Or, you can ask if it is a strength or weakness, with points assigned if it is a strength. For comparison purposes, ask questions where you can quantify the responses as much as possible.
You’ll want to customize the questions to your situation, but some basic questions might include:
- How long have you used the agency?
- Are you currently using the agency? (I like this one because many times I have found the reference company is not currently using the agency.)
- In your opinion, in evaluating their strengths and weakness, could you please rate the following areas into strength, weakness, average (neither a strong point or a weakness) or not applicable? (Here is where you’ll really need to understand the biases of the rater.) Rate: Collection Results, Responsiveness (to your requests, accounts placed in error, sense of urgency, etc.), Turnover of Personnel, Administrative Details, Complaints, Skiptracing, Suing Debtors, Timeliness of Remittances, Security, Compliance with Regulations, Staying Up-To-Date with New Regulations
- Overall, how would you rate them in comparison with your other agencies? (Be sure to provide how you want the answer for comparison purposes – such as Above Average, Average or Below Average, or a scale from 1 – 5 with identifying which is best rating.
- Would you recommend them?
Then, it’s important to ask the Housekeeping questions ‑ to verify that the information provided is a good match for your company. (And I stress that all of their information will be completely confidential.)
- What type of accounts do you place for collection?
- How long have you used the agency. (I remember being asked for a reference once for an agency that we were still onboarding, for a totally different industry, and wondered why they didn’t have any long-term references that were more applicable.)
- What is the average balance?
- What is your monthly volume?
- At approximately what age do you place accounts?
- Do you consider your accounts difficult to collect?
- Do you provide detailed documentation at placement?
- Can you think of anything else that would be useful to me in my decision-making process?
You don’t have to, but I like my score system to add up to a score of 100. For example, establish the points to be awarded for the reference itself; such as 15 points for a similar reference, 0 points for a non-similar reference company. For length of doing business, maybe 10 points for three years or more, 5 points for less than a year, no points for under six months. For currently using, maybe 5 points. For each category I asked them to rate, I provide a number for strengths, and no points for weaknesses. For housekeeping questions, I also use a point system – rating the type of accounts and similarity to my work.
So, by determining the scoring system prior to the references, it provides you an objective way to evaluate the responses. Scores don’t have to follow mine, but be sure to think through what is important to your organization, and develop a rating system like you might for scoring a call or evaluating success.
Evaluating The Responses
Don’t assume that because the agency does a fine job for another company it would automatically do the same for your company. Compare and contrast their situation with yours. Failure to get a good reference from one check may not necessarily mean that the agency wouldn’t perform well for your company. In fact, just the opposite may be true, depending on the reference. We conducted one research study where respondents names their top performing agencies and their worst performing agencies – you won’t believe it… One third of the agencies were on both lists!
By developing a scoring system for reference checking and assigning point values to the various categories, the credit professional can quantify the different responses for each agency under consideration. Using a quantitative approach assures the credit professional of a comparable method and helps assimilate the data necessary to make that all important decision of which agency to utilize.
Conclusion
By incorporating a structured and thorough reference-checking process into your vendor selection strategy, you set the foundation for a successful and sustainable partnership. Reference checks not only validate a vendor’s claims but also uncover insights that can help you make informed decisions aligned with your organization’s goals. Ultimately, the time and effort spent on due diligence will pay off by minimizing risks, avoiding costly mistakes, and ensuring your chosen vendor is the right fit for your operational, compliance, and reputational needs.
Taking the time to verify a vendor’s claims and performance through structured reference checks is a vital step to ensuring a successful, long-term partnership that meets your organization’s unique needs.
And One Caveat:
Reference checking is an important part of the puzzle of vendor management, but it is also just a point in time check. Be sure you don’t rest on the references, but develop a solid vendor oversight program to make sure the future is as successful as the history!
Note: Resource Management Services, Inc. provides Third Party Auditing/Oversight services for Creditors. For more information, contact me at judy.hammond@resourcemanagement.com or check us out at ResourceManagement.com
Author: Judy Hammond
Judy Hammond is founder and President of Resource Management Services, Inc. The corporation was founded in 1986 and specializes in auditing and consulting, serving the collection and recovery industry. As President of Resource Management Services, Inc., she has more than 35 years of experience with an emphasis on operational reviews for compliance and operational effectiveness of collection operations, both for creditors’ internal collection and recovery operations as well as collection agencies and attorneys. She has worked with top banks and financial institutions, utilities, credit unions and telcoms, (and their vendors) and has conducted many Best Practices projects. She is author of various industry publications: “Comprehensive Agency/Attorney Usage Study,” “Comprehensive Agency/ Attorney Usage Study II” and “Collect More From Collection Agencies”. Her work with creditors who were looking to sell debt for the first time, and subsequent Buyer/Seller research was the foundation for the second corporation, The Debt Marketplace, Inc. She worked with Dennis Hammond as co-founders of the Debt Buyers’ Association, (now RMAi), building the foundations for industry standards, as well as the original code of ethics. She developed and produced two industry conferences, Collection and Recovery Solutions and Debt Connection Symposium & Expo, from their inception in 2002 and 2006, respectively, to 2022. Prior to starting her own company, she worked with two large collection agencies.
Sign Up for the Twice Monthly Newsletter
Just enter your email address at the top orange bar at:
Collection Compliance Experts – “The Power of Expertise: Oversight Perfected”
It’s that easy! Twice a month – we provide blog updates and Resources for the Collection and Industry Professional.
Your email is just for this newsletter. We never sell your information. No fee. Opt-out at any time.
