October 13, 2020

Line Dempsey: Welcome to our podcast, Regulation Matters: a CLEAR conversation. Once again, I’m your host, Line Dempsey. I am currently the chief compliance officer with Riccobene Associates Family Dentistry here in North Carolina, and I serve as the chair of the National Certified Investigator Training Committee with CLEAR.

To those that are maybe not familiar with CLEAR, CLEAR is the Council on Licensure, Enforcement and Regulation, and it's an association of individuals, agencies and organizations that comprise the international community of professional and occupational regulation. Our podcast is just an opportunity for you to hear about the latest and greatest in our community. And today, I'm joined by a friend of mine I've known for quite a number of years, Peter Mackey with Peterbridge Communications LLC. I'm glad to have you with us today, Peter.

Peter Mackey: Thanks, Line. I'm glad to be here.

Line: Absolutely, and thanks for our listeners for joining us as well. Today, what we'd like to talk about is an article that you wrote for the latest issue of CLEAR Exam Review, and that article is “Conflicts of Interest in Credentialing,” specifically conflicts of interest that subject matter experts might encounter when serving in exam-related roles for credentialing organizations. So I guess to start off our conversation, how did you get interested in this topic, Peter?

Peter: Well, it has a long history. The early part of my career was in Washington DC, which is a place where conflicts of interests is a way of life.

Line: (laughter) For sure!

Peter: Some of my responsibilities there were involving procurements, and so we had to be very sensitive to whether the bidders had conflicts that we needed to be aware of. More recently in the latter part of my career working for a credentialing organization, CFA Institute, and actually before that, I served as a subject matter expert on a lot of different panels and committees. And as a consultant and as a staff member, I worked with subject matter experts in many different functional areas, and I was constantly surprised about the level of, I call it, unconsciousness or unawareness with regard to conflicts that a subject matter expert might have and not be sensitive to.

Line: Certainly, just not aware. So I guess looking at your paper itself, what were the objectives? I guess, what were your objectives in writing it?

Peter: So the primary objective was just to... I figured, and I actually knew from networking with other credentialing staff, that my experience was fairly common, so my hope was to raise awareness and share ideas for managing conflicts among subject matter experts. I didn't see a lot of treatment out there in other articles or papers.

Line: So for our listeners that maybe aren't CLEAR members or haven't had a chance to read the article, I guess, can you give us a little summary of what the paper is about?

Peter: Sure, sure. It's essentially about my experiences working with volunteer or consultant subject matter experts on various exam-related activities. The most sensitive activities would be those directly involving exam content--item writing and reviewing and standard setting--because of the stakes involved. But I think the issue is also relevant for subject matter experts working on practice analysis endeavors and curriculum and learning design and materials development. I think all these present an opportunity for people to have potential conflicts, so that's what it's about. And again, it's somewhat informed by my discussions with colleagues from other credentials about issues that they experienced.

Line: Right, so I guess maybe to start things off, how do you define a conflict of interest?

Peter: So again, my experience is with the CFA credential, Chartered Financial Analyst. And most of our members are money managers or analysts of investments that advise other people about investments. And so we have a very strong code of ethics and standards of practice that define your behavior in practice, not necessarily geared towards subject matter experts in exam-related activities, but I think they have a lot of relevance to those activities. In the Code of Ethics for the CFA Institute for Chartered Financial Analyst, they define conflict as “any situation in which a person has a personal or an outside interest substantial enough to affect the person's judgment or interfere in any way with their duty to act in the best interest of the organization or the assigned activity or mission.” And that's not necessarily the definition everyone would adopt, but I think it's fairly close to many other variations on that definition.

And I guess I would just add in defining it, there's two things that are very important, I think, or to be clear about when we talk about conflicts of interest, and one is simply that all of us have conflicts. If you have a job, if you have professional colleagues, friends, relatives, relationships, membership organizations you belong to, affiliation with groups, you're going to have some conflict. The second most important thing is about understanding that the existence of a conflict doesn't imply any wrong doing, simply the potential for people behaving in a way or in ways that might harm one of the interests that you're associated with.

Line: Right, and sometimes even just the recognizing that there may be a conflict, I guess, can probably influence people along that as well. So, you mentioned subject matter experts. I guess, is there anything unique about subject matter expert activities and conflict?

Peter: I would say that... Well, first, I want to say a couple of words about subject matter experts. The vast majority, and we can for this purpose consider all of them, are well motivated generally to do something "to give back" (that’s an expression we often hear) to the profession. And secondly, it would be very rare to find a subject matter expert involved in some of these volunteer or paid activities for the exam that are going to do anything purposely to harm the profession or the integrity of the credential. But I think it's important again to go back to the prevalence of conflicts for almost all subject matter experts. Their activity for the credential is not their main allegiance. They almost always have a job, hopefully, other professional relationships and affiliations, so I think that's an important distinction for the subject matter expert role.

Just a couple of quick illustrations in case I haven't been clear up to this point. So to illustrate a conflict, every credential holder, everyone who's achieved in our case the Chartered Financial Analyst designation, they have almost an inherent conflict to restrict entry to the profession. That's not to say that most of us behave that way or even think that way, but it's a potential issue. And I call that the guild mentality—pull up the ladder so no one else can compete with you in your profession—and it does happen. You do see reports of this kind of behavior in some professions occasionally. It's not very prevalent, but I think that's just an illustration of how we all can have conflicts. And then on a flip side of that, so if I am interested in making the credential harder to achieve, I'm going to do things like write harder exam questions, contribute or try to influence a cut score to the exam that's higher than maybe it reasonably should be or fairly should be.

Then you get to the flip side of that, of course. And we all have personal relationships. When a credential is very successful like the CFA or in many of the licensing programs that CLEAR deals with, you're going to know people in that profession, you're gonna meet people, and you're often gonna meet people who are taking the exams, who are trying to achieve the credential. So my daughter could be taking the exam this year, or my best friend's son could be taking the exam this year. So those are some very general examples of why this could be an issue.

Line: Well, I know when investigators get around a table and chat, we like to talk about some of the great experiences that we've had. I guess, from your perspective, what has been one of the worst situations that you've encountered with conflicted subject matter experts?

Peter: Sure, let me say... Let me just mention some of the roles that might be of issue. In most cases, credentials have training programs, or in some cases, it's even getting the university degree, for example to become a nurse, to prepare for the exam. So you could have anybody affiliated with a prep course or a professor teaching the course, a tutor. People could be involved in multiple credential organizations, some of which might be considered competitors. And then back to the personal relationship, you could have a boss or a direct report who's taking the exam, work colleagues, family members, friends.

And then specifically, to answer your question, in our case, the conflict situations we've found that we didn't catch ahead of time were people involved with grading exams and then also doing exam-related work for another credential, or an item writer who's also doing some work for another credential. A grader that was teaching a prep course on our credential. But probably the worst case that I've personally experienced was discovering that an individual participating in the standard setting workshop to set the cut score for our exams was actually an employee of a prep course that had not disclosed that to us.

And all these situations I described, I think we managed to avoid serious harm, but it would have been much better had these conflicts been identified before the individuals became involved in our activities. And I think I'd add one other comment that I think leads to these situations where you discover too late that the person had a conflict, and that's the mentality I think that all of us come into these activities with. We're good people; we consider ourselves good people; all the activities I'm involved with are good; I'd never do anything to hurt the credential. And then secondly, if you talk to people about conflicts, you often find they're pretty defensive about it. They think that you're accusing them of doing something wrong because they have this conflict of interest, or that you don't trust them because you're talking to them about some other relationship they might have. And sometimes the tendency, and I think this was the case in all of the examples that we've experienced, the individuals have a tendency to just rationalize their activities and convince themselves that it's okay. How could anything be wrong with these two activities? And that leads them to conclude that they don't need to disclose what else they're doing. And that's where we've gotten into trouble, where there's a non-disclosure and then later we discover there was something that should have been disclosed but the individual didn't think it was important.

Line: Right. Well, what are the risks that you're hoping to address with conflict management?

Peter: So at a very high level, I think that all of the risks involve potential harm to the integrity of the program, the integrity of the exam and the reputation of the organization. The worst case risk would involve a purposeful or even an inadvertent sharing of exam content, which might lead to some candidates having an unfair advantage on the test. Another risk is that people come into your standard setting workshop to help set the cut score, and they've got a strong bias based on their other relationships or activities to either argue for a very high or very low cut score. Of course, in all these activities you're looking for experts that are going to exercise independent and objective judgment and try at least to be aware of biases so you can manage them. And even in activities that might be considered less risky or less prone to influence by a conflict, for example practice analysis participants or people involved in determining the critical concepts and topics competencies that are important to your profession in your credential. An example in our credential: What if my specialty is managing futures to help people manage their risk and their investment portfolios? Well, I might like to see a whole study session or certainly an extensive reading on my particular area of expertise. So it could be more subtle than just a blatant financial conflict.

Line: Well, are there any guidelines out there? It seems like there might be some specific to specific organizations, but are there any guidelines out there for those that are involved in testing, licensing and credentialing, in that industry itself?

Peter: Yes, Line, there are some. In writing the paper and researching the paper, I wanted to answer that question. The publication called The Standards for Educational and Psychological Testing is what we in the testing business consider the bible, and rightfully so. But when they talk about subject matter experts, mostly what they're emphasizing is to make sure that your subject matter experts are competent. And you have some mentions here and there of potential conflicts. In my view, (and with all due respect to the bible, I think it deserves its reputation) but some of the treatment is a bit subtle or may be lacking in details. So they, for example, say that people "should have an awareness of personal and contextual factors that may influence the testing process." That could easily, I think, go unnoticed. Another sentence they have is "all professionals in the testing process should follow the ethical guidelines of their profession." Again, very, very general. The NCCA Standards of accreditation of certification programs has some very specific requirements and guidelines that I think are worth mentioning. They have cautions against conflicts or potential conflicts in members of governing boards, and they also, in Standard 10 and Standard 11 of the standards, they have very specific discussions about preventing and managing conflicts with all individuals involved in the testing process. I think that document is well worth reading and specifically Standard 10 and 11.

Line: Well, I guess as maybe a parting question, what recommendations do you have to improve conflict management when you're dealing with SMES, with subject matter experts?

Peter: As a general suggestion, I would say just focus on promoting awareness. Again, I think that's kinda the genesis of my interest in this topic, is the lack of awareness that people might have conflicts. Second is a very strong focus on disclosure, requiring or strongly encouraging subject matter experts to tell us all they can about the other activities they are involved with and the other relationships they might have. And finally, taking the time to evaluate the information that you receive to determine the nature of the conflict and whether there's likelihood of an issue or a breach and what are the consequences if there is an issue. Specifically in the paper, towards the end, I make some more specific suggestions and one is, again, I would recommend that, in your profession, if you have a code of ethics for that profession, that's a great source to leverage. The subject matter experts would be aware of those guidelines or those rules, so you don't have to start from ground zero. You can start with those rules, and many of them, and certainly in our case, have a lot of relevance to how you behave as a subject matter expert.

I certainly would strongly encourage using written agreements or contracts with your volunteers or your consultants, subject matter experts. That's where you can specifically lay out your concerns for what conflicts might exist and your disclosure requirements, and your expected behaviors, what information is sensitive that should never be discussed outside the meeting room.

I also think when it comes to making judgments about whether this conflict is manageable or whether it should make the subject matter expert ineligible to participate, particularly where there's any doubt, I think it's very helpful to enlist other experts outside your area—maybe it will be legal counsel, maybe it would be ethics staff if you have one—to get their viewpoint on whether this is a conflict and how serious it might be.

And then it comes down to, I think, many forms of communication. Communicate to the subject experts frequently. Remind them of their obligations. Give them examples of past instances where a conflict caused a problem. Engage your veteran subject matter experts to talk about their experiences and how they avoid or manage conflict situations. And then you just can't communicate enough about this topic, I think. It doesn't mean you have to take over meetings to talk about it, but a little mention here and there every other meeting reminding people about these issues, I think, is very helpful.

Line: That's great advice. I think, again, you were point on talking about how just because there's a conflict, that just doesn't necessarily mean that they can't do their job. But I guess looking at it from an outside perspective of what is the perceived conflict. How would someone else from the outside looking in potentially react to someone's conflict or a potential for a conflict? So that's great information.

Peter: Yeah, I think that's a great point, Line. It's not only what could happen, but it's how does the rest of the world and particularly other stakeholders in your profession, how they might look at that situation. And in the end, of course, it's a balancing act. We need subject matter experts to produce our content, and we need to recognize that conflicts exist. So how do we get there? How do we manage those issues to a successful conclusion? And in our case, and I think in most credentials that I've been associated with, you really wanna achieve a culture and an environment of trust, mutual trust, so you certainly don't want people to always think they're under suspicion. But I think it's not too hard to get to that place if you just collegially promote awareness and disclosures and discussions.

Line: Well, as you said, communication is the key. Well, I think this has been a great discussion. So Peter, I wanna thank you for your time and being a part of CLEAR’s podcast. It is always great to have the chance to talk about these issues and learn from each other. So again, I just wanna thank you for speaking with us today.

And I also wanna thank our listeners. We'll be back with another episode of Regulation Matters: a CLEAR conversation very soon. Thank you to our frequent listeners. And if you're new to the CLEAR podcast, please subscribe to it. It's available in a lot of different areas. It's available on Podbean, Apple Podcasts, Google Podcasts, Stitcher, Spotify, and TuneIn. And if you've enjoyed this podcast episode, please leave a rating or comment in the app. Those actions actually help us improve our ranking and make it make it easier for new listeners to find us and join the conversation.

So feel free to visit our website at www.clearhq.org for additional resources, as well as a calendar of upcoming training programs and online events. Finally, I wanna thank our CLEAR staff, specifically Stephanie Thompson, our content coordinator and editor for this program. Once again, I’m Line Dempsey, and I hope to be speaking to you again very soon.

The audio version of this podcast episode is available at https://podcast.clearhq.org/e/conflicts_of_interest_SMEs/

Copyright © 2021 CLEAR
Please read the following disclaimer.  |  
Website by Nicasio LLC and CLEAR
108 Wind Haven Dr., Ste. A
Nicholasville, KY 40356
(859) 269-1289
Fax: (859) 231-1943
Powered by Wild Apricot Membership Software