The ExpertLens– Building bricks of evidence

Original content from the Mapping Pathways blog team

Caroline Viola Fry

The ongoing International AIDS Conference (AIDS 2012) in Washington D.C. is an opportunity for Mapping Pathways team members to disseminate information and findings on the project. 

In the second of this two-part post, Ohid Yaqub and Caroline Viola Fry from RAND Europe discuss the work involved for the ExpertLens, a cornerstone of the Mapping Pathways project. Read part one here.

Ohid Yaqub

MP: How did you narrow down on the stakeholders or participants in the ExpertLens?

OY: We chose the participants from the U.S., South Africa and India, the three focus countries of the Mapping Pathways project. We aimed to assemble an expert panel across multiple stakeholder groups in each country. So we were looking for policy makers, academic researchers, clinical researchers and people from industry.

One more thing to note is that we weren’t targeting extremely top-level policy makers who may not necessarily be familiar with the details of the situation on the ground and the context in which the proposed care strategy might be deployed. We were looking for people with the appropriate knowledge and the time to think very carefully about their answers, think about them yet again in relation to the group, justify their answers to the group, discuss them and maybe refine their answers afterwards over the course of three weeks in an online forum. Our participants needed to be very engaged with the topic.

CVF: We wanted our participants to have some background knowledge and expertise in their area but not necessarily be experts in everything there is to know about the HIV landscape. The benchmark on which we were trying to look for people to engage with was if they knew about the HPTN052 trial or about Truvada.

MP: How much time and effort went into drafting the questions for the ExpertLens?

OY: We drafted the questions very carefully and lot of work went into giving questions the right tone so that we could generate useful discussions in the online forum. There were more than 20 versions of the question set. 

We were very aware when thinking about the questions that the situation many decision-makers might face is not one where they can say yes to everything but one where they may have to make choices from a limited pot of resources. There may be a situation where they have to choose one over the other and so that’s why some of those questions were ranking questions.

CVF: The questions ended up drawing out some of the more nuanced views on issues, which highlighted the fact that they were not straightforward questions and that policy decisions are not black and white.

MP: What, in your view, were some of the most palpable benefits of using the ExpertLens?

OY: Our initial impression was that we’d get quick-fire, reactionary opinions being thrown around on this topic. The ExpertLens forced people to be little more reflective, and take on board what other people were saying.

One thing everyone does agree on is that HIV is a very interdisciplinary problem where concerns of policymakers, researchers and clinicians need to taken into account since they are all stakeholders. Being able to get different stakeholder groups to be able to talk to each other is very important since some forms of media and discussion forums don’t allow that. A prime example is journals, where disciplines and professions talk to each other in very considered, robust ways. But the issue there is that they talk to their own disciplines and tend to talk to people who read the same journals. So the ExpertLens is a great way to have a different kind of conversation.

CVF: One of the useful aspects of it being online on forums rather than being a conversation in person is that one can track these discussions and analyse what contributed to someone changing mind if they did. We also looked at what may not have had an effect and left people where they were. We went through comments very carefully and stepped into some of these conversations as moderators to prompt some of the discussions and ask relevant questions.

MP: Is the ExpertLens an ongoing process? Is there a key finding that you could share?

OY: The iterations have all been done and the ExpertLens has been closed. It ran for roughly about a week for each round. We then analysed the data and are in the process of writing up the findings, which will be published in an upcoming RAND book on the Mapping Pathways project (forthcoming on www.randeurope.org).

CVF: The findings are so complex that it’s impossible to sum up here. The key finding is that there’s very little information out there and very little consensus. It’s not a bad thing because it’s still interesting to ask where the points of divergence are and why do people disagree.

OY: Phase two plans to build on what we’ve assembled here in phase one. ExpertLens is one of the bricks of evidence we’re pulling together. We’ll be synthesising evidence from literature reviews, surveys, interviews and this ExpertLens to make up series of assumptions that can feed into better starting conditions on a model and we can then try and model for behavior. 

[Content that is linked from other sources is for informational purposes and should not construe a Mapping Pathways position. Please look for us on Facebook here www.facebook.com/MappingPathways and you can follow us on Twitter @mappingpathways as well.]