Module 11 Appraise evidence from stakeholders

This episode accompanies Module 11 of the course, which is about appraising evidence from stakeholders – the people affected by our decision or the people who are able to influence it. The acquisition of this evidence is covered in Module 10 and its corresponding podcast episode.

In this episode we are reminded that sometimes we engage with certain parties as practitioners (those with experience in the problem or solution we’re working with) and sometimes as stakeholders – with feelings and perceptions about how our decision will impact them, and how they might want to affect it, if they have sufficient power. 

We hear several examples from the guests about stakeholder situations  which show how things can be quite complicated, with lots of things at stake. We consider a number of ethical considerations, because at the end of the day, even though it may be possible to make a decision and implement a solution, it is also necessary to ask – should we? 

Stakeholder evidence has the potential to throw us off course, perhaps slowing down our timetable and complicating matters. But consider the consequences if we don’t know their perspectives and concerns or if we haven't weighed up whether their views are representative. There may be ethical, political, organisational and societal outcomes that could do irreparable harm. 

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Steven ten Have, Partner TEN HAVE Change Management & Professor of Strategy and Change at VU University Amsterdam
• Dr Lisa J Griffiths, CEO, OzChild National Support Office

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

Transcript

00:00:00 Karen Plum

Hello and welcome to the Evidence-Based Management Podcast. This episode accompanies Module 11 of the course, which is about the appraisal of stakeholder evidence. There's more on stakeholder evidence in Module 10 and its corresponding podcast episode. 

Module 11 focuses on the evaluation of the evidence collected from stakeholders and a determination of whether it's representative of the different stakeholder groups. 

I'm Karen Plum, a fellow student of evidence-based management, and in this episode we hear from the guests that contributed to Episode 10, Eric Barends, Managing Director of the Center for Evidence-Based Management, Professor Denise Rousseau from Carnegie Mellon University; Steven ten Have, an organizational consultant at TEN HAVE Change Management and Professor of Strategy and Change at VU University Amsterdam; and finally Dr Lisa Griffiths, CEO of Australian childcare organization, OzChild.

Let's get going. 

00:01:13 Karen Plum

In this episode I was keen to include some practical examples of stakeholder situations. Does stakeholder evidence and potential outcomes lead to different decisions being taken, and if so, how does that work out and what are the broader ethical questions we should be mindful of? 

Before we get into that, I wanted to clarify something that distracted me when I was doing the stakeholder modules and that's why we take into account the feelings and perceptions of stakeholders, with all the biases they come with, but we don't do the same with practitioners. 

I asked Eric and Denise why subjective evidence is OK for one evidence type, but not another. 

00:01:52 Eric Barends

In the module we make the point - stakeholders are not always stakeholders and that's the kind of weird statement to make. But the point is that for instance, take employees. When you would ask them, we're considering implementing this new way of working - do you think that will be effective or is it a good idea? And when they, based on their experience say, well, probably not because we run into this or it's actually a bad solution for that. That is evidence from practitioners. 

These are practitioners that have experience with the situation, and although they are affected by the decision and they are supposed to work in a new way, they are a practitioner and you should consider this as evidence from practitioners. It is different when there are emotions evolved and subjective irrational feelings or whatsoever. They say, I don't know and I need to work with the computer - I usually have just a pencil on a piece of paper. And I don't know, maybe this computer will monitor me and record what I do behind screen or I don't know. But that is a stakeholder concern and it may be very irrational, it may be subjective, but it may lead to the situation that this person doesn't want to work with the computer anymore and wants to stick to a pencil and a piece of paper. 

Irrational, yes, and you may have evidence, very strong evidence from all the other sources that working with this new method or computer or machine or protocol will improve efficiency and go faster, but when people feel uneasy by it and refuse to work with it, then the effect will be zero. 

So that's why you always need to take into account. It feels a bit weird because it's subjective and it may be irrational and you may get angry, like that's a crazy idea, what makes you think that? 

That doesn't matter it has an impact, and when it's one or two people, stakeholders, employees, yeah, you may get away with it. But when it's a broad felt subjective belief, irrational feeling towards this decision - that's a whole different situation. 

00:04:21 Denise Rousseau

I think a critical issue with regard to stakeholders and why feelings first matter is that feelings are not facts but they are social reality. And the idea that we have to deal with people where they are. A decision or a change may move them to a different place but the facts on the ground are, they have this feeling or response to a problem tackled in this way or to a solution that involves these costs and that has to be factored into both our statement of the problem and potentially also into how we solve it.

Because that's going to be the facts on the ground, the social facts, and we don't argue with social facts. It's like you don't argue with somebody over their identity. You know that just is. It's a fact on the ground - may change, may develop, may grow, may be different, but it is right now. 

Other thing is that opinions from stakeholders. Stakeholders are typically they're not necessarily experts in the decision, right? They're not necessarily even experts in the solution, but they might be quite expert in the consequences. 

00:05:30 Karen Plum

So we have to be clear about any group. Are we asking for their input as experts in their field or as people affected by the decision or with the power to influence it? When we critically appraise the two types of evidence, we use different approaches. Trustworthiness in the case of practitioners and representativeness in the case of stakeholders.

As we talked about in Episode 10, stakeholder evidence can provide a rich seam of feedback and perspectives about the problem or solution we're exploring. Not only does this contribute to the decision we're trying to make, but it helps inform the implementation of the solution and the change management effort that’ll be required.

As Steven hinted last time, it's important to be clear about the different stakeholder groups, and here he emphasizes the importance of neither lumping too many together or overly splitting them apart. 

00:06:25 Steven ten Have

I think of seeing them in the right way is very important to have an overview of all the stakeholders. But also to aggregate them in the right way, because I think Mintzberg makes a split between splitters and lumpers, and he says you can categorize and categorize and that's the way splitters are working and lumpers are saying well, we are all people or you have just men and women. And so on. 

And I think for example, in company context, employees are a group of stakeholders and shareholders are a group of stakeholders. But I think talking about splitting, it's very important to understand that within the group of employees there are also a lot of differences and you can't split them till you are at an individual level of course. But you can split and insert groups and I think US Professor said about shareholders in the current context, they talk a lot about stakeholders versus shareholders, and they are trying to show that the shareholders are one group with one stake, having the shares at the highest level.

But I think that the problem with, for example corporate purpose, is that shareholders also can be differentiated because you have the short term and the long term, you have the people who are interested in a company also from societal motives for example. 

And you have people who are just there for the money indeed. But you have to be very precise to make your stakeholder management precise. Understanding people in general is a difficult thing, but if you start with respect and you work with principles like principle of fair process, you are in the right trick and you make a good start and I think in that way you can do stakeholder management in a very effective and also respectful way. 

00:08:16 Karen Plum

As a way of thinking about understanding the perspective of shareholders, Denise gives an example about, of all things, water quality. 

00:08:24 Denise Rousseau

I'd like to give you an example that I thought was - it stuck with me, it was shared with me when I was doing some work in Canada and one of my people that I whose path I crossed with was part of a negotiation team negotiating between oil and gas in. part of Canada, the provincial government and the indigenous people. 

And it was a project that had to do with water and land rights and a contract that was going to be made because the tribes had resources that they wanted to use, but they wanted to use in a way that they thought was appropriate for their community. And my informant said, you know our scientists brought all this data to the tribe that says after we do this construction, the water quality is going to be this. It will be this pure. It'll have these features. It'll be, it'll be fine.

And the tribal leaders said but you don't understand. To us, water quality is - can our grandchildren fish in it?

Now sit back down and think about - what does it take to have good water for fish? And that was a huge pivot. All of a sudden people went, oh! Added a level of dimensionality to what the end goal was. 

00:09:42 Karen Plum

I love that story. I would never have come up with that on my own! You have to hear and really understand the perspectives of the people that are affected. And indeed the impact that they are most concerned about - on their own grandchildren. But that said, it also concerns me that in the hands of the unethical, some stakeholder evidence could be used to manipulate situations. 

I guess at the end of the day, we're better off if we have that evidence. It raises our level of awareness and that's a key thing. Knowing about stakeholder concerns and impacts informs us. What we do with it will remain on our conscience. 

Moving onto ethical aspects, I asked Eric and Denise about the key considerations when we're engaging with stakeholders. 

00:10:27 Eric Barends

I would argue that the ethical perspective of evidence-based decision making in terms of stakeholder evidence, that simple situation that that you need to consult your stakeholders in advance. Because these ethical consequences are not always clear, it's not always clear in advance who will be harmed by your decision or unequally affected by the decision. 

So this is something that you need to discuss with your stakeholders and ask them whether there are any unforeseen consequences or people may be harmed in a way that is actually not fair, compared to all the other employees or other stakeholders. 

So you need to ask them - that that's my strong advice. And another argument to make that you need to have multiple sources of evidence, not only one, because we're not always aware of the impact our decision has on certain small subgroups, and this is a way to find out and take that into account in your decision making process. 

00:11:39 Denise Rousseau

Well, this is a very important issue in terms of just the fact of reaching out to stakeholder groups has ethical implications - so underscore that! 

You know it's a moral act in a way, and I believe one of the factors that we're seeing in the practice of evidence-based management, is that reaching out to stakeholders is not a one and done. 

That the idea that we go and we contact these folks for this decision and we never talk to them again. But we contact these folks for this decision once. The idea that stakeholder evidence will be of better quality in a context of quality relationships between the decision makers and the community or the stakeholder groups. It may take a little time, to build a quality relationship. 

So if we're talking about a kind of a momentous decision that's not going to be made by Friday, we have a little bit more time to build a relationship, and it would not at all be surprising that people at first would say, OK, what are you doing here, and what do you want from us? 

It's the second and third time you come back that matters. The second and third time you attend a meeting, so I would suggest that if the decision is important, to build a bridge. And to have traffic on that bridge. I think that's the most important idea, 'cause it's likely to be the case that stakeholders for your decisions are repeatedly interdependent. 

So part of being an evidence-based manager is it's just like having experts on speed dial - knowing how to reach a reference librarian on speed dial; being able to access your stakeholders and have decent quality relationships with them is really critical. Recall the old three cups of tea book that you know you need to be there long enough to have drunk 3 cups of tea. That's sort of an idea. 

A second issue is if we have to make decisions rather quickly, I really advise the idea of honesty with regard to what the scope of the decision is, as you know it; what your authority is, as you know it; what you're hoping to make happen in the interaction; acknowledging what you know and don't know; acknowledging what you may or may not be able to do. 

Almost all disclaimers initially build suspicion rather than confidence, but over time disclaimers can be informative to people about what the scope of the decision is and the latitude of the person making the decision. And I advise that, so I'm going to underscore this idea. The first encounter with stakeholders should not be the last. 

Because what people think initially and how they react initially, may not be how they will react ultimately, once the relationship is on a better footing. You might have to say your disclaimers more than once before they have any value or merit, because people have some reason maybe to be skeptical. 

00:14:37 Karen Plum

I think it's so important to realize that trust takes time to form, particularly if people are suspicious or they've been burned in the past, as we've discussed in Episode 10. 

As I mentioned earlier, I was interested to know how and in what ways stakeholder evidence can really influence a decision or an outcome, particularly if it runs counter to original plans or ideas. Steven gave me a great example from the defense industry. 

00:15:05 Steven ten Have

It's an example from the defense industry in which there was large restructuring program. It was in the 90s, so to speak, after ending the Cold War and for defense industry this was very challenging because they had fairly clear customer base for years - countries with their armies. But after that a lot of their technology became in a way useless because the attacks were not from country to country, but for example, from smaller terrorist groups. 

So for example, defending your coast regions was more and more important and asked for other kinds of technology. So those companies had to restructure and we worked for a Dutch company. It's a part of a French larger company, Thales. And they struggled with restructuring their operations in the Netherlands. And they are located in - this very important detail and also very important if you talk about stakeholder management - they were located (in the) east of the Netherlands and in the east of the Netherlands there were also two other larger companies located partly - Ericsson and Philips.

And both Ericsson and Philips for other reasons than (for this) defense company restructured their operations here in the Netherlands. So this Eastern region of the Netherlands was confronted with a lot of restructuring in three large companies. And I think Thales was the third in line then of course, as you can imagine, a lot of people in that Eastern region from different families, are working in those three companies.

And at a certain moment the CEO of Thales was confronted with the overview of the people who would be part of the layoffs and he said, OK, I think our own consultants have done a proper job, they have applied a very clear set of criteria and they had also designed a fair process and works council and also the unions are on board, so that's a very big thing and also very good illustration of the quality of the process. 

But he said also I want a second review of the whole list and it was at least with 500 people. And he said some people are from families who are already damaged or confronted in the layoff process of Ericsson and Phillips with a lot of grief and a lot of problems, social, and also from an economic point of view. 

So he said, can we do a second review in order to assess if some families within our own company, because, for example, three people from the same family are on the list of layoffs or from the aggregate overview from Philips, from Ericsson and Thales are attacked in a way, you could say, in disbalanced or disproportional way. And of course, this also asks for the very intense and precise conversations with the unions and the works council, because it is a very good thing to take somebody from the list, but it's also very clear that he has to be replaced by somebody else who was earlier not on the list, so it's a very delicate process. 

But the CEO he stick to this plan and this intervention and it had also a lot of positive consequences because people understood his criterion and understood also his approach, and I think it's a very good illustration of a very sophisticated and also a very engaged and social stakeholder management in a very demanding situation. 

00:18:53 Karen Plum

It's a fascinating story which illustrates the difficulty in safeguarding the interests of all stakeholders. We must respect each stakeholder in their situation, but sometimes tradeoffs need to be made. In this case, the CEO of Thales had taken the time to understand the situation in the wider area, establishing the pattern of layoffs at the two other companies that were already impacting the workers, their families and the wider society. 

Lisa gave me another example of the need to address ethical dilemmas. 

00:19:25 Lisa Griffiths

Yes, so there's always ethical dilemmas in paying attention to some sources of evidence and not necessarily having the other sorts. So one very powerful piece of data in Australia is the over representation of Aboriginal children in the out of home system. So if you're an Aboriginal child, you are - in some states in Australia - 17 times more likely to be removed from your family than if you're a non Aboriginal child. 

And in the Australian Capital Territory (ACT) the removal rate for children was the highest in the nation a few years ago. And there were no programs that worked directly with Aboriginal families to help prevent removal of children. And it's quite well known and documented problem across the nation and the Aboriginal community and Aboriginal advocacy organisations continually remind the various governments in Australia of that problem. 

And we knew of a program that has had multiple randomized controlled trials done around the United States in working with lots of different ethnic communities in helping prevent children being removed from families and addressing some of the factors that prevent family functioning. So we wanted to put this program on the ground in the ACT where this problem was really prevalent. 

However, we aren't an Aboriginal community controlled organization, so we chose to instead look at the Aboriginal stakeholders and build a relationship with one of the local providers who provide services to Aboriginal youth. And for two years we developed and nurtured a relationship with the Aboriginal leadership of the organization as well as members of my team in my organization developing relationships with the other organization’s team as well and speaking with other stakeholders in government and even the minister themselves. 

And we got to a point where the leader of the Aboriginal community controlled organization said that they would like to enter into a partnership to look at bringing a western model to the ACT, but they were only comfortable with doing this if we would talk to the Aboriginal community and explain what that model was about. 

And of course, we didn't develop the model ourselves, so we invited from America the developers of the model to come to Australia and meet with the Aboriginal community and meet with the local elders and talk through the family strengthening approach that was used and hear from the elders about what was important culturally for it to be able to be put on the ground in the ACT. 

And they provided lots of feedback to the model developers and to ourselves as an organization. So the model was adapted to take on a cultural lens and we commenced the model in partnership with the local Aboriginal community controlled organization, us bringing the clinical experience and them bringing the cultural lens. And we also changed the way people could enter the program to suit the Aboriginal community needs, having listened to the Aboriginal voice.

And the program has been running for four years now and 100% of the children that have been through the program had stayed safely at home. 

00:23:37 Karen Plum

I was struck by how easily the Aboriginal communities could have felt done too, if a heavy handed approach had been taken – “Here, this works in the US, we’ll do the same”. 

Lisa explained to me that in other parts of Australia, slightly different approaches have been taken, so it all comes down to local context and the receptiveness of the key stakeholders on the ground. Lisa said that while organizational data, scientific evidence, and practitioner expertise are all important, she feels it's the stakeholder evidence that's worth spending more time on. 

00:24:11 Lisa Griffiths

I think that it's really important in leadership in regards to stakeholders to never underestimate the value of stakeholder engagement and in your decision making because normally with stakeholders there's sort of a human at the end of it. Whereas if you're looking at the scientific data, or you know you're not really offending anybody if you're just reading a bit of research and if you're looking at the organizational data or sector data and looking at trends and there's not anybody really at the end of it. When you're talking to practitioners, you do get more sensitivity because you know they're the experts in their domain. But you can have a kind of a healthy exchange. 

But stakeholders, they all have different values and those values often driving what's best for oneself, and you're probably coming out at what's best for the problem you're trying to solve, which might not align with what's best for oneself with the stakeholders. So it's a complicated area and one that's worth spending more time on probably, than less. 

00:25:22 Karen Plum

So there are a lot of ethical considerations, particularly when we operate in the types of environments that our examples came from. I think it's also important to recognize that while many people may think stakeholders can be a source of delay, detail, pickiness and irritation, given an open mind and patience, they also bring resources that can increase the richness of the decision making, as Denise explains. 

00:25:47 Denise Rousseau

One of the things I have noted in evidence-based practitioners broadening their attention to stakeholders is the resources that it can bring to the decision maker and to the group of people trying to make a decision that stakeholders sit on ideas, they don't mean to, but they also may have identified local problems or concern that tie into the decision you're making that might actually broaden its appeal. Make a decision more powerful because of its relationship to other things going on in the organization. 

We don't know what we don't know. We often fear stakeholders 'cause they'll make the decision more complex, but I can promise you they can also make the decision richer and more resourceful. So with that hope, that it's not all complexity you get by reaching out to and building rapport with stakeholders. 

Expect to learn expect gains to come to you and your organization by that stakeholder involvement. But not if it's a one and done. If it's a repeat effort to build quality relationships, that really is where the benefit is, and it changes really the context and the facts on the ground of the decision. 

00:27:16 Karen Plum

And to sum up, Eric points out that evidence-based decision making is like a double-edged sword. It can be a force for good, but it can also inflict harm. 

00:27:27 Eric Barends

Evidence-based decision making, evidence-based management as based practice, whatever you call it is a very sharp, two-sided knife. Meaning that it's like a chef's knife. You can brilliantly, perfectly cut vegetables, meat, or whatsoever, but you can also inadvertently cut someone else or cut yourself.

So the evidence-based approach is very effective in many ways, but it can also be effective in a way that it really harms people. It's like being intelligent, being smart, you can do brilliant things with clever people and clever people and can invent brilliant medical interventions that save the lives of hundreds of people, but they can also come up with, you know, invent weapons of mass destruction with their brain, with their intelligence and have a devastating impact on society at large. 

So evidence-based practice in itself is neutral. It's amoral, it's the way how you apply it, how it will work out and whether it's unethical or effective or not. So I think that is specifically of importance when you think about stakeholders, because that's where it finally will be determined whether your decision may be effective but completely unethical or not effective at all because everyone refuses to support the decision or do the things that's agreed upon. 

So that's where it all comes together with the stakeholders. And often a lot of people specifically in the beginning when we brought in the stakeholder perspective, the evidence from stakeholders, were critical towards the idea of stakeholder evidence, they say, but you bring in all kinds of subjective stuff. I mean the scientific research that is hard facts, you can figure out whether it's trustworthy yes or no. 

Same counts for organizational data and practitioner experience. I mean, it's true or not, or it's strong or not, to trust or not. But subjective beliefs and feelings of stakeholders don't go there. But if you are, for instance a change manager, you know that resistance to change even from irrational subjective beliefs, can really, you know, hinder and stop the change from happening and then the change outcome is actually zero. 

So therefore it is something you always need to take into account. Either from the perspective of being effective, having impact, or ethical considerations. 

00:30:15 Karen Plum

Finally I thought it was worth emphasizing something that all the guests mentioned, and that's the importance of engaging with stakeholders early on. Certainly before the decision is taken or the solution chosen. Their input can provide rich information that you would otherwise be unaware of. 

Ethical considerations that could get you into a whole lot of hot water. Opportunities and support to carry everyone through the change journey, helping to unlock and address the underlying factors that contribute to resistance. Also, it seems to me that the further through the journey you go, the more disinclined you are to want to move away from the decision or decisions that are already forming. 

I’ll leave you with Denise’s reaction to that thought as she describes ‘path dependence’. 

And in the next episode, we turn our attention to the aggregation of all the different sources of evidence that contribute to evidence-based decision making. 

00:31:10 Denise Rousseau

In management research, we call that path dependence. You know we got here because of all the things we did before. And if we don't know all the things we did before, sometimes you can't figure out how the heck did we get here?

Critical idea is that your opportunities your decision latitude is greatest early on and that's where in an earlier podcast we talked about the importance of generating alternatives in decision making and part of generating alternative ways of thinking about a problem can be recognizing what stakeholders know, fear or see as opportunities and so that can actually be synergistic with our generation of alternative ways of thinking about the problem, or alternative ways of solving it.