Evidence-Based Management

Module 15 Building the capacity for evidence-based management

• Season 1 • Episode 15

Share episode

This episode accompanies the last module of the course – module 15, which is all about building an evidence-based management capacity in organisations. 

There are 3 levels – you as an evidence-based manager; your colleagues who you can influence and guide in evidence based approaches; and the skills, practices and processes in your organisation.

We hear from several experts about ways to build on what you’ve learned, and how others have approached the development of evidence-based practices in their organisations, including the CIPD, the professional HR body, about how it promotes this practice among its membership.

And you’re never alone, there are always others forging a path in evidence-based management who you can reach out to, learn from and share with.


Host: Karen Plum

 Guests:

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

00:00:01 Karen Plum

Hello and welcome to the evidence-based management podcast. This episode accompanies module 15, which is the final module of the course. It addresses ways to build the capacity for evidence-based decision making in organisations. 

I'm Karen Plum, a fellow student of evidence-based management, and in this episode we hear from Eric Barends, Managing Director of the Center for Evidence-Based Management; Professor Denise Rousseau from Carnegie Mellon University; Jonny Gifford, Senior Researcher at CIPD; David Creelman, CEO of Creelman Research; Stefanie Nickel, Global Head of Diversity and Inclusion at Sandoz; and Steven ten Have, partner at Ten Have Change Management and Professor of Strategy and Change at VU University, Amsterdam. 

There's a lot to cover, so let's get started. 

 

00:01:07 Karen Plum

In this episode, there are three key areas - how we build our own skills and continue to learn as evidence-based practitioners; how we share our knowledge and influence our colleagues; and how we develop evidence-based skills and approaches in our organisations. Here's Eric to go through those three levels. 

00:01:27 Eric Barends

When it comes to building capacity in your organization, it's important to distinguish three perspectives or levels. It starts with yourself. Do you have an inquisitive mindset? You need to develop that to become an evidence-based manager and develop the skills - how to acquire the evidence, how to critically appraise it, whether it applies, assess et cetera. All the things that we teach in the modules. So that's the first step. Do the modules, complete the course so you develop your evidence-based mindset and your evidence-based skills. 

The second level or the second perspective is where you educate your peers in taking an evidence-based approach. Small steps, educate them about research findings that all research isn’t created equal. Explain about the pitfalls of organizational data and evidence - stuff like that. When they have these weird beliefs or they have strong assumptions that are maybe biased, make that a part of your discussion and ask them politely, though respectfully, what's the evidence, how do you know and stuff like that. So that's your second circle of influence. 

So first yourself, then it broadens, when you're knowledgeable enough when you are certain enough about your own skills and have practiced it, you go through your peers and then finally the third level is the organization itself. This is more technical, it's about - is your organization in a position to generate organizational data in a reliable and valid way?

And you can point that out, like listen, we should have more information or have a system in place that generates these KPIs or whatsoever. It would be great if we would have people in our organization that could have a look at the research findings as a sort of service for managers in organization. Wouldn't it be great if we would have a group of practitioners or stakeholders - a panel that we could, you know, survey on certain points in a more structural way, et cetera. 

That's building more technical, instrumental capacity in your organization. So these are three different perspectives, three different circles of influence. And you may be very successful regarding your peers, but struggle to get more structural resources in place within your organization. You need to make a point or set up a whole project, maybe, to make sure that the data or the evidence you need to make decisions, or your colleagues need to make decisions, are generated. 

So these are three different areas with all their barriers and pitfalls and challenges. 

00:04:28 Karen Plum

So the first level is us, and if you're like me, you've done the modules, done a lot of thinking, some agonizing, adjusted your perspectives and started to ask more questions and gently to challenge people's assumptions and biases. 

Clearly, having the right mindset and attitude is helpful for us as evidence-based managers. If we weren't interested in making better decisions, we wouldn't be here, right? But it's not just about having the right mindset. As Eric explains, we have a big toolkit to help us to be more evidence-based. 

00:05:01 Eric Barends

There are skills that you can master and apply and that you should learn. An evidence-based approach is a systematic six step approach with specific skills and things you do in a certain order, and now when we get at the end of the process, we say yeah, but it's also an attitude!

Because in each step you need to have the same attitude that was there in Step 2 - asking questions. That attitude of an inquisitive mindset should be there in the six step and the seventh step and the eight step, whatever. When it's about applying the evidence, ask people - how do you know, based on what? What's the evidence? You take critical approach toward the quality - how trustworthy is this judgment of this person? 

00:05:50 Karen Plum

And as with anything, once you become more practiced, you can run through the steps more quickly and with more confidence. The steps become like habits, rather than us having to think them through fully every time. Although we covered this in an earlier episode, I think it's worth emphasizing that when we're starting out, taking gentle steps and sharing your knowledge respectfully is probably the best way to be helpful and recognized for what you can contribute. 

00:06:18 Eric Barends

Make a constructive recommendation or come up with a proposal like - this is very interesting, I think it's important to have a look at the organizational data. I'm sure there is information that may be helpful. Would it be helpful if I look into this because I would love to do this? I did a course so I know a little bit about organizational data, et cetera. That is a good approach and we hear from students that are relatively at the beginning of their career, so they don't have that much power or a very strong position in the organization, but making those suggestions really seemed to resonate. 

If you would say in a discussion whether we should do X or Y and you bring forward hey, I know that there is actually quite a lot of research on this topic. Would it be helpful if I look into this and maybe next time when we come together, I summarize some of the findings? Maybe that's helpful for the discussion. So in a constructive way, in a positive way, offer your skills and your expertise as an evidence-based manager.

That is the first step rather, than to try to transform the whole organization through this brilliant, perfect evidence-based decision-making machine. That's not going to happen. 

00:07:39 Karen Plum

In the early stages, if we do nothing more than ask questions, clever critical questions, then we'll already be making a difference. Eric describes this as the evidence-based managers superpower, an inquisitive mindset, always asking questions, taking nothing for granted. 

Let's move on to consider the role evidence-based managers can play in influencing and encouraging colleagues to be more open to evidence-based practice. I asked Denise how she would advise managers. 

00:08:12 Denise Rousseau

The idea, first is what managers model to others, to their staff, their employees, are likely to influence how the employees and peers make decisions. And we have some good evidence that indicates that how managers share the way their own decisions are made (what information they used, why they made it) with their employees, increases the trust in that manager and also their capacity to do problem solving locally because they have more information about why the boss made a decision a certain way. But they also see oh, it's good to ask stakeholders. It's important to look at the metrics. She did it, so I'll try it. So the modeling part is very important for spreading the word. 

Second issue is managers are trainers, and the idea that working with especially your immediate direct reports on some decisions that are of consequence and helping coach them through the process of framing the issues, gathering information. So I like to use the example of a friend of many years, who was the director of a home for the aged here in Pennsylvania in the United States and I think he was the first evidence-based manager I ever met. 

And what he did was very interesting. He had seven direct reports - he met with them every Thursday and they met for two hours and in the first hour of their meeting they read something that they had worked on in the previous week. It might be an article from the Journal of Applied Psychology, it might be the goal or a popular management book of some years ago. They would talk about what they learned from that. What were the things that struck them, how were decisions approached, what would be the input to decisions they're making, without necessarily trying to target to a specific decision.

It was a developmental conversation where they got different perspectives on what the management issues were in the reading. And then they did, in the second hour, the decisions they were there to make. And he said that part of what that process did is not only did it expose people to different literatures and different ways of making decisions, but it created deeper conversations about the decisions they were making. People asked more questions because he was modeling inquiry to them in that first hour. 

And learning to improve the quality of their reflection on the bigger picture - strategic and also tactical decisions - his top management team made. So thinking about how we prime the pump with our people. 

Change the conversation, you change the way people think or certainly the way the information they have access to. That would be, I think, one of the most important things.

00:11:00 Karen Plum

In a world where we always seem to be in a rush, spending two hours a week investing in deeper thinking and developing people’s critical thinking skills, seems an amazing investment which will surely pay off over time. 

Denise suggested another approach, which would be to pick a routine decision - like how we make selection decisions or how we onboard new people - so something we do over and over again and treat it as an opportunity to learn more about what works best. 

00:11:31 Denise Rousseau

Have people read a little of the research on that routine, like onboarding newcomers or you know having them shadow current employees, there's good deal of research on that. Talk about the experience people are having and then use that to redesign the processes. He'll pick something people care about that's sort of accessible to them that they can get quick feedback on and model and evidence-based practice there. But there's always going to be a learning component to introducing evidence-based practice in an organization and to sustaining it. 

00:12:03 Karen Plum

Denise's third suggestion is about the assessment of outcomes. You're not an evidence-based manager unless you're assessing outcomes, as we discuss in episode 14. 

00:12:13 Denise Rousseau

You're not an evidence-based manager unless you're assessing outcomes. I've had to take this to heart in my own teaching, where I now assess very different things as my students go through a course on change or on evidence-based practice because I have to walk the talk and also I really do want to learn and if I don't assess outcomes, I'm not going to learn and I'm not a good model to them. 

So the idea also is beginning to create metrics that are regularly used in your organization, that people can then gather weekly, monthly, quarterly, whatever it is, and then sit down and reflect on what they say. It's not so much a matter of keeping score as it is monitoring the outcomes you're getting and talking about why. And then monitoring the outcomes that you get over time and learning what works. 

And that sustained attention to metrics used in a constructive developmental way, I think is a powerful model for evidence-based managers and you're not an evidence-based manager unless you are assessing results. 

00:13:14 Karen Plum

I certainly wish I'd worked for managers who took this approach, who took the time to review, reflect, and learn from outcomes. I asked change management consultant Steven ten Have for his suggestions for evidence-based managers. Reflecting that many people in organisations are open, in theory, to different concepts, ideas and approaches (hence the prevalence of management books which people buy and then never get around to reading or finishing). 

He thinks it's good to acknowledge the concepts people are aware of, while gently steering them to a more evidence-based approach. Here's how he explains it. 

00:13:52 Steven ten Have

If I talk about practical issues in companies, I always try to relate to the concepts that they are aware of, or that they embrace. And then I say OK, for example, you talk about the learning organization which was there in the 90s and is now, I think back and which is I think a positive thing, because it's positive concept that learning is very important. But people say OK, learning organization and then then I say OK, I advise you to do two things. 

One is try to find an empirical based journal article on learning organizations in order to see what the evidence is and the other thing is, try to imagine which other discipline than management is important for this concept and in case of the learning organization, that's very clear that you have to look at psychology, and in particular, psychology of learning. 

So I always advise my customers and my students to contributions to the concept they already embrace and give a critical angle by reading about the evidence which is there. And sometimes people say although it's very difficult to get to that evidence, and then I explain that it's not more difficult than go to the bookstore and buy a management book.

And I always advise them to read a book from a discipline which has nothing to do in direct way with management, but has to do with principle which is applied in the context of management. If you talk about the development of people in organizations in particularly in cases of change. And in this way, without being too confronting, you develop I think, the thinking of people and they get into the evidence-based thinking and also the reflection you need without that you disqualify the things they already use. Instead, we make contact on the concepts that they already use. 

And sometimes, well, maybe the intervention can be more brutal because for example, I work for organizations which are into self-steering autonomous teams and then I ask them do you think that this works in all cases and situations, better than working with in hierarchy or with more classic concepts of workgroups. And most people are believers, and they say yes, it is the ideal concept and it works very good and people are happy and the performance is better and so on. And then the first thing you need to do, of course, and that's also very evidence-based perspective, is look at the data. Is this really true? 

But also again at the evidence, scientific evidence, which is there. And for example, in this case, in some situations, classic concept of a team or workgroup may work better than the modern concept of the autonomous or self-steering teams. And I think most people, after being irritated in first instance, are happy to hear that their concept can work, but not in all situations and to having provided guidance with regard to alternative ways of working, so that they can develop their organization really in a in a good and evidence-based way. 

00:17:20 Karen Plum

It's a salutary reminder that people don't like to be told. But helping them come to their own conclusion increases the likelihood they'll buy into the idea, the decision or the solution. 

So what about building capacity in the organization - the level of skills and capabilities available to support evidence-based management. For David Creelman, the answer is having a team of analysts to gather scientific evidence and organizational data. Because in all probability, managers just don't have the bandwidth. 

00:17:52 David Creelman

Well the most successful and most direct way, is you hire a team of analysts and I'm going to focus on the HR function and people analytics, where they actually say we're going to hire a few people and they're going to be called people analytics specialists and their job will be to look at data and evidence to help support decision making.

And typically a manager, say a vice president and maybe they were vice president of operations before or vice president of talent before. And suddenly they're told, oh guess what your title’s changed you’re now vice president of operation and analytics or vice president of talent and analytics. 

So now you have somebody responsible for bringing evidence into the organization, so at a senior level, so it's part of their responsibility and then they have a staff to support them who have the time. Because if we were looking, let's imagine that recruiting, you know talent acquisition - if you say this is a really important function and there's lots of evidence of what we can do to improve it functioning, let's start doing that.

Well, if you're the head of recruiting, you already have very, very busy days. You really don't have very much time and the recruiters are likewise, so everybody in your organization is already running at 100%. So I actually think that you have most success in evidence-based management when you bring on extra resources specifically tasked with doing that kind of research and reporting to a senior person who’s ultimately accountable for bringing evidence-based thinking into the organization. 

00:19:31 Karen Plum

David believes that's the most successful route, but he's also seen people gather together allies and like-minded people from across the organization. People who are sympathetic to an evidence-based approach, because they have master’s degrees or PhDs and have been exposed to the practice during their academic careers. And Stefanie Nickel at Sandoz agrees. For her, it's all about finding like minds and forging alliances. 

00:19:58 Stefanie Nickel

We rolled out the training with Eric and Denise which was appreciated. It is a very nice training to really guide you through the thinking and why it makes sense and how can you apply it. But I think even before that, there's a step around understanding that just thinking practically doesn't make us effective, right? Unless there is a strategic element to this to say really, what's the vision? How do we most effectively get there? 

And if you're not asking that question around impact, it's difficult to find, I would say, the commitment for evidence-based management, right? It's all about approaching things strategically. I think about finding alliances, about finding like minds and to make the study happen for Sandoz, for example, we worked with our ethics, risk and compliance colleagues and the global Novartis team because there we had this like-minded thinking around how can we drive impact? Let's test and experiment. 

So I think it's about at the moment, individuals understanding the opportunity. And being willing to invest time and resources. It's not necessarily that data analytics shares the vision just by their function, right? Still in HR, and I think if you're looking at actions right away, which is often the case, it's difficult to understand I think the value of this approach. So if you have any other contributors that has the answer to this, I'd love to hear it. It's not a simple one, but I think it's definitely worth it. 

00:21:22 Karen Plum

David pointed out that the term ‘evidence-based management’ is also less understood than ‘analytics’. For one thing, most people believe that they use evidence already, so they think that's a tick in the box. But analytics is something that's quite trendy right now, even though we're trying to avoid fads and fashions. 

00:21:40 David Creelman

Right now the road to evidence-based management in organizations will probably run through analytics. That's the function that's being funded, that's what managers are talking about. It's an unusually narrow way to approach evidence, I mean evidence is much broader than just analyzing data, but nonetheless I would suggest recognizing that if you talk about evidence-based management, you may get a blank stare; if you talk about analytics, then everyone says oh yeah, that's really cool and important and we should be doing that. 

00:22:19 Karen Plum

Jonny Gifford at CIPD agrees that the word ‘evidence’ can be abused and that it's perhaps better to focus on ‘research’. 

00:22:28 Jonny Gifford

It's interesting because the word evidence clearly resonates with people. The fact that it gets not only used but also abused, I think shows that it's a powerful rhetorical word. And yet on the other hand, the phrase evidence-based practice is a real turn off for people. So I know that some people find it more useful to talk about drawing on the scientific research, or drawing on science, rather than drawing on the principles of evidence-based practice. 

But I think that there is a greater appreciation that there are better and worse ways to engage with research, and if we’re canny about this and rigorous about this, then we can do it in a way which, as individuals, makes us both more effective and more influential. And from the point of view of our organisations, makes them both better performing and more enjoyable healthy places to work. 

00:23:30 Karen Plum

Of course, it's not all about research. There are three other sources of evidence we need to take into account, but I suspect the academic research is where a lot of people start. You may remember in episode six of the podcast ‘A Short Introduction to Science’, I mentioned my frustration that academics don't provide more guidance for managers in their research studies. Rob Briner at the time explained that many researchers are simply not interested in the practical implications, that's not why they did the research. And in any case, they’re academics, so why would they have that knowledge anyway? 

I discussed this with Jonny, who felt that certainly with some academic journals, this is changing. 

00:24:11 Jonny Gifford

I think it's changing to some extent. So you have some journals, like the Human Resource Management Journal, which insists on having kind of practitioner notes up at the beginning. So like bullet pointed things and they're trying to push the issue by ensuring that the practical recommendations aren't literally two sentences at the at the very end of the paper, but they are thought about and more proactively kind of drawn out.

It's something that we spend a lot of time doing within our evidence reviews, so we try and summarize in bullet points. It does need to be a little bit speculative at some point, but we need to try and make inroads. We need to be clear about what we're sure about and what we're not sure about so we can say - we know that for example, social support has really big outcomes in terms of performance and well-being. 

Then we need to go a bit further and say OK, in what ways might you be able to strengthen social support within your organization, within your team and that's where it starts to get a little less scientific, because we don't necessarily have research on that. But it's a really important step to make. 

00:25:34 Karen Plum

In terms of building internal expertise, Jonny’s role embraces the promotion of evidence-based management and evidence-based HR, to the HR membership community, as well as encouraging the use of more evidence in internal decision making. 

So how has this professional HR body gone about developing its skill base? 

00:25:55 Jonny Gifford

The CIPD is not an academic institution in itself, but we do have a research team, so we have researchers with postgraduate level training in social science research. Nonetheless, we've spent time developing our skills internally in critical thinking, in searching for literature in systematic ways, in appraising studies, and so on. We found that the CEBMa online modules invaluable in that sense. 

We also work with partners, external partners like CEBMa to conduct reviews, so it's often helpful to have that external resource. We also work with subject experts to help interpret findings and guide sort of questions in the first place. So we have subject experts internally within CIPD, but also academics through our networks. 

And from the practical point of view, we have colleagues who look after our relationships with Members. We find this particularly useful when we want to engage with senior HR stakeholders to find out what sorts of questions they want to know the evidence on and to talk with them about how we can draw out recommendations from practice that are likely to hit the ground in a positive way; so they're not just based on the research, they're also based on what managers know is likely to work in their organisations. 

00:27:31 Karen Plum

So this is evidence-based management on a larger scale - embracing many organisations rather than one. I think it really helps reinforce the broadening of the learning and sharing that exists in this field. It may feel like a lonely place if you're a lone voice in your organization. But then again, as you role model and encourage a more questioning mindset in others, you'll hopefully see change start to happen. You can't expect to be skilled at everything at the start but make use of the toolkit and strengthen your capability as you go. 

And that's it for this episode, and indeed for this series of podcasts that accompany the evidence-based management course. I'd like to thank all of the guests, academics and practitioners alike, that have been on the show, for sharing their wisdom and experiences. All the discussions helped deepen my understanding and I hope you've had a similar experience. 

If you enjoyed the podcast, why not share it with colleagues or fellow students? Thank you for listening and I'll leave you with a final thought from Denise Rousseau, who reminds us that once you're part of the evidence-based community, you're never alone. 

00:28:43 Denise Rousseau

There's nothing more powerful than being part of a community of practice because we learn from each other's experience, so there's a lot of reflection that's not possible. Managers can be evidence-based practitioners, even in a sea of something very different. However, your own mental health, growth and resources really are sustained by having ties to others who have similar values and also through whom you can access facts and information. 

The idea that none of us can read everything, but we can be alerted because our network has pointer knowledge, you know somebody has worked on this issue of performance feedback, somebody’s worked on this issue of agile in my network.

I think it is inherent in evidence-based practice that we will be part of social networks, because there's so much to learn, so little time and there's a great deal of power and efficiency that comes from having a community that helps you practice.