Module 2 ASK critical questions

This episode accompanies Module 2 of the course, which is all about asking questions – critical questions to understand what the problem is that we are trying to solve. This is because so often in organisations we leap to solutions without being clear what the problem is.

This approach can be driven by a desire to do “stuff”, to get things done and to feel that we are making progress. But how many times is a solution delivered and it doesn’t seem to do the trick? Maybe that’s because not enough time was taken to understand the nature of the problem before jumping to solutions.

So we need to ask questions – lots of them. Asking them in the right way, of the right people, at the right time is vital.

By being constructive in our questioning, we can make a contribution to the decision making process even if we are not experienced, not powerful or not senior in the organisation. Sometimes the inexperienced have a better perspective because they don’t make so many assumptions and will be more easily forgiven for asking the "daft questions". 

In other words “question everything”!

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Centre for Evidence Based Management (CEBMa) 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University 
• Dr Lisa J Griffiths, CEO, OzChild National Support Office

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/

Transcript

00:00:02 Karen Plum

Hello and welcome to the evidence-based management podcast. This episode accompanies Module 2 of the course, which is all about asking critical questions to identify the problem you're trying to solve and the potential solutions under consideration. 

Asking the right questions, of the right people, at the right time isn't as easy as it sounds. The ideal outcome is that you make a constructive contribution and not damage your career prospects and work relationships along the way. 

I'm Karen Plum, a fellow student of evidence-based management and to discuss the ASK module, I'm joined by Eric Barends, Professor Denise Rousseau and Dr Lisa Griffiths, CEO of OzChild. Let's get going. 

00:00:58 Karen Plum

Probably like most students, I arrived at this module fresh from understanding the principles of evidence-based practice and keen to know more. Here's Denise Rousseau. 

00:01:08 Denise Rousseau

Because I've learned people go through module 1, which is the introduction and they love it. Bells and whistles, this is going to be so cool! Then they go through module 2 which is the module on asking and they still like it, but they don't like it quite as much and its harder self-work. 

If we do this early part well, it opens up so many options 'cause all the research evidence on decision making says the beginning of the decision process is the richest greatest opportunity, to realize gains and benefits and improve the quality of the outcome of the decision you make. 

And we shortchange the front end, especially in kind of time starved, performance-oriented environments. People want to cut to the chase, but the chase starts in the beginning and get the problem frame right, what follows can be very straightforward and perhaps even less contentious, because the motives will be clearer. 

00:02:09 Karen Plum

I found the second module really interesting, but also quite challenging and at times uncomfortable. Reflecting on it made me realize that I've stopped asking questions in many aspects of my life and work. It opened up a whole realization as I looked deeper that asking questions is a skill and not one that I think is fully recognized or practiced.

So often questions are delivered or received as a challenge. Sometimes we feel that they are just a segue to the questioner having the perfect opportunity to put their point of view across, or to show us how wrong we are. 

Politicians ignore questions and just say what they want to say, regardless of what they were actually asked! When did questions become a challenge rather than a genuine respectful inquiry and a desire to know more? 

In this episode we're exploring the need to ask questions, questions that are critical to our pursuit of an evidence-based approach, but asking them in such a way that we aren't perceived as critical in the way we question. I talked to Eric Barends about why asking questions is so important. 

00:03:13 Eric Barends

The reason why this step in the evidence-based approach is so important is because what you're doing here is not just being critical for the sake of being critical and show that you're smarty pants and have brilliant questions to ask, or that you're engaged, no! It is - you want to know whether there are gaps in the evidence base. That's why you ask the questions. 

What is the problem we're trying to solve? Because you want to know whether it's clear what the problem is for all people involved, so we're not solving the wrong problem. Or it's just, you know, a pet theory or model or new thing that the manager or leadership of the organization fell in love with and they just want to implement because they think it's a nice idea.

Or you want to know what actually it is known how experienced managers, professionals in the organizations feel about this problem. So that's the aim, but it absolutely may come across as being over critical. 

00:04:24 Karen Plum

Denise Rousseau puts this very well. 

00:04:27 Denise Rousseau

For many people and organizations, critical thinking was either never the strong suit or it was sort of beaten out of them by the pressures of performance or the hierarchical cultures in which they're exposed. 

00:04:38 Karen Plum

I definitely suspect that I've been in that category for a while, but given that this course is encouraging students and evidence-based practitioners to question everything, how can we do that in a respectful way so as not to irritate everyone or become the “evidence police” as Denise puts it. Here's Eric. 

00:04:56 Eric Barends

My recommendation, my advice to everyone who just finished this module is practice a little bit. Ask two or three questions in a situation at the organization, in your team, or with your boss or whatsoever in a polite, respectful way. To see what's the evidence base, is it clear what the problems is, et cetera? 

Because you are right, if you ask at the wrong moment when the decision was already made by the top of the organization, and we're now in the implementation phase and you start asking questions, but what is exactly the problem and how do we know, etc., it's going to be a career limiting exercise and you'll be in a bad position and your colleagues will get annoyed, and that's absolutely understandable. 

00:05:49 Karen Plum

The more I reflect on the challenge of asking questions, I realize that this is a skill that many people have lost or maybe didn't have in the first place. Being curious and being encouraged to be curious, rather than every question being received as a challenge, depends very much on the culture that we operate in and whether it even feels comfortable to ask those questions and to learn from the answers. Here's Denise. 

00:06:14 Denise Rousseau

As individual practitioners of evidence-based management, our role I see it is to get evidence into the conversation. Because conversations with the coworkers and with managers around us and others in our community, broaden the space in which evidence can be sought, gathered problems framed. 

So how do we talk about these things? Well, we get started, you know, we raise issues about why might a person think that? What's the support, what's the evidence that this is based on? Oh, it's your experience. Well, tell me a little about your experience. How successful was it? How many times did you do that? Is this something you think generalizes? Will it hold in a different setting? 

The idea of beginning to explore alternative ways of thinking about the problem. Remember our focus is learning. Learn to think more critically, learn to gather more sources of evidence, and that's important because all evidence isn't created equal, and most evidence is imperfect. So multiple sources is really important from the science, from organizational data, from the stakeholders, and experts in judgment - that broadening the sources of evidence is a process, but once people get the benefit of that, they start seeing it. I think of this as a healthy inoculation of a good virus of something that can multiply because you will find that it does work. 

00:07:50 Karen Plum

It's also clear that as practitioners we may be frustrated by the actions of our managers or the wider organization, chasing the latest fad or fashion or the latest flavor of the month. Being able to take a moment and ask what are we trying to solve here is vital to start the journey of understanding in terms of defining the problem and ultimately its solution. In seeking multiple sources of evidence we need to turn to different parties as Eric explains. 

00:08:20 Eric Barends

It helps if you can ask questions - so what exactly are you trying to solve here? How certain is it that we have this problem? Do we all feel we have this problem? How do our colleagues feel about this? How do the people involved; do they agree that this is the main problem? 

Try to be helpful and to add value. Rather than just being critical for the sake of being critical. On the other hand, we often get questions from young managers or people that are not yet in a managerial position because they're young, or they don't have that much experience, or they're just at the beginning of the career and for them it's hard. 

But it's also, I think, a way to signal that you're interested in the stuff that's going on in the organization, and you're a critical thinker because you ask clever questions that actually make sense and you can be helpful by asking those questions. So please, do ask them and don't you know, back off and say ah well, you know, I'll do what they tell me and etc. 

Be a little bit more critical, don't take anything for granted. Challenge assumptions, but to a certain level that it still becomes helpful, rather than being a barrier damaging your relationship with your manager. 

00:09:46 Karen Plum

I think it's vital that we get better at asking questions in a collegiate way, not a challenging or I want to make you look silly kind of way. As Eric says that could be very career limiting. If through our learning, we can help our colleagues and managers to ask better questions and to be more discerning about the evidence they use in decision making, then everyone wins, right? 

To do this, we have to build or maintain trust with our colleagues, and so keeping an eye on whether what you are asking has the capacity to help at this time, is really important. If we damage our relationships with others, then that won't do anyone any good. 

I talked to Dr Lisa Griffiths, CEO of OzChild, an organization that operates in the child welfare sector in Australia about what she and her leadership group do differently now that they've been through the course. Here's what she said. 

00:10:38 Lisa Griffiths

We do ask a lot of questions - they remain very open and curious. And it's really a skill for them to acquire, to be able to hold themselves there, not jump to a solution. Because particularly managers and I know that in some of the managers that I've worked with in the organization, they often want to provide a solution immediately to one of their team members that might come up to them and say or I'm struggling with this. 

And they have to hold the discipline and saying, OK, well tell me more. You know, to really kind of get to the root of what it is they're trying to achieve before jumping to offering their advice 'cause that helps with growth and development as well. 

But I think what's really been empowering for managers - they don't waste a lot of energy in the wrong areas. Because sometimes somebody might think there's a problem and they'll jump straight to a solution and go hurtling down a pathway to something and then you know only to find out that somebody else has already done that before and then it's duplication of effort. 

00:11:47 Karen Plum

This strikes me as being very consistent with a good management style and a coaching approach. We don't need to know all the answers - we need to help others to explore, to learn and to develop. To create a safe space for people to ask questions, evaluate the trustworthiness of the evidence and weigh the different sources when reaching a decision about the way forward. 

In OzChild’s case they are using evidence-based principles internally and are also becoming known in their sector as practicing an evidence-based approach. So that's spreading into the way they work with other agencies as well. 

As a change manager, I've heard many people in organisations say, “we don't do change well in this organization”. And as I worked through module 2, I started to ask myself, well, I wonder whether when change initiatives are implemented, whether people were trying to address the right problem. It's so easy to identify a problem and rush to a solution, implement some change management, and then be upset that the outcome isn't as expected. 

So I asked Eric, are we often just trying to fix the wrong problem? 

00:12:54 Eric Barends

I think it would be very helpful if you would start with the proper diagnosis and ask questions first - but what is the problem we're trying to solve and what causes this problem? Well, this is because you know our operations are actually very inefficient and we should, really redesign them and do some lean management. And then you talk with the people involved and they say, what do you mean? That’s nothing to do with our processes. 

Look at my employees, the manager then says. They're all very young. They are inexperienced, they’re straight from school or from their education or training. And the reason is we don't have enough budget to hire experienced people. We can only pay, you know, a limited amount of money, so we're forced to hire young, inexperienced people. 

As a result, they make mistakes and things are not done in the most efficient way, so it has nothing to do with lean management or redesigning our processes. It's a budget problem and now we can solve this by you give me more money so can hire more experienced people. Or we train these people. We should invest in education and training so it's not a process optimization problem. 

It's probably a financial problem, or maybe it's an educational problem or professional development problem, so that's why we ask in the module - we have the example of the door to needle time where you say what could be the reason for this problem? Because we all agreed it’s a problem, but the underlying cause could be different, and that is the same in every change management or change project. 

You really need to make 100% sure that you're solving the correct problem or you're focusing on the right cause of the problem. 

So take an evidence-based approach. Try to figure out what is exactly the problem we're trying to solve? What's the evidence we have this problem and what do we think is the root cause of this problem? And do we all agree, is there indeed evidence supporting this? Or did we jump on the change project or went into change mode and started changing things? 

00:15:17 Karen Plum

This underlines for me the importance of stakeholder involvement. When I worked in organisations, often management would hire an expensive consultant and they would end up telling them what I could have told them if they'd only asked me! And I know that sometimes hiring consultants is absolutely the right thing to do, but in any case, we need to be hearing from our stakeholders. 

Here's Eric. 

00:15:40 Eric Barends

They often have a very clear idea of what the problem is, and also probably whether a solution that is suggested - whether it will work, yes or no? Because there are a lot of context specific situations and what works for a large insurance company may not work for, I don't know an academic hospital or whatever. So you should really discuss this with your stakeholders, the people that will be affected by the decision or have experience with the situation. Do you agree that this is a problem?  Do you agree that the main cause of this problem is X? And how do you feel about the solution? Will it help? Will it be helpful? 

I'm surprised every time over and over again how knowledgeable stakeholders are and how helpful it is to just have a check with them. Just check your assumptions. 

00:16:37 Karen Plum

Similarly, practitioner evidence needs to be questioned, something that's hard to do in organisations where hierarchy and power also come into play, and it's easy to take things as read. 

Here's Lisa Griffiths again. 

00:16:51 Lisa Griffiths

When I joined the organization that was the first question I asked people - how do you know what you're doing is working? And most practitioners at the time, could say – “I don't know”. It feels like it's working and it's a relationship sector, so we tend to sort of feel that if the relationships are good then it must be working. 

00:17:13 Karen Plum

We'll look more at practitioners in the next module. Right now I just want to spend some time on takeaways from module 2. 

Like many other students I expect, I was studying the scenarios in the quiz and found myself wanting to ask more questions before picking my answers, but there was nobody to ask! And I expect everyone draws on their own experience when considering those scenarios. 

So I put this to Eric.

00:17:39 Eric Barends

It is true that you probably bring in your own experience and history and context and based on that you will probably have ideas – oh it could be this, could be that and you want to ask more questions.  When you have limited experience - we noticed when we asked questions like - is the problem sufficiently clear / very clear / somewhat clear, that they are more likely to say, yeah, I think it's clear. 

Well people would a lot of experience say, well hold on - it could be X, could be Y, or it could be, et cetera, et cetera. 

So we can see that the question, there's a lot of variation in how the questions are answered. Actually again here, take a learning approach and don't take a performance approach. The goal here is that you learn stuff. 

So answer the question, then have a look at the feedback and if you say yeah, I think it's pretty clear and the answer says, well, you know, I'm not sure because this and that etc., it's still unclear whether it could be caused by A, B or X, learn from it. 

And of course, when you're a very experienced consultant and you know when your job is to interview your client and ask a lot of questions, there are probably zillion other questions you could come up with, but keep your eye on the ball. The ball is again the evidence-based approach. 

And you want to know whether there is sufficient evidence to substantiate the assumptions being made in the organization. That's what you want to know and of course, there are other questions like you know, how does the top of the organization feel about it, did you check this, did you check that? Which are all valid good questions. 

But this is about whether the evidence base is sufficiently strong before we start changing things in organizations or before we make this big decision, and that's what the questioning is about. 

00:19:44 Karen Plum

Another aspect I wanted to explore was the logic model, and in a sense, this is about telling the story of the situation. 

Sometimes the rational, logical approach isn't the best way to tell the story, but as with many aspects of the process, this introduces some discipline so that you ensure you cover all the bases. Eric explains. 

00:20:05 Eric Barends

The main point here is you need to have a narrative behind why you want to make this decision, and that starts with the problem. We have a problem with X. As a result, this affects the performance of our organization or the well-being of our employees or it scares away customers or it harms patients or whatsoever. 

As a result, dah di dah di dah. So therefore we should do X - because if we do X, then X will go better and as a result dah di dah di dah. So there needs to be a narrative so when you ask questions again, you're asking this to figure out what is the evidence supporting the assumption? 

So first you need to know what is the problem we're trying to solve and how does this problem occur, et cetera, and then assumptions will surface and then you need to ask so, do we have any evidence? And how do we know that this assumption is actually correct? 

And then a story develops, emerges and then you have a coherent - I mean in other we call it logic model, you can also refer to it as causal mechanism or I think in science they refer to - theoretical framework - it's an hypothesis. I think this is going on as a result this occurs and if we therefore do this then that will go better. 

You need that narrative, that logic model, this hypothesis, to understand the in's and out of what you're going to do in the organization. 

But also it gives you clues about what kind of evidence you need to consult, to see whether this narrative, this logic model, these assumptions, are actually correct, or maybe they're misleading because they turn out to be actually incorrect or even flat out false. 

00:22:06 Karen Plum

Identifying the assumptions that underlie some of these steps is clearly very important. There could be some really big ones hiding in there that otherwise won't be outed. My next question to Eric was about the different options we're given when we're assessing a particular scenario.

We have options like “somewhat” or “very little”, and sometimes it's difficult to know which is correct and depending on your experience and personality, you might be more or less challenging or skeptical about those options. 

Essentially, I'm coming to the understanding that although I have to get 70% of the questions correct to pass the course, the percentage really isn't the point. The point is, that I think deeply about the answers and why there might be a better answer. 

00:22:50 Eric Barends

Because we want to trigger you and see whether there is maybe even a better answer. Again, have a look at the feedback and see what the feedback says regarding your answer and why we consider this to be an even better answer.

So that's the reason why it's there. It is actually, throughout the course and you will notice that in the module on the critical appraisal of research findings, you will be asked - how trustworthy do you consider this finding from this study? And again we have about 6 categories, like highly trustworthy, fairly trustworthy, somewhat untrustworthy, et cetera, et cetera. 

And to the despair of students - like oh my God, really? I mean what's the difference? But it's there from an educational perspective and not so much to make sure that you get it right, but that you think about stuff. I mean, the quiz is not the end part. I mean, you're supposed to learn things there. It's not only to check whether you know the answer. And by the way we tried it out with less categories, but students didn't like it and said if there is a better answer, a more nuanced answer, or when you have other thoughts about this, we would like to hear them. So actually that's why they're still there. 

00:24:19 Karen Plum

And depending on how skeptical and critical we are, maybe the evidence never looks good enough to us, and so we'll never pass those questions. Here's Eric talking about Rob Briner, CEBMa’s Scientific Director, who he doubts would pass the course. 

00:24:33 Eric Barends

Absolutely, I mean Rob Briner, he's our Scientific Director. He's there for a reason because he is one of the most critical thinkers you will ever encounter. And he's also a true academic so he will always ask questions. So why is that? And why do you think that? And what's the evidence for this? But that's an assumption. Et cetera. So yes, you can go on and on and on asking questions, and you can always be even more critical, that's absolutely true. 

But you know we're practitioners. At a certain point we need to stop and make a decision. I mean we need to get on with our business and you know, same as a physician even when you deal with patients that are seriously ill, there's always a margin of uncertainty and you can't go on and diagnose and diagnose and diagnose, and, you know, postpone your decision. 

Sometimes you just have to go and make your decision. However, ask these questions. I mean, we know that a lot of organizations don't ask any questions because they feel they need to make decision fast. That's the other end of the spectrum. I mean take some time, ask these questions, try to figure out what's the evidence base and then you move on. That's actually the best approach. 

00:25:54 Karen Plum

Finally, I think it's helpful to appreciate that some students of the course are going through the material on their own, some are undertaking it as part of a group at college or university and others are part of an organization like OzChild where they're able to discuss things amongst themselves following the completion of each module. 

When you're doing this course with colleagues, you build up a common language and understanding of what you're trying to achieve. And you can provide support and mutual coaching to each other. 

Here's our final thought from Lisa Griffiths. 

00:26:28 Lisa Griffiths

You can hear the common language that's been picked up throughout the course, and people are allowed to challenge, and one of the roles in leadership is you have to give permission for people to use the things you teach them, correct? So I don't need to remind people to ask the question, what's the problem we're trying to solve? 

Because they've learned very, very quickly that they're authorized and empowered to push back and my own executive push back on me if I ever go into, you know, let's do this. They'll say, well, let's stand back a minute now and ask, is there a problem here in the first place, or are you just coming up with something you wanted to do? And it's nice and people are very empowered to do that. 

00:27:12 Karen Plum

And that's all for this episode. I hope you've enjoyed hearing more about the ASK module and it's given you some food for thought and deeper understanding. Here's a final thought from Denise Rousseau. 

00:27:23 Denise Rousseau

The idea is to say to somebody, have you ever thought about what else there could be? You know that there is some evidence of this. I'm doing it more gently to expand their mental space, and one of the things I'm concerned with is better understanding the issue before we proceed ahead.

Being able to create that kind of space for discussion is not evidence police, but it’s certainly evidence advocate and to be trustworthy advocates that we not only are developing our skills, we're helping to develop others and develop our organization’s capability. 

So I see this as a, a constructive movement to expand beyond the current categories for thinking about problems and organizations. Evidence from experience, and from the cases that we've written, suggest it can work, but there's background preparation to make it work.