Soft Systems Methodology: The Use of Rich Pictures from Evaluation - 이중 자막
and I'll turn it over to you.
Thank you.
Thank you.
Thank you for that Danielle.
So today I'm going to be talking about the use of rich pictures and evaluation.
Firstly, I'm going to give you a little bit of
background about how rich pictures sort of work and where they come from in the soft systems methodology.
Then give you some tips for running a session.
And a few ideas about how you might analyze the information that comes out of the pictures.
Why might we use rich pictures?
This is for evaluation practitioners looking for a tool to help visualize some of those problematic situations.
You see at the beginning of an evaluation that you want to unpack to aid in the development of an evaluation framework.
Over the next quarter of an hour, I'll discuss how rich pictures can be a useful tool when you seek to isolate key issues quickly.
when you want to avoid being overwhelmed in complex situations,
when you need to take into account emerging or changing circumstances,
when you want to ensure all key stakeholders are represented and their perspectives heard,
when you need to understand interconnections that might not be initially apparent and when you need to understand problem medical situations.
Now what we've found is that rich pictures can help you get on track first.
I found them to be a very time efficient way of generating a lot of understanding about a
particularly when you have the right people involved in developing them.
And as an aside,
I if you're struggling with the rich picture it may actually be because the
people involved don't have enough knowledge so they can be a bit of a check in that way as well.
There's very useful ways of collecting information at the beginning of the project when as an external evaluator you may not have a
deep knowledge about that material.
as it's possible to facilitate them even when you don't have that knowledge.
Our rich pictures are a really good way of showing complicated situations.
They attempt to capture a situation using a no holds barred cartoon representation of all the connections, relationships, influences, and ideas and cause and effect.
Now rich pictures are also a for us to connectively learn not only about the obvious
facts of a situation but also about the abstract or emotional things like the social atmosphere among the different actors and stakeholders.
Now, we've found that integrating the use of rich pictures into our evaluation practice
is helpful when we're setting the scope for an evaluation, when we're framing the key evaluation questions or informing the development of the evaluative criteria.
and we have written a little bit more on these things not so much using
rich pictures in this area but how we do this more in general and I think there's a
reference at the back of this slide set for you on this.
So we tend to use rich pictures at the start of an evaluation to help us better understand the territory.
Now what seem to be the barriers to using rich pictures?
If rich pictures are so hard to do well, then why might we bother with them?
Well the rich pictures come from soft systems methodology and this methodology has existed for over 40 years.
A Bob Williams has consistently said that rich pictures offer considerable value to evaluate his and indeed the soft systems methodology does.
However, it's still rich.
versus evaluators.
So one of the things I'd say about the soft systems methodology is that it contains a number of useful tools.
And what's not commonly known is that you don't need to use the whole soft systems methodology.
You can just cherry pick the parts you want.
And the literature confirms that many people are doing just this.
just using some of the tools from the methodology.
So in this presentation,
I'm suggesting just using the rich pictures tool from the soft systems methodology and using
the rich pictures at the start of an evaluation to help get a sense of the scope,
identify the key evaluation questions and feed into the development of the evaluation.
framework.
OK, but rich pictures still have a reputation for being hard to do well, is this justified?
Possibly.
In this next section we'll look at some aspects of setup and analysis that are important and will help you to use rich pictures well.
So what are the pragmatics of organising a Dura-rich picture?
As an any qualitative group,
you need to consider the group dynamics,
and as we all know, when you're working with groups, they need to go through those stages of forming, norming and storming.
People are more likely, I've found, to be able to do a rich picture once the group is formed.
That means this is not a good starting exercise and I generally do some other
kind of warm-up exercise first if I have to do them fairly early on in the session
or ideally I would actually wait till about halfway through a workshop to do
the rich picture and leave it as one of the exercises we do later on when people have formed.
Now, when drawing rich pictures, I'd suggest you generally want about three to six people working on each drawing.
Once you get beyond about six people, it's 10 to split into two groups anyway, and so it's better to just plan for that.
Also at times it's important to think who's in each group drawing the pictures and if you have a group with senior managers and support staff and a range of operational staff you will want to
mix them up at the table so that you get those different perspectives within each group drawing the pictures.
Another thing you need to consider is how confidentiality will be managed.
Some of the representations that people draw are going to be the unmentionable,
the elephants in the room, the holy cows, however you describe them in your culture, the things that people don't normally about.
and often we know that people have started to draw these when you hear a loud laugh at a
table and then you find that that's what's happened.
Someone's drawn one of these pictures and often at that point other people all pick up pens and start contributing to the drawing as well.
So it can be quite a nice process of actually getting a whole lot more people engaged in the drawing side of the picture.
So, one of the things, because we know we're going to have those unmentionables come up,
is we need to consider at the start of the process how we're going to manage confidentiality.
One of the ways that we often do this is we just stay,
what's said in the room stays in the room,
and we ask people to use what we call chatter-pouse rules,
which very much an English convention, we probably have a different way of describing this in the states.
We also need to consider the
And one of the things we've found is that sometimes people like to get us in to draw these pictures drawn rather than doing them from inside the organization,
particularly as they know that some of these issues that are going to come up may be sort of things that the evaluator may not want to address.
So I've had you to have someone like us actually running that part of the session.
Okay, so explaining what's required in a rich picture can be a bit tricky at the start.
How might you introduce the idea of a rich picture and how might you get people to know how to do it?
Luckily for us all,
the UK Open University has a great page with instructions on how to do rich pictures and how to introduce them to participants and you'll see on the chart
that you've got in front of you that there's a link at the bottom of the page and if you
click on that it will take you to all that open university.
I tend to follow their approach but not everybody does and if you have a look in Bob's latest book
he's got a different way of setting up a rich picture and running it so there are different
ways of doing these the way that I'm suggesting here is just one way and I appreciate that lots
of people have different approaches to doing them.
The key thing is that respondents are clear what the task is going to be and so I kind of break this down into two stages.
introduce the idea of rich picturing and that we want them to do a rich picture and then we tell
them about the question we want them to answer.
Now I actually use this little drawing that you can see here where you've got people going,
oh I don't know where to start, oh no, anything about rich pictures, how can I draw this?
Because these are the kind of responses,
that you get from people as they're drawing them, and injecting a little bit of it as quite a good way to see them.
that.
Once we get into the picture,
once we get to the drawing, one of the key challenges of doing rich pictures well in my view is actually getting a good question.
And I think it's worth testing the question before you round the session to make sure that there aren't
any ambiguities that are more unintended consequences of it.
And of course, that's the same as any question.
So there are two parts to the question,
I guess, and that it both needs to be general enough to capture structure, processes, issues and concerns, and the participants' roles and relationships.
and yet also specific enough to understand the problem being addressed.
In a lot of the systems thinking world, people talk about not making the whole system, but just mapping the area that is considered problematic.
And is what Sheklin talks about as well.
They if we don't need to drink the ocean, let's just focus on the part that's the area we need to be in.
Okay.
So you've given the instructions.
What happens next?
Well, often, not for sure.
And this can be the really scary part in Dona Rich picture is that there's this enormous pause after you've explained what needs to happen as people think about what they're going to do.
And I have to admit,
the first time I did a rich picture when this happened,
I thought, oh my goodness, if I actually got my question right, what's happening here?
But people do just need that time to also think about how they're going to.
And so what we find is that we just need to allow that sort of 15 seconds or so,
it will feel like the longest 15 seconds in your life possibly.
As I've already mentioned,
some people may not be sure how to start,
and the experience of drawing a rich picture can feel quite uncertain for some,
and particularly senior managers might be a little bit anxious about their performance in this kind of space.
So sometimes...
participants to suspend judgment and trust in the process and give me 15
minutes just I promise you this is going to work just just give us a of
time for it to get started and normally if we do that we find
The good news is that no matter where people start on their picture, the key thing is for them just to get started.
Some people won't want to pick up a pen,
others will love being described,
and often I find in rich picturing for the first sort of five or ten minutes,
mainly one person draws and then often after the un-mentionable comes up, other people start to join in.
And you may find by the end of this session,
most people are drawing,
but there may be one or two that really hold back from that, and I've learnt that's okay and I just go with that.
Now along the way,
it's great to mitigate feelings of uncertainty and provide reassurance that the process is working fine and that the drawings that they're doing.
Now one thing I can't stress strongly enough is that the drawing is the medium and in this
context it's used as a way to enable people to express their ideas, insights and feelings.
It's important to reassure people we're not after a masterpiece,
we just want their ideas on paper and so I often have little drawings on my slides like the one you can see here that
to make it clear that a messy drawing is fine and that all the teams are good and stick
figures and quick ways of conveying ideas are particularly what we're after.
Now, as people are drawing the pictures, there may be some moments of unsuiting to your frustration
for some who want perfect pictures, or who struggle with just drawing stick figures.
These may be people that are normally very articulate, and they find this process is kind of slowing down their communication as a woman.
I mean, it's not, but that's how they feel.
Participants have told us that it's a bit like going whitewater rafting.
You're not sure where you're going or if you can manage what's around the next corner.
So it's important to reassure them that feelings are normal during the process and that the process will be worth your while.
Now as people get more drawings on the page they can start to feel a bit over the world.
I've had people say,
oh my goodness,
I knew this was complex, but now I see it on the page, it just makes me feel tired or just gives me a headache.
Again, reassure them this is a normal way to feel during this part of the process and you know we often
find people feel that way at that stage.
Okay, so you've managed to get your group to They've been working on it for about 40 to 50 minutes.
What's next?
How are we going to analyze the data that we've collected?
Here's an example of a rich picture.
As you can see, without explanation, it's actually fairly unintended.
But those who constructed it could tell you a compelling story about why they went to the conference,
the activities they selected, and a whole lot of other stuff about this conference.
So, how are we going to interpret the information in the pictures?
As one of my colleagues has said, this is where the magic happens.
happens.
In our practice, we get participants to tell the story of their pictures once they're completed.
And I generally allow five to eight minutes per picture for these descriptions.
This is where the real richness of the method becomes apparent, is people describe their views the problematic situation to others in the room.
So there's also a shear and to us as others explore their thoughts and insights and what's happened.
And we ran a session last week where we had 24 people in the room and we had them drawing four rich pictures.
And they all had different things in their pictures, put all together.
They could really see the benefit of what they've done and we've got a really rich Now how do we capture this?
Well we don't guess what people's pictures mean.
We don't read anything into their drawings.
We audiotate the descriptions and so we'll get them to stand up.
One person in the group to stand up, describe the picture.
Two others will hold and so they can be pointing to aspects of the So we're really clear which bits of the
picture they're talking to and then we audiotape that description.
We then have that transcribed and we use that as data for our evaluation.
And we tend to look for themes, be the boundaries, perspectives or interrelationships that might inform how we proceed from here.
Now, one of the questions I had last year when I presented a brief demonstration workshop
on this at AEA was how do we show the drawings in our reporting.
And to date, we haven't used the drawings in our reporting.
For a start, we probably have to redraw them because they've got all these unmentionable things in them.
And as you saw from this, actually, I'm going here.
They're pretty unintelligible really in their raw state.
And as we said, they're actually a medium.
They're not meant to be the message itself.
They're the way we're capturing information.
So we're looking for these abstract and emotional bits of feedback that we might get, unlike the special atmosphere, amongst the different actors in stakeholders.
But in front of the start getting your questions.
And you would,
because we're finished up about 9 minutes and we want to make sure we have all the questions,
the answer that people want to ask, okay, if didn't keep going.
Okay, so it's time to wrap up now.
You're right, Johnny.
I've given you a whistle-stop tour of how you might use rich pictures at the start of an evaluation.
I've shown you that the territory is a bit challenging,
but the rich pictures can be a valuable tool in your toolkit if you persevere and accommodate the uncertainty.
I've given you some hints for how to do rich pictures well, traps to watch out for, and ideas on rich picture.
I hope you'll give rich pictures a try.
Thank you.
Okay, over to you, darling.
Thank you very much, Judy.
We a couple of questions and so bear with me a bit of a sore throat but hopefully you can hear me.
The first question is Can you explain the difference between a rich picture and a non-rich picture?
What is a rich picture?
A rich picture to me is a picture that's describing a problematical situation and so you've got
a group of people working on it and it's one way they're telling you about what's happening in the system.
So if we think back,
we were talking about in that little picture that we had of the rich picture,
we were talking about structure, processes, issues concerns and including yourself.
So the rich picture is, I mean they look very rough, but these are the components that they capture.
Got it.
Okay.
So the next question we have, David is asking, how long do you meet that people go through this exercise?
How long do you actually to take pencil to pay for?
So, typically, I'd allow between 30 and 40 minutes.
It depends on the complexity of the situation.
The one we were doing last week,
we thought we'd allow them 45 minutes, but in fact, we ended up allowing 55 minutes because we could see that they weren't finished.
So we had another activity at the end that was optional that we could pull out if we needed to, and that's what we did.
Okay, great.
We have another question.
Have you ever used rich pictures in a pre-post design that people can do their pictures and then you do an intervention or whatever and then
you can look at the rich pictures again and see what the differences are?
Now, I haven't done that, but I've done something that was fascinating, which was where we got
people who were the most implementers,
the people that developed the policy and the people that implemented the policy to come together and draw a rich picture together of what happened.
And that was really useful, actually, that was one of the most useful ones I think we've ever done.
So that's a great idea during the pre and post.
I really like it.
Great.
Can you say something about the instructions that you give people?
He says, Alan, what do you tell them when you get this thing started?
I think that to do that would actually take quite a lot of time,
so I'd refer you to the website,
have a look there,
and this is something that I might do a little,
another session on at some point in a bit more detail, it would actually take me longer than I could do here right now.
Okay, fair enough, what can you tell people about the boundaries of a rich picture?
I mean, they're presumably the thing you want in and out.
Well, if you framed your question right, that will come out OK.
And the point, is getting that question right so that those boundaries are there.
And where they put the boundaries,
where we're interested and where they see the boundaries being around a because sometimes things are happening because of something you may not know of.
And going to be one of the magics of the picture.
So we don't put boundaries around them like that.
All right, now this is an interesting question, mostly because it's about something I've never heard of before.
The question is how rich pictures compare with anecdote circles.
I don't know what anecdote circles are.
Make up an answer for us.
I can't, sorry.
Okay.
Can you give an example of the kinds of instructions of bombs that you give people to get them?
Okay, so, well I normally just actually stand back and give them space and one of the things
I've found in those first sort of two or three minutes when they're really struggling,
it's best to just keep out of it actually and let them almost sort of self-setal and write themselves
a bit because it can be a little bit tricky for them getting into the space.
Sometimes if they say, you know, we don't know whether to draw this or whether to draw that, I'll say, well, what do you think?
What would you put at the future of this?
Or I might say it doesn't matter if it's in the middle of the picture,
this is all going to work out anyway, just get started.
So I tend to forcing them on on whatever they're doing and just just encourage them at that point
because it will kind of sort itself out.
Later on you might give them a prompt of the something that they haven't raised in the picture that other people seem
to have in their pictures but at the same time what we really wanting to do here is get their perspective.
So it's kind of like interviewing in away.
It's to be a bit careful not to prompt too much, but just to keep them on track.
Okay, fair enough.
Let's see,
we have an endorsement here about how to use this with the new tour involved in the polling program and she loves
it and would do it again as they say.
Is it possible to use that kind of a electronic media instead of markers and crayons and pencils?
Have you ever tried using smart cores or anything?
I haven't tried that yet.
When we did the ones in NAEA last year,
we crayons and afterwards Bob said to me, for goodness sake, get marker pens, they're much better and I agree.
I tend to use big fat marker pens but I also give people a wide range of coloured pens so that they can choose what they like.
And if I've got someone who's not participating but clearly needs to be,
I'll do something silly like give them a pink pen or you know and jolly them along a bit.
And often they'll reject the pink pen, but they might pick up another one.
I have never used the rich pictures with individuals rather than with groups.
I haven't yet and that's a really interesting idea.
That what I talked with Bob about and he says that sometimes he'll draw a rich picture himself to try and make a situation.
So I haven't tried that yet but you know that could be interesting.
Okay, here's the question, can you give the talk a little bit about sort of carrying the rich picture through the evaluation and you do it upfront, but presumably to generate some knowledge or information that's
used to be evaluation goes on or not.
Yeah no we do tend to use them as it goes on because it is unclacking the problematic situation for people.
So we did some rich pictures last year when we were doing an evaluation for one of our clients and it really unpacked what a
lot of the issues were and gave us a much deeper understanding than we would have had in an hour and a half otherwise.
The other reason we've started using them is there's such a quick way to get out a lot of information and I don't know about you
but we're finding that we can never get more than about two hours max these days with senior management teams.
So finding something really what actually drove me to really work with them and grip them up.
Okay, excellent.
But we have a couple minutes left, so parting words of wisdom in the assembled multitudes here?
I read quite a bit at the start of using rich pictures about how you
might analyze the actual drawings themselves and I have given you some resources at the
end I'll just put these up now so people can see
I've put some results at the end that talk about how people use the ideas about the icons and how you might read into them,
but I've tended personally not to do that myself.
I came out of a market research background where we used enabling techniques for branding
and marketing and saw the power and letting people just express how they see things and
then also learnt very early on in an example where a young boy had an
drawing for me and I thought it meant one thing and it actually meant something completely different to him,
that it's so important that we actually ask people to unpack their pictures for us.
As I said,
one of my colleagues,
when he first came to one of the rich pecturing sessions, he afterwards, wow, I didn't know this was going to work so well.
And he said, the magic really happened when you got those people to explain their pictures to each other.
There was such a transferring of knowledge and we got so much out of it as well.
So that's the part of the process,
I guess I'd recommend that you don't skimp on, allow plenty of time for that in your first view we can throw.
Okay, well, it's 12.30 and we didn't promise to finish up by 12.30, I guess, if people want to stay off longer, that be great.
Let me put it up to our, we are always looking for speakers.
So if you want to be a presenter,
or know people who might want to be presenters, if have subjects and topics, then you think we ought to cover.
As I like to say, you know where we are.
Send us an email.
Give us a call.
I promise we'll accommodate people.
We'll certainly try.
Then, as you can see, we do have, we are finished, although, Julie, if want to
stay on for a few more minutes in case anybody has any questions, hope all we have are kudos.
Thanks, Julie.
Excellent presentation.
I would certainly, certainly agree with you.
Ah, someone else is very helpful.
I mean, you quite a fan club today.
Well, listen, thank you to everybody for tuning in here.
Ah, are you able to get copies?
Yes, the entire presentation, the slides and the audio will be on the website, on the webinar website.
And I don't know if you all the speakers,
by the way, you can see us on YouTube, and you can get copies of our slides.
And so I don't know.
So, when Danielle will put that link up, I guess pretty soon, I don't know.
Yeah, we usually do it right away.
Good.
As I said, thank you to everybody.
We're glad you got something out of this.
We are looking for copies.
We looking for speakers.
Don't be shy.
Give us recommendations.
Thank you all again.
And Judy, you Bye bye.
We forget up early, I suppose, for all of us people in very different time cells.
Okay.
I will affect all the questions.
더 많은 기능 잠금 해제
Trancy 확장 프로그램을 설치하면 AI 자막, AI 단어 정의, AI 문법 분석, AI 구술 등을 포함한 더 많은 기능을 사용할 수 있습니다.
인기 있는 비디오 플랫폼과 호환
Trancy는 YouTube, Netflix, Udemy, Disney+, TED, edX, Kehan, Coursera 등의 플랫폼에서 이중 자막을 지원하는데 그치지 않고, 일반 웹 페이지에서 AI 단어/문장 번역, 전체 문장 번역 등의 기능도 제공하여 진정한 언어 학습 도우미가 됩니다.
다양한 플랫폼 브라우저 지원
Trancy는 iOS Safari 브라우저 확장 프로그램을 포함하여 모든 플랫폼에서 사용할 수 있습니다.
다양한 시청 모드
극장, 읽기, 혼합 등 다양한 시청 모드를 지원하여 전체적인 이중 자막 체험을 제공합니다.
다양한 연습 모드
문장 청취, 구술 평가, 선택 공백, 테스트 등 다양한 연습 방식을 지원합니다.
AI 비디오 요약
OpenAI를 사용하여 비디오 요약을 생성하여 핵심 내용을 빠르게 파악할 수 있습니다.
AI 자막
3-5분 만에 YouTube AI 자막을 생성하여 정확하고 빠른 자막을 제공합니다.
AI 단어 정의
자막에서 단어를 탭하면 정의를 검색하고 AI 단어 정의 기능을 활용할 수 있습니다.
AI 문법 분석
문장에 대한 문법 분석을 수행하여 문장의 의미를 빠르게 이해하고 어려운 문법을 습득할 수 있습니다.
더 많은 웹 기능
Trancy는 비디오 이중 자막 뿐만 아니라 웹 페이지의 단어 번역 및 전체 문장 번역 기능도 제공합니다.