Learning While Building eLearning: Part 3 – Facilitating Online

Scholar Project -8This is the third of four pieces reflecting on the experiences of Emilio, a subject matter expert who was tasked with converting his successful F2F training into an elearning offering. This one focuses on the facilitation aspects of the course! You can find the context in part 1 , and part 2. (Disclaimer: I was an adviser to the project and my condition of participation was the ability to do this series of blog posts, because there is really useful knowledge to share, both within the colleague’s organization and more widely. So I said I’d add the blog reflections – without pay – if I could share them.)

I want to kick this off with a quote from the amazing Beck Tench talking about facilitating online learning:

Learning and change are super complex. Consider we may never know the effects of our work. Every snapshot lacks context in some way. Proceed with listening, kindness, observation, and experimentation. Accept that there will be uncertainty, as in all things, and move forward anyway.

I love this quote because it reminds us that facilitating online learning is about the teacher’s expertise. And about engagement. And about our stance as an online facilitator – something I think is often invisible or ignored.  Emilio stepped into that stance with a lot of grace, tolerance for the unknown and comfort with trying, learning, and even with a little failure. In my experience this is not that common!

Nancy:  Let’s talk a bit about stepping into reality, the launch of the course. This was your first time facilitating an online learning course. What happened?

Emilio: The beginning was very stressful. There was a moment where I had to reset my vision that I had created at the beginning of this project. We thought we had everything planned by the Thursday before the course. We were prepared to send a message out  to the people who had signed up for the course, expecting them to register on the actual Moodle site, and begin surfing the site and get fully on board on the first Monday of the course.

Then our partner failed to send us the list of participants in time and we had to postpone the launch. Once we got the list, we sent the welcome message on a Thursday. And yet by Monday people had not surfed the website and registered. I had to say, “wait wait, convince yourself, just don’t get frustrated.” This is what we were paying for: a pilot to experience everything, anything that can go wrong. It is better to experience it now. Next time we will do it better. That will be the real start.

This process takes a little bit of emotional intelligence. You can’t lose your focus. You have to learn in the experience. Don’t focus on the idea that this is the official worldwide launch of your elearning program, but a learning experience. So it was not a big deal. Just a couple of hours of freaking out.

Nancy: Now that you have had the experience what reflections do you have about moving and facilitating your successful F2F course? How did you engage people?

Emilio: Other than wanting to respond more quickly? (Laughter: Emilio was amazing – he was not only teaching online for the first time, but he was doing it WHILE he was on the road for work!) Here are some of my lessons.

First, what should I do about participants that belong to a group not responding to each other? I see the first person in that group posts and gets no response. I wondered, should I intervene? I wondered about how to  group participants in some way, to point out some challenges and invite others to react. But I didn’t hoping they would eventually engage. There were two groups where no one commented at all. If I were to do it again I would immediately ask others to post something.  

Nancy: There are more experiments with gamification in online, where, for example, you get points towards badges for responses. I’m not always sure of the long term benefit of these kinds of incentives and if they actually support the learning, but they appear to get people engaged in the moment. Maybe it can trigger learner socialization quicker and be something useful to explore.  Because as you noted, participation in the design of this course assumes people will interact with each other. So socialization of the group is the first step towards that participation, and later is essential for successful group work.

Emilio: Second, I can teach from anywhere. I could see that in our pilot. I was travelling like crazy. Another take away is the real leverage of technology. I could be doing different things in different places in the world and still deliver a course. You see people are learning from anywhere. If you compare that to level of effort for a F2F course, it is a trade off. But the value is there and you as an officer, can become much more productive. Once you invest in the up front work of design and planning, which was more than I expected.

There are some challenges to this anytime/anywhere though! I feel a bit guilty. I could have done a better job dedicating a bit more time overall. Once I woke up I did not realize the time difference in the office hours and had to wake up at 3am. There are a couple of times I knew I was responding two days later. I know that shouldn’t happen, how I wanted it to be. I wanted to respond within 24 hours.

Emilio: Third, include a synchronous element. The most effective tool I feel I had was our weekly synchronous “Office Hours.”  They gave me an opportunity to introduce a dose of F2F interaction which is fantastic.

During the office hours I got a chance to interact with the participants. They would post several questions. The sharing the screen was super critical. I surfed and took them where we wanted to go, to a question related to a graph or slide and explain it. You can sense by the comments – “oh yes, thank you this clarifies a lot.” We quickly solved problems.

Also, just by hearing their questions I could pinpoint those slides where the message may not be that clear and I would edit a couple of things right away. So it helped me get clearer as well.

We tried to record and post the recordings for those who could not attend due to work or time zones, but we had some technical problems. We will try and fix that next time. But I will also really encourage the participants to attend, because it brings the passion for the subject matter and the collegiality which is needed for the group work and active participation. The people who attended office hours were also the people who completed the course!

Some ideas for next time is to expand the use of office hours to help better set up the groups and the process for the group work. Maybe teams could have a private chat or meeting once a week and I could use some questions to help them get to know each other in the context of the course. That leads to my fourth learning: group work requires building relationships. Our group exercises need to be reconsidered (design) and I need to figure out how to get people comfortable enough with each other to actually engage in the group work.

Nancy: Yes, that is really hard, particularly when the participants have allocated an hour a day for three weeks and there is a lot of material to cover!

Emilio: Fifth, don’t do this alone! Milica was my assistant and she was always there. One time I could not log into the office hours and Melicia took care of it. In hindsight, we should have included her earlier in the facilitation conversations and planning. Part of the team. You and the other consultants Cheryl, Terri and everyone were very helpful.

Nancy: What was the facilitation highlight for you?

Emilio:  The first and second Office Hours were critical. The course was mostly asynchronous. I knew people were coming in. I logged in and I saw people logging in and that made it real. There are people there! They had interest, and were  asking questions, actually reading the slides. I could see the numbers (page views). But until you talk to them, see them asking questions, it is hard to see if they really are reading the material. When we held our weekly synchronous Office Hours, this became much more real.

Nancy: So would you keep doing this?

Emilio: Absolutely yes, I’ll keep doing this. Reflecting on it now, and putting into perspective from an administration standpoint,  what I produced during those four weeks of the course, there is an increase in efficiency. I delivered a course – granted for 7 people – but while I was working Bangkok, Mexico and then Peru. Pretty impressive. Amazing, yeah. I had good connectivity fortunately.

Up Next: Reflections from the whole team

Learning While Building eLearning: Part 2 – Learning Objectives & Assessment

signsThis is the second (slow!) of four pieces reflecting on the experiences of Emilio, a subject matter expert who was tasked with converting his successful F2F training into an elearning offering. Emilio has let me interview him during the process.

This piece focuses on the thorny issue of learning objectives at the front end of an elearning project and assessment at the other end. You can find the context in part 1 here. (Disclaimer: I was an adviser to the project and my condition of participation was the ability to do this series of blog posts, because there is really useful knowledge to share, both within the colleague’s organization and more widely. So I said I’d add the blog reflections – without pay – if I could share them.)

Nancy: Looking back, let’s talk about learning objectives. You started with all of your F2F material, then had to hone it down for online. You received feedback from the implementation team along the way. What lessons came out of that process? How do we get content more precise when you have fewer options to assess learner needs and interests “in the moment” as you do face to face, and with the limited attention span for online?

Emilio: I realize now that I had not thought about being disciplined with learning objectives. I had created them with care when I first developed my F2F offering. Once I had tested the course several times, I recognized that I forgot my own initial learning objectives because in a F2F setting I adapted to student’s interest and knowledge gaps on the spot, and I was also able to clarify any doubts about the content. Therefore, over time, these learning objectives become malleable depending on the group of students, and thus lost presence in my mind.

This became apparent as  I was doing the quizzes for the online work and got comments back from Cheryl (the lead consultant).  She noted where my quiz questions were and were NOT clarified in the learning objectives and content. I realized I was asking a bunch of questions that were not crucial to the learning objectives.

With that feedback, I narrowed down the most important questions to achieve and measure the learning objectives. It was an aha moment. This is something that is not necessarily obvious or easy. You have to put your mind into it when you are developing an e-learning course especially. It applies to the F2F context as well, but in an e-learning setup you are forced to be more careful because you cannot clarify things on the spot. There is less opportunity for that online.  That was very critical. (Note: most of the course was asynchronous. There were weekly “office hours” where clarifications happened. Those learners who participated in the office hours had higher completion rates as well.)

It was clear I had to simplify the content for elearning set up – and that was super useful. While my F2F materials were expansive to enable me to adapt to local context, that became overload online.

Nancy: What was your impression of the learners’ experiences?

Emilio: It was hard to really tell because online we were  dealing with a whole different context. Your indicators change drastically. When I’m in F2F I can probe and sense if the learners are understanding the material. It is harder online to get the interim feedback and know how people are doing. For the final assessment,  we relied on a final exam with an essay question. The exam was very helpful in assessing the learner’s experience, but since it is taken at the end of the course, there are no corrective measures one can take.

Nancy: Yes, I remember talking about that as we reviewed pageviews and the unit quizzes during the course. The data gives you some insight, but it isn’t always clear how to interpret it. I was glad you were able to get some feedback from the learners during your open “office hours.”

We used the learning objectives as the basis for some learner assessment (non graded quizzes for each unit and a graded final exam which drew from the quizzes.) How did the results compare with your expectations of the learners’ acquisition of knowledge and insights? How well did we hit the objectives?

Emilio: We had 17 registered learners and 7 completed. That may sound disappointing. Before we started, I  asked you about participation rates and you warned me that they might be low and that is why I am not crying. The 7 that completed scored really well in the final exam and you could see their engagement. They went through material, did quizzes and participated in the Office Hours. One guy got 100% in all of the quizzes, and then 97% in the exam.

We had 8 people take the final exam. One learner failed to pass the 70% required benchmark, but going deep into it, Terri (one of our consultants) discovered the way Moodle was correcting the answers on the multiple choice was not programmed precisely. It was giving correct answers for partially correct answers. We need to fix that.  Still, only one failed to pass the 70% benchmark even with the error.

The essay we included in the exam had really good responses. It achieved my objective to get an in depth look at the context the learners  were coming from. Most of them described an institutional context. Then they noted what they thought was most promising from all the modules,  what was most applicable or relevant to their work. There were very diverse answers but I saw some trends that were useful. However, it would be useful to have know more of this before and during the course.

Nancy: How difficult was it to grade the essays? This is something people often wonder about online…

Emilio: I did not find it complicated, although there is always some degree of subjectivity. The basic criteria I used was to value their focus on the question asked, and the application of all possible principles taught during the course that relate to the context described in the question.

Nancy: One of the tricky things online is meaningful learner participation. How did the assessment reflect participation in the course?

Emilio: We decided not to give credit for participation in activities because we were not fully confident of how appropriately we had designed such activities for an e-learning environment in this first beta test. I think this decision was the right one.

First, I feel that I did not do a good job at creating an atmosphere, this sense of community, that would encourage participation. Even though I responded to every single comment that got posted, I don’t really feel that people responded that much in some of the exercises. So I would have penalized students for something that is not their fault.

Second, we had one learner who did every exercise but did not comment on any of the posts. He is a very good student and I would have penalized him if completion relied on participation. Another learner who failed did participate, went to the office hours and still did not pass the final exam.

We failed miserably with the group exercise for the second module.  I now realize the group exercise requires a lot of work to build the community beforehand.  I sense this is an art. You told me that it is completely doable in the elearning atmosphere, but after going through the experience I really feel challenged to make it work. Not only with respect to time, but how do you create that sense of community? I feel I don’t have a guaranteed method for it to work. It is an art to charm people in. I may or may not have it!

Nancy: The challenges of being very clear, what content you want to share with learners, how you share it, and how you assess it should not be underestimated. So often people think it is easy: here is the content! Learning design in general  is far more than content and learning design online can be trickier because of your distance from your learners – and not just geographic distance, but the social distance where there is less time and space for the very important relational aspects of learning.

Up Next: Facilitating Online

#SKiP16 – My Crazy Experimental Offering on Space, Media and Constraints in Visual Teaming

IMG_20160624_142629841Last Friday was a GAS! I spent the day at Sketching in Practice, an amazing offering from Simon Fraser University and led by the incredible Jason Toal. In my next post I’ll recap more of the event as a whole, but I had an amazing time offering a one hour experimental session in the afternoon exploring the impact of the arrangement of space, offering of media and provision of (or not) of task constraints in how a group works together using visual practices. This is part of my preparation for the workshop Michelle Laurie and I are offering this September in Rossland, BC, My Pens, Our Pens: engagement through participatory visualization. More and more I want more than the visual harvest of graphic recording. I want to really dig into the practices that use collaborative and shared visuals for doing real learning and work. So this was a great opportunity.

Here was the description of the session along with my initial planning sketch:

Session title: What if? An experiment to explore if/how structure, format and media influences our interactions

IMG_20160627_074439769_HDRSession overview: Are you ready to be the principle investigator and subject for a short experiment? Join us in this hands on, fingers dirty, experiment on the impact of structure, format and media influences on our interactions with each other. You will be assigned a cohort upon entering the room, with some degree of instruction and materials. You will participate for 20 minutes with that cohort. Then we’ll do a gallery walk and debrief of the experience. Magic or mayhem? Or both? Let’s explore.

As people entered the room (about 35) I assigned them a number. The stations were preset around the room with a number showing on a card. I gave the briefest of brief introductions, as I wanted this to be about the experience, not explanations. The teams dispersed and upon “go” they turned over their cards which provided a brief set of directions (or lack thereof.) The teams then had 20 minutes. The stations included:

  • Station 1: On the floor, a rich assortment of media, and task of simply drawing with no talking allowed. Media included regular markers, pan pastels and crayon markers.
  • Station 2: On a table, no specified task and use only the media provided (dark and light chocolate bars in a bag!)
  • Station 3: On table, provided media (regular markers and crayon markers), no task specified, team can only communicate by singing
  • Station 4: On table, plan a trip from Vancouver BC to Seattle using only images/no words (but you can talk) with regular markers
  • Station 5: Chairs in a circle, flip chart and pens nearby, task to identify a peer’s challenge and offer peer consultation.
  • Station 6: On table, no directions or other constraints.

It was fascinating to watch. The table with just singing as the team communication directions quickly became a pretty frustrated set of people. One person made an effort at singing, but no one else joined in. The table paper became covered with words and images of frustration, along with some comments on their appreciation of the squishy markers that go on like lipstick. The trip planning team talked about their task. When one person put down the latitude demarcation, the whole flow just burst forward and everyone started drawing. The chocolate coloring folks, after a few moments of disbelief, jumped in and we all smelled the chocolate. (I have to admit, after a while the bars looked less like chocolate than some other substance…) The no directions team started out slow but got into the swing of things with a fairly broad and abstract image.  The peer consultation group did not pull in the flip chart until I mentioned it halfway through but they appeared to be deep in conversation.

After the 20 minutes, teams had a few minutes to identify their key insights, then we did a gallery walk to each station to share insights. It was really interesting.

  • IMG_20160624_142639563Station 1: People for the most part stuck with the area of paper they started with, and did beautiful, amazing images. One participant worked bigger and provided some marks that offered opportunities for connecting the individual areas. The team felt that if they had more time, this integration direction would have really kicked in. There were some comments about how yummy the richness of media were. (PAN PASTELS!!)
  • IMG_20160624_142746673Station 2: The chocolate team had really dirty fingers! Despite the early disbelief, they embraced their medium. I think they also ate a bit of the chocolate, which seemed like a smart thing for me! Again, the initial marks by one member provided the start, role modeling that “embracing.” As we walked from station to station, this sense of the role of the “first person to make a move” proved very strong.
  • IMG_20160624_143702970Station 3: The singing only table had a fascinating discussion about the fear of singing in public and we contrasted that with the fear of drawing in public. We realized that the fear of singing made the fear of drawing seem less intense, so maybe moving people WAAAAY beyond their initial comfort zones (to singing), then stepping back (to drawing) might be a way to frame and reflect on our fears. That said, there were some lovely individual marks on the paper. AND a lot more text than I’ve seen when I’ve done this exercise before.
  • IMG_20160624_142247972_HDRStation 4: The trip planners said they really bonded as a team. They were worried they were going to be broken up to other teams and did not like that idea. Of all the teams, there was the greatest sense of “team!”
  • IMG_20160624_141959599_HDRStation 5: The consultation team had to pull in some additional chairs which were higher than the initial comfy chairs, raising the observation about power as manifest in the set up – the higher chair people felt they had to lean in more, and the lower chairs were perhaps more quiet. That said, they had a productive peer consultation. Visuals appeared to be a minimal part of their experience. It made me wonder about how explicit we need to be with both the provision of visual tools and suggestions for use. They don’t appear to be a default.
  • IMG_20160624_143218959Station 6: One tweet out of the no directions group cracked me up – something to the effect of “this is my favorite kind of direction!” They noted that there was some sense of wanting direction or leadership, but after one person made some marks, again, things flowed. However, they wanted chocolate, so they traded some markers with the chocolate folks.

My sense is that most of the people enjoyed the experience in the end, even if they experienced frustration or remorse that they were not in a particular group. There was a lot of interest in both the dynamics of the space/constraint/media question, but the new element that came up for me was the role of the first mark, who makes it, and how they make it. This sets the tone.

If this intrigues you, consider joining us for the workshop My Pens, Our Pens: engagement through participatory visualization.

Here is the photo set on Flickr:

Exercise: Implications of medium, space and constraints

Embracing Uncertainty, Diversity and Othering for Learning

For a little over 2 years I’ve been supporting an internal learning program for my colleague Pete Cranston of Euforic Services. As we wrap up this project on “Learning about learning” in an international development context, Pete and I have debriefed and he is writing a series of comprehensive blog posts on our reflections. I want to pick up on one particular aspect of our conversation that struck me through the work, and which has also come up in a number of other contexts: the role of others in our learning. How does otherness help us learn? (Dr Zuleyka Zevallos, thanks for your great resource site on otherness!)

What we observed

Instead of starting with defining otherness, I want to share my experience. One important way to learn is with and from others. This is particularly relevant in workplace learning. In international development, regardless of the domain (agriculture, health, water/sanitation/hygiene, etc.) each investment of money, time and practice holds the potential for useful learning – useful to those implementing the work themselves, and to others interested in the same work. Often these lessons get stuck locally, or even remain invisible or unconscious because they don’t get a chance to be examined by ourselves or others. It is no surprise that many find a stark lack of reflection time, alone or with others. We learn through reflection, conversation, and the modeling and mentorship of others.

Our hypothesis in the learning to learn project (LAL for the acronym lovers out there) was that simply by creating more space and opportunity for conversational interaction between project staff, new insights would emerge and existing insights would gain wider visibility and application. We had a particular emphasis on conversational practice, both face to face and online. Learning with and from each other. While there was no explicit intention for mentorship, that possibility was always open.

Through the process, we noted the following three things that made a difference in conversational learning:

  1. Come out as a learner. Declare our intention and orientation towards learning by how we prioritize time and how we practice as learners. This includes being open about not knowing, willing to take learning risks (what, you don’t KNOW the answer!!!) and practice the art of asking questions and LISTENING to answers.
  2. Recognize what we are doing is reflective practice. We call it M&E (monitoring and evaluation), reviews, and all those business process words. It is more fundamental: Stop. Think. What did we actually learn? What do we know more about now? What new questions does that uncover? Unless we are intentional about our reflective practice, the learning drips by. Another way to view this is the idea of  catching ourselves learning. Simply noticing triggers the potential and recognition of learning.
  3. Create space and leadership for learning.  We can think about making it happens as about space. Or spaces. The fastest, but perhaps least rich is surveys. The shrink the time commitment and we know time is the biggest impediment. But what about actually making spaces for reflection? Can we commit 10 minutes in a weekly meeting? Something once a month? An annual learning review? We need to PUT that stuff in, notice when and how it becomes intentional. We must lead with it from from top by role modeling and recognizing reflection as a valid, yes, IMPORTANT use of our time. Yes, stuff t happens without leaders, but leaders make it happen better by lending legitimacy. We must lead from every direction and in every corner. Reflection and learning can be recognized by donors. Without being too literal, what about 10% for M&E, 10% for communications, and 10% for learning. That leaves 70% for the DOING and probably will increase the effectiveness and value of that doing.Don’t get me wrong. Learning is also random, chaotic, free flowing, emergent, and what we learn may often be different than what we set out to learn. The LAL project included very basic funded Learning Exchange Visits where grantees visited each others projects and went into the field. They had real time with each other and stuff just happened. The immersion triggers experiences and memories that take you conceptually to a new point. They can suggest new practices, behaviors and even ways of monitoring and evaluation. In the pursuit of learning with an open agenda, we embrace uncertainty and find new learning entry points.

Adam Frank writes about the Liberating Embrace of Uncertainty, which resonates with these three things Pete and I observed:

For science, embracing uncertainty means more than claiming “we don’t know now, but we will know in the future”. It means embracing the fuzzy boundaries of the very process of asking questions. It means embracing the frontiers of what explanations, for all their power, can do. It means understanding that a life of deepest inquiry requires all kinds of vehicles: from poetry to particle accelerators; from quiet reveries to abstract analysis.

So what about otherness?

When we discussed #3, I asked Pete, what did we notice about the culture of the organizations and individuals involved? We talked about the huge value of the leader’s openness, despite a position of positional and resource power (the funder!) We talked about the diversity of project staff – geographically, organizationally, amount of experience (age), etc. Then I was reminded about a question Bron Stuckey had recently asked on Facebook :

I am wondering how you would respond to the proposition “The best person to influence the practice of a novice is someone who was most recently a novice themselves” It’s some research bubbling up for which I have a real passion. I do have to admit it is more driven by intuition and first hand experience than a body of literature or prior research 🙂 Just wondering what my respected colleagues think of this proposition. Does it fit with your research, experience and vision?

The discussion that followed Bron’s question was diverse and fascinating. I’m sharing the thread,  Bron Stuckey Learning With Others with her permission. Through it came the thread that learning happens in a (diverse) ecosystem of people, contexts and content and that who we learn with influences our learning.

The other we know and who knows us: Sometimes we learn best from people who’s experience is close to or just advanced of ours. It is a place of relative comfort.

The other we respect (and/or fear): Sometimes we learn from people with influence and power because that causes us to pay attention. Sometimes we learn from the deep experience of a expert or long-time practitioner.

The other we aren’t so sure about: Sometimes we learn from people who are different from us, hold different opinions and experiences because that asks us to examine our own understandings, biases and assumptions. This is a powerful but often more challenging set of others when we can step across mistrust and even animosity and step curiously into learning from someone who really is the “other” from us.

How otherness feeds into #1, #2, and #1

Coming out as a learner means we have to trust ourselves, regardless of the “others” we engage with. So part of our identity has to be “learner.” Reflective practice asks us to question our own assumptions, not just those of the “others.” And creating space for learning asks us to invite in the “others.” So for me, the bottom line is a strong look inward into identity, reflection and invitation. This requires us to stop our often unwitting exclusion of others, of not inviting, not listening and not engaging. Thus the opening picture. I bet you wondered why I picked it, right?

Stuff that Inspired this Post:

Symmathesy = Learning together.(Pronounced: sym- math-a-see) A working definition of symmathesy might look like this:Symmathesy (Noun): An entity composed by contextual mutual learning through interaction. This process of interaction and mutual learning takes place in living entities at larger or smaller scales of symmathesy.Symmathesy (Verb): to interact within multiple variables to produce a mutual learning context….

The way in which we have culturally been trained to explain and study our world is laced with habits of thinking in terms of parts and wholes and the way they “work” together. The connotations of this systemic functional arrangement are mechanistic; which does not lend itself to an understanding of the messy contextual and mutual learning/evolution of the living world.

Reductionism lurks around every corner; mocking the complexity of the living world we are part of. It is not easy to maintain a discourse in which the topic of study is both in detail, and in context. The tendency is to draw categories, and to assign correlations between them. But an assigned correlation between a handful of “disciplinary” perspectives, as we often see—does not adequately represent the diversity of the learning fields within the context (s). The language of systems is built around describing chains of interaction. But when we consider a forest, a marriage, and a family, we can see that living entities such as these require another conceptual addition in their description: learning.

 

P.S. Added on Friday – Catherine mentioned an image in her comment below. Here is the image!

tempoeespacios