Category: learning

Learning While Building eLearning: Part 3 – Facilitating Online

Scholar Project -8This is the third of four pieces reflecting on the experiences of Emilio, a subject matter expert who was tasked with converting his successful F2F training into an elearning offering. This one focuses on the facilitation aspects of the course! You can find the context in part 1 , and part 2. (Disclaimer: I was an adviser to the project and my condition of participation was the ability to do this series of blog posts, because there is really useful knowledge to share, both within the colleague’s organization and more widely. So I said I’d add the blog reflections – without pay – if I could share them.)

I want to kick this off with a quote from the amazing Beck Tench talking about facilitating online learning:

Learning and change are super complex. Consider we may never know the effects of our work. Every snapshot lacks context in some way. Proceed with listening, kindness, observation, and experimentation. Accept that there will be uncertainty, as in all things, and move forward anyway.

I love this quote because it reminds us that facilitating online learning is about the teacher’s expertise. And about engagement. And about our stance as an online facilitator – something I think is often invisible or ignored.  Emilio stepped into that stance with a lot of grace, tolerance for the unknown and comfort with trying, learning, and even with a little failure. In my experience this is not that common!

Nancy:  Let’s talk a bit about stepping into reality, the launch of the course. This was your first time facilitating an online learning course. What happened?

Emilio: The beginning was very stressful. There was a moment where I had to reset my vision that I had created at the beginning of this project. We thought we had everything planned by the Thursday before the course. We were prepared to send a message out  to the people who had signed up for the course, expecting them to register on the actual Moodle site, and begin surfing the site and get fully on board on the first Monday of the course.

Then our partner failed to send us the list of participants in time and we had to postpone the launch. Once we got the list, we sent the welcome message on a Thursday. And yet by Monday people had not surfed the website and registered. I had to say, “wait wait, convince yourself, just don’t get frustrated.” This is what we were paying for: a pilot to experience everything, anything that can go wrong. It is better to experience it now. Next time we will do it better. That will be the real start.

This process takes a little bit of emotional intelligence. You can’t lose your focus. You have to learn in the experience. Don’t focus on the idea that this is the official worldwide launch of your elearning program, but a learning experience. So it was not a big deal. Just a couple of hours of freaking out.

Nancy: Now that you have had the experience what reflections do you have about moving and facilitating your successful F2F course? How did you engage people?

Emilio: Other than wanting to respond more quickly? (Laughter: Emilio was amazing – he was not only teaching online for the first time, but he was doing it WHILE he was on the road for work!) Here are some of my lessons.

First, what should I do about participants that belong to a group not responding to each other? I see the first person in that group posts and gets no response. I wondered, should I intervene? I wondered about how to  group participants in some way, to point out some challenges and invite others to react. But I didn’t hoping they would eventually engage. There were two groups where no one commented at all. If I were to do it again I would immediately ask others to post something.  

Nancy: There are more experiments with gamification in online, where, for example, you get points towards badges for responses. I’m not always sure of the long term benefit of these kinds of incentives and if they actually support the learning, but they appear to get people engaged in the moment. Maybe it can trigger learner socialization quicker and be something useful to explore.  Because as you noted, participation in the design of this course assumes people will interact with each other. So socialization of the group is the first step towards that participation, and later is essential for successful group work.

Emilio: Second, I can teach from anywhere. I could see that in our pilot. I was travelling like crazy. Another take away is the real leverage of technology. I could be doing different things in different places in the world and still deliver a course. You see people are learning from anywhere. If you compare that to level of effort for a F2F course, it is a trade off. But the value is there and you as an officer, can become much more productive. Once you invest in the up front work of design and planning, which was more than I expected.

There are some challenges to this anytime/anywhere though! I feel a bit guilty. I could have done a better job dedicating a bit more time overall. Once I woke up I did not realize the time difference in the office hours and had to wake up at 3am. There are a couple of times I knew I was responding two days later. I know that shouldn’t happen, how I wanted it to be. I wanted to respond within 24 hours.

Emilio: Third, include a synchronous element. The most effective tool I feel I had was our weekly synchronous “Office Hours.”  They gave me an opportunity to introduce a dose of F2F interaction which is fantastic.

During the office hours I got a chance to interact with the participants. They would post several questions. The sharing the screen was super critical. I surfed and took them where we wanted to go, to a question related to a graph or slide and explain it. You can sense by the comments – “oh yes, thank you this clarifies a lot.” We quickly solved problems.

Also, just by hearing their questions I could pinpoint those slides where the message may not be that clear and I would edit a couple of things right away. So it helped me get clearer as well.

We tried to record and post the recordings for those who could not attend due to work or time zones, but we had some technical problems. We will try and fix that next time. But I will also really encourage the participants to attend, because it brings the passion for the subject matter and the collegiality which is needed for the group work and active participation. The people who attended office hours were also the people who completed the course!

Some ideas for next time is to expand the use of office hours to help better set up the groups and the process for the group work. Maybe teams could have a private chat or meeting once a week and I could use some questions to help them get to know each other in the context of the course. That leads to my fourth learning: group work requires building relationships. Our group exercises need to be reconsidered (design) and I need to figure out how to get people comfortable enough with each other to actually engage in the group work.

Nancy: Yes, that is really hard, particularly when the participants have allocated an hour a day for three weeks and there is a lot of material to cover!

Emilio: Fifth, don’t do this alone! Milica was my assistant and she was always there. One time I could not log into the office hours and Melicia took care of it. In hindsight, we should have included her earlier in the facilitation conversations and planning. Part of the team. You and the other consultants Cheryl, Terri and everyone were very helpful.

Nancy: What was the facilitation highlight for you?

Emilio:  The first and second Office Hours were critical. The course was mostly asynchronous. I knew people were coming in. I logged in and I saw people logging in and that made it real. There are people there! They had interest, and were  asking questions, actually reading the slides. I could see the numbers (page views). But until you talk to them, see them asking questions, it is hard to see if they really are reading the material. When we held our weekly synchronous Office Hours, this became much more real.

Nancy: So would you keep doing this?

Emilio: Absolutely yes, I’ll keep doing this. Reflecting on it now, and putting into perspective from an administration standpoint,  what I produced during those four weeks of the course, there is an increase in efficiency. I delivered a course – granted for 7 people – but while I was working Bangkok, Mexico and then Peru. Pretty impressive. Amazing, yeah. I had good connectivity fortunately.

Up Next: Reflections from the whole team

Learning While Building eLearning: Part 2 – Learning Objectives & Assessment

signsThis is the second (slow!) of four pieces reflecting on the experiences of Emilio, a subject matter expert who was tasked with converting his successful F2F training into an elearning offering. Emilio has let me interview him during the process.

This piece focuses on the thorny issue of learning objectives at the front end of an elearning project and assessment at the other end. You can find the context in part 1 here. (Disclaimer: I was an adviser to the project and my condition of participation was the ability to do this series of blog posts, because there is really useful knowledge to share, both within the colleague’s organization and more widely. So I said I’d add the blog reflections – without pay – if I could share them.)

Nancy: Looking back, let’s talk about learning objectives. You started with all of your F2F material, then had to hone it down for online. You received feedback from the implementation team along the way. What lessons came out of that process? How do we get content more precise when you have fewer options to assess learner needs and interests “in the moment” as you do face to face, and with the limited attention span for online?

Emilio: I realize now that I had not thought about being disciplined with learning objectives. I had created them with care when I first developed my F2F offering. Once I had tested the course several times, I recognized that I forgot my own initial learning objectives because in a F2F setting I adapted to student’s interest and knowledge gaps on the spot, and I was also able to clarify any doubts about the content. Therefore, over time, these learning objectives become malleable depending on the group of students, and thus lost presence in my mind.

This became apparent as  I was doing the quizzes for the online work and got comments back from Cheryl (the lead consultant).  She noted where my quiz questions were and were NOT clarified in the learning objectives and content. I realized I was asking a bunch of questions that were not crucial to the learning objectives.

With that feedback, I narrowed down the most important questions to achieve and measure the learning objectives. It was an aha moment. This is something that is not necessarily obvious or easy. You have to put your mind into it when you are developing an e-learning course especially. It applies to the F2F context as well, but in an e-learning setup you are forced to be more careful because you cannot clarify things on the spot. There is less opportunity for that online.  That was very critical. (Note: most of the course was asynchronous. There were weekly “office hours” where clarifications happened. Those learners who participated in the office hours had higher completion rates as well.)

It was clear I had to simplify the content for elearning set up – and that was super useful. While my F2F materials were expansive to enable me to adapt to local context, that became overload online.

Nancy: What was your impression of the learners’ experiences?

Emilio: It was hard to really tell because online we were  dealing with a whole different context. Your indicators change drastically. When I’m in F2F I can probe and sense if the learners are understanding the material. It is harder online to get the interim feedback and know how people are doing. For the final assessment,  we relied on a final exam with an essay question. The exam was very helpful in assessing the learner’s experience, but since it is taken at the end of the course, there are no corrective measures one can take.

Nancy: Yes, I remember talking about that as we reviewed pageviews and the unit quizzes during the course. The data gives you some insight, but it isn’t always clear how to interpret it. I was glad you were able to get some feedback from the learners during your open “office hours.”

We used the learning objectives as the basis for some learner assessment (non graded quizzes for each unit and a graded final exam which drew from the quizzes.) How did the results compare with your expectations of the learners’ acquisition of knowledge and insights? How well did we hit the objectives?

Emilio: We had 17 registered learners and 7 completed. That may sound disappointing. Before we started, I  asked you about participation rates and you warned me that they might be low and that is why I am not crying. The 7 that completed scored really well in the final exam and you could see their engagement. They went through material, did quizzes and participated in the Office Hours. One guy got 100% in all of the quizzes, and then 97% in the exam.

We had 8 people take the final exam. One learner failed to pass the 70% required benchmark, but going deep into it, Terri (one of our consultants) discovered the way Moodle was correcting the answers on the multiple choice was not programmed precisely. It was giving correct answers for partially correct answers. We need to fix that.  Still, only one failed to pass the 70% benchmark even with the error.

The essay we included in the exam had really good responses. It achieved my objective to get an in depth look at the context the learners  were coming from. Most of them described an institutional context. Then they noted what they thought was most promising from all the modules,  what was most applicable or relevant to their work. There were very diverse answers but I saw some trends that were useful. However, it would be useful to have know more of this before and during the course.

Nancy: How difficult was it to grade the essays? This is something people often wonder about online…

Emilio: I did not find it complicated, although there is always some degree of subjectivity. The basic criteria I used was to value their focus on the question asked, and the application of all possible principles taught during the course that relate to the context described in the question.

Nancy: One of the tricky things online is meaningful learner participation. How did the assessment reflect participation in the course?

Emilio: We decided not to give credit for participation in activities because we were not fully confident of how appropriately we had designed such activities for an e-learning environment in this first beta test. I think this decision was the right one.

First, I feel that I did not do a good job at creating an atmosphere, this sense of community, that would encourage participation. Even though I responded to every single comment that got posted, I don’t really feel that people responded that much in some of the exercises. So I would have penalized students for something that is not their fault.

Second, we had one learner who did every exercise but did not comment on any of the posts. He is a very good student and I would have penalized him if completion relied on participation. Another learner who failed did participate, went to the office hours and still did not pass the final exam.

We failed miserably with the group exercise for the second module.  I now realize the group exercise requires a lot of work to build the community beforehand.  I sense this is an art. You told me that it is completely doable in the elearning atmosphere, but after going through the experience I really feel challenged to make it work. Not only with respect to time, but how do you create that sense of community? I feel I don’t have a guaranteed method for it to work. It is an art to charm people in. I may or may not have it!

Nancy: The challenges of being very clear, what content you want to share with learners, how you share it, and how you assess it should not be underestimated. So often people think it is easy: here is the content! Learning design in general  is far more than content and learning design online can be trickier because of your distance from your learners – and not just geographic distance, but the social distance where there is less time and space for the very important relational aspects of learning.

Up Next: Facilitating Online

Embracing Uncertainty, Diversity and Othering for Learning

For a little over 2 years I’ve been supporting an internal learning program for my colleague Pete Cranston of Euforic Services. As we wrap up this project on “Learning about learning” in an international development context, Pete and I have debriefed and he is writing a series of comprehensive blog posts on our reflections. I want to pick up on one particular aspect of our conversation that struck me through the work, and which has also come up in a number of other contexts: the role of others in our learning. How does otherness help us learn? (Dr Zuleyka Zevallos, thanks for your great resource site on otherness!)

What we observed

Instead of starting with defining otherness, I want to share my experience. One important way to learn is with and from others. This is particularly relevant in workplace learning. In international development, regardless of the domain (agriculture, health, water/sanitation/hygiene, etc.) each investment of money, time and practice holds the potential for useful learning – useful to those implementing the work themselves, and to others interested in the same work. Often these lessons get stuck locally, or even remain invisible or unconscious because they don’t get a chance to be examined by ourselves or others. It is no surprise that many find a stark lack of reflection time, alone or with others. We learn through reflection, conversation, and the modeling and mentorship of others.

Our hypothesis in the learning to learn project (LAL for the acronym lovers out there) was that simply by creating more space and opportunity for conversational interaction between project staff, new insights would emerge and existing insights would gain wider visibility and application. We had a particular emphasis on conversational practice, both face to face and online. Learning with and from each other. While there was no explicit intention for mentorship, that possibility was always open.

Through the process, we noted the following three things that made a difference in conversational learning:

  1. Come out as a learner. Declare our intention and orientation towards learning by how we prioritize time and how we practice as learners. This includes being open about not knowing, willing to take learning risks (what, you don’t KNOW the answer!!!) and practice the art of asking questions and LISTENING to answers.
  2. Recognize what we are doing is reflective practice. We call it M&E (monitoring and evaluation), reviews, and all those business process words. It is more fundamental: Stop. Think. What did we actually learn? What do we know more about now? What new questions does that uncover? Unless we are intentional about our reflective practice, the learning drips by. Another way to view this is the idea of  catching ourselves learning. Simply noticing triggers the potential and recognition of learning.
  3. Create space and leadership for learning.  We can think about making it happens as about space. Or spaces. The fastest, but perhaps least rich is surveys. The shrink the time commitment and we know time is the biggest impediment. But what about actually making spaces for reflection? Can we commit 10 minutes in a weekly meeting? Something once a month? An annual learning review? We need to PUT that stuff in, notice when and how it becomes intentional. We must lead with it from from top by role modeling and recognizing reflection as a valid, yes, IMPORTANT use of our time. Yes, stuff t happens without leaders, but leaders make it happen better by lending legitimacy. We must lead from every direction and in every corner. Reflection and learning can be recognized by donors. Without being too literal, what about 10% for M&E, 10% for communications, and 10% for learning. That leaves 70% for the DOING and probably will increase the effectiveness and value of that doing.Don’t get me wrong. Learning is also random, chaotic, free flowing, emergent, and what we learn may often be different than what we set out to learn. The LAL project included very basic funded Learning Exchange Visits where grantees visited each others projects and went into the field. They had real time with each other and stuff just happened. The immersion triggers experiences and memories that take you conceptually to a new point. They can suggest new practices, behaviors and even ways of monitoring and evaluation. In the pursuit of learning with an open agenda, we embrace uncertainty and find new learning entry points.

Adam Frank writes about the Liberating Embrace of Uncertainty, which resonates with these three things Pete and I observed:

For science, embracing uncertainty means more than claiming “we don’t know now, but we will know in the future”. It means embracing the fuzzy boundaries of the very process of asking questions. It means embracing the frontiers of what explanations, for all their power, can do. It means understanding that a life of deepest inquiry requires all kinds of vehicles: from poetry to particle accelerators; from quiet reveries to abstract analysis.

So what about otherness?

When we discussed #3, I asked Pete, what did we notice about the culture of the organizations and individuals involved? We talked about the huge value of the leader’s openness, despite a position of positional and resource power (the funder!) We talked about the diversity of project staff – geographically, organizationally, amount of experience (age), etc. Then I was reminded about a question Bron Stuckey had recently asked on Facebook :

I am wondering how you would respond to the proposition “The best person to influence the practice of a novice is someone who was most recently a novice themselves” It’s some research bubbling up for which I have a real passion. I do have to admit it is more driven by intuition and first hand experience than a body of literature or prior research 🙂 Just wondering what my respected colleagues think of this proposition. Does it fit with your research, experience and vision?

The discussion that followed Bron’s question was diverse and fascinating. I’m sharing the thread,  Bron Stuckey Learning With Others with her permission. Through it came the thread that learning happens in a (diverse) ecosystem of people, contexts and content and that who we learn with influences our learning.

The other we know and who knows us: Sometimes we learn best from people who’s experience is close to or just advanced of ours. It is a place of relative comfort.

The other we respect (and/or fear): Sometimes we learn from people with influence and power because that causes us to pay attention. Sometimes we learn from the deep experience of a expert or long-time practitioner.

The other we aren’t so sure about: Sometimes we learn from people who are different from us, hold different opinions and experiences because that asks us to examine our own understandings, biases and assumptions. This is a powerful but often more challenging set of others when we can step across mistrust and even animosity and step curiously into learning from someone who really is the “other” from us.

How otherness feeds into #1, #2, and #1

Coming out as a learner means we have to trust ourselves, regardless of the “others” we engage with. So part of our identity has to be “learner.” Reflective practice asks us to question our own assumptions, not just those of the “others.” And creating space for learning asks us to invite in the “others.” So for me, the bottom line is a strong look inward into identity, reflection and invitation. This requires us to stop our often unwitting exclusion of others, of not inviting, not listening and not engaging. Thus the opening picture. I bet you wondered why I picked it, right?

Stuff that Inspired this Post:

Symmathesy = Learning together.(Pronounced: sym- math-a-see) A working definition of symmathesy might look like this:Symmathesy (Noun): An entity composed by contextual mutual learning through interaction. This process of interaction and mutual learning takes place in living entities at larger or smaller scales of symmathesy.Symmathesy (Verb): to interact within multiple variables to produce a mutual learning context….

The way in which we have culturally been trained to explain and study our world is laced with habits of thinking in terms of parts and wholes and the way they “work” together. The connotations of this systemic functional arrangement are mechanistic; which does not lend itself to an understanding of the messy contextual and mutual learning/evolution of the living world.

Reductionism lurks around every corner; mocking the complexity of the living world we are part of. It is not easy to maintain a discourse in which the topic of study is both in detail, and in context. The tendency is to draw categories, and to assign correlations between them. But an assigned correlation between a handful of “disciplinary” perspectives, as we often see—does not adequately represent the diversity of the learning fields within the context (s). The language of systems is built around describing chains of interaction. But when we consider a forest, a marriage, and a family, we can see that living entities such as these require another conceptual addition in their description: learning.

 

P.S. Added on Friday – Catherine mentioned an image in her comment below. Here is the image!

tempoeespacios

Resource Guide to Learning Delivery Methods

resource guide to learning

What learning delivery methods are there and how do you consider them? In 2014 Scott Leslie and I supported an analysis for decision making on training strategies for the International Labour Organization’s Microinsurance Innovation Facility, now the Impact Insurance Facility (http://www.impactinsurance.org/ ).

As I moved on to other related works, I kept drawing on these initial documents. Finally, I’ve gotten around to a) getting permission from the ILO to share (THANK YOU!) and trying to pull out some of the domain specific stuff so it might be more widely useful.

The Resource Guide to Learning Delivery Methods  (pdf for now) is a version of one of the outputs of our work, shared with permission from the Facility with the hope that it adds value to your work. (The other elements include a Glossary and a Strategic Options documents.) We hope to also put these online for crowd-sourced critique and revisions. (TBA!) In the meantime, I just want to get this OUT THERE!

From the Introduction:

This document is a version of one of the outputs of our work, shared with permission from the Facility with the hope that it adds value to your work. (The other elements include a Glossary and a Strategic Options documents.) We hope to also put these online for crowd-sourced critique and revisions. (TBA!)

The Facility team was evaluating their e-learning options to expand capacity building for microinsurance for the poor. An early identified need was to understand elearning in the wider context of delivery mechanisms.

The basic content can be useful when starting to consider capacity development, training or specifically an e-learning strategy. It is not exhaustive, and some things have become dated since it was written. Understanding that, it can help you understand the range of learning options, and where they might be most effective.

We surveyed 20 different learning delivery methods across five major categories:

  • Face-to-face delivery methods
  • Online delivery methods:
    • Traditional online courses
    • MOOCs, communities & self-directed learning
    • Synchronous methods
    • Mobile delivery methods
  • Offline delivery methods
  • Blended and hybrid methods

This guide is the detailed analysis of these methods in the context of the Facility’s domain. It includes a general description, domain-related examples, and provides insight and comparisons on these methods based the qualities the organization identified as key to assessing any proposed solution. These qualities include:

  • the implications of this method on scalability and adaptability of content
  • implications on quality control
  • any effects the method may have on motivating learners’ completion of training and achievement of learning objectives
  • the ease and costs of implementing any of these methods

Your qualities may be different, so you may wish to consider your needs and how the methods may or may not meet your needs. This document is offered as a Free Cultural License as defined by the Creative Commons https://creativecommons.org/licenses/ .

If Teaching is Relational, How Does That Inform Online Teaching?

learningI was in my car the other day, thinking about all the workplace learning projects I’m currently involved in. At times, I feel like I’m moving a giant rock up a hill, with gravity being “lets push content to our learners.” I keep pointing to the importance of practice, context, reflection, informal learning. But I realize I have not sufficiently highlighted the importance  of teaching as a practice about relationships. About being human. About, liberation and yes,  about LOVE.

On our local public radio station, KUOW, ran a story about Nate Gibbs-Bowling reflecting on segregation in Washington state schools and what he is doing about it. At the core of his thesis was that teaching was relationship centric. His success with students of color was not dependent on his subject matter expertise, but his relationship with those students. If you have five minutes, take a listen: Washington Schools Are Segregated And That’s Not OK | KUOW News and Information

Nate’s experience in K-12 education rang a bell with me in my experience with workplace learning. To get the engagement that leads to gains in the application of learning, I use two things. The first is to work as hard as I can to make sure the learning offering has real, applicable relevance to the learner in doing their job. The second is to get to know them and use that relationship to engage with the learners to co-discover ways to liberate the application of the learning.

So if we believe (and hopefully can prove) that teaching is relational, what does that imply for online learning and teaching online, especially the proliferation of self paced, content centric elearning? Or worse, online teaching as enforcement and control.

It means we have to challenge the status quo of content-centricity! This does not mean throwing out content, but it means starting with relationship in the appropriate contexts.

My friend and colleague, educator Brad Beach of Australia and I have been having a years long, very slow conversation on what unlocks learner engagement online and if it varies by domain. I have changed a lot of my thinking about online interaction over time, both as I’ve learned and the environment has evolved, but one thing has always been central. Treat people like real, human beings. Use what the Dali Lama calls being “heard, seen and loved.” We may use the word “respected,” but I think it really is about love. But suit yourself! 🙂

A week ago during one of these conversations (they usually happen very early my Friday morning, Brad’s late Friday night) Brad came back again after working with some folks trying to do more vocational education online, with, as they call them in Australia, the “tradies.” He said “Nancy, you were right.” There had been a lot of push back that tradies don’t hew to this idea of relationship building online. Well heck, they do. It might look and sound different, that’s all.

I asked him, “so what does that say about the facilitation of online learning?” Brad, smart man that he is, answered “it is about good teaching. Period. Online or offline.” And together, our experience tells us that good teaching is, among other things, relationship centric.

handsRelationship Centric Practices in Online Teaching

So let’s name some of these practices. I’ll share a few of mine. Please share some of yours in the comments.

  1. Bring your whole self. A workplace learner is juggling many things. Compartmentalization takes more energy to maintain. Bring a little bit of who you are, and find out a little bit of who they are. This helps identify opportunities to liberate learning, often in unexpected ways. For me, a little goofiness goes a long way. Just a little bit.
  2. Bring your unknowingness and curiosity along with your knowledge. As adults, we are co-learners. We learn with and from each other through our conversations, activities and reflections. If we are “know it alls” we often block this co-learning.
  3. Bring human expression into all forms of communication. Use text based body language. (I’m jumping up and down in my chair as I read your response!) Add pictures and images (even silly little sketches) that not only contextualize the content, but our engagement with it and its application in our lives.
  4. Keep the content tap turned to low. This is really, really hard for me. Look at the length of this blog post. I’ll be working on this until the day I die. But pouring more on rarely is the key to engaged teaching and learning.

What do you suggest?

Edit on 2/11: Some related links: