Assignment

Choose any digital learning tool currently on the market.  Explore it, poke at it, twist it and see if you can break it (in a pedagogical sense, not a technical one).  When you have a good sense of what it does, write a description of the tool, including the intended learners, content, and approach to learning.  What are its strengths and weaknesses?  How should it be evaluated? How could it be improved or extended?  2-3 pages

Response

Synopsis

Udemy is an online course marketplace who’s mission is to “help anyone learn anything” according to their website which also states that every course is “available on-demand, so students can learn at their own pace, on their own time, and on any device.” The platform caters to learners and businesses offering over 35,000 courses ranging from photography to mobile development. At it’s core, Udemy offers an online course publication tool that allows instructors to create their courses and put them up for sale both on Udemy’s marketplace and the instructor’s own website. The instructor sets the selling price and shares the revenues with Udemy at varying rates, depending on who initiated the sale. The instructor keeps 97% of the revenue if the sale originated from their own website and 50% if the sale originated from Udemy’s website.

Learning

Besides the obvious focus on the students, Udemy has a significant focus on the instructor, offering several resources to aid instructors in creating courses. To start with, Udemy offers a free “How to Create Your Udemy Course” which utilizes the platform itself to deliver it. A support website is also available offering several articles such as “Getting Started: How do I create my Udemy Course?”. There is also a closed Facebook group is available for instructors to share experiences, get help and learn from each other. These resources focus on planning, producing, publishing, and promoting the instructor’s courses.

The designers seem to believe that instructors, as learners of the tool, need to understand how teaching online is different from teaching in a classroom. The support material focuses on guiding the instructors on best practices, media quality, and pedagogical styles that best work in this environment. On top of these resources prior to creating a course, Udemy enforces a course review process once the course is ready. This process entails a detailed inspection of the quality of the media, the course content organization, as well as the frequency of different media utilized. For example, a course with only text, only slides, or only videos – will be rejected. A mix of media, quizzes, and presentation styles is therefore valued by Udemy as essential for the learners (students) to succeed.

Content

Judging by the content presented, designers see as barriers to developing an understanding of the subject matter is course planning and digital literacy. Starting with guiding the instructors on learning objectives and general planning of the course, the designers offer basic pedagogical knowledge. Moving on to the production of the course, the designers offer detailed instructions and specifications on audio and video size and quality as well as filming and editing tips, for example. Publishing instructions are also offered guiding instructors on pricing strategies, free course previews and other information about how to make the course more attractive to students. Finally, Udemy provides suggestions on how to drive sales of the course.

Following the content on creating the course, Udemy continues with guides on how to utilize the tools they offer on their digital platform. A strict “Course Quality Checklist” is presented as well as the “Udemy Studio Code of Conduct” which details what is allowed, encouraged as well as what is frowned upon. Interestingly enough, the last session in the “Udemy Teach” section of their site includes “Coding Exercises” which talks about how to create exercises, validate and checking student’s code, and a few example exercises for Javascript, Html, and CSS. This shows a tendency of online courses be heavily geared towards programming courses. My personal guess to why this happens: programming instructors and students have a higher digital literacy and comfort around technology. It is probably harder to find a tech savvy Yoga instructor that publishes an online course as it is to find a Yoga student looking for a strictly online course on Udemy. A quick search shows 155 Yoga courses versus 557 ‘programming’ courses along with 683 ‘development’ courses. Times are changing. 

Technology

The features the designers are leveraging in this implementation revolve around cloud storage and Ajax. Cloud storage means that all the content is uploaded to Udemy’s platform and stored in their environment – including videos – for no extra charge to the instructor. This allows complete control of the content and delivery quality of the courses. Ajax is a ‘modern’ technique of creating web pages that allow dynamic loading of content, draggable elements, and addition of new sections without the necessity of reloading the page. This provides a fluid and intuitive interface that makes the job of creating the course content actually pleasurable.

On the student’s end, the interface is also intuitive, clean, and easy to use. Each section of the course is presented without distractions and provides clear actionable items to control the playback.

Assessment

The success of this tool is publicized on their web page with numbers such as 9 million students, 35 thousand courses, 19 thousand instructors, 35 million course enrollments, 8 million minutes of video content, and 80 languages. Although these are all big numbers by any standard, I would also be interested in looking at the following numbers:

1. Growth rate of the number of instructors joining the platform
2. Time between account creation and course publication
3. Average number of courses published per instructors
4. Average revenue per course
5. Course completion rates by students

I would also be very interested in interviewing instructors who have published courses on other platforms to understand what Udemy’s course publication tool is doing right or wrong. From personal trial and error, I’ve found Udemy’s interface the easiest to use and the one that provides the most scaffolds for the instructor. Their review process is also extremely helpful with attention to minimal details showing that there are actual humans reviewing the course content. This ensures course quality for the students and gives the company a high level of credibility as well as showing their care towards the learner.

Evaluation

Scale: 0 – Absent, 1 – Minimal, 2 – Strong, 3 – Exemplary

The tool is making effective use of unique features of this technology.

2 – Strong: Udemy’s uses the latest Html techniques to provide a good user experience. I would have judged it exemplary if there were a drawing tool embedded in the platform – something like a white board that would record my strokes and voice over from within the tool.

The features of the tool demonstrate an understanding of the target learner.

3 – Exemplary: Udemy course publication tool is setup in a way that it asks for information from the instructor in a structured and familiar manner using terminology commonly used by teachers such as course goals, course summary and other features one would expect in a pedagogical tool.

The design of the tool suggests an understanding of the challenges unique to learning the target content.

3 – Exemplary: Udemy’s wide variety of content, tools and possible interactions amongst instructors show a great care towards the main driver in education – the instructor. They understand that teachers, educators, and subject matter experts may not have all the TPACK necessary to become an online instructor. To supplement this, they try to provide content in various formats with several examples and support for them.

“Activities, tasks, functions, and understandings do not exist in isolation; they are part of broader systems of relations in which they have meaning. These systems of relations arise out of and are reproduced and developed within social communities, which are in part systems of relations among persons. The person is defined by as well as defines these relations. Learning thus implies becoming a different person with respect to the possibilities enabled by these systems of relations. To ignore this aspect of learning is to overlook the fact that learning involves the construction of identities.” (Lave and Wenger, 1991, Ch. 2)

Regardless of your current profession and experience, you have been impacted by education and technology. As we progress in our society, we must think how might we deliver the best educational content, implement the most effective teaching methodologies (pedagogy), and the utilize tools that engage both learners and educators in meaningful learning experiences. Education is one of the most complex issues in our society and has been since the beginning of civilization. Without education, how does a community, a company,  a country, and the human race progress? This paper, along with the Pedagogical Compass (https://prezi.com/zgdhgwrlealw/) will present an overall view of who are the stakeholders, how education happens for educators, how learning happens, and what might we select as relevant content for the future.

Even if you are not directly involved in education, you certainly have faced the need to teach someone, explain how something works, train a new employee, present your research, your work, or your thoughts. With this in mind, we propose to look at educational tools with a set of lenses that might provide an encompassing view when designing effective learning tools. The Pedagogical Compass looks at what we teach (North), how we teach (South), how we learn (East), and who we learn from (West). Through these four cardinal positions we might facilitate and hopefully stir your thought processes based on current research, learning theories, and experiments done in the field.

If we look at user experience designers, we generally consider a tool’s graphical layout, the affordances provided by the tool, it’s usability or ease of use, and finally the service and/or outcome the tool offers. Game designers go a step further in looking at how the user repetitively engage with the tool, reward systems, and how the gamer learns and progresses in the gameplay. One effective framework to use is the “Core Loop” which looks at every step of engagement one has within a game. It involves a cycle which starts with 1) assessment of the current scenario, 2) choosing the correct action, 3) aiming your action appropriately, 4) launching your action, 5) being rewarded (or not) by the consequences of your action. Once rewarded, you go back to step 1 where you assess your next move. By identifying the elements in each of the loop’s nodes we are able to better visualize the process and hopefully improve it. What happens between these nodes should also be considered in order to change the speed of the loop’s cycle.

This approach can be particularly useful in designing a learning tool. The learner, when engaging with new content or knowledge that must be acquired, will first assess what is known, what resources are available and what needs to be achieved. Second step is to choose a potential approach to absorbing the content such as reading, taking notes, and discussing the subject matter with colleagues. Once the action is chosen, one must aim at the appropriate content to engage with, launch your action and finally be rewarded by learning, understanding, and/or comprehending the content. We then continue back to the first step where we assess once again what we know, what we should do, how to apply it, take action, and be rewarded by the results. Yet designing a learning tool is not limited to the learner’s core loop. Learning happens to someone, within a social and cultural context, setup by a teacher, guide, or environment, and the interactions of these elements.

Going back to our Pedagogical compass, let’s first look at what we teach. Is it useful teaching quantum physics to a learner who’s talents lie primarily in the artistic realm? Will a certain content be helpful to get a job or to function better in society? It seems more than plausible to “focus what and how we teach to match what people need to know” (NETP, 2010). Therefore, when designing a learning tool, we must first consider the ultimate goal – the learning objective and outcomes. This approach has been coined by Walters & Newman, 2008 as backward design:

“This backward approach encourages teachers and curriculum planners to first think like an assessor before designing specific units and lessons, and thus to consider up front how they will determine whether students have attained the desired understandings.” (Walters & Newman, 2008)

“One starts with the end—the desired results (goals or standards)— and then derives the curriculum from the evidence of learning (performances) called for by the standard and the teaching needed to equip students to perform.” (Walters & Newman, 2008)

By preemptively defining how we will evidence the intended learning, we might do a better job when designing and refining each step and activity along the learning/teaching experience.

Now that we have our learning plan in place, how might we effectively transmit this to our learners? How do we teach more effectively? Can we simply use technology to do so? Can we eliminate the teacher from the process? This technocentric approach, where one believes that technological tools alone will transfer knowledge to students is widely criticized. Yet technology makes us rethink education and the role of the teacher in a more profound way:

“Combating technocentrism involves more than thinking about technology. It leads to fundamental re-examination of assumptions about the area of application of technology with which one is concerned: if we are interested in eliminating technocentrism from thinking about computers in education, we may find ourselves having to re-examine assumptions about education that were made long before the advent of computers. (One could even argue that the principal contribution to education made thus far by the computer presence has been to force us to think through issues that themselves have nothing to do with computers.) ” (Papert, 1987)

Therefore we must not only look at the tool but how we use it, and how we interact with the learners when engaging with the content. Learning is a continuous process, a technique acquired that will leverage further and future learning – learning how to learn. Learning that it is possible. One is not born with a certain and immutable level of intelligence. Believing this fixed notion of intelligence is potentially harmful and limits learners to put in the effort into the task. If the learner believes that progression is not possible, it becomes a self-fulfilling prophecy. The simple act of praising or criticizing one’s ‘intelligence level’ instead of nurturing the process of learning may prevent learners from having a ‘growth mind-set’ and promoting self-guided interest in development of one’s knowledge base:

“I think educators commonly hold two beliefs that do just that. Many believe that (1) praising students’ intelligence builds their confidence and motivation to learn, and (2) students’ inherent intelligence is the major cause of their achievement in school. Our research has shown that the first belief is false and that the second can be harmful—even for the most competent students. ” (Dweck, 2007)

“Understanding that interest can develop and that it is not likely to develop in isolation is essential. Further articulating the contribution of interest to student learning and its relation to other motivational variables has potentially powerful implications for both classroom practice and conceptual and methodological approaches to the study of interest. ” (Hidi & Renninger, 2006)

Now that we have glossed over what we should learn and how we might teach it, we can look at how do we actually learn. Learning is a natural and innate process. We learn how to speak, how to walk, how to interact with our environment, and how to behave in society. We learn not only in formal environments such as schools and training centers, but also in the interaction with others. If we look at children playing video games, research shows that they are naturally learning how to play the game, how to collaborate, and interact with each other with the goal of enriching their experience:

“For these reasons, we do not appeal to the games-are-highly-motivating explanation, but we do see a reason that young people play games and get them tangled up with the rest of their lives, and this reason is cultural. The phrase that best helps us explain it comes from one of our participants, Mikey, who in talking about games said, “It’s what we do.” The “we” he was referring to was kids these days, the young people of his generation.” (Stevens, Satwicz, McCarthy, 2008)

Another powerful concept is that we learn by teaching. What better way to understand a concept but to explain it to someone else? Not only must we utilize our metacognition to access the key elements, but we must articulate in a clear manner so that others can grasp the knowledge at hand. On top of that, humans naturally seem to care more about helping others than helping themselves. An increased level of responsibility and engagement with the content when teaching others is tapped into – it’s called the Protégé Effect. The research looked at how children taught a Teachable Agent (TA) and how this affected their own content acquisition.

“We then introduce TAs, which combine properties of agents and avatars. This sets the stage for two studies that demonstrate what we term the protégé effect: students make greater effort to learn for their TAs than they do for themselves.” (Chase, Chin, Oppezzo, Schwartz, 2009)

“Given our hypothesis that the protégé effect is due to social motivations, we would expect students in the programming condition to be less inclined to acknowledge ” (Chase, Chin, Oppezzo, Schwartz, 2009)

Finally, but not less importantly, we should look at who we learn from, beyond the teacher in it’s most traditional definition. Research shows, along with our common knowledge, that we learn from our peers, from our environment, from the media that we consume and the interactions we engage in while doing so: Joint Media Engagement – the new co-viewing (Takeuchi and Reed Stevens, 2001):

“The variety of ways that we saw young people arrange themselves to play games surprised us, especially since most of these ways were interpersonally and emergently organized by the young people themselves.” (Stevens, Satwicz, McCarthy, 2008)

“Parents, teachers, and other adults may wish to share educational resources with their children, but teaching with media and new technologies doesn’t always come naturally, not even for experienced instructors. Provide guidance for the more capable partner in ways that don’t require a lot of prior prep or extra time, actions that can help ensure that the intended benefits of the resource are realized. ” (Takeuchi and Reed Stevens, 2001)

With this is mind, the role and actions of the teacher is greatly expanded and complicated since it must consider not only what is happening inside the classroom but also outside the classroom. Engaging students, triggering and maintaining their interest in the content is a great challenge that can be modeled by the Four-Phase Model of Interest Development developed by Hidi & Renninger, 2006, which looks deeply into how interest progresses from an initial casual level of engagement to a more deeply involvement with the subject matter, where the teacher’s role is to provide positive feelings towards the content, generate curiosity to encourage further research, provide opportunities for learning by offering content and pointers towards meaningful resources, and a guide on research to enable learning progression. By providing this, the interest level of a learner will move from Triggered Situational Interest to Maintained Situational Interest to Emerging Individual Interest and finally to a Well Developed Individual Interest.

Designing learning tools might be the most complex challenge we face in our society, not only from a pedagogical standpoint. We must look at the scalability of teaching, content relevance, socio-cultural implications, cognitive developmental stages, interaction with peers, policy, assessments, teacher professional development, costs, and implementation – to list a few. We invite you to become part of this ever evolving field, take on the challenge of creating a better future for humanity, develop, implement and research how might we help spreading knowledge across the world in an effective, considerate and meaningful way. We need designers, teachers, engineers, developers, psychologist, philosophers, doctors, lawyers, leaders, and anyone with a desire and drive to share knowledge and improve the tools we have to do so.

Reading Notes

National Education Technology Plan

• Focus on technology but need to use it for PD
• Focus Areas:
  • Learning
  • Assessment
  • Teaching
  • Infrastructure
  • Productivity

Understanding by Design

• Backwards design or backwards planning
• Clear learning objectives
• How could we incorporate game design practices into education?

Computer Criticism vs. Technocentric Thinking

• Ed Tech is not the silver bullet – must come with pedagogy and PD

In-Game, In-Room, In-World

• Kids learn plenty from each other
• Kidification of education

 The Perils and Promises of Praise

• Growth mindset
• Constructive praise – effort and process not ability itself (you’re so smart!)

Four-Phase Model of Interest Development

• Model
  • Triggered Situational Interest
  • Maintained Situational Interest
  • Emerging Individual Interest
  • Well Developed Individual Interest
• Teacher’s interest is probably best predictor of effective teaching
• Teacher’s role is to provide:
  • Positive feelings
  • Generate curiosity
  • Provide opportunities
  • Guide on research

 The New Coviewing: Joint Media Engagement

• Design Guide
  • Mutual engagement
  • Dialogue inquiry
  • Co-creation
  • Boundary crossing
  • Intention to develop
  • Focus on content, not control
• Challenges
  • Parents too busy
  • Parents unaware of needs
  • Don’t enjoy the same content
  • Desired interactions not always triggered
  • Little continuity into other family activities
  • Distraction are always present
• Design principles
  • Kid driven
  • Multiple plains of engagement
  • Differentiation of roles
  • Scaffolds to scaffold
  • Trans media storytelling
  • Co-creation
  • Fit
• “What goes on between people around media can be as important as what is designed into the media”

Teachable Agents and the Protégé Effect

• Care more about pleasing others than oneself, so having someone you need to help enhances learning through teaching this person

 Tangible Bits: Beyond Pixels

• Tangible User Interfaces

 Horizon Reports

• re-teaching our teachers how and what to teach

Paper Planning

Pedagogical Compass

North – what we teach

• Content relevance
  • “focus what and how we teach to match what people need to know ” (NETP, 2010)
  • “It leverages the power of technology to provide personalized learning and to enable continuous and lifelong learning. ” (NETP, 2010)
  • “Build tools and experiences that revolve around a child’s existing interests, not just prescribed topics. To do so, producers need to design mechanisms that make children’s interests visible and can assist adults in responding to them. ” (Takeuchi and Reed Stevens, 2001)
  • “Joint media engagement can be a useful support for developing literacy, including basic reading ability, cultural literacy, scientific literacy, media literacy, and other 21st century skills.” (Takeuchi and Reed Stevens, 2001)
• Assessment
  • “technology-based assessments can provide data to drive decisions on the basis of what is best for each and every student and that, in aggregate, will lead to continuous improvement across our entire education system. ” (NETP, 2010)
  • “This backward approach encourages teachers and curriculum planners to first think like an assessor before designing specific units and lessons, and thus to consider up front how they will determine whether students have attained the desired understandings.” (Walters & Newman, 2008)
• Teacher’s interest
  • Teacher’s interest is probably best predictor of effective teaching – Lucas

 South – how we teach

• Teacher Professional Development
  • “Professional educators are a critical component of transforming our education systems, and therefore strengthening and elevating the teaching profession is as important as effective teaching and accountability. ” (NETP, 2010)
• Curriculum construction
  • Backwards design or backwards planning – Clear learning objectives
    • “One starts with the end—the desired results (goals or standards)— and then derives the curriculum from the evidence of learning (performances) called for by the standard and the teaching needed to equip students to perform. ” (Walters & Newman, 2008)
• Using technology wisely
  • “Assigning roles to participants so that tasks and content match up to individual maturity is another way of ensuring that everyone is suitably challenged and/or entertained.”  (Takeuchi and Reed Stevens, 2001)
  • “Parents, teachers, and other adults may wish to share educational resources with their children, but teaching with media and new technologies doesn’t always come naturally, not even for experienced instructors. Provide guidance for the more capable partner in ways that don’t require a lot of prior prep or extra time, actions that can help ensure that the intended benefits of the resource are realized. ” (Takeuchi and Reed Stevens, 2001)
  • “Mark Weiser’s seminal paper on Ubiquitous Computing [54] started with the following paragraph:
    “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”
    I do believe that TUI is one of the promising paths to his vision of invisible interface. ” (Ishii, 2008)
  • “Combating technocentrism involves more than thinking about technology. It leads to fundamental re-examination of assumptions about the area of application of technology with which one is concerned: if we are interested in eliminating technocentrism from thinking about computers in education, we may find ourselves having to re-examine assumptions about education that were made long before the advent of computers. (One could even argue that the principal contribution to education made thus far by the computer presence has been to force us to think through issues that themselves have nothing to do with computers.) ” (Papert, 1987)
• Student feedback
  • Praise effort and not ability:
    • “I think educators commonly hold two beliefs that do just that. Many believe that (1) praising students’ intelligence builds their confidence and motivation to learn, and (2) students’ inherent intelligence is the major cause of their achievement in school. Our research has shown that the first belief is false and that the second can be harmful—even for the most competent students. ” (Dweck, 2007)
    • “Maybe we have produced a generation of students who are more dependent, fragile, and entitled than previous generations. If so, it’s time for us to adopt a growth mind-set and learn from our mistakes. It’s time to deliver interventions that will truly boost students’ motivation, resilience, and learning. ” (Dweck, 2007)
• Managing motivation and student interest
  • “In fact, teachers often think that students either have or do not have interest, and might not recognize that they could make a significant contribution to the development of students’ academic interest (Lipstein & Renninger, 2006)” (Hidi & Renninger, 2006)
  • “In general, findings from studies of interest suggest that educators can (a) help students sustain attention for tasks even when tasks are challenging—this could mean either providing support so that students can experience a triggered situational interest or feedback that allows them to sustain attention so that they can generate their own curiosity questions; (b) provide opportunities for students to ask curiosity questions; and (c) select or create resources that promote problem solving and strategy generation. ”  (Hidi & Renninger, 2006)
  • “Understanding that interest can develop and that it is not likely to develop in isolation is essential. Further articulating the contribution of interest to student learning and its relation to other motivational variables has potentially powerful im- plications for both classroom practice and conceptual and methodological approaches to the study of interest. ” (Hidi & Renninger, 2006)
• Trends (NMC Horizon Reports)
  • Blended Learning
  • Open Educational Resources
  • Digital Literacy
  • Integrating Technology in Teacher Education
  • Rethinking Roles of teacher

 East- how we learn

• Access to education
  • “The underlying principle is that infrastructure includes people, processes, learning resources, policies, and sustainable models for continuous improvement in addition to broadband connectivity, servers, software, management systems, and administration tools.” (NETP, 2010)
• Growth mindset
  • “Other students believe that their intellectual ability is something they can develop through effort and education. They don’t necessarily believe that anyone can become an Einstein or a Mozart, but they do understand that even Einstein and Mozart had to put in years of effort to become who they were.” (Dweck, 2007)
• Learn from culture
  • “For these reasons, we do not appeal to the games-are-highly-motivating explanation, but we do see a reason that young people play games and get them tangled up with the rest of their lives, and this reason is cultural. The phrase that best helps us explain it comes from one of our participants, Mikey, who in talking about games said, “It’s what we do.” The “we” he was referring to was kids these days, the young people of his generation.” (Stevens, Satwicz, McCarthy, 2008)
• Four-phase model of interest development
  • Triggered Situational Interest
  • Maintained Situational Interest
  • Emerging Individual Interest
  • Well Developed Individual Interest
• Learn by teaching – protégé effect
  • “We then introduce TAs, which combine properties of agents and avatars. This sets the stage for two studies that demonstrate what we term the protégé effect: students make greater effort to learn for their TAs than they do for themselves. ” (Chase, Chin, Oppezzo, Schwartz, 2009)
  • “Given our hypothesis that the protégé effect is due to social motivations, we would expect students in the programming condition to be less inclined to acknowledge ” (Chase, Chin, Oppezzo, Schwartz, 2009)

 West – who we learn from

• Technocentric views
• Learn from peers
  • “The variety of ways that we saw young people arrange themselves to play games surprised us, especially since most of these ways were interpersonally and emergently organized by the young people themselves. ” (Stevens, Satwicz, McCarthy, 2008)
  • “In fact, shared attentional focus on media in real time is a powerful interactional resource not found in most contemporary asynchronous social media, and researchers across a range of disciplines highlight the importance of joint attention for learning and meaning- making (e.g., Barron, 2000, 2003; Brooks & Meltzoff, 2008; Bruner, 1983, 1995; Goodwin, 2000; Meltzoff & Brooks, 2007; Stevens & Hall, 1998; Tomasello, 1999, 2003). ” (Takeuchi and Reed Stevens, 2001)
  • “Stevens, Satwicz, and McCarthy’s (2008) naturalistic studies of siblings and friends playing video games together at home examined the spontaneous instances of teaching and learning that players set up among themselves during gaming sessions, as well as how their in-room interactions connect with what’s going on inside the game and in their lives outside the home (e.g., school). ” (Takeuchi and Reed Stevens, 2001)
• Learn from teachers who’s roles are to provide: (Hidi & Renninger, 2006)
  • Positive feelings
  • Generate curiosity
  • Provide opportunities
  • Guide on research
• Parents (coviewing)
  • “To get families to use a new platform with any regularity, it should easily slot into existing routines, parent work schedules, and classroom practices. There are, after all, only so many hours in the day to accommodate new practices.” (Takeuchi and Reed Stevens, 2001)
  • “What children learn and do with media depends a lot on the content of the media, but they depend perhaps as much on the context in which they are used or viewed, and with whom they are used or viewed.” (Takeuchi and Reed Stevens, 2001)
• Society

Pedagogical Compass Planning

Act 1 – Why should you read this paper?

Want to become a learning tool designer? Care about learner? Care about teachers? Care about reducing the digital literacy gap?

Who are you? Teacher? Policy maker? School leader? Designer? Engineer? Developer?

The compass:

– North – what we teach

– South – how we teach

– East – how we learn

– West – who we learn from

 Act 2 – Evidence

How is LX design similar and different from:

UX designer – consider:

– User

– Usability

– Task at hand

Game Designer

– Learning the game – onboarding instructions

– Engagement – motivation, interest, reward systems, core loop

– Game mechanics

Learning Experience Designer

– Learning objectives

– Differentiation

– Cognitive developmental stages

– Cultural context

– Joint media engagement and co-viewing

– Learning from peers – protégé effect, learn by teaching

Act 3 – Conclusion

LX is probably the most complex type of design there is. Have to consider:

– The learner

– The teacher

– The environment

– The peers

– The cultural context

– Assessment

– Learning objectives

– Policy

– Costs

– Implementation

– Scalability

1. Product/prototype

Our target learner was Achu, a calm and smiley 12 year old boy who is fond of playing basketball, watching HotWheels videos on YouTube and painting. He follows instructions well yet rarely initiates activities on his own. That is also the case with communicating with others, unless he needs to go to the bathroom or needs more paint for example. He responds to questions but is not always sure about his answer. He often repeats the last words heard when answering. Our impression was that he knows the answer yet has trouble externalizing it appropriately.

We immediately focused on the idea of helping Achu initiate verbal communication in so far as it would help him express his desires and needs more effectively. Our initial brainstorms revolved around using technology such as VR and games that would prompt him for verbal responses or would require verbal input to be utilized. We generated a few statements that helped us focus on the learner’s needs and the solution:

How Might We

• HMW help him say more words?
• HMW motivate him to want to communicate?
• HMW stimulate him to produce original words?
• HMW make him comfortable sharing words with others?
• HMW make him feel like his words have value?

This lead us to the following Needs Statement:

“Achu is a shy pleaser who needs to practice creating his own words in order to facilitate him communicating with others.”

To achieve that, we created a low-resolution prototype which consisted of playing a video with no sound on the laptop and prompting him to narrate what was going on. The final goal was to have a video with his voice narrating the events. We were able to engage him in the activity and on a few occasions, he actually generated new words, when prompted. We felt that the prototyped achieved some of the initial goals but there was still something missing to be considered truly effective.

After this initial test, we were able to get feedback from Marina. She thought the prototype worked but partially because narration is a technique that has been used extensively before by his speech therapists. He generated new words but still needed prompting from us. She also stimulated us to think more about how could he transfer what he learned within our product to his everyday life. With this in mind we evolved our learning goal to:

“We want Achu to learn the value of communicating with others.”

After presenting our finding from our initial prototype, we dug deeper into what was missing and discussed some more potential solutions. We finally connected the idea that the value of communication is shown more evidently when helping others. We could use a teachable agent in the product and elicit the Protégé Effect (Chase, Chin, Oppezzo, & Schwartz, 2009). We built on the idea that while Achu might not find it always natural to speak for himself, he might find it compelling to speak to someone to help them.

We introduced Tom in our prototype. A blind cat who asked Achu for help figuring out what was on the screen. We scaffolded the experience by creating a simple learning progression. We start with a single word on the screen. Tom asks Achu what is the word. Once Achu says the word, Tom thanks for Achu’s help. We wanted to ensure that we were ‘valuing the process and not only the final result’ (Dweck, 2007). After 3 words, we moved on to 3 short sentences, 3 pictures, and finally 3 videos.

On the day of the test we were unsure about the results and therefore also took a few other activities to gauge Achu’s engagement and levels of communication we could elicit from him.

We had him play with an App that records what you say and plays it back with a funny voice through a character. He was soon bored with the activity.

We moved on to observing him assembling a jigsaw puzzle with a phrase instead of a picture as the complete set. He was very fast at combining the scattered words into a perfect sentence. He was also prompted to read it out loud, which he did with ease with the exception for one word he did know how it sounded. He seemed embarrassed but was reassured by the teacher that it was ok to say that he did not know – which he finally did. This episode showed us that the Protege Effect might actually work on him since he would not want his ‘friend’ to not know something.

Our final activity prior to testing our prototype was to engage him with text messaging. He clearly understood what was going on and responded by typing onto Alex’s phone while I was in another room with the other phone. His trouble was dealing with the small keyboard on the phone but it showed promise in that he might engage well with this form of communication with a larger keyboard.

Finally we tested our prototype. Achu was immediately fond of Tom, the cat and rapidly replied to his prompts. The words, sentences, and pictures we verbalized promptly. The video also succeeded in promoting verbalization yet it took some more time for him to think about what to say. Once he did it and Tom thanked him, his energetically and positive reaction was priceless and strong evidence that the Protege Effect worked. He even clapped his hands and said “Achu is helping the cat!”. 

What surprised me the most during the process was how a small adjustment in the product resulted in such a big change in the levels of engagement. The process of narration was still the same, yet the purpose and motivation it was made clear to him. Narration for narrations sake did not have value for him. Helping Tom did. It also reminded me that we were eliciting in a small way Joint Media Engagement (Takeuchi & Stevens 2011) between Achu and Tom the cat. They were both consuming media and helping each other out – feeding off of each other – learning from each other. A lesson learned that I will carry onto all my future design processes.

One thing I felt was missing in the process was a greater level of engagement with Achu’s teachers, Marina, and eventually his parents. To fault was lack of time, schedule conflicts and few attempts on our part to communicate more frequently with the stakeholders. Yet for the purposes of the course and the learning process, the interactions were fruitful and thought provoking leading always to new iterations and fine tuning of the product.

2. Collaboration 

The collaboration within our team was effective. I assumed the creative and technical role while Soren looked at our product through a more pedagogical lens and Alex with the documentation and write ups. It was a fruitful process where I felt each one in the group contributed effectively and pulled their own weight throughout. My multimedia skills helped us to rapidly create the prototypes, presentations and video. Soren’s teaching background helped us selecting the appropriate language, level of complexity, and scaffolds towards learning. Alex helped us with summarizing and documenting our meetings, tests, and findings.

Our process was very much guided by our class activities. We met only twice outside of class, not counting our three visits at OMS. This does not mean that we did not communicate outside of class. Through Google Docs we constantly collaborated with the elaboration of the presentations, texts and ideas. This demonstrated the effectiveness of the scaffolds we received as designers from our professor as well as our groups efficiency to generate ideas and agree with the path to take.

Next time around I will certainly work again with all the collaborative digital tools we used to document and brainstorm our ideas. I will also take the lead in creating the multimedia content since it is something I enjoy doing and see how valuable it is. As far as doing things differently, I would only wish to have more time to interact with the stakeholders and the learner. I will push harder to communicate more effectively with the intended audience and try to get more insights as to what the learner’s needs are.

3. Learning

My learning experience during the project was more one of trying to apply learning theories to the project than trying to be overly creative, as was the case in some previous projects I’ve worked on. The challenge was to design for a learner which we knew very little about, but using the educational lens we were able to apply and test learning theories with a certain success. A core motivator for me was  actually a little bit of the Protege Effect mixed with the Four-Phase Model of Interest Development. Looking back at the quarter, I noticed that much of my effort towards creating a better product was drawn from wanting to please the other, to teach, and to provide a benefit to his life. Not to mention the desire to please the teacher as well in the process. As for the interest development aspect of learning, I feel I reached Emerging Individual Interest, close to Well Developed Individual Interest – depending on if I am able to evolve the product in the future.

More importantly I believe I improved my skills and techniques of rapid prototyping. The pressures of creating a functional prototype to be placed in the unguided hands of a user were removed by the “Wizard of Oz” technique. It allowed me to create more freely and rapidly. It allowed me to continue thinking freely about potential solutions instead of being vested on a product because of all the time I spent in detailing a quasi-product. Yet I also learned that being able to design this way also requires some previous experience with prototyping. You must be able to predict user’s interactions that might completely break the desired effect. Therefore, even in a free-formed rapid prototype has a Minimal Viable Product.

References

Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18(4), 334-352.

Dweck, C. S. (2007). The perils and promises of praise. Kaleidoscope, Contemporary and Classic Readings in Education, 12.

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational psychologist, 41(2), 111-127.

Takeuchi, L., & Stevens, R. (2011). The new coviewing: Designing for learning through joint media engagement. In New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.

Chatted with Omair about my learning problem while walking to Vaden Medical Center… got there late so had to reschedule the TB shot for next monday.

Omair’s desire was to have a tool that would help him remember all the topics he must cover. Using keywords and a mind map for each piece of content, the system would alert him if any node was not covered yet. His main concern is covering the entirety of the content.

The more I look at the problem, the more I see that I would need to focus on a subject matter due to the particularities of each, pedagogically speaking. To try to create an overarching generalized system that helps you create ‘better’ courses seems like an outreach and a  technocentric view of a solution.

My initially narrow view of wishing to create a magical tool that would help me create a course by giving me tips and suggestions along the way might have the be reconsidered.

I am starting to see that a larger challenge that has to be addressed is teacher professional development. How do we scale it? How do we integrate it into their daily routines?

… rethink… review… research…

Assignment

Response

Team SAL: Soren, Alex, Lucas 

Lessons Learned

• Time with the intended user and rigorous user testing are critical to success. Every interaction our team had with our OMS student yielded more ideas and insights.

• It can be challenging to figure out whether an idea is not good or whether its implementation was the issue. It’s quite possible that a user will not engage well with a prototype even though the basis of the prototype is a worthwhile idea. Consequently, it is important to identify the critical learning mechanism to be tested and give the learner different ways to engage with it.

• What would be motivating to us is not necessarily as motivating to other learners. For example, while our team might be excited to narrate videos, our OMS learner was much more motivated to help a blind cat understand the video.

• It is important to use a learner’s strength to help him improve upon his weakness. With our OMS learner, we found greater success when we played off of his strength (caring for others) to motivate him to talk.

• The Wizard of Oz technique enabled our team to rapidly test, change, and evolve our prototype. The freedom to build just part of our prototype and simulate the rest of the prototype experience also allowed us to test several backup ideas, which helped us gather additional insight on our learner’s motivations and interests. 

• We realized how important it is to record and review testing sessions. By closely analyzing when our learner showed engagement, we were able to pinpoint the specific mechanisms through which our prototype encourages participation and positive affect. 

• The “protege effect” – or more broadly the task of teaching someone else – is a powerful motivator. Our strongest prototype turned our learner into a teacher that would explain videos to a blind cat who couldn’t see them. Our learner was quite responsive as he saw real purpose in generating words to help the needy creature.