LDT Seminar – Master's Project First Draft

Learning Experience Designer (LXD)
Lucas Longo
v.1, Feb 27, 2016

“In spite of the proliferation of online learning, creating online courses can still evoke a good deal of frustration, negativity, and wariness in those who need to create them.” – Vai, M. & Sosulski, K. (2015)

ABSTRACT

The trend towards online learning environments is irreversible and an increasing number of higher educational institutions are going in that direction. It is a labor intensive task for professors who must transition from a traditional classroom or lecture hall model to an online environment. Aside from the learning curve into any e-learning authoring tool or learning management systems (LMS) such as Moodle, OpenEdX, Coursera, and Udemy, new content must be created and organized: pdfs, images, videos, links, and animations to list a few. The challenge is to make it easier for professors or subject matter experts (the user) who, for the most part, do not have formal pedagogical training or multimedia content creation skills to publish their courses adopting research-based best-practices.

Learning Experience Designer (LXD) embeds curriculum construction techniques, tips, guides, and content within the usually blank template you are presented with. It utilizes research-based heuristics to suggest course formats, pedagogical strategies, learner activities, assessments and challenges. During the process of creating a course, LXD will prompt the user for information as well as analyze the content created in real-time to suggest what the next steps could be. Instead of taking a “course on how to create a course on platform X”, LXD integrates this course into the content publishing tool making it a seamless, more pleasing, and easier experience for the user.

As a proof-of-concept, I propose to utilize as a base, an existing LMS and add onto its interface a virtual student / coach that interacts with the user throughout the course creation process. It will start off by asking the user what the course goals and learning objectives are. It will also provide examples, analogies, and recommended steps to develop them. It might recommend assessment points after the user created a certain number of content modules. It will also suggest content for further reading, links to the learning communities or social networks for extended feedback. The idea is to emulate an expert teacher, posing as a student, who is asking guiding questions and providing insights throughout the process of planning the course, creating the content, organizing the learning progression, and creating a learning community with the students.

CHALLENGE

Needs

In 2009 I started a mobile app development school in Brazil targeting developers and designers who needed to acquire these new hot skills. For the first year or so I taught the iPhone app development course while looking for more teachers to meet the large demand. Pedagogically, I was going on instincts, using a very hands-on approach: explain the concept, model it, and have the students do it themselves. I presented the content through guiding slides and shared my screen when demonstrating. There was no assessment activities except for walking around the classroom making sure everyone was able to copy the modeled activity. The students seemed to enjoy it and it was straightforward enough to explain to new teachers.

The challenge came when I decided I to start selling the courses online. A ‘real’ curriculum had to be designed and new written and video content had to be created. This task proved to be daunting for me and the developers who had no guidelines as to what works or not. The developer-now-teachers were slow to produce the material, and it was usually of poor quality: slides with too many details, missing or confusing explanations of key concepts, badly sequenced, amongst other quality issues. How could I give them guidance as to what quality material looks like? Where do I start? How much video versus written material should I use? How will students ask questions? How will we manage all these students? What are the best practices? All questions that could be resolved by a well designed software that would scaffold the process of creating the curriculum and course content.

Most LMSs offer a “course on how to create a course” yet only provide a blank template with you hit the “new course” button. The need is to have an in-line guide of the steps needed to plan, create, and manage the course. LDX should be adaptable to the user’s interactions and level of expertise as to what questions, scaffolds and suggestions it makes. A user would be exposed to more frequent interventions at the start of the process. The interventions would gradually become less intrusive, yet always available.

LEARNING

Benefits:

LDX will make the user more proficient in the art of sharing their knowledge, stimulating them to repeat the process, and create new and better courses. Users will benefit from theory grounded strategies that promote effective learning in online environments. The virtual student will lead the process by posing provocative questions and requesting content, assessment, and reflection activities to be inserted into the course progression.

The main learning outcome will be that online teaching requires a different set of approaches, content, media, interactive experiences, and assessment methods to be effective. The virtual student will serve as an instructor and coach for the user during the process. Teaching and learning will occur during the process of creating a course.

To demonstrate the effectiveness of LDX I propose to survey the users pre- and post-utilization of the tool with questions that will inform me of the following characteristics of the user:

Pre-utilization:

  • Previous knowledge/experience with pedagogy
  • Previous knowledge/experience with online teaching
  • Perception of online course effectiveness
  • Personal beliefs on the challenges of creating an online course
  • Confidence level for creating an online course

Post-utilization:

  • New pedagogical content acquired
  • New online teaching content acquired
  • Perception of online course effectiveness
  • Personal beliefs on the challenges of creating an online course
  • Confidence level for creating an online course

During the utilization of the tool I intend to collect the following data:

  • Webcam video recording
  • Screen video recording w/ mouse tracks and clicks
  • User will be asked to think-aloud throughout the process

I also intend to test LDX with users who have already created online courses and interview them to get the following:

  • Perception of how much LDX actually helped them in the process
  • What would they do different now if they were to redo their existing courses
  • Input and feedback on what worked, what didn’t work, and suggestions

The results will be interpreted using the grounded theory qualitative research method. Theories of how to improve the tool will emerge from the evidence coding and proposition creation. The conclusion will address issues such as the viability of the concept, effectiveness, and suggestions for future improvement.

Approach for Learning:

The approach to learning that informs my design is a combination of the Protégé Effect, Understanding by Design, and TPACK.

The Protege Effect posits that students make a greater effort to learn in the benefit of someone else rather than for themselves, and thus end up learning more in the process of teaching someone else (Chase, Chin, Oppezzo, & Schwartz, 2009). This effect will be elicited through the virtual student who will prompt the user to teach him by asking leading questions, making suggestions, and warning the user about excessive use of one style of teaching as well as the lack of content, reflection opportunities, or detailing of previous knowledge. The virtual student closes the gap between the content ideation and the actual student’s experience. Through immediate feedback, the virtual student will elicit the user to think deeply about content choices and aid in the process of deciding the learning progression that must be in place.

Understanding by Design is a methodology (Wiggins & McTighe, 2005) that anchors the curriculum construction process on the end learning goals and evidences of learning. The virtual student’s interactions will guide the user through these steps which provide a structure and sequence that ultimately produces a more cohesive and effective learning experience. By starting with what the student will end up with in terms of knowledge, skills, or understandings, the user will be focused on a clear end goal thus producing better directed content.

Technological Pedagogical Content Knowledge (TPACK) is a framework that builds on Lee Shulman’s construct of pedagogical content knowledge (PCK) to include technology knowledge (Koehler & Mishra, 2009). LDX aims to act precisely within this space insofar as it integrates the complex interaction among three bodies of knowledge: content, pedagogy, and technology. LDX “produces the types of flexible knowledge needed to successfully integrate technology use into teaching.” (Koehler & Mishra, 2009)

DESIGN OF THE LEARNING EXPERIENCE

Existing solutions (“competition”):

LXD is a construct that, for the purposes of this project, will build upon an existing LMS avoiding the enormous task of creating from scratch an entire system to handle content and course organization. It will also build upon the existing “course on creating a course”, which is in effect the main competition of the platform. Users might rather go through such a course and then plan out and publish their own course, without any intrusion from the virtual student. With this in mind, the goal of LDX is to test if such an integration or synchronicity of learning and doing is effective.

Most LMSs offer, within their course creation templates, simple text fields or spaces to fill out the learning objective for each section or lesson for example. LDX takes it a step further by providing leading questions and best practices for each step of the process. It also introduces adaptive feedback based on the content already published on the platform. LDX proposes to analyze the content, media types, and sequence of activities in order to suggest what the user might do next.

Looking at Udemy for example, I’ve found several areas/content that aim at helping the user through the process, yet they are always clearly external to to content publication area. These external areas provide further readings, suggestions, and helpful tips yet they are static in nature.

  • Automatic Messages
    • Welcome Message
    • Congratulations Message
  • Course Goals
  • Intended Audience
  • Course Requirements
  • Instructional Level
  • Course Summary

Approach for Design:

LDX will be a web-based tool which will overlay the existing LMS with the virtual student along with texts, images, and video triggered by analyzing the steps and content being published in the course. Let’s say that the user has published a 30 minute video – LDX might suggest that the video should be shorter. If the user publishes 50 pages of text with no images, LDX might suggest that images illustrate concepts more powerfully that text alone. LDX might prompt the user to insert a knowledge-check or reflection activity once the user has published 5 pieces of content. The idea is to provoke the user to think about how the learner will be processing the content towards learning.  

The key features of LDX are:

  • Virtual Student
    • 3D character that talks to the user
    • Guides the user through the process of creating the content
    • Asks questions about the content and format of the course as it is created
  • Course Publication Tool
    • Existing LMS’s features
  • Curated Content
    • Access to similar courses to get examples
    • Ability to link to external material for student’s reference

EVIDENCE OF SUCCESS

I propose to assess the effectiveness of my solution by testing LDX with users with online teaching experience and users with no experience. The goals are to judge if such features improve the experience of creating the course and if the resulting course positively affects the learning outcomes. For practical purposes, I will narrow the content area to a beginner’s programming course at the undergraduate level since it is a subject matter I am familiar with and have access to potential users of the system.

APPENDIX A: TIME

Milestones and deliverables

When do you need to do what, in order to finish on time? Example:

Winter quarter Observe target learners

Develop ideas

Write proposal

March 20, 2015 Proposal draft submitted to advisor
Date Participants for user testing and learning assessment arranged
Date Low-res learning assessments complete
Date Low-res prototype studies complete
Date Round 2 learning assessments complete
Date Round 2 prototype design complete
Date Final user testing and learning assessment complete
July 20, 20165 Project logo and video submitted
July 2931, 20165 EXPO presentation, demo
August 46, 20165 Draft report submitted
August 113, 20165 Signed Master’s Project Form submitted

 

APPENDIX B: MONEY

Item Approximate Cost
Virtual student – design $ 300
Virtual student – development $ 3000
Total: $3300

APPENDIX C: PEOPLE

Supporters

  1. Candace Thille – online teaching platform pedagogy
  2. Grace Lyo – VPTL – Associate Director of Instructional design
  3. Pedro Cunha – graphic design
  4. Eduardo Cremon – software architecture & development
  5. Karin Forssell & Paulo Blikstein – feedback & support

APPENDIX D: SCHEDULE

Month Day Item
February 26 Define survey questions and send them out
March 4 Define problem and target audience
March 11 Collect Research
March 18 Hot to measure success
March 25
April 1 Feature list
April 8 Feature list
April 15 Wireframes
April 22 Define technologies
April 29 Database
May 6 APIs
May 13 User Interface
May 20 User Interface
May 27 User Interface
June 3 Final Adjustments
June 10
June 17
June 24 User Testing
July 1 User Testing
July 8 Analyze User Testing Data & Feedback
July 15 Final Adjustments
July 22 Final Adjustments
July 29 V1 LDT Expo

APPENDIX D: REFERENCES

Keywords

  • Hybrid Online Learning
  • Instructional Design
  • Train the Trainer
  • Professional Development
  • TPCK & TPACK

Research / Citations

Essentials of Online Course Design https://www.routledge.com/products/9781138780163

Towards Best Practices in Online Learning and Teaching in Higher Education http://jolt.merlot.org/vol6no2/keengwe_0610.htm

EXPLORING FOUR DIMENSIONS OF ONLINE INSTRUCTOR ROLES: A PROGRAM LEVEL CASE STUDY https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CB4QFjAAahUKEwjQ4te54tfIAhUL1GMKHcGSCxA&url=http%3A%2F%2Fonlinelearningconsortium.org%2Fsites%2Fdefault%2Ffiles%2Fv9n4_liu_1.pdf&usg=AFQjCNHtnYf76HkFI-YrIcLhxBWoNPXhRw&sig2=RQVCKYoBJvqv-Gtu8oyCdw

(MY) THREE PRINCIPLES OF EFFECTIVE ONLINE PEDAGOGY http://files.eric.ed.gov/fulltext/EJ909855.pdf

Source Effects in Online Education http://research.microsoft.com/en-us/um/people/thies/las15-source-effects.pdf

The Five stage Model http://www.gillysalmon.com/five-stage-model.html

From face-to-face teaching to online teaching: Pedagogical transitions http://www.ascilite.org/conferences/hobart11/downloads/papers/Redmond-full.pdf

From On-Ground to Online: Moving Senior Faculty to the Distance Learning Classroom http://er.educause.edu/articles/2017/6/from-onground-to-online-moving-senior-faculty-to-the-distance-learning-classroom

Why some distance education programs fail while others succeed in a global environment http://www.sciencedirect.com/science/article/pii/S1096751609000281

Case Study: Challenges and Issues in Teaching Fully Online Mechanical Engineering Courses http://link.springer.com/chapter/10.1007/978-3-319-06764-3_74

TPCK and SAMR – Models for Enhancing Technology Integration (2008) http://www.msad54.org/sahs/TechInteg/mlti/SAMR.pdf 

SAMR and TPCK in Action http://www.hippasus.com/rrpweblog/archives/2017/08/28/SAMR_TPCK_In_Action.pdf

SAMR: Beyond the Basics http://www.hippasus.com/rrpweblog/archives/2017/08/26/SAMRBeyondTheBasics.pdf

From the Classroom to the Keyboard: How Seven Teachers Created Their Online Teacher Identities http://www.irrodl.org/index.php/irrodl/article/download/1814/3253

A structure equation model among factors of teachers’ technology integration practice and their TPCK http://www.sciencedirect.com/science/article/pii/S0360131515000949

Examining Technopedagogical Knowledge Competencies of Teachers in Terms of Some Variables http://www.sciencedirect.com/science/article/pii/S1877042815006990/pdf?md5=1d1ccf6d1fb7088d7fda105f66d677c6&pid=1-s2.0-S1877042815006990-main.pdf

The Technological Pedagogical Content Knowledge-practical (TPACK-Practical) model: Examination of its validity in the Turkish culture via structural equation modeling http://www.sciencedirect.com/science/article/pii/S0360131515001189

Using TPCK as a scaffold to self-assess the novice online teaching experience http://www.tandfonline.com/doi/abs/10.1080/01587919.2015.1019964#aHR0cDovL3d3dy50YW5kZm9ubGluZS5jb20vZG9pL3BkZi8xMC4xMDgwLzAxNTg3OTE5LjIwMTUuMTAxOTk2NEBAQDA=

What Is Technological Pedagogical Content Knowledge? http://www.editlib.org/p/29544/

The role of TPACK in physics classroom: case studies of preservice physics teachers http://ac.els-cdn.com/S187704281201779X/1-s2.0-S187704281201779X-main.pdf?_tid=cf1faf84-81bf-11e5-8938-00000aacb35f&acdnat=1446509831_08753d5dcf76ed3f790bd4382aae1e31

Handbook of Technological Pedagogical Content Knowledge (TPCK) for Educators https://books.google.com/books?hl=en&lr=&id=lEbJAwAAQBAJ&oi=fnd&pg=PP1&dq=tPCK&ots=-p0TWk4RCI&sig=FElDYqBq7xyKcFWehvVRZ91LrNE#v=onepage&q&f=false

When using technology isn’t enough: A comparison of high school civics teachers׳ TPCK in one-to-one laptop environments http://www.sciencedirect.com/science/article/pii/S0885985X14000229

Teacher Education Programs and Online Learning Tools: Innovations in Teacher http://www.igi-global.com/gateway/book/63882

A Blended-learning Pedagogical Model for Teaching and Learning EFL Successfully Through an Online Interactive Multimedia Environment https://journals.equinoxpub.com/index.php/CALICO/article/view/23157/19162

The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature https://www.sri.com/sites/default/files/publications/effectiveness_of_online_and_blended_learning.pdf

How to Do More with Less: Lessons from Online Learning http://files.eric.ed.gov/fulltext/EJ982835.pdf

Build It But Will They Teach?: Strategies for Increasing Faculty Participation & Retention in Online & Blended Education http://www.westga.edu/~distance/ojdla/summer172/betts_heaston172.html

The design and development of an e-guide for a blended mode of delivery in a teacher preparation module http://reference.sabinet.co.za/webx/access/electronic_journals/progress/progress_v36_n2_a6.pdf

Lessons from the virtual classroom : the realities of online teaching [2013] https://searchworks.stanford.edu/?q=836557457

Essentials for Blended Learning: A Standards-Based Guide http://www.lybrary.com/essentials-for-blended-learning-a-standardsbased-guide-p-412451.html

Design and development process for blended learning courses http://www.inderscienceonline.com/doi/pdf/10.1504/IJIL.2013.052900

Pearl Jacobs, The challenges of online courses for the instructor http://www.aabri.com/manuscripts/131555.pdf

Developing an Online Course: Challenges and Enablers https://www.academia.edu/7511220/Developing_an_Online_Course_Challenges_and_Enablers

Considerations in Online Course Design http://ideaedu.org/wp-content/uploads/2017/08/idea_paper_52.pdf

 

Curriculum Construction – Week 9 – Reading Notes 

Jacob, B. (November 2001). Implementing Standards: The California Mathematics Textbook Debacle, Phi Delta Kappan, pp. 264-272.

  • Math Textbooks
    • “I examine California’s recent ‘standards-aligned” mathematics textbook adoption process, which provides a lens to scrutinize the impact of high-stakes policies on classroom practice”
  • Background on California
    • Standards by content areas
    • Revised every 7 years
    • K-8 only, High schools chose their own
  • 1996-1997 Math Adoption
      • Drafted by 4 voluntary Standards Commission
      • Met for 1 year with researchers, mathematicians, and educators
      • Published for review and comment
      • State Board of Education rejected and created a new one
        • 4 Stanford mathematic professors wrote it
        • Removed examples and clarifications
        • Done without input from the teachers
        • Problem solving techniques substituted by extensive practice and direct instruction!!
      • Dixon report
        • Douglas Carnine of the University of Oregon
        • Research review – selection biased towards board’s ideology – rote computation, not mathematical reasoning
      • 1999 Adoption
        • Story repeats itself – still rote computation + tell teachers exactly what to do
      • 2001 Adoption
        • Professor Hung-Hsi Wu from UC: could not explain his math reasoning “I’m puzzled as to why this is so difficult…”
        • Scripted instruction – “Teacher stupidification movement”  Richard Allington

Woodward, A., & Elliot, D. (1990). Textbooks: Consensus and Controversy. In Textbooks and Schooling In the United States, 89th Yearbook of the National Society for the Study of Education. Chicago: University of Chicago Press. pp. 146-161.

  • Stakeholders are resistant to change in education
  • Publishers have to adapt content to political ideologies
  • Cost effective to have a national, neutral, and bland textbook as opposed to local books

Teacher PD – Week 9 – Reading Response Assigment

Teacher PD – Reading Response
2016, Winter, Week 9
Lucas Longo

Prompt

What insights about the nature and design of online PD programs and the research on online PD did you gain by reading the set of articles by Fishman & colleagues and Moon & colleagues? What is the value of this type of “conversation” among scholars in a journal?

Response

I was pleasantly surprised initially with the results, even if conservative, that showed no “significant differences” between the online PD and the face-to-face modalities (Fishman et al, 2014). High dropout rates in MOOCS for example, might suggest that the same would occur with online PD. The population in ‘regular’ online courses are wildly different than that in online PD, yet there was a suspicion that teachers might not engage as deeply with the content as they would in face-to-face PD. The readings presented the several affordances of the online model but made clear that. I do believe that face-to-face is essential but it can be scaled down in favor of online content delivery and even community building. Further research as to what content should go online and what works best in face-to-face.

The Fishman et al 2013 piece focused on the “experimental comparison of PD delivered in two different media.” (Fishman et al, 2014). It showed that both methods of delivery work, and it “was welcomed by many as a sign that it is “safe” to employ online PD.” (Fishman et al, 2014). This research finding hopefully stimulates the market and researchers to further develop this modality to be able to scale both the training of PD facilitators and PD itself in an effective manner. The focus might start to become what subject areas and PD activities are better suited for each medium. For example, we can infer from the results that general SCK and PCK can be effectively delivered online for teachers. Yet if we think about training PD facilitators to support deep discussions around the subject or to press teachers for further inquiry, face-to-face might be more effective.

“The relative merits of online versus face-to-face conversation may vary across activities depending on the type of work planned for the teachers and the nature of sense-making and collaboration required.” (Moon, Passmore, Reiser, & Michaels, 2014, p2)

Moods’ Next Generation Science Exemplar System (NGSX) (Moon et al, 2014) seems to be a promising and solid start in exploring the hybrid modality of PD. Particularly interesting is their model of providing ‘pathways’ that enable facilitators to lead organized study groups. “There is expertise embedded in the system, in the structuring of tasks and discussions and in the ongoing commentary in embedded videos.” (Moon et al, 2014, p174) In other words, it focuses on providing research-based best-practices for leading and conducting face-to-face discussions using online content as a source of information and a scaffold for activities. This is the first I’ve seen such an effective and directed integration of the online and face-to-face methodologies; granted that my knowledge around the subject is limited, yet they strive for utilizing the best of both worlds.

The most interesting aspect of these types of “conversations” among scholars is the richness of information that emerge from it, educating us about the current state of affairs in the field in a summarized and passionate manner. Reading research papers alone you eventually form an opinion about the subject matter. This kind of dialogue between the experts you exposes to what are the controversies, the agreements, what is important or not, what needs to be changed, and what are the current theories and strategies used to tackle the field.  

One thing that was not mentioned in these readings was TPCK. My thought is that by simply exposing the teachers to the affordances of the online platforms, they might start to wonder how they might incorporate them into their own practices. Even if not explicitly a subject matter in the PDs we read about, I wonder if it was secondary effect of the teacher’s learning. They might start showing videos of other kids discussing math problems to establish norms and model the behavior. The online content can be used as an aid for teachers who are less proficient in the multiple ways of explaining a certain subject.

Finally, these papers have encouraged me to continue investigating how might we embed PCK scaffolds for instructors within online course authoring tools, beyond the ‘course on creating a course’. The idea is to have a virtual coach and some virtual students that will prompt for information, provide supporting content, and ask questions about the content at the appropriate stages of planning, developing, and submitting the online course. These papers showed that learning how to teach can happen online as well.

References:

Fishman, B., Konstantopoulos, S., Kubitskey, B.W., Vath, R., Park, G., Johnson, H., & Edelson, D.C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64 (5), 426-438.

Fishman, B., Konstantopoulos, S., Kubitskey, B.W., Vath, R., Park, G., Johnson, H., & Edelson, D.C. (2014). The future of professional development will be designed, not discovered: Response to Moon, Passmore, Reiser, and Michaels, “Beyond comparisons of online versus face-to-face PD.” Journal of Teacher Education, 65 (3), 261-264.

Moon, J., Passmore, C., Reiser, B.J., & Michaels, S. (2014). Beyond comparisons of online versus face-to-face PD: Commentary in response to Fishman et al., “Comparing the impact of online and face-to-face professional development in the context of curriculum implementation.” Journal of Teacher Education, 65 (2), 172-176.

 

LDT Seminar – Master Project Draft Review with Karin

Had a great meeting with Karin to discuss the first draft of the Master’s Project.

Main takeaways:

  • Find research that supports the notion that
    • Higher Ed is focusing everyday more on online courses
    • Instructors find putting courses online a challenge and a ‘large’ task.
    • Situated learning is better than parallelized
    • They why of the ‘course to create a course’
  • Virtual student
    • Have more than one, allow instructor to give them characteristics
    • Have an explicit virtual agent for the coach/expert
  • TPCK
    • Focus more on PCK
  • Competition
    • Facebook Groups
    • Hired Instructional Designer
  • Evidence of Success
    • Add what may not work

Brazilian Education – Week 9 – Class Notes

We had a very controversial and heated discussion in class today. The presentation was lead by Professor Grün:

“Professor Roberto Grün is the author of 4 books and 50 articles on immigration, economic and financial sociology. Professor Grün teaches Organizational Behavior and Sociology in the Federal University of São Carlos, Brazil, where he leads the “Center for Economic and Financial Sociology (NESEFI)”. He is currently the Joaquim Nabuco Chair in Brazilian Studies at Stanford’s Center for Latin American Studies.”

He believes that scandals, regardless of guilt, happen when stakeholders, the opposition, the media, and the public opinion are in synchronicity and start acting at a fast pace. Loosely quoting him: ‘We as scientists look at the mechanisms, not at guilt.’

Regardless of the phenomena, guilt should be evidenced and punished, regardless of the size of the scandal, in my opinion.

IMG_2155.JPG

Teacher PD – Week 9 – Class Notes

Did a great group activity of proposing an online PD to the rest of the class:

  • Affordances and limitations of going to scale via online PD and by preparing PD facilitators
    • Affordance is something you may afford / buy – it’s on the shelf, it’s available but you might choose or not be able to buy it.
    • Limitation is something that is absent from the shelf – it’s not available
  • Were the teachers required to participate in the PD?
  • Was there data on the online participation?

Teacher PD – Week 9 – Reading Notes

Borko, H., Koellner, K., & Jacobs, J. (2011). Meeting the challenges of scale: The importance of preparing professional development leaders. Teachers College Record, Date Published: March 04, 2011. http://www.tcrecprd.org ID Number: 16358.

  • What Must Math Professional Development Leaders Know and Be Able to Do?
    • Engaging teachers in productive mathematical work
      • Have to give more than SCK but also multiple forms of representations and how to lead discussions about common misconceptions
    • Leading discussions about student reasoning and instructional practices
      • Must maintain focus and anchor discussions
      • Analyze student work and videos is practice
    • Building a professional community
      • PD leaders must establish norms for constructive discussions
      • Create a safe space
  • Challenge is to train PD leaders but little support from stakeholders

Borko, H., Koellner, K., & Jacobs, J. (2014). Examining novice teacher leaders’ facilitation of mathematics professional development. The Journal of Mathematical Behavior, 33, 149–167.

  • Facilitators are generally successful with
    • Workshop culture
    • Video clip selection for use in the PD workshops
  • Facilitators have a hard time with
    • Supporting discussions to foster SCK and PCK
      • Mathematical Knowledge for Professional Development (MKPD)
  • Need to prepare novice PD facilitators
    • Need to show what quality PD looks like
    • Draw candidates from practicing mathematics teachers for local, site-based PD
      • Working with adults is different than working with kids
  • 3 phase agenda for designing, implementing, and investigating scalable PD
    • 1) Design for positive impact on teacher learning
    • 2) Design for repeatability in different contexts and with different facilitators
      • Specify role of facilitator
      • Develop resources and training for facilitators
    • 3) Compare multiple PD programs on impact on teacher and student learning
  • Conceptual framework: Scalable high-quality mathematics PD
    • Structure
      • Opportunities to engage with learning community, situated in the practice of learning
      • Select and use artifacts such as student work and videos
      • Safe space: trust and respect to be able to look at own’s work
      • Model instructional strategies
    • Content
      • Mathematics Knowledge for Teaching (MKT)
        • Common content knowledge
        • Specialized content knowledge
        • Knowledge of content and students (KCS)
        • Knowledge of content and teaching (KCT)
    • Preparation of PD leaders
      • Often a missing step in educational reform efforts
      • Need to know the content and how to lead
        • “leaders must be able to identify mathematics problems and discussion prompts that promote in-depth conversations focused on the mathematics content, support productive social interactions, and orchestrate discussions that help teachers unpack their often highly symbolic or incomplete reasoning (Elliott et al., 2009).” (Borko, Koellner, & Jacobs, 2014, p151)
  • Project Design: preparing teacher leaders to facilitate the Problem-Solving Cycle (PSC)
    • How well do teacher leaders (TL) implement PSC with fidelity
      • What did they enact well?
      • What was hard to enact?
    • PSC
      • PD for TLs
      • PD for teachers
      • Improved quality of teaching
      • Improved student learning
    • Cycle
      • Teachers collaboratively solve math problems & develop plans for teaching it
      • Implement plan with own students & video tape lessons
      • Facilitators select video that represent key moments
      • Analyze and discuss material
    • Supports needed
      • Create a professional learning community
      • Facilitate discussions with teachers
      • Facilitate video-based discussions to examine student thinking and classroom instruction
  • Results
    • Workshop culture
      • TL were able to establish a safe space and engage teachers in discussions
        • “As Mandy explained, “I already knew the teachers, so the comfort level and things like that were pretty much set.” Jordan agreed, noting that “the group came together rather quickly.” (Borko, Koellner, & Jacobs, 2014, p157)
      • Eliciting thoughts about their own students’ work helped in engagement
      • Challenge is to get participants to share ideas and take intellectual risks
    • Specialized Content Knowledge
      • Solution strategies are rated higher than mathematical representations
    • Pedagogical Content Knowledge (PCK)
      • Video clip selection
        • Often hard to find examples from teacher’s practices – easier when videos are selected by PD designers
        • TL did a good job at selecting the videos though
          • BUT were not as successful in leading discussions on instructional practices or student thinking
        • TL’s training focused on video selection & launching questions
          • Have to focus more on leading discussions
      • TLs find it harder to lead discussion where teachers critique their instructional practices
        • Easier to analyze student’s mathematical reasoning
  • General Discussion and Implications
    • TLs were able to replicate PSC model in PD
    • Easily enacted characteristics
      • Climate of respect and trust
      • Collaborative working relationships
      • Suggestions for establishing and maintaining community
      • Video clip selection – teach teachers to select their own videos and share
    • Hard characteristics to enact
      • Difficulty in supporting deep analysis in discussion to foster SCK, PCK: KCS & KCT
        • They did get extensive preparation and support… but still might need more
      • TL’s content knowledge must also be increased
        • They were able to show multiple solution strategies or mathematical representations
          • BUT had a hard time discussion the relationship, affordances, and limitations of the solutions/representations
          • High- and Low-Press Exchanges
            • “The three indicators on which the TLs were rated more highly—generating and analyzing ways to solve the task, discussions of various representations, and discussions of various solution strategies—are similar to what Kazemi and Stipek (2001) referred to as “low-press exchanges” in the elementary mathematics classrooms they studied, exchanges such as solving open-ended problems in groups and sharing solution strategies. In contrast, the five indicators that entail analyzing reasoning, discussing relationships among representations or solution strategies, and discussing affordances and constraints of representations or solution strategies are similar to the “high-press exchanges” they described.” (Borko, Koellner, & Jacobs, 2014, p164)
      • More opportunities for practice
        • It’s an art just like teaching
        • Need skillful improvisation
  • Conclusion
    • Scale PD, scale PD Facilitators
    • Create a research body on MKPD
      • “We believe that these three possible domains of MKPD—specialized content knowledge, pedagogical content knowledge, and learning community knowledge—go beyond and look different than the knowledge that a typical mathematics classroom teacher holds. Because PD leaders are expected to promote the development of teachers’ knowledge in these domains, they must hold a deeper and more sophisticated knowledge of mathematics than their colleagues, just as teachers must hold a deeper and more sophisticated knowledge than their students.” (Borko, Koellner, & Jacobs, 2014, p165)

Jackson, K., Cobb, P., Wilson, J., Webster, M., Dunlap, C., & Appelgate, M. (2015). Investigating the development of mathematics leaders’ capacity to support teachers’ learning on a large scale. ZDM Mathematics Education, 47, 93-104.

  • Revised learning goals for math leaders’ learning
  • Principles for supporting math leaders’ capacity to design and lead high-quality PD
    • Sustained over time and involve the same group of teachers working together (Darling-Hammond, Wei, Andree, Richardson, & Orphanos, 2009)
    • Supports for teachers’ learning should be close to practice (Ball & Cohen, 1999)
    • Co-participation with accomplished others
    • Pedagogies of investigation and of enactment (Grossman et al., 2009)
      • Video-cases for teaching (Borko, Koellner, Jacobs, & Seago, 2011; Sherin & Han, 2004)
    • Pressing productively on teacher’s ideas
  • Worked with district over 4 cycles of PD design and deployment
  • Treating teacher learning as a progression is a must
  • Productive types of PD activities that are useful to enact in Teacher PD
    • Modeling a lesson with follow-up discussion
    • Viewing video-recordings with follow-up discussion
  • Discussion and conclusion
    • PD leaders understood and supported teachers’ learning progression
    • PD leaders used more “show-and-tell” than deep discussions and facilitations
    • Need to add a goal that focuses on the “development of new practices as a process of reorganizing their current practices that requires explicit guidance.” (Jackson, Cobb, Wilson, Webster, Dunlap, & Appelgate, 2015, p102)
    • Investigate prior Teacher PD sessions (video) and jointly plan for upcoming sessions with accomplished others, worked.
    • Need to work on supporting PD Designer “in learning how to press on particular teacher understandings and specific aspects of their practice.” (Jackson, Cobb, Wilson, Webster, Dunlap, & Appelgate, 2015, p102)
    • Use videos of the PD session itself to inform improvements in their design

Fishman, B., Konstantopoulos, S., Kubitskey, B.W., Vath, R., Park, G., Johnson, H., & Edelson, D.C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64 (5), 426-438.

  • Online vs face-to-face PD sessions
    • No significant differences in outcomes
    • Used the same content as a base for comparison
  • Research question and subquestions:
    • How does online PD compare with face-to-face PD in terms of effects on teachers and students when the PD content is held constant?
      • Are there differences in teachers’ learning in terms of changes in beliefs and knowledge as a function of different PD modalities?
        • Changes in teacher CK
        • Teacher’s beliefs about self-efficacy to teach
        • Teacher’s beliefs about teaching in general
      • Are there differences in teachers’ classroom practice?
        • Videotaped teacher’s practice
      • Are there differences in student learning outcomes as a function of PD modalities?
        • Student test scores
  • The Evolution of Research on Teacher Learning from PD
    • From studies of attitude and beliefs (self-reported) to teacher and student learning (classroom practice and student learning data)
    • New curriculum providers offer PD on how to implement/adopting their curriculum
  • Online PD has many benefits
    • Accommodate teachers’ busy schedules
    • Access to powerful resources no available locally
    • Supports ongoing learning
    • No need to assemble in same location
    • BUT – how about building a community amongst teaches?
  • Online PD in the research had a face-to-face general orientation session: blended/hybrid online model
  • PD standards that were being taught:
    • Making connections
    • Evidence-based decision making
    • Technology use
  • Online PD self-pacing is a great benefit
    • Teachers can go at their own pace
    • Teachers can access material in a need-to-know basis
    • Looking at material while teaching, not in a separate session

Moon, J., Passmore, C., Reiser, B.J., & Michaels, S. (2014). Beyond comparisons of online versus face-to-face PD: Commentary in response to Fishman et al., “Comparing the impact of online and face-to-face professional development in the context of curriculum implementation.” Journal of Teacher Education, 65 (2), 172-176.

  • Review of Fishman et all study
    • Design of the PD itself was underspecified
    • Assumptions on PD learning goals were not made clear
    • Not much detail about evaluation measures
  • Face-to-face vs online benefits might vary depending on specific activities and goals
    • “The relative merits of online versus face-to-face conversation may vary across activities depending on the type of work planned for the teachers and the nature of sense-making and collaboration required.” (Moon, Passmore, Reiser, & Michaels, 2014, p2)
  • PD tenets
    • Embedded in the subject matter
    • Involve active sense making and problem solving
    • Connected to issues of teachers’ own practice
  • Next Generation Science Exemplar System (NGSX)
    • Web-based learning environment for teachers
    • Does not require a skilled facilitator on-site
  • Online PD
    • Better affordances…
      • “There are affordances of online systems that simply cannot be matched in a traditional setting. However, as a field, we know little about how these web-enabled and social media capacities interact with teacher learning and whether or how they are in line with established ideas about professional learning in general.” (Moon, Passmore, Reiser, & Michaels, 2014, p3
    • Limitations of community building but the next generation of teachers is used to that!

Fishman, B., Konstantopoulos, S., Kubitskey, B.W., Vath, R., Park, G., Johnson, H., & Edelson, D.C. (2014). The future of professional development will be designed, not discovered: Response to Moon, Passmore, Reiser, and Michaels, “Beyond comparisons of online versus face-to-face PD.” Journal of Teacher Education, 65 (3), 261-264.

  • Response to Moon’s response to Fishman
  • Looked at different media in PD delivery
    • “The heart of our study was the experimental comparison of PD delivered in two different media. Our study was a “media comparison” study, the value of which has been hotly debated in the field of educational technology (Clark, 1983, 1994; Kozma, 1994).” Fishman et al, 2014)