Teacher PD – Final – Research Notes

oTPD (Online Teacher Professional Development

NGSX – Next Generation Science Exemplar System for Professional Development


  • June 2012 started partnering with
    • School districts
    • State departments of education
    • Informal science education providers
    • Math/science partnerships
    • 9 states to pilot the beta version of the NGSX
      • “Argumentation, Explanation, and Modeling the Behavior of Matter.”
  • The Mosakowski Institute for Public Enterprise at Clark University
  • Sarah Michaels: Clark University and Tidemark Institute Associate
  • Jean Moon: Tidemark Institute
  • Brian J. Reiser: Northwestern University and Tidemark Institute Associate


  • K-12 Science Education
  • Based on National Research Council’s (NRC) “A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas”


  • “NGSX (Next Generation Science Exemplar System) is a face-to-face learning environment, in which the participants in a study group draw on an on-line system that poses tasks for each session and provides rich cases, supportive materials, and scaffolding tools to guide the work.”
  • “teachers support students in using science and engineering practices to develop, apply, and refine disciplinary and crosscutting ideas. Teachers engage in these practices and investigate classroom cases to explore how to bring these approaches into their own classrooms.”
  • 3 hour units
  • Period of 10 weeks
  • Facilitator has his or her laptop connected to a projector for the group to view video and task prompts embedded in the site.
  • An introductory video
    • a teacher, scientist, or researcher
    • theme for the unit
      • Nature of modeling
      • Support for classroom discourse
      • Difficulties students face in reasoning about the nature of matter
  • Include classroom cases to analyze
    • 5 min clips of teachers and students engaged in modeling practices
    • Prompts for discussion
      • Modeling tasks
      • Student thinking
      • Teaching strategies
    • Work to do between sessions
      • Readings about the science practices
      • Readings about students’ learning of the subject matter
      • Directions to try out aspects of what they have learned in the participants’ own classrooms

Professional learning system

  • Digital resources
  • Guided activities
  • Interactivity with colleagues

Encourages a mix of participants

  • K-12 teachers
  • Administrators
  • Science coaches
  • Higher education pre-service faculty

Major ideas

  • NRC Framework
  • Next Generation Science Standards (NGSS)

Theoretical Underpinning

  • Community of inquiry framework

Research Tool

  • 3D learning
  • Enactment in classroom

Learning pathway for facilitators

  • Create and support a knowledge-building culture
  • Teacher leaders, coaches, and PD providers
  • Model of peer-leadership in PD that scales

NGSX Design Approach

  • Instead of learning about NGSS, teachers are learning to teach with NGSS
    • Organize PD around teacher sensemaking of classroom cases
    • Focus on high leverage practices: engaging in argumentation to develop and use explanatory models.
    • Organize teacher study groups working to apply reforms to their own practice
    • Combine focus on science, student thinking, and pedagogy
    • Develop capacity for peer-led facilitation

NGSX Pathways

  • Argumentation, Explanation, and Modeling the Behavior of Matter: Supports teacher learning about modeling, argumentation and explanation in the context of disciplinary core ideas about the nature of matter.
    • Units 1-3: Modeling and three-dimensional learning. Teachers develop and use models to explain matter phenomena (science) and explore how this reflects the shifts in the Framework and NGSS (science pedagogy).
    • Unit 4: Analyzing classroom cases to learn to build a discourse community to support modeling, argumentation, and explanation (student learning, science pedagogy).
    • Unit 5: Teachers analyze a high school classroom case of student developing and refining models to explain air pressure phenomena (student learning, science pedagogy).
    • Unit 6: Teachers analyze a middle school classroom case of students engaging in argumentation to develop particle model of matter (student learning, science pedagogy).
    • Units 7-8: Taking it back to our own classrooms: Teachers work in teams to adapt existing instructional units to integrate science and engineering practices (science pedagogy). (Currently a face to face workshop, now being embedded in two NGSX units)
  • The Facilitator Pathway: Supports facilitation strategies to guide productive discussion in teacher study groups, and to help teachers grapple with the challenges of incorporating three dimensional learning into their own classrooms.
  • Additional pathways targeted for development include a pathway in life sciences in which teachers learn to support students in argumentation, explanation, and modeling population interactions and natural selection.

More Research


Penuel, W. R. (2015). Infrastructuring As a Practice for Promoting Transformation and Equity in Design-Based Implementation Research.

  • Some teachers took went through some of the NGSX sessions
  • Challenges of research
    • High turnover in large districts
    • Shifting assignments with disregard to teacher’s background or experience
    • “These lead not just to high levels of attrition in research studies; they undercut investments by district and school leaders in subject-matter focused initiatives aimed at improving teaching and learning (Shear & Penuel, 2010).” (Penal, 2015)

Smart, S. T. E. M., & Schools, L. L. F. S. (2016). Teaching and Learning Under the Next Generation Science Standards.

  • Two initiatives combine technology-enabled case analysis within a study-group format
    • Science Teachers Learning through Lesson Analysis (STeLLA)
    • NGSX


Fusco, J., Gehlbach, H., & Schlager, M. (2000). Assessing the impact of a large-scale online teacher professional development community. InSociety for Information Technology & Teacher Education International Conference (Vol. 2000, No. 1, pp. 2178-2183).

    • closed in 2013
    • Recommends EdModo
  • Full research on this – maybe use this as the focus?

Fusco, J., Haavind, S., Remold, J., & Schank, P. (2011). Exploring differences in online professional development seminars with the community of inquiry framework. Educational Media International, 48(3), 139-149.

  • 4 sessions of 2 professional development seminars were offered to members of an organization.
  • The seminars were voluntary, free of charge, and participants did not receive credit for their attendance
  • Community of inquiry framework
    • Essential elements of an educational experience:
      • Social presence
        • How members share and interact
      • Cognitive presence
        • Engaging with the content:
          • Brainstorming
          • Exploring topics
          • Integrating information
          • Constructing understanding
          • Reflection and dialogue about understandings.
      • Teaching presence
        • Sustains the learning experience
        • Encourages inquiry
        • Consists of:
          • Design
          • Facilitation
          • Direct instruction
  • Findings
    • First year
      • Did not figure out how to motivate participants
    • Second year
      • Corrected with more interventions
      • Demonstrated the importance of seminar leader or facilitator
      • Initial activities must provide easy success
      • Motivated participants help increase social and cognitive presence in inquiry discussions
      • Social presence = higher satisfaction ratings
    • Supports CoI framework as a planning tool for the development of seminars

Farooq, U., Schank, P., Harris, A., Fusco, J. & Schlager, M. (2007). Sustaining a community computing infrastructure for online teacher professional development: A case study of designing Tapped In. Journal of Computer Supported Cooperative Work, 16(4-5), 397-429. Norwell, MA: Klewer Academic Publishers.

  • Design principles used to design 4 interventions on Tapped In
    • Amy Jo Kim’s (2000) design principles that characterize successful, sustainable online communities:
      • Build flexible, extensible gathering places.
      • Design for a range of roles.
      • Develop a strong leadership program
      • Facilitate member-run subgroups
      • Create and maintain feedback loops
  • Conceptual components utilized:
    • Multiple interaction formats and technologies.
      • Support work practices of large numbers of different groups
      • Enable users to know with whom they are interacting and what is going on around them
      • Allow users to create, store, and share discourse objects (e.g., notes, overhead slides)
      • Communicate in real time or asynchronously, as the need arises
      • Engage in group activities hosted by designers as well as their own circle of colleagues
    • Identity and trust.
    • Ownership and empowerment
    • Heterogeneity
    • Community management, leadership, and sustainability.
  • Research Method
    • Participatory design (PD)
      • Socio-technical systems theory (Mumford 1983)
        • Importance of including the membership of a community in the design process

Online PD Offerings:

ASCD: http://www.ascd.org/

  • Positive
    • Global and long standing
    • Members – constituents, journals, videos, conferences, institutes, onsite and online PD programs
    • Focused on PD
    • Wide range of content offering & well organized
      • What we teach
      • How we teach
      • Who we teach
      • How we lead
    • All modes
      • Online
      • On-Site
      • Blended
      • Literature
  • Negative
    • Could not find research done on it

Edmodo: https://www.edmodo.com

  • Positive
    • Many digital tools available
      • Classroom management
      • Calendar
      • Community creation tools
      • App marketplace
      • Integration to Cloud Services
  •  Negative
    • Now overarching teaching or learning framework
    • More of a tool than an set of learning resources
    • Poor quality of content materials

PBS TeacherLine: http://www.pbs.org/teacherline/

  • Positive
    • Facilitated and self-paced
    • Beginning and experienced teachers
    • Research based
    • Award winning
  • Negative
    • Graduate & CEU credits upon verification of a higher-ed institute
    • Only one research partner
    • Free course had no videos – just basically formatted text
    • Old fashioned navigation

ReadWriteThink http://www.readwritethink.org

  • Positive
    • Marketplace for resources
      • Classroom resources
      • Parent & Afterschool resources
      • Links to Online PD programs
    • Strong partnerships
      • NCTE
  • Negative
    • Very little videos
    • Little content
    • No community building tool

The University of North Dakota: PD for Educators http://educators.und.edu

  • Positive
    • Marketplace for several online PD programs

Intel Education: http://www.intel.com/content/www/us/en/education/intel-education.html

  • Self-paced but old-school

Annenberg Learner http://learner.org

  • All free material
  • Tons of content
  • Well organized
  • All modes

Library of Congress: http://www.loc.gov/teachers/professionaldevelopment/

  • Free content
  • Limited online self-paced material
  • Mostly PDFs

TeacherFirst http://www.teachersfirst.com/

  • Free resources
  • Links to several other providers

University of the Pacific http://www.pacific.edu/Academics/Professional-and-Continuing-Education/Courses-for-Teachers.html

  • Several courses online self-paced

University of Phoenix http://www.phoenix.edu

  • All online

University of Wisconsin STOUT http://www.uwstout.edu/soe/profdev/courses.cfm

  • Online Graduate Course

Professional Development Institute https://www.webteaching.com

  • Online PD + tools to be used in the classroom

Sophia https://www.sophia.org/professional-development

  • Online PD

Harvard GSE Online Programs https://www.gse.harvard.edu/ppe/programs/online

The Teacher’s Academy http://www.theteachersacademy.com

The Heritage Institute http://www.hol.edu

CE Credits Online http://www.cecreditsonline.org/

International Baccalaureate Online PD http://www.ibo.org/professional-development/which-type-of-training-is-right-for-me/online-pd/

Adam State University Colorado https://www.adams.edu/extended_studies/professional-development/

TeachingChannel https://www.teachingchannel.org/

PepperPD https://www.pepperpd.com/ in association with WestEd https://www.wested.org/)

Professional Learning Board http://k12teacherstaffdevelopment.com/tlb/

TeachMe PD http://www.teachmeceus.com

San Francisco State University https://cel.sfsu.edu/education

Stanford Center for Professional Development http://scpd.stanford.edu/

LDT Seminar – Week 10 – Class Notes

Due to the success of Graph Learner in the BBA Expo and my enthusiasm towards building and evolving the app, I voiced my temptation of switching my Master’s Project to it.

The LXD platform aims at creating virtual agents that help instructors along the process of creating online courses. It is a very interesting problem to tackle but has a limited scope in so far as scale and potential impact.

Graph Learner on the other hand could impact a much wider population, it’s scalable, has the potential for a real revenue stream, and teaches essential skills needed by all students and even adults.

“Pay attention to what you are enthusiastic about,” said one of my colleagues.

So this is what I will be reflecting upon over the Spring Break 🙂

LXS vs. Graph Learner.


Teacher PD – Week 10 – Class Notes

Group Presentation

  • National Writing Project
  • Writing assignment
    • Prompt
      • What do you think about prior to developing a writing assignment?
    • Response (5 minutes to do it)
      • First thing I would do is create a learning objective for the assignment and a rubric for assessing it. I would share this with the students and ask if they understand the assessment and prompt for any suggestions or things they might want to change. I would then show some examples of quality work and have them assess them using the rubric. Lastly I will ask them to list 3 topics they might want to write about. Finally, I would show them some techniques of how to structure their writing in terms of content and flow of ideas.
    • Share with peer
      • Interesting: teacher would do an initial draft then work with the students to polish it up

2nd half

  • National Academy of Sciences Expert Committee Reports (nap.edu)
  • The New Teacher Training Program (tntp.org)
  • National Education Policy Center (nepc.colorado.edu)

Hilda’s tips on oTPD


Teacher PD – Week 10 – Reading Notes

National Academies of Sciences, Engineering, and Medicine. (2015). Science Teachers Learning: Enhancing Opportunities, Creating Supportive Contexts. Committee on Strengthening Science Education through a Teacher Learning Continuum. Board on Science Education and Teacher Advisory Council, Division of Behavioral and Social Science and Education. Washington, DC: The National Academies Press. [Read Summary, Chapters 6 & 9; also recommended: Chapter 8]

  • “A Nation at Risk” – book
    • Warned of the risks of neglecting improvements in the quality of teaching in public schools
    • Next Generation Science Standards (NGSS) was a response to the book
  • Teachers are the ones who will deliver these new standards
  • Conclusions
    • Will require teachers to alter the way they teach
    • Teachers do not have adequate CK
    • Teacher PD is selected and opted-in by individual teachers
    • Teachers need better preparation
      • Differentiation
      • CK
      • PCK
    • PD programs features
      • Analysis of own practice and student work
      • Content focused
      • Alignment with district policies and practices
      • Duration
    • Online PD is effective
    • Learning occurs within and outside of school
      • PD
      • Learning communities
      • Coaching
      • etc…
    • Schools need to be supportive of PD
    • PD must be contextualized
    • Administration’s support is central to advance learning
    • Teacher leaders – support and train new
    • Closing the gap – must attend to teacher’s different learning needs
    • US lacks a coherent and well-articulated system for PD
  • Recommendations for Practice and Policy
    • Understand what are the current teacher learning opportunities
    • Support teacher PD within own context
    • PD outside of school and embedded in the work day
    • Use research-based PD
      • Clear learning goals for teachers
      • Content specific
      • Student specific
      • Link to classroom instruction analysis
      • Interact with peers
      • Analyze data on student’s learning
      • Opportunities for collaboration
    • Develop internal capacity & partner with external expertise
    • Revise policies to encourage teacher learning opportunities
    • Explore new formats and media for teacher learning
  • Chapter 6 – Professional Development Programs
    • Definition
      • Purposefully designed to support particular kinds of teacher change
      • Include a focused, multi day session for teachers that takes place outside of the teacher’s classroom or school
      • May include follow-up opportunities over the school year
      • Have a finite duration (although they can take place over a period of 2 to 3 years)
    • Several PD opportunities in catalogue (over 1000 in 1 year)
      • Disjointed and incoherent
    • Core features of research-based PD
      • Focus on content
        • Subject matter content
        • How students learn that content
      • Active learning
        • Observing expert teachers
        • Reviewing student work
        • Leading discussions
      • Coherence with learning activities
        • Aligned with other learning opportunities
        • Aligned with school, district, and state policies
    • Structural features of research-based PD
      • Form of the activity
      • Collective participation of teachers from the same school, grade, or subject
      • Duration of the activity
        • Total number of hours
        • Span of time
    • Summary
      • PD can lead to sustainable changes in teacher’s knowledge and beliefs and their instruction
      • Little research that links directly to student outcomes – but seems to work
      • More to add to the consensus model
        • Content learning is intertwined with pedagogical activities such as analysis of practice. (Heller et al., 2012; Roth et al., 2011).
        • Analysis of student learning – artifacts of practice: student work and lesson videos (Greenleaf et al., 2011; Heller et al., 2012; Roth et al., 2011)
        • Focus on teaching strategies (Greenleaf et al., 2011; Johnson and Fargo, 2010; Penuel et al., 2011; Roth et al., 2011).
        • Reflect on and grapple with challenges to their current practice (Greenleaf et al., 2011; Johnson and Fargo, 2010; Penuel et al., 2011; Roth et al., 2011).
        • Scaffolded by knowledgeable professional development leaders (Greenleaf et al., 2011; Heller et al., 2012; Penuel et al., 2011; Roth et al., 2011).
        • Analytical tools support collaborative, focused, and deep analysis of science teaching, student learning, and science content (Greenleaf et al., 2011; Roth et al., 2011).
      • Must have better designed PD research
        • Research Design
          • “Few studies used strong research designs incorporating pre-post measures of both sets of outcomes shown in Figure 6-1 (teachers’ knowledge and instruction and students’ learning) and a control or comparison group.”
        • Research Scope and missing data
          • No studies look at schools organization and context
          • No studies look at role and expertise of PD providers and facilitators
      • Online Programs
        • Explosion of PD programs offered in this media
        • Still little research on it – early stages
        • Most programs use social constructivist approach
          • Problem-based learning
          • Inquiry-based learning
          • Mentoring
          • Communities of Practice
        • Most research is Qualitative – need more Quantitative?
        • Still need expert guidance and facilitation – open discussion forum is not enough (same as face-to-face efforts)
        • Teachers are more reflective online than on face-to-face
        • Technocentricity – online is not enough – has to be well designed in the first place
    • Conclusions
      • Evidence is still not very robust
      • Few studies employ control or comparison groups
      • Most studies look at 1 study, in 1 location, with few volunteer teachers

Screen Shot 2016-03-06 at 11.31.41 AM.png

  • Chapter 9 – Conclusions, Recommendations, and Directions for Research
    • Need better designed PD
    • Need better policies and practices in schools
    • Need support from Administrators
    • Recommendations for Research
      • Create system to collect data on
        • Current workforce
          • Qualifications
          • Experience
          • Preparation
        • General trends and consensus of best-practices
        • PD opportunities
      • Improve research methods
        • Include more quantitative data
        • Use control or comparison groups
      • More research on
        • Professional learning communities
        • Mentoring
        • Coaching
        • Online Learning
        • Teacher Networks
        • Teacher evaluation
        • School Organization
        • School Context
        • Principal and Leaders influence in PD
      • Formal Recommendations
        • 1: Focus Research on Linking Professional Learning to Changes in Instructional Practice and Student Learning
        • 2: Invest in Improving Measures of Science Instruction and Science Learning
        • 3: Design and Implement Research That Examines a Variety of Approaches to Supporting Science Teachers’ Learning
        • 4: Commit to Focusing on Meeting the Needs of Diverse Science Learners Across All Research on Professional Development
        • 5: Focus Research on Exploring the Potential Role of Technology
        • 6: Design and Implement Research Focused on the Learning Needs of Teacher Leaders and Professional Development Providers

The New Teaching Project (2015). The Mirage: Confronting the hard truth about our quest for teacher development. New York: TNTP. [Read through the Recommendations; no need to read appendices.]

  • Findings (“In short, we bombard teachers with help, but most of it is not helpful—to teachers as professionals or to schools seeking better instruction.)
    • Districts are making a massive investment in teacher improvement—far larger than most people realize.
    • Despite these efforts, most teachers do not appear to improve substantially from year to year—even though many have not yet mastered critical skills
    • Even when teachers do improve, we were unable to link their growth to any particular development strategy.
    • School systems are not helping teachers understand how to improve—or even that they have room to improve at all.
  • Recommendations
    • REDEFINE what it means to help teachers improve
      • Define “development” clearly, as observable, measurable progress toward an ambitious standard for teaching and student learning.
      • Give teachers a clear, deep understanding of their own performance and progress.
      • Encourage improvement with meaningful rewards and consequences.
    • REEVALUATE existing professional learning supports and programs
      • Inventory current development efforts.
      • Start evaluating the effectiveness of all development activities against the new definition of “development.”
      • Explore and test alternative approaches to development.
      • Reallocate funding for particular activities based on their impact.
    • REINVENT how we support effective teaching at scale
      • Balance investments in development with investments in recruitment, compensation and smart retention.
      • Reconstruct the teacher’s job.
      • Redesign schools to extend the reach of great teachers.
      • Reimagine how we train and certify teachers for the job.

Hill, H.C. (2015). Review of The Mirage: Confronting the hard truth about our quest for teacher development. Boulder, CO. National Education Policy Center.

  • “… public districts invest heavily in teacher professional development, what is offered is often a poor fit to teacher needs and ultimately ineffective as a means to improving teacher evaluation scores.”
  • “… mis-match between the behaviors rewarded by teacher evaluation and the professional development features…”

Story 2: Fired, Hired, and Inspired by Kathleen Aldred

  • The story of “Mr.Z”, a ‘late-hire’ who missed the PD and involvement with the the New Teacher Institute
  • Rowdy students would not respond to his instructions
  • Previously a chef – received no teacher training
  • “Do now” strategy to quiet students down – shook hands of all incoming students
  • Teacher evaluation went bad
  • Fired
  • Re-hired by a more ‘progressive’ school

LDT Seminar – Week 9 – Class Notes

Discussed our projects in our regular groups.

My report:

  • Got an internship next quarter at Stanford’s Vice Provost for Teaching and Learning (VPTL)!! Going to work on the course on creating a course using OpenEdX as the platform! Perfectly aligned with my Master’s Project!
  • Survey for teachers – fell behind on this one – have to update the questions based on Karin’s feedback and send it out.
  • Challenges ahead: technical implementation of solution

Then we went to the Cactus Garden to do some group activities… fun : )


Beyond Bits and Atoms – Final Project Notes

Our “Graph Learner” has been getting very positive results and the app development has been coming along nicely.



Screen Shot 2016-03-06 at 10.18.25 AM.png


  • Name sliders
  • Name axis
  • Bold x & y when slider is moving
  • Create levels changer
    • y=m*x
      • Control X and Y
    • y=m*x+c
      • Adjust c
      • Adjust m
      • Control X and Y
    • y=x^2
      • Auto X
      • Control X and Y
    • y=sin(x)
      • Auto X
      • Control X and Y
    • y=a*sin(x)
      • Auto X
      • Control X and Y
    • y=a*sin(b*x)
      • Auto X
      • Control X and Y
    •  y=a*sin(b*x+c)
      • Auto X
      • Control X and Y
    •  y=a*sin(b*x+c)+d
      • Auto X
      • Control X and Y


  • Axis markings
  • Labels for x & y values
  • Home screen with the mode
    • y = x
    • y = x + b
    • y = m * x + b
  • Baseline turn on or off
  • Grid lines
  • Clear drawing button
  • Auto speed
  • Draw the trace
  • Auto x & y
  • m & b sliders (only in 3rd mode)
  • sin/cos wave
  • Table with values
  • Color code table
  • Create levels changer
    • Demo y=x
      • Auto Y
      • Auto X
      • Control X and Y

LDT Seminar – Master's Project First Draft

Learning Experience Designer (LXD)
Lucas Longo
v.1, Feb 27, 2016

“In spite of the proliferation of online learning, creating online courses can still evoke a good deal of frustration, negativity, and wariness in those who need to create them.” – Vai, M. & Sosulski, K. (2015)


The trend towards online learning environments is irreversible and an increasing number of higher educational institutions are going in that direction. It is a labor intensive task for professors who must transition from a traditional classroom or lecture hall model to an online environment. Aside from the learning curve into any e-learning authoring tool or learning management systems (LMS) such as Moodle, OpenEdX, Coursera, and Udemy, new content must be created and organized: pdfs, images, videos, links, and animations to list a few. The challenge is to make it easier for professors or subject matter experts (the user) who, for the most part, do not have formal pedagogical training or multimedia content creation skills to publish their courses adopting research-based best-practices.

Learning Experience Designer (LXD) embeds curriculum construction techniques, tips, guides, and content within the usually blank template you are presented with. It utilizes research-based heuristics to suggest course formats, pedagogical strategies, learner activities, assessments and challenges. During the process of creating a course, LXD will prompt the user for information as well as analyze the content created in real-time to suggest what the next steps could be. Instead of taking a “course on how to create a course on platform X”, LXD integrates this course into the content publishing tool making it a seamless, more pleasing, and easier experience for the user.

As a proof-of-concept, I propose to utilize as a base, an existing LMS and add onto its interface a virtual student / coach that interacts with the user throughout the course creation process. It will start off by asking the user what the course goals and learning objectives are. It will also provide examples, analogies, and recommended steps to develop them. It might recommend assessment points after the user created a certain number of content modules. It will also suggest content for further reading, links to the learning communities or social networks for extended feedback. The idea is to emulate an expert teacher, posing as a student, who is asking guiding questions and providing insights throughout the process of planning the course, creating the content, organizing the learning progression, and creating a learning community with the students.



In 2009 I started a mobile app development school in Brazil targeting developers and designers who needed to acquire these new hot skills. For the first year or so I taught the iPhone app development course while looking for more teachers to meet the large demand. Pedagogically, I was going on instincts, using a very hands-on approach: explain the concept, model it, and have the students do it themselves. I presented the content through guiding slides and shared my screen when demonstrating. There was no assessment activities except for walking around the classroom making sure everyone was able to copy the modeled activity. The students seemed to enjoy it and it was straightforward enough to explain to new teachers.

The challenge came when I decided I to start selling the courses online. A ‘real’ curriculum had to be designed and new written and video content had to be created. This task proved to be daunting for me and the developers who had no guidelines as to what works or not. The developer-now-teachers were slow to produce the material, and it was usually of poor quality: slides with too many details, missing or confusing explanations of key concepts, badly sequenced, amongst other quality issues. How could I give them guidance as to what quality material looks like? Where do I start? How much video versus written material should I use? How will students ask questions? How will we manage all these students? What are the best practices? All questions that could be resolved by a well designed software that would scaffold the process of creating the curriculum and course content.

Most LMSs offer a “course on how to create a course” yet only provide a blank template with you hit the “new course” button. The need is to have an in-line guide of the steps needed to plan, create, and manage the course. LDX should be adaptable to the user’s interactions and level of expertise as to what questions, scaffolds and suggestions it makes. A user would be exposed to more frequent interventions at the start of the process. The interventions would gradually become less intrusive, yet always available.



LDX will make the user more proficient in the art of sharing their knowledge, stimulating them to repeat the process, and create new and better courses. Users will benefit from theory grounded strategies that promote effective learning in online environments. The virtual student will lead the process by posing provocative questions and requesting content, assessment, and reflection activities to be inserted into the course progression.

The main learning outcome will be that online teaching requires a different set of approaches, content, media, interactive experiences, and assessment methods to be effective. The virtual student will serve as an instructor and coach for the user during the process. Teaching and learning will occur during the process of creating a course.

To demonstrate the effectiveness of LDX I propose to survey the users pre- and post-utilization of the tool with questions that will inform me of the following characteristics of the user:


  • Previous knowledge/experience with pedagogy
  • Previous knowledge/experience with online teaching
  • Perception of online course effectiveness
  • Personal beliefs on the challenges of creating an online course
  • Confidence level for creating an online course


  • New pedagogical content acquired
  • New online teaching content acquired
  • Perception of online course effectiveness
  • Personal beliefs on the challenges of creating an online course
  • Confidence level for creating an online course

During the utilization of the tool I intend to collect the following data:

  • Webcam video recording
  • Screen video recording w/ mouse tracks and clicks
  • User will be asked to think-aloud throughout the process

I also intend to test LDX with users who have already created online courses and interview them to get the following:

  • Perception of how much LDX actually helped them in the process
  • What would they do different now if they were to redo their existing courses
  • Input and feedback on what worked, what didn’t work, and suggestions

The results will be interpreted using the grounded theory qualitative research method. Theories of how to improve the tool will emerge from the evidence coding and proposition creation. The conclusion will address issues such as the viability of the concept, effectiveness, and suggestions for future improvement.

Approach for Learning:

The approach to learning that informs my design is a combination of the Protégé Effect, Understanding by Design, and TPACK.

The Protege Effect posits that students make a greater effort to learn in the benefit of someone else rather than for themselves, and thus end up learning more in the process of teaching someone else (Chase, Chin, Oppezzo, & Schwartz, 2009). This effect will be elicited through the virtual student who will prompt the user to teach him by asking leading questions, making suggestions, and warning the user about excessive use of one style of teaching as well as the lack of content, reflection opportunities, or detailing of previous knowledge. The virtual student closes the gap between the content ideation and the actual student’s experience. Through immediate feedback, the virtual student will elicit the user to think deeply about content choices and aid in the process of deciding the learning progression that must be in place.

Understanding by Design is a methodology (Wiggins & McTighe, 2005) that anchors the curriculum construction process on the end learning goals and evidences of learning. The virtual student’s interactions will guide the user through these steps which provide a structure and sequence that ultimately produces a more cohesive and effective learning experience. By starting with what the student will end up with in terms of knowledge, skills, or understandings, the user will be focused on a clear end goal thus producing better directed content.

Technological Pedagogical Content Knowledge (TPACK) is a framework that builds on Lee Shulman’s construct of pedagogical content knowledge (PCK) to include technology knowledge (Koehler & Mishra, 2009). LDX aims to act precisely within this space insofar as it integrates the complex interaction among three bodies of knowledge: content, pedagogy, and technology. LDX “produces the types of flexible knowledge needed to successfully integrate technology use into teaching.” (Koehler & Mishra, 2009)


Existing solutions (“competition”):

LXD is a construct that, for the purposes of this project, will build upon an existing LMS avoiding the enormous task of creating from scratch an entire system to handle content and course organization. It will also build upon the existing “course on creating a course”, which is in effect the main competition of the platform. Users might rather go through such a course and then plan out and publish their own course, without any intrusion from the virtual student. With this in mind, the goal of LDX is to test if such an integration or synchronicity of learning and doing is effective.

Most LMSs offer, within their course creation templates, simple text fields or spaces to fill out the learning objective for each section or lesson for example. LDX takes it a step further by providing leading questions and best practices for each step of the process. It also introduces adaptive feedback based on the content already published on the platform. LDX proposes to analyze the content, media types, and sequence of activities in order to suggest what the user might do next.

Looking at Udemy for example, I’ve found several areas/content that aim at helping the user through the process, yet they are always clearly external to to content publication area. These external areas provide further readings, suggestions, and helpful tips yet they are static in nature.

  • Automatic Messages
    • Welcome Message
    • Congratulations Message
  • Course Goals
  • Intended Audience
  • Course Requirements
  • Instructional Level
  • Course Summary

Approach for Design:

LDX will be a web-based tool which will overlay the existing LMS with the virtual student along with texts, images, and video triggered by analyzing the steps and content being published in the course. Let’s say that the user has published a 30 minute video – LDX might suggest that the video should be shorter. If the user publishes 50 pages of text with no images, LDX might suggest that images illustrate concepts more powerfully that text alone. LDX might prompt the user to insert a knowledge-check or reflection activity once the user has published 5 pieces of content. The idea is to provoke the user to think about how the learner will be processing the content towards learning.  

The key features of LDX are:

  • Virtual Student
    • 3D character that talks to the user
    • Guides the user through the process of creating the content
    • Asks questions about the content and format of the course as it is created
  • Course Publication Tool
    • Existing LMS’s features
  • Curated Content
    • Access to similar courses to get examples
    • Ability to link to external material for student’s reference


I propose to assess the effectiveness of my solution by testing LDX with users with online teaching experience and users with no experience. The goals are to judge if such features improve the experience of creating the course and if the resulting course positively affects the learning outcomes. For practical purposes, I will narrow the content area to a beginner’s programming course at the undergraduate level since it is a subject matter I am familiar with and have access to potential users of the system.


Milestones and deliverables

When do you need to do what, in order to finish on time? Example:

Winter quarter Observe target learners

Develop ideas

Write proposal

March 20, 2015 Proposal draft submitted to advisor
Date Participants for user testing and learning assessment arranged
Date Low-res learning assessments complete
Date Low-res prototype studies complete
Date Round 2 learning assessments complete
Date Round 2 prototype design complete
Date Final user testing and learning assessment complete
July 20, 20165 Project logo and video submitted
July 2931, 20165 EXPO presentation, demo
August 46, 20165 Draft report submitted
August 113, 20165 Signed Master’s Project Form submitted



Item Approximate Cost
Virtual student – design $ 300
Virtual student – development $ 3000
Total: $3300



  1. Candace Thille – online teaching platform pedagogy
  2. Grace Lyo – VPTL – Associate Director of Instructional design
  3. Pedro Cunha – graphic design
  4. Eduardo Cremon – software architecture & development
  5. Karin Forssell & Paulo Blikstein – feedback & support


Month Day Item
February 26 Define survey questions and send them out
March 4 Define problem and target audience
March 11 Collect Research
March 18 Hot to measure success
March 25
April 1 Feature list
April 8 Feature list
April 15 Wireframes
April 22 Define technologies
April 29 Database
May 6 APIs
May 13 User Interface
May 20 User Interface
May 27 User Interface
June 3 Final Adjustments
June 10
June 17
June 24 User Testing
July 1 User Testing
July 8 Analyze User Testing Data & Feedback
July 15 Final Adjustments
July 22 Final Adjustments
July 29 V1 LDT Expo



  • Hybrid Online Learning
  • Instructional Design
  • Train the Trainer
  • Professional Development

Research / Citations

Essentials of Online Course Design https://www.routledge.com/products/9781138780163

Towards Best Practices in Online Learning and Teaching in Higher Education http://jolt.merlot.org/vol6no2/keengwe_0610.htm

EXPLORING FOUR DIMENSIONS OF ONLINE INSTRUCTOR ROLES: A PROGRAM LEVEL CASE STUDY https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CB4QFjAAahUKEwjQ4te54tfIAhUL1GMKHcGSCxA&url=http%3A%2F%2Fonlinelearningconsortium.org%2Fsites%2Fdefault%2Ffiles%2Fv9n4_liu_1.pdf&usg=AFQjCNHtnYf76HkFI-YrIcLhxBWoNPXhRw&sig2=RQVCKYoBJvqv-Gtu8oyCdw

(MY) THREE PRINCIPLES OF EFFECTIVE ONLINE PEDAGOGY http://files.eric.ed.gov/fulltext/EJ909855.pdf

Source Effects in Online Education http://research.microsoft.com/en-us/um/people/thies/las15-source-effects.pdf

The Five stage Model http://www.gillysalmon.com/five-stage-model.html

From face-to-face teaching to online teaching: Pedagogical transitions http://www.ascilite.org/conferences/hobart11/downloads/papers/Redmond-full.pdf

From On-Ground to Online: Moving Senior Faculty to the Distance Learning Classroom http://er.educause.edu/articles/2017/6/from-onground-to-online-moving-senior-faculty-to-the-distance-learning-classroom

Why some distance education programs fail while others succeed in a global environment http://www.sciencedirect.com/science/article/pii/S1096751609000281

Case Study: Challenges and Issues in Teaching Fully Online Mechanical Engineering Courses http://link.springer.com/chapter/10.1007/978-3-319-06764-3_74

TPCK and SAMR – Models for Enhancing Technology Integration (2008) http://www.msad54.org/sahs/TechInteg/mlti/SAMR.pdf 

SAMR and TPCK in Action http://www.hippasus.com/rrpweblog/archives/2017/08/28/SAMR_TPCK_In_Action.pdf

SAMR: Beyond the Basics http://www.hippasus.com/rrpweblog/archives/2017/08/26/SAMRBeyondTheBasics.pdf

From the Classroom to the Keyboard: How Seven Teachers Created Their Online Teacher Identities http://www.irrodl.org/index.php/irrodl/article/download/1814/3253

A structure equation model among factors of teachers’ technology integration practice and their TPCK http://www.sciencedirect.com/science/article/pii/S0360131515000949

Examining Technopedagogical Knowledge Competencies of Teachers in Terms of Some Variables http://www.sciencedirect.com/science/article/pii/S1877042815006990/pdf?md5=1d1ccf6d1fb7088d7fda105f66d677c6&pid=1-s2.0-S1877042815006990-main.pdf

The Technological Pedagogical Content Knowledge-practical (TPACK-Practical) model: Examination of its validity in the Turkish culture via structural equation modeling http://www.sciencedirect.com/science/article/pii/S0360131515001189

Using TPCK as a scaffold to self-assess the novice online teaching experience http://www.tandfonline.com/doi/abs/10.1080/01587919.2015.1019964#aHR0cDovL3d3dy50YW5kZm9ubGluZS5jb20vZG9pL3BkZi8xMC4xMDgwLzAxNTg3OTE5LjIwMTUuMTAxOTk2NEBAQDA=

What Is Technological Pedagogical Content Knowledge? http://www.editlib.org/p/29544/

The role of TPACK in physics classroom: case studies of preservice physics teachers http://ac.els-cdn.com/S187704281201779X/1-s2.0-S187704281201779X-main.pdf?_tid=cf1faf84-81bf-11e5-8938-00000aacb35f&acdnat=1446509831_08753d5dcf76ed3f790bd4382aae1e31

Handbook of Technological Pedagogical Content Knowledge (TPCK) for Educators https://books.google.com/books?hl=en&lr=&id=lEbJAwAAQBAJ&oi=fnd&pg=PP1&dq=tPCK&ots=-p0TWk4RCI&sig=FElDYqBq7xyKcFWehvVRZ91LrNE#v=onepage&q&f=false

When using technology isn’t enough: A comparison of high school civics teachers׳ TPCK in one-to-one laptop environments http://www.sciencedirect.com/science/article/pii/S0885985X14000229

Teacher Education Programs and Online Learning Tools: Innovations in Teacher http://www.igi-global.com/gateway/book/63882

A Blended-learning Pedagogical Model for Teaching and Learning EFL Successfully Through an Online Interactive Multimedia Environment https://journals.equinoxpub.com/index.php/CALICO/article/view/23157/19162

The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature https://www.sri.com/sites/default/files/publications/effectiveness_of_online_and_blended_learning.pdf

How to Do More with Less: Lessons from Online Learning http://files.eric.ed.gov/fulltext/EJ982835.pdf

Build It But Will They Teach?: Strategies for Increasing Faculty Participation & Retention in Online & Blended Education http://www.westga.edu/~distance/ojdla/summer172/betts_heaston172.html

The design and development of an e-guide for a blended mode of delivery in a teacher preparation module http://reference.sabinet.co.za/webx/access/electronic_journals/progress/progress_v36_n2_a6.pdf

Lessons from the virtual classroom : the realities of online teaching [2013] https://searchworks.stanford.edu/?q=836557457

Essentials for Blended Learning: A Standards-Based Guide http://www.lybrary.com/essentials-for-blended-learning-a-standardsbased-guide-p-412451.html

Design and development process for blended learning courses http://www.inderscienceonline.com/doi/pdf/10.1504/IJIL.2013.052900

Pearl Jacobs, The challenges of online courses for the instructor http://www.aabri.com/manuscripts/131555.pdf

Developing an Online Course: Challenges and Enablers https://www.academia.edu/7511220/Developing_an_Online_Course_Challenges_and_Enablers

Considerations in Online Course Design http://ideaedu.org/wp-content/uploads/2017/08/idea_paper_52.pdf