Qualitative Research – Week 7 – Class Notes

Was my turn to bring some snacks – brought a bag of nicely cut celery and hummus – delicious. One friend had never tried celery before!

Here we are discussing the Propositions our groups drew from the interview data presented in the readings.

IMG_1102

And I almost ate the Play Dough Gold Fish made by Saia! She almost tricked me into eating it… kkk

IMG_1103

The real thing!! Below… way too similar!

Can-I-give-my-baby-Goldfish-crackers

Class Notes:

IMG_1104 IMG_1105

 

Tech 4 Learners – Week 7 – Class Notes

Had some of the parents and kids from OMS come watch our presentations. We were actually the first ones to go – Soren presented and I controlled the slides… went a little over time but went well.

At the end of class, stayed back with Alex, Karin and Marina to try to get to some kind of breakthrough or decide upon next steps that we could make with our project. Karin thoughtfully asked if there was anything Marina never tried and would want to. She said Virtual Reality… got the juices flowing…

How could we have A engage, practice, and develop his verbal communication skills within a virtual world. Would he verbalize in order to interact with the game? Could he drive a car with voice commands? Would he engage in the task of teach an agent within this world – or engage in verbal play with a virtual character?

Back home I thought of Minecraft – would he engage in that level of focus requiring activity? Could he interact with the game using an existing world and voice commands?

Some of the feedback we got:

IMG_1106 IMG_1107

Qualitative Research – Week 7 – RDR Assignment

Finished the transcription for 15 minutes of our hour long interview… uff!


Interview Transcript

Interviewee: Anita Lin, ‎iHub Manager at Silicon Valley Education Foundation (SVEF)
Audio file: download
Interviewer: Lucas
Interview Date: Oct 29 2015
Transcription start time: 26:40
Transcription end time: 41:55

[Lucas] You’re good?

[Anita] umhum

[Lucas] Alright so… hum… you guys good… hum… so… I think, hum… we’re gonna dive into a little bit more about the model you mentioned

[Anita] Ok

[Lucas] So if you could tell me in your own words what’s the process that the startups go through with iHub prior to Pitch night?

[Anita] So we’re recruiting startups that are early stage so, what I would say we define that between Seed and Series A, hum… but I think it’s probably more broadly interpreted than that and so… We kind of reach out to contacts we have in the Bay Area and maybe a bit nationally and ask them to help us pass on the message that we are kind of looking for Ed Tech companies that are, that could be used in some classrooms specifically.

[Lucas] O

[Anita] From there companies apply online through a, like, a Google Forms. It’s pretty simple. It’s a very short application process, but I do think we’ll probably add to that next year. hum. And then following a certain time period I convene the invites of different people to be part of a short list committee. And so our short list committee consists of venture capitalists, it consists of accelerator partners and then also people from the education community so that typically is maybe a like an Edtech coach of the school or an IT Director at a school. Hum. Potentially some educators as well. So we kind of bring together this, this committee that, from all of our applications we f it down to 12. Then we ask those 12 to pitch other pitch game and we kind of ask them “Hey, focus on things you used in STEM classrooms” and we, we invite judges that are business judges. So typically CEOs of big companies in the area and then also education leaders. so we had [name of person] one. There’s also, hum… we’ve also had people who (whispers) trying to think of who else… (normal) we’ve had educators, we’ve also had IT Directors as well cause we kind of think, you know, they’re different slices of the education world so we have both of those be there and then they pitch and then the judges ultimately select the pool of companies that we work with for the round.

[Lucas] So you mentioned there’s 12 applicants… 12 selected [uhum] and then how many go to hum, the actual orientation?

[Anita] We pick between 6 to 8 companies [6 to 8] This last round we picked 6 hum, I think we have 11 pitches so that’s probably what we have.

[Lucas] Uhum… Is there any, hum… reason for this number?

[Anita] For us its capacity of teachers [capacity of teachers] so we support, in our last round we supported 25 teachers. And this round we have ‘bout 13 teams of teachers. And so we kind of didn’t wanna companies to support more than 2 or 3 although I think… we… we… we didn’t wanna it to be super challenging for companies to support and also since they are early stage products, we found that some companies as they’re taking off, like, they get really busy and they’re like, completely immersive so… I think it’s to balance both the support aspect but as well is kind of the teachers that we can support also.

[Lucas] Uhum… so let’s go a little bit back, uh, what happens between the pitch night and orientation in terms of your interactions?

[Anita] So we send out to our teachers and they’ve kind of, I would say vaguely, have defined the problem of need, and we’d like to kind of like focus them on the future. Make ‘em define it a lot more clearly. Hum… but have… we send out… I send out a form that kind of says, you know, “You’ve seen these companies at the pitch round. Here’s some more information about them if you’d like”. And I ask them to preference these different companies. So like, 1, 2, 3, 4 I mean we have them preference them whether or not, they’re going to work with all of them. And so, we… then they preference them and I kind of match them typically if I can I just give them their first choice of company that they’d like to work with cause I think that (mumbles), builds a lot of  investment in our process, hum… and then by orientation they know who they are working with and then we kind of tell them that all of… We’ve already told them all the program requirements before but we kind of go over them at orientation and then go over… let them meet their companies for the first time.

[Lucas] Great. And how about after orientation? What happens?

[Anita] So after the orientation we kind of let them go and set up their products for about a week or two depending on the time crunch we have from the end of the year and then… for the next couple of weeks they use the products in the classroom. There might be some observations but I would say these observations are mostly from a project management perspective more than like, an evaluative one. And then they submit feedback. And so we have some templates that we give them that we ask them to submit feedback from. There’s probably have a guiding question for each one and each week we’ll update that guiding question. Also we use a platform called Learn Trials which kind of gets qualitative feedback generally from these teachers about the product and includes comments but also has a rubric that they kind of use. And we’ve asked for pre and post assessments in the past that our teacher created ahm, but this probably hasn’t been… we have not been doing that. I think we need to find a better way to incorporate, so…

[Lucas] So, so… tell me a little bit more about this tool for the Qualitative Assessment. You said the name was?

[Anita] Learn Trials – and so they have a rubric that assesses an ed tech company across different strands whether that’s usage, whether that’s how easy was it for it to set up. And they kind of just rate them almost like grading it you know, like give it an A, give it a B. So like kind of like over time. And we ask them to submit it in different, like different… on a routine, so every 2 weeks or something. Where you’re able to kind of see how the product performs over time.

[Lucas] And am I correct to assume that after orientation the process goes towards, until the end of the semester?

[Anita] Yes – so it’s only until the end of this semester. So typically December, I want to say like  18th

[Lucas] And then what happens?

[Anita] And then at the end of this orientation we SVF maybe with the help of some of our partners like LearnTrials will aggregate some of this data and will share that out with the community. Additionally for this round something we’d like to do is maybe then from our 6 companies that we work with, work with a few of them and help them… help support implementation in the school versus just a couple classrooms that a school. So we’re still figuring this spring what that exactly looks like in terms of the implementation of the, these products but that’s something that we’d like to do.

[Lucas] And when you say community you mean both teachers, schools and the EdTech companies? You share it with everyone?

[Anita] Yeah

[Lucas] Hum… so what other events or resources you provide that have like similar goals or priorities? Or is this the only…

[Anita] Like within our organization? [yes] Well, in terms of teachers support, like, our Elevate Program I know… I talked about how it helps students but really a big point, I think a big selling point for districts is that it helps teachers, we give a lot of teacher professional development during that time. And so I think our program is also to help teachers who are early adopters of technology, help them kind of meet other teachers at different school for early adopters, and build a cohort that understands that and kind of can refer to each other. Humm… so we also do some, some I would say professional development is not as extensive as all of it is but we kind of want to help teachers understand how to use it, EdTech in their classroom. Potentially, referencing… Our goal is to reference the SAMR model. So..

[Lucas] Uhum… And is this whole process the first cycle you guys are going through, or you have been doing this for a while?

[Anita] So we started our first round in the Spring of 2014 – so this is technically round 4 but we’ve itter… like… it changes… little pieces of it change each round. So in the past when we’ve done it, when I run it, it was just I would recruit individual teachers from schools and so then I would form them onto a team so maybe a school, a teacher from school A, a teacher from school B, and a teacher from school C. And in this round I re…, we did recruitment where I recruited teacher teams. So now it’s like 3 teachers from school A, 3 teachers from school B, and then they are all using the same product at their school site so I think that helps with the piece of collaboration that was harder to come by earlier.

[Lucas] And how was it harder?

[Anita] Yeah, so I think for our teachers we would like them to meet up kind of weekly. And when you’re not at the same school it’s a lot harder to meet on a weekly because maybe one night one school has their staff meeting and then the other night the other school has their staff meeting and then, you know, I think it was a lot of commitment to ask and I think a lot of teams found it really challenging and maybe would not always be there because of that. Hum… So… that was a big shift from that. But I think it really builds a community within their school. And I think, what we have heard from teachers and from districts, is that a lot of times for a school for adopt or you know, use a product across their school, its because a group of teachers have started of saying “I’ve been using this product. I really like this product. Hey, like friend over there! Please use this product with me,” and they are like, “Oh! Yeah we like it” and kind of builds momentum that way [uhum]

[Lucas] So yes, so I guess that talks to the implementation phase of the, of the software that they were trialling. Hum… could you tell of us of a, a specific hum, aaaaa, ww… what do you call this phase after orientation? the pilot? [the pilot] phase. The Pilot Phase. So. Yeah. Could you tell us one story that things went really well or things went really badly?

[Anita] Sure! So, there was a product that as used in the last round where I felt like, it was really… we saw a lot of interesting things happen, hum… but they’re all like lot of qualitative metrics. So it”s called Brain Quake, and actually the CEO and cofounder, he’s the… he actually was an LDT graduate, hum… but…  it’s basically this game on an iPad or… whatever, where you can play… you have little keys and you have to line the keys to get this little creature out of a jail essentially, but,  what was interesting is when you turn the gears it also kind of teaches an eight number sense, so it’s like, this interesting puzzle that kids kind of enjoy solving. And so he was using this in some classrooms in the Bay Area. Also one in Gilroy and this teacher was a special ed teacher. And so she was kind of showing them this and so… What I think was really, really successful that I found was that for one of her students, they had trouble with like motors skills. And so one of the skills that they had trouble with was kind of like turning the gear on the iPad. hum. But the student actually learned to turn the gear like to the left. Cause you can turn it both ways and they were able to like, learn that skill moving like, doing a different motor skill than they had before because they really wanted to play the game. And so I thought that was like a really wonderful example of how technology can be really inspirational or like really helpful versus I think their other, you know. Well, lots of examples in literature where technology just like, you know, it’s just a replacement for something. Hum… and so… I think also a lot of the other teachers who worked with that product really, their students really enhoyed it, cause it is really engaging and they were making like, connections between the fact that, you know, I’m doing math and they could see h… they could understand that, you know, if I redid this into an algebraic expression… like they were coming up with that terminology and then they were like “we could just rewrite this into an algebraic expression”. And I think that was a like a really wonderful example of a product that went really well.

[Lucas] Did that product end up being hum, adopted or implemented in the school [Yeah, so…] effectively?

[Anita] That just happened this spring and I don’t think it has been yet. Hum… they’re still also like an early, you know like an early stage company so they’re, I think they’re still growing and figuring out exactly what it looks like. But I think that we are trying to support companies in that way. And we’re still figuring that out. So…

[Lucas] And was there ever hum… a big problem in a pilot?

[Anita] Yeah, let me think… typically I would say the problems that we run into in a pilot is where, companies are like working with their teachers and it’s going well but then sometimes companies get really… I guess it depends, now that I think about it. In the fall of last year the was one where the company like, the developers got really busy cause they’re just, start-up just took off. And so the became pretty unresponsive with our teachers. The teachers like, emailing me, and I’m like trying to get in contact with it, and so typically when there’s not communication between these parties, it would… the pilot would not be as successful as it could because they weren’t communicating. Things weren’t changing. Hum… In the spring, one of the things… The biggest challenge we found was actually testing. So testing was happening for the first time for Common Core and so what would happen was these teachers that email me, being like “I can’t get a hold of the Chrome Book carts”. Like, they just couldn’t get access to the technology they needed to run their pilot. And so… one teacher… her district told her this before she like committed to the pilot. And she just like pulled out. Like she’s like “I just can’t do this” like “I don’t have access to these, to like, the technology that I need”. Hum… But some other teachers, they were like, one of them told me she had to like had to go to the principle and like beg to use the Chromebooks on a like… on a day that they weren’t being used, but, I think because it wa the first year of testing, a lot of schools and a lot of districts were very, hum, protective of their technology cause they just wanted to make sure it went smoothly. And that totally makes sense. And so… for… because the… the testing when it… kind of… varied like when this would happen for the different schools but, some schools were more extreme in like saying, you know, were just gonna use it for all this quarter… like we… like, you know, we’re gonna lock it up and then others were like “Well… we’re not testing now so feel free to use it” So… That was a big challenge in our pilot this spring.  

 

Qualitative Research – Week 7 – Qualitative Research Critique

Last week I saw Janet Vertesi speak about her research “Seeing Like Rover” and found it so interesting that I decided to use it for the assignment this week – a Critique of a Qualitative Research Paper:


 

EDUC 200B – Qualitative Research Critique
“Seeing like a Rover: Visualization, embodiment, and interaction on the Mars Exploration Rover Mission”, Janet Vertesi, 2012 Society of Fellows, Princeton University, Princeton, NJ, USA
http://sss.sagepub.com/content/42/3/393.full.pdf+html
Lucas Longo

The study’s purpose is to investigate how physical representations of objects (embodiment), plays a role in a team’s social order and organization. New members for example, need time to be able to connect affectively to a remotely operated thing to finally “see like a rover”. This skill connects the team members socially and physically to the Rover. “Jude, too, explained to me that when something is not right with the Rover, ‘We feel it in our bodies.” (Vertesi, J., 2012, pg. 11). It makes them talk in the first person plural pronoun, “we”, to talk about the Rover. “We expect to turn around and take images of [the target] … . We’re about four meters from the outcrop…” (Vertesi, J., 2012, pg. 13)

Her laboratory ethnography consisted of more than 80 interviews, 360 meetings, and 10 different locations over a period of 2 years. The NASA Mars Exploration Rover team consists of several hundred scientists scattered all over the world who have daily meetings to decide what the Rovers’ next actions are going to be. The use of the body to simulate or explain moves the Rover’s “eyes”, “arm”, and “fingers” should do was observed during these meetings but more frequently while working alone through a problem or planning a task.

Her conclusions were that embodiment connects every team member into the body of the Rover, enunciating a sense of one. A shared responsibility, complicity and solidarity towards one mission and a common goal. She also argues that visualization of data and it’s subsequent interpretation and understanding, along with the visual nature of embodiment, serves as an instrument of social and organizational order. “Each [scientist] interacts with an image in a way appropriate to his or her position, consistent with both disciplinary heritage and with the organizational protocols of their laboratory, shop floor, or teleconference line.”, (Vertesi, J., 2012, pg. 17).

Even though her conclusions and findings were interesting, I felt that the research purpose was unclear as far as what practical applications, consequences, or impact it could have. The abstract itself states that this study looks further into understanding the “importance of embodiment to visualization” and how it plays a role in “maintenance of social order within the organizational context of the laboratory”, “contributing to our understanding of representation in scientific practice.” What are some practical recommendations or activities that other scientific teams could do to improve team-work, social orders, and/or communication?

I was baffled at how much data she must have collected with the many interviews, meetings and site visits. She did not mention how she specifically collected her data during the interviews yet her precise descriptions of body movements, objective and impartial description of observations show that her methods were effective. “She then raises both hands to either side of her head, forearms perpendicular to the floor, head tilted slightly down, fists open but fingers lightly curled.” (Vertesi, J., 2012, pg. 2). Along with these descriptions, came a few figures showing the positions described, enhancing our visualization.

I applied the “Criteria for a Good Ethnography” (Spindler & Spindler, 1987, pg 18-21)  to this piece and the top 3 criteria that were best met:

  1. “Criterion II. Hypothesis emerge in situ.” The unique organizational arrangement of this team of scientists contributed towards the construction of her hypothesis. She states; “I seek to broaden our understanding of visualization and embodiment from individual perception or dyadic interactions to include the collective work…” Through her observations she was then able to focus this general statement into a more specific hypothesis that embodiment contributes to the social order of the group.
  2. “Criterion III. Observation is prolonged and repetitive.” I feel that the amount of data she must have collected is enormous. Not only she interviewed more than 80 people in 2 years – almost a person per week – she participated in over 360 meetings – one every two days. She does not mention that se was a participant observer. I actually attended a class where she presented this study and she mentioned that she had a job or function there. That is how she was able to have participated in that many activities – she went there every day.
  3. “Criterion IV. The native view of reality is attended…” The very subject of her observation is how the native, in this case the scientists, expressed themselves through physical gestures when talking about the Rover. She was exposing to us how they viewed their world at the same time as how the Rover saw Mars. One other point that stood out for me in this study was how well she used previous research and articles throughout her introduction and conclusion. It showed me how deeply one can analyze a niche behavior and extract relevant generalizations about human behavior and group dynamics. My initial negative reaction towards the research’s purpose transformed into a sense of awe at the extrapolation capabilities we can attain when closely and attentively observing phenomena.

 

Tech 4 Learners – Week 7 – Prototype Presentation

Prototype presentation: 

Text for presentation: 

Learning goal: 

We want ‘A’ to learn the value of communicating with others.

Theory of learning:

Our tool intends to offer him practice in generating new words and then rewarding him with the playback experience. This will reinforce that his words have meaning, power and entertainment value. If he is interested in the material presented, he will engage in narrating it. Ideally we want him to transfer these skills into real life situations.

If the program:

  • Utilizes engaging content for ‘A’
  • Replays ‘A’s words so he and others see their value (and enjoy them!)
  • Initially offers prompts for ‘A’ to speak then gradually reduces them

Then ‘A’ will overtime…

  • Generate spontaneous and increasingly complex sentences.
  • Transfer those skills to real life situations.

Question:

  • How do we build on what we’ve learned to create a tool that encourages ‘A’ to express his thoughts with less prompting?
  • What additional mechanism or feature could support ‘A’ in transferring these skills to real-world situations?
  • Did some videos but there would be no time to present it – we had 60 seconds!

Videos

I prepared a stop-motion video for presenting the prototype as well:

Which is the final result of some thinking and prototyping:

LDT Seminar – Week 7 – Expert Interview

I interviewed Candice Thille, Ann Porteous, Sara Rutheford-Quach, and Karin Forssell regarding my learning problem. The greatest take-away for me was the vast amount of research and practice that has already happened. The depth and width of the problem space is enormous yet it helped me immensely in understanding the need to become more specific and focus on areas that interest me the most and with which I have the most familiarity with.

Learners’ goals: From the learner’s perspective, I understood from the interviews that there is a real challenge in knowing how to effectively create, publish, run and/or lead an online course. Very few tools aid the content-expert in acquiring PK and PCK  during the process of publishing a course for example. There are few lesson planning tools embedded in the existing educational technologies out there.

Context: I was ‘forced’ to learned about all the distinct online course types and methodologies that exist in the market to be able to ‘talk-the-talk’ during the interviews. This showed how rich the field is and how much there is to advance in it still. A major take-away was that human interaction has to be designed into a course, be it teacher-student or student-student along with a temporal dimension so that the course is effective in terms of course completion and depth of learning.

Special Needs: The learner’s needs are very specific to the subject matter I found out through the interviews. A statistics course has different needs than a linguistic course and different needs than a programing course. The scaffolding required for each subject matter would be different. The teaching strategies are different. The activities recommended to be done with the learners will be different.

In conclusion, the interviews were a tremendous source of information and served a great purpose of directing further inquiry, research and the desire to repeat the interviews in order to evolve the debate once I can get a better grasp of the material. Inspiring.

Learning Environments – Week 7 – Class Notes

The Reading Czars this week outdid themselves 🙂 The group was divided into two. My group met in the classroom.

Lego Run:

legos_render

We were divided into 2 competing groups. Each group was given a set of Lego pieces – the same required to build a model. This model can only be seen by the Observer, who describes to the Runner  what it looks like. The Runner then tells the Builder about the model, who tries to build it as precisely as possible. In the middle of the exercise, an extra Runner was added, which sped up the process immensely. Less waiting time for the Builder. More bite sized instructions.

Model -> Observer -> Runner -> Builder -> Replica

IMG_1055

Windhover

The second half was going to the Windhover Contemplative Center. No electronics allowed inside. We all got one of two texts to reflect upon.

IMG_1041

b._1105-mm-0337_-_1200_px_0

matthew-millman-cover-page

After reflection period inside we debriefed at the Papua New Guinea Sculpture Garden

o-2  o

Second half of class we discussed the readings.

IMG_1043

IMG_1057