Posts that are olin-ish

QualMIP: interim game plan clarification

Part of the QualMIP series, information pills introduced here.

This is public documentation of a message I recently sent to the group; there have been lots of changes/info floating around lately, side effects so I brought them all into one place for clarity.

This email contains 5 parts:

  1. What you need to bring to Monday
  2. What I need to bring to Monday
  3. What we will be doing Monday
  4. What you’ll need to do between Monday and returning from spring break
  5. What I expect you to bring back to studio after spring break

What you need to bring to Monday:

  1. A data collection schedule for the rest of the project; your data collection time (observations, physician interviews, time spent with artifacts) should be solidly on each of your individual calendars. We’ll be creating a shared one on Monday.
  2. An annotated/scaffolded selection of your fieldnotes to share. Your scaffolding should include specifications of what you’d like your teammates to give back to you — think of it as giving them a small homework assignment. Your teammates will be working with it for the next 2 weeks, so point out hard parts, ask them good questions; put them to work for you. (I’ll be modeling this process on Monday, so we’ll leave time at the end of Monday to update your annotations and print them out — but you should have your data and your first attempt at scaffolding/annotating them before arriving, even if that scaffolding is a piece of paper with 2 sentences of context and 3 questions for the group.)

What I need to bring to Monday:


  1. My data, scaffolded for you to engage with it.
  2. Questions about your data collection schedule and your process and evolving focus.
  3. Encouragement. :)


What we’re doing on Monday:

12:00-1:00: (in office) Mel models data-sharing
1:00-1:30: (at lunch) — brain break time
1:30-2:00: (at lunch) — reflection/comparison discussion; what was I doing as I modeled? What can you do with your own data?
2:00-2:30: (on your own) — artifact updating/printing
2:30-3:00: (in office) Project forward planning, including another round of clarification on your research questions/data/methods, master schedule creation, and sending a participation opportunity list to potential collaborators from another professor’s research group.

What you’ll do between Monday and when we get back from spring break:

  1. Respond to your teammates’ data, using the prompts they wrote to guide you. Ask them for clarification if necessary; don’t feel like you need to engage with this all on your own.
  2. Collect/analyze more of your own data. Remember to memo! Leave traces that you think will help you later. Remember that your teammates’ data/prompts may inspire some of your own thinking.

What you’ll bring back from spring break:

  1. Responses to your teammates’ data, in whatever format they’ve requested.
  2. All your data, in digital or physical form. If digital, write what it is on a post-it so we have a physical representation (if you come a few minutes early, our office has plenty of post-its). We’ll be creating a master data inventory/taxonomy during the post-spring-break studio.

Braindump edition: playing with language to describe my research

I’ve been playing with language to describe my research. One of my old notebook pages describes it this way:

  1. Making surprises visible (about what, infertility and to whom?)
  2. Helping others make sense of the surprises.

The first implies a sensitization process and an emphasis on emergence (without excluding a priori possibilities; we’re not blank slates). My approach tends to leave room for serendipity; I am curious about the unknown-unknowns, neurosurgeon the things we don’t (yet) know we don’t know. The emphasis is on the self as sensor, the team as sensor network; we are viewing the world reflected in each other, and we don’t need to be able to understand or articulate something to mark it as possibly important to pursue. I think of research as embodied, situated, personal, communal, full of tensions and contradictions we don’t need to resolve right away. This part is more about the “us” — experiences with and within what we are already close to.

The second is more about enlarging that “us” — looking at that which is currently “other,” and linking it to ourselves. It’s where the development of a shared vocabulary comes in. It’s not that language doesn’t factor into the first part; these aren’t hard boundaries. The shared vocabulary may be glances, words, scholarly references, names, stories. It’s mutable. It’s not centrally controlled, and it’s not a perfect capture of the territory that it maps.

These thoughts aren’t terribly worked out yet, but I’m trying to practice ways of writing down the fluid. It feels exactly like that — drawing a pen through water, trying to cup something inside my hands that keeps on flowing out. I’m trying to immerse myself in the writing of people who do this well, who signpost tracks behind them with a wry reminder not to take their trail too seriously. I’m working on it. I’m trying. I’m doing what I know: vigorous sprints of exercise to clarify my thought, Ritalin, drawing… it’s hard, thinking. Thinking in new ways is just hard.

One snippet I found recently and liked: Noffke describes action research as a place where “understandings and actions emerge in a constant cycle, one that highlights the ways in which educators are partially correct, yet in continual need of revision, in their thoughts and actions. The process does not end, as with traditional notions of research, with richer understandings of education for others to implement; rather, it aids in an ongoing process of identifying contradictions, which in turn, help to locate spaces for ethically defensible, politically strategic actions.” (p. 4 of Action research and democratic schooling: Problematics and potentials.)

Another snippet, where I respond to the questions of what impact I want to have on students here, on Olin itself, and on the world — my answers, in a thin scribble:

Students: Teach attention and awareness of habits of perception, and ways to both create tentative categories and question them. Training on improvisation and communicating in the moment as a team.

Olin: Sharing our stories with each other and playing with language for describing them. Looking at parallels between student growth environments and faculty/staff growth environments and experiences.

World: Radically transparent qualitative research, postmodernism in engineering education, and (to my surprise — did I actually write this down?) deaf gain in engineering education. (The last one I wrote down in all lower-case, and I don’t remember if the un-capitalization was significant to my past self.)

There we go. Braindumps.

I’m glad this little notebook on the web is mine; I’m glad I decided long ago that it was for my future self, and that not everything I put out here has got to be coherent. I’m glad I keep upholding that decision at a stage in my career where classmates feel their presence needs to be polished, their publications need to be thought through — I’ll get there too, I also have edited publications — but I like putting some of the earlier stages out there, because something about it feels liberating to my mind; once others can potentially see something I’ve made, once someone else can take it on, then I can much more easily let it go.

QualMIP week 6: first project-focused “research meeting” and getting “better” at “this,” for some value of “better” and “this”

Part of the QualMIP series, pills introduced here.

Today was our first “research meeting” in studio – last week we wrapped up the technique-focused exercises and plunged full-steam-ahead into data collection, so this week was the first hurtling-back from that and was entirely driven by… what the group brought in. Sharing and co-analyzing qualitative data is a skill, and over the next few weeks we’ll be practicing getting “better” at “this.” I deliberately haven’t defined what “better” is, nor what “this” is — looking at the emergent definitions for those two terms will be part of our praxis for the data collection phase of QualMIP.

It’s a messy process, and a self-revising one, and that’s part of what makes these discussions so difficult; as the famous quote goes, “if we knew what we were doing, it wouldn’t be called research.” Everyone selected a subset of their data, fieldnotes, or memoes to copy and share. Artifacts analyzed this week included slide decks, photos of signs in the studied environment, and fieldnotes. Cesar and Emily were initially worried they wouldn’t be able to gather data because of their transport limitations in getting off campus, but found that the internet provided them with a wealth of complex documents to analyze.

The discussion was difficult but good, and one of my thought is “wow, how am I going to scale this up for more students, when I can’t be with every group?” I feel a tension between trying to be a good model vs. having students actively engage and drive. It’s something I’ll be watchful for as we go along, because I know it’s not a binary opposition (I can do both at the same time!) but I’m not sure how to actualize it yet.

From my perspective, one temptation that came up within the group today is the tendency to present “too much” data for the group to engage with. (I kinda expected this.) Giving others context for analysis, scaffolding discussion, and choosing which pieces to select for sharing are all sub-skills that take practice. We… will practice them. I’m pretty sure this blog post will prompt me to build more scaffolding for this activity for the later, larger run of the class next time around (that’s part of why I’m taking these notes now).

Another challenge is continuing to seek alignment between research question(s), dataset(s), and analysis methodolog(y/ies) as we journey through the mess. Technically, I should put paradigm in there as well, but last week’s discussion seemed to indicate everyone wanted to pursue a mostly-interpretative paradigm this time around (as opposed to critical, postmodern, etc). In future iterations of this course, I’d like to play more with the different paradigms, but interpretative is as good a place as any to begin.

Next week’s assignment is to continue converging on the data bounds (topic/site/type), research questions, analysis plans, and their alignment. The data collection schedule for the remaining two weeks should be crystal-clear, and the team will be bringing in another round of artifacts and memos and scaffolding for everyone to review “at home” so we have more time to look over and dive into one another’s work. We’ll also be revisiting the question of what “better” and “this” are in terms of getting “better” at “this” (whatever we’re doing in our studio meetings). We’ll be revisiting that question a lot.

That means that next week’s studio time will largely be occupied by… my data. This wasn’t planned from the beginning of the semester, but now that the team has gone through their first round of scaffolding others through engaging with their in-progress data and analysis… now it’s my turn. Part of the motivation is to give us a brief look at what a more experienced researcher does (I’ve already told them I’m not the best at this yet, but I’ve at least had more practice), part of it is to offer my practice up for critique because I’m not perfect either and that’s important to know, and part of it is to give us time to look over the project data “at home.”

I’ve offered the team a choice among some of my projects (at various stages in the process of completion) and will be modeling being on the other end of the co-analysis/critique/review/etc. process next week, and then we’ll step back and compare. I will not do it perfectly, and that’s the point; I’m hoping they’ll spot the weak spots I already know of, and also guessing they’ll catch things I have not yet seen. I’m looking forward to that last one.

Learning activities for when your students are exploring areas you don’t know (inspired by open source)

Preparing to teach a class where you “don’t know the material” is tricky, cardiology especially when the thing is so big and so vast that you can never truly be an “expert” in it. If you think about expertise as knowing all the things, that is. If you think of expertise as the ability to be productively lost, that changes the entire game — the trick is how to help your students get through the same sort of territory.

I don’t have an overarching framework of strategies, but we do have a few things to share from the Teaching Open Source world. I wish we had a better write-up of our overall philosophy, but it wasn’t sufficiently developed (that’s something I’d love to work on with others… later… after… thesis…) but you can infer a bunch from some of our artifacts, so here you go, in order of least interesting to most interesting in my heavily biased opinion.

1. We have generic project-helpful activities.

We have a number of learning activites that (1) are highly likely to be useful for students regardless of what project topic they are on and (2) have clear criteria for successful completion, aka they are assessable. Analogous college-type things might be stuff like “make a test plan for your object” or “determine the appropriate formulae for predicting the behavior of your material” (I’m obviously making these examples up in a domain — materials science — I don’t know much about).

2. We have specific tool walkthroughs.

We also have activities that are walkthroughs of specific skills/tools common to most projects, often on a known setup (in the case below, a dummy server). This would be things like “do this intro exercise to use the Instron for the first time with our pre-cut samples.”

3. We have activities about critiquing the work of others (not other students in the class — this is not about peer assessment).

Moving into more interesting stuff: we also have activities that are about looking at other people’s work — not making new things, but critiquing existing things, to start developing a sense of how experts see things. In college, that might be: “Look at these pages from 3 lab journals, compare/critique; what makes the good ones good, the bad ones suboptimally helpful?” The reason it’s not peer critique is that you want to curate the examples to be rich and to have a range of things you can pull out in the discussion. (For those of you with qualitative research backgrounds, you can think of this as artifact analysis.)

4. We have resources (not just live demos) that lay out expert thinking.

We have think-alouds, where (more) experienced people demo their thought process being “productively lost” and then unpack it to newcomers, so they can start comparing metacognitive strategies. Every time you think out loud to students with their project, you’re doing this — but sometimes making artifacts is helpful, too. Also, accessibility is super-important for this… if you’re making videos, caption them. If you’re including images, describe them, and so forth. (My images are screenshots of the linked webpages, so I didn’t put image descriptions.)

5. We frame their mindset explicitly.

We have documents that explain the state of mind / viewpoint / psychological priming we want students to take towards things.

QualMIP week 5: fumbling into project mode

Part of the QualMIP series, buy information pills introduced here.

This week marked a transition point for QualMIP. Instead of doing exercises that focused on particular aspects/types of qualitative fieldwork (artifact analysis, interviews, observations), we are transitioning into fieldwork for the group’s project (site: social dancing venues).

This week’s report has 2 parts: (1) Debrief from last week’s scavenger hunt and (2) what’s happening in the transition to focusing on the project.

Debrief from last week’s scavenger hunt

Last week’s scavenger hunt was… fun. One of my (unstated) working hypotheses for this group independent study is that I should err on the side of too little scaffolding rather than too much. Instead of pre-loading them with a lot of information about document/artifact analysis, I wanted to get them into doing it, knowing that they’d play around. Afterwards, we stepped back and look at what they had done. My job is mostly to give them language for the techniques they independently invented so that they can hook their ideas to broader qualitative methodology literature.

As it turned out, today’s theme was postmodernist ideas/language, largely because that’s where my brain is these days. Here’s what we played with:


  1. Everything is a text that can be analyzed, not just things that are “words on paper.” (This is one of Derrida’s famous ideas.) That old copy of the Honor Code? Text, obviously. But the arrangement of the Honor Code copies on the display wall? Also text! The machine shop punchcard, the photo of the dining hall in 2003, the words we spoke today in 2016, the clothes on people’s bodies? Also text. During the scavenger hunt, Emily and Cesar were analyzing all these things and more. There aren’t hard boundaries on what you may and may not examine in document/artifact analysis.
  2. The idea of intertextuality — that texts are entangled with/in each other. (In other words, there aren’t absolute hard boundaries between texts, either.) For instance, the Honor Code’s 2003 and 2016 versions are… one document? Two documents? The 2003 document’s traces clearly show up in the 2016 ones, but there are differences. If we add the github page that tracks the differences between the versions, that page links to the 2003 and 2016 versions as “separate” documents… but now, how many texts do we have? One? (The Honor Code!) Two? (The Honor code in 2003 and 2016!) Three? (The 2003 version, the 2016 version, and the diff page!) All the questions! Allllll the questions.
  3. The shifting-ness of meaning and words, which I wrote about recently. Each seemingly innocuous or simple artifact (even just a phrase or word) has winding rabbit-holes of depth, but it is difficult to see this depth without context. Things we dismiss as ordinary or fancy-schmancy and simply drift by… often have depths behind them; the long-winded plaque declaring that “Wherefore, Formally Named Person has Contributed to the Success of XYZ” or the terse piece explaining changes to a policy are likely to have aeons of hidden stories and complex discussions behind them. (Another overarching thesis of this independent study: NOTHING IS BORING.)


When Paige raised the question of how to capture/represent history, I almost brought up the map-territory relation, but decided we didn’t have enough time.

Transition to project

Cesar, Emily, and Paige drove most of this discussion about goals and scheduling, and the end result (from my perspective) was (1) we know we have too many ideas and don’t know how to scope yet, (2) let’s do fieldwork for 4 weeks and then figure out deliverables, and (3) this is still kind of a mess.

I was suppressing a grin the whole time. It’s hard to resist the temptation to jump in and scaffold them and tell them what to do, and instead let them figure it out themselves. The tension and discomfort is pretty obvious — it’s not bad, they’re handling it well, and in hindsight I should have let them know earlier that the tension/uncertainty is absolutely normal and exactly what they’re supposed to be practicing.

That, plus… there are so many ways in which they both are and aren’t on the same page, and I suspect this will come out in the next few weeks. Excellent. Also, I know they’re probably reading this blog post right now. Also excellent. (Hi, team!)

Next week’s assignment is to bring whatever public subsets of their private fieldnotes they want to share with the team, prepared with whatever annotation/questions/context-in-general they think would be of help (aka “don’t just photocopy a page from your notebook with handwriting we can’t read; make it readable, give us context, tell us what you want to do with it.”) We’re going to get our hands dirty and spend most of our time in the next few weeks in everybody’s data, seeing where things shape up.

What does the word “maker” mean?

I wrote a long reply to a facebook thread on Olin colleague Debbie Chachra’s piece, more about “Why I Am Not A Maker.” Reposted here with light editing for context.

First, information pills this response is written by someone who built her early career inside the hacker/maker world, working with self-identified hackers/makers and being seen by them as “one of us.” I still identify as such, and it’s an important community — or rather, loose collective of communities — for me.

It is also written by a postmodernist researcher in engineering education, and the response below is my attempt to bring some of those ideas to bear on “making” in an engineering-accessible way. Specifically, part of the debate in the conversation (that I was responding to) is about what “maker” can or should mean (this is a massive oversimplification and not an accurate summary, but it’s the shortest way to provide context for my two-part response, below).

Point 1: Language matters tremendously. The word “Maker” has a shifting, socially constructed meaning — it’s not that the meaning has changed over time, it’s that the meaning is always changing over time, and that it’s always multiple contradictory meanings at once… because so many people use that word in so many contexts and with so many intents.

So when we argue about what the word “maker” means, or what it “really” is or what it “should” be — we’re clutching at solidity that isn’t there. There’s no Platonic Solid of the Definition of Making. We’ve made it, and we continue to make and unmake it, and there is no firm ground.

Point 2: Action matters tremendously. As others have pointed out, language can be reclaimed both by its usage and by taking actions that can then be pointed to by that language. All of us can do this. Language belongs to anyone who speaks (or signs, or writes) it; action belongs to anyone with a will. (Some actions cost some people more than others; some people can take actions others can’t, but everyone can do something, even if it seems to be trivially small.)

In other words: if you think of words as signs pointing to things/events/actions, you can either write different things on the signs, move the signs to point at different (or more or fewer) things, or make more things/events/actions for the signs to point to. And anyone can do one or more of these things in some way.

I see Debbie working to point out the holes in the Big Grand Ol’ Story of Making — the one that holds it up as shiny and wonderful, without acknowledging how it and its claim of “meritocracy” and “everyone can do it” is also dangerously exclusionary. There are people being made invisible by this narrative. Other narratives of making are possible. Debbie’s piece points towards this.

In my mind, there isn’t a “better” narrative of making. Any single narrative that claims to be the only one is going to have flaws. Multiple narratives, multiple viewpoints, multiple contradictory ways of seeing things — there is no single self-consistent system we can make where all is “equal” and “all voices are heard.” That’s why we see people pushing so hard for multiplicity, for getting voices out there; whether they know it or not, they’re working to unmake “The Story Of Making” into thousands of tiny stories.

QualMIP week 4: artifact analysis scavenger hunt

Part of the QualMIP series, doctor introduced here.

I’m on a research trip this week, symptoms so QualMIP gets a scavenger hunt (of sorts; strictly speaking, it doesn’t fit the definition, but they are roaming around campus to look for things) in my absence as a way to start folding document/artifact analysis into the things they’re doing. I’ve included the entirety of their clues document below.

Scavenger hunt start!

Welcome! You’ll find the relevant materials in my office, on the table; there are 4 things labeled with post-it notes (A/B/C/D) for each section. Take them with you as you go about the day.

You’ve got a hard stop at 3pm, so pace yourselves; each section should take about the same amount of time (this accounts for you eating during the first section), but you’ll want to make the last one shorter so you have time to do the reflection questions (maybe 10-15 minutes).

First thing to do: send me your 100 research questions.

Second thing to do: create a shared google doc with public editing and drop the link into the QualMIP chat. Name it whatever you like. You’ll be putting shared notes in this document as you go around; whenever I talk about a doc or document in the instructions later, that’s the document I mean.

Now go to the dining hall, get some food, and use to decode the first message. You can leave your other stuff behind in the office — you’ll come back here later — but you’ll want (1) somebody’s laptop and (2) somebody’s smartphone.


Lbh’er ybbxvat ng n tyvzcfr bs Byva cnfg — gur qvavat unyy va Snyy 2003 (vapvqragnyyl, guvf jnf bar bs zl qvtvgny cubgbtencul cebwrpgf sbe Cebs. Qbavf-Xryyre).

1. Svaq gur crefcrpgvir gur cubgb jnf gnxra sebz. Gnxr gur fnzr cubgb (be nf pybfr nf lbh pna trg) jvgu fbzrobql’f cubar, naq fraq vg gb gur DhnyZVC tebhc, naq cnfgr vg va gur qbphzrag.

2. Pbzcner gur vzntrf gb gur fprar lbh frr orsber lbh. Jung unf punatrq? (Crbcyr? Pybguvat? Sheavgher? Sbbq? Cyngrf?) Znxr na vairagbel bs punatrf — erzrzore, jr ner ybbxvat sbe bofreinoyr punatrf evtug abj, abg vasreraprf bs punatrf be fcrphyngvbaf nf gb jul guvatf unir punatrq. (Va bgure jbeqf, n gbgny fgenatre gb Byva fubhyq or noyr gb ybbx ng gur fnzr cubgb naq gur fnzr ivrj nf lbh’er ybbxvat ng evtug abj, naq nterr pbzcyrgryl jvgu lbhe vairagbel.) Chg vg va gur DhnyZVC qbphzrag, cersnprq jvgu “vairagbel bs punatrf orgjrra cubgbf naq qvavat unyy.”

3. Svavfu rngvat lbhe yhapu naq zbir ba gb gur frpbaq negvsnpg ol jnyxvat gb gur jbbqra jngresnyy. Bapr lbh trg gurer, qrpbqr gur arkg zrffntr.


Fb… gur Ubabe Pbqr.

1. Ybbx hc ng gur jnyy jvgu nyy gur fvtangherf naq cyndhrf ba vg. Jurer ner gur qbphzragf cynprq? Ubj ner gurl neenatrq? Jung qbrf guvf cynprzrag nssbeq — qb gurl nssbeq ernqvat? Gbhpuvat? Jung zvtug gurfr pubvprf gryy lbh nobhg jung lbh’er “fhccbfrq” gb trg sebz frrvat gurfr qbphzragf urer — jung zrffntr vf guvf fraqvat? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC qbp cersnprq jvgu “Ubabe Pbqr cynprzrag gubhtugf.”

2. Ubj ner gur qbphzragf cynprq naq zbhagrq ba gur jnyy? Ubj ner gurl cebgrpgrq (be abg cebgrpgrq)? Jung qbrf gung gryy lbh nobhg gur vzcbegnapr bs gurfr qbphzragf naq/be gur rkcrpgngvbaf sbe gur raivebazragny pbaqvgvbaf gurl’yy raqher? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC qbp cersnprq jvgu “Ubabe Pbqr cebgrpgvba gubhtugf.”

3. Tb onpx gb gur rkcrevrapr lbh unir bs fvtavat bar bs gur qbphzragf hc ba gur jnyy. Ubj jnf guvf jnyy, gur qbphzrag, naq gur phygheny snpgbef vg flzobyvmrf rkcynvarq gb lbh onpx gura? Jung zngpurf, be qbrf abg zngpu, gur guvatf lbh’ir orra bofreivat? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC pung cersnprq jvgu “Ubabe Pbqr rkcrevrapr gubhtugf.”

4. Jr’er tbvat gb purng n yvggyr: nf zrzoref bs guvf pbzzhavgl, jr xabj gur ahzore bs qbphzragf hc ba guvf jnyy vaperzrag ol bar rirel lrne. Vs lbh qvqa’g xabj gung, jbhyq lbh thrff vg? Jung pyhrf jbhyq lbh hfr? Tvira gur pheerag gvyvat bs qbphzragf ba gur jnyy, jura jvyy jr eha bhg bs ebbz? Qb lbh guvax gur qbphzragf ba gur jnyy jrer nyjnlf neenatrq va gurve pheerag gvyvat? Ubj zvtug lbh svaq bhg (N) jung zvtug unccra jura gur jnyy ehaf bhg bs ebbz va vgf pheerag neenatrzrag naq (O) vs vg jnf neenatrq qvssreragyl va gur cnfg (Qba’g npghnyyl uhag guvf qbja, ohg gryy zr ubj lbh jbhyq tb nobhg vg — n srj qvssrerag jnlf vf rira orggre). Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC pung cersnprq jvgu “Ubabe Pbqr vaperzragvat gubhtugf.”

5. Gur qbphzragf hc ba gur jnyy unir fyvtug qvssreraprf va gurve grkg — gurl’ir orra rqvgrq guebhtu gur lrnef. Vs jr jrer nepunrbybtvfgf, jr zvtug chyy gurz nyy qbja naq ybbx guebhtu gurz yvar ol yvar gb svaq gur qvssreraprf. Ohg evtug abj, jr qba’g unir gung zhpu gvzr. Fb… chyy hc uggc://jjj.byva.rqh/npnqrzvp-yvsr/fghqrag-nssnvef-erfbheprf/fghqrag-yvsr/ubabe-pbqr/ naq qb n pbzcnevfba. Jung unf punatrq? Gnxr jungrire abgrf lbh arrq sbe guvf va gur qbphzrag, cersnprq jvgu “Ubabe pbqr qvssreraprf.”

6. Zvav-fghql qrfvta rkrepvfr gvzr! Vs lbh jrer tbvat gb vairfgvtngr gubfr punatrf — jung gurl zrnag, jub znqr gurz, jul gurl unccrarq, jung qvssrerapr vg znqr — jung birenepuvat erfrnepu dhrfgvba(f) jbhyq lbh nfx, jub jbhyq lbh gnyx gb, naq jung dhrfgvbaf jbhyq lbh nfx gurz? Guvax nobhg 2-3 crbcyr naq orgjrra 3-5 vagreivrj dhrfgvbaf sbe rnpu — gur dhrfgvbaf pna or gur fnzr be qvssrerag sbe rnpu crefba. Jevgr qbja gur erfrnepu dhrfgvba(f), gur crbcyr lbh’q gnetrg gb vagreivrj (rvgure ol anzr be ol trareny qrfpevcgvba/ryvtvovyvgl pevgrevn) naq gur vagreivrj dhrfgvbaf, jvgu n engvbanyr sbe rnpu pubvpr (jul guvf crefba, jung jbhyq lbh ubcr gb nppbzcyvfu? jul guvf dhrfgvba, jung ner lbh gelvat gb trg ng?) Glcr gurz vagb gur qbphzrag, cersnprq ol “Ubabe pbqr vagreivrj cebgbpby.” Erzrzore bhe rkcrevzragf jvgu Ryvmnorgu’f cebgbpby ynfg jrrx; jung ryrzragf qb lbh jnag gb chg vagb lbhe dhrfgvba qrfvta?

7. Purpx lbhe jbex sebz #5. Ybbx ng gur pbzzvg uvfgbel ba uggcf://tvguho.pbz/byva/ubabepbqr — qbrf vg zngpu? Jung qvq lbh zvff, vs nalguvat? Lbh ner rssrpgviryl ybbxvat ng gur fnzr vasbezngvba jvgu gjb qvssrerag vagresnprf… jung qvssrerapr qbrf vg znxr? (Sbe nzhfrzrag naq ybym, frr uggcf://tvguho.pbz/byva/ubabepbqr/chyy/1 — nygubhtu vg qbrf envfr vagrerfgvat dhrfgvbaf nobhg ubj gb ercerfrag npgvbaf va uvfgbel.) Gnxr nal abgrf lbh jnag ba guvf va gur qbphzrag, be srry serr gb fxvc vs gurer’f abguvat lbh jnag gb funer/obhapr bss zr.

8. Jnyx vagb gur NP naq qbja gbjneqf gur znpuvar fubc. Bapr lbh trg gurer, qrpbqr gur arkg zrffntr.


Nyy evtug, guvf vf jurer V fgneg rnfvat hc ba gur fpnssbyqvat n ovg. Lbh’er jnezrq hc… xrrc gelvat gb trg nf zhpu vasbezngvba nf lbh pna ol ybbxvat ng lbhe negvsnpgf naq cynlvat qrgrpgvir.

1. Gur yvggyr oyhr pneq lbh’er ubyqvat… jung vf vg? (Jung ner gur qvssrerag jnlf lbh pbhyq rkcnaq ba gung dhrfgvba — rknzcyrf: jubfr vf vg? jub znqr vg? haqre jung pvephzfgnaprf jnf vg perngrq? jung jnf vgf vagrag? jung qbrf vg gryy lbh nobhg culfvpny rdhvczrag, crqntbtvpny nffhzcgvbaf, yrneavat erdhverzragf, rgp. ng gur gvzr?) Lbhe dhrfgvbaf, nal nafjref lbh pna fcrphyngr, naq gur qngn/bofreingvbaf gung jneenag gung fcrphyngvba, gb gur qbphzrag, haqre gur urnqre “oyhr pneq dhrfgvbaf.”

2. Jung shapgvba qbrf gur yvggyr oyhr pneq freir? Jung flfgrz freirf gung shapgvba ng Byva abj? Qrfpevor vg gb zr, naq gura pbzcner/pbagenfg qrfvta genqrbssf orgjrra gurz. Nafjref gb gur qbphzrag, znxr hc na nccebcevngr urnqre. ;-)

3. Lbh znl jnag gb tb onpx gb gur bssvpr abj. Guvaxvat bayl nobhg gur 3 negvsnpgf lbh’ir hfrq naq gur 3 cynprf lbh’ir bofreirq fb sne, jung vf gur fpbcr — gur obhaqf — bs jung lbh pbhyq znxr pynvzf nobhg ng guvf cbvag? (Sbe vafgnapr: lbh pna’g ernyyl fnl zhpu nobhg cneragny creprcgvbaf bs Byva sebz nal bs gur qngn lbh ybbxrq ng gbqnl… jung pbhyq lbh fnl fbzrguvat nobhg? Fbzr bs gurfr jvyy or jrnxre/fgebatre guna bguref. Juvpu, naq jul?) Nafjref gb qbphzrag, znxr hc na nccebcevngr urnqre. Guvf vf na rkrepvfr va yrneavat ubj gb obhaq lbhe fpbcr naq fgngr lbhe pynvzf.

4. Ybbxvat ng gur abgrf lbh’ir gnxra sbe negvsnpgf N guebhtu P, jung vasreraprf pna lbh znxr nobhg Byva va 2003 if 2016? Xrrc va zvaq gur fpbcr lbh’ir ynvq bhg va #3. Jung ulcbgurfrf pbzr hc va guvf fcnpr, naq jung shegure qngn jbhyq lbh arrq gb tngure va beqre gb grfg lbhe pynvzf? Nafjref gb qbphzrag, znxr hc n urnqre.

Bapr lbh ner qbar jvgu guvf, qrpbqr gur arkg zrffntr.


Yrg’f frr ubj lbh qb jvgubhg fpnssbyqvat — ybbx ng guvf, naq znxr lbhefryirf n frevrf bs dhrfgvbaf (nf va gur cevbe negvsnpgf) gb jnyx guebhtu, gura jnyx guebhtu gurz. Erzrzore, jura lbh tvir zr n ulcbgurfvf be vasrerapr, lbh arrq gb nyfb tvir zr qngn/ernfbaf jul lbh’ir pbzr gb gung pbapyhfvba. Nafjref gb qbphzrag, znxr n fgehpgher gung jvyy jbex sbe lbh.

Bapr lbh ner qbar jvgu guvf, qrpbqr gur ynfg zrffntr.


Teno cbfg-vgf sebz gur jvaqbj oruvaq zl qrfx. Rnpu crefba fubhyq nafjre rnpu dhrfgvba ba bar be zber cbfg-vgf. Svaq n perngvir cynpr gb fgvpx ‘rz… jurer V’yy frr ‘rz jura V trg onpx.

0. Jung fhecevfrq lbh nobhg guvf rkrepvfr? (vs nalguvat; “abguvat” vf n svar nafjre)
1. Jung vf qbphzrag/negvsnpg nanylfvf?
2. Jung pna vg qb — be gryy lbh — gung bgure zrgubqf pna’g? (Juvpu zrgubqf?)
3. Jung pna vg ABG qb — be gryy lbh — gung bgure zrgubqf pna? (Juvpu zrgubqf?)
4. Ubj zvtug lbh hfr gur fgengrtvrf lbh rkcrevzragrq jvgu gbqnl… ba lbhe cebwrpg sbe DhnyZVC?
5. Ubj zvtug lbh hfr gur fgengrtvrf lbh rkcrevzragrq jvgu gbqnl… bhgfvqr bs DhnyZVC?
6. Ubj jbhyq lbh erivfr guvf fpniratre uhag sbe gur arkg DhnyZVC ongpu?

Jura lbh’er qbar, ybbx ng gur zhfvp fgnaq gung’f evtug oruvaq zl bssvpr qbbe. Gurer’f n terra furrg bs cncre ba vg. Ybbx oruvaq gur cncre, naq rawbl.

Cvat zr ba gur pung gung lbh’er qbar, naq tb ubzr. :) Fcraq lbhe svryqjbex gvzr guvf jrrx jbexvat gbjneqf lbhe cebwrpg — srry serr gb nfx, rgp. va pung vs lbh’q yvxr srrqonpx ba nalguvat.

QualMIP and Insper: finding our habitual scripts

Part of the QualMIP series, order introduced here.

What would happen if you had 15 minutes to create a skit portraying a classroom and could only say the word “unicorn”? That was the exercise facing a mixed group of students from Olin and Insper (a new engineering program in Sao Paolo, discount rx Brazil) last week. By taking dialogue away, we wanted to see what other clues the 3 teams used to show us what they were doing.

The results were hilarious — and telling. All three teams had identical staging for their skits, even if they set them up independently. All three skits featured one actor at the front and several actors looking bored in the back, and the dialogue for all of them sounded like this:

ACTOR AT THE FRONT: (sternly) Unicorn. Unicorn, unicorn unicorn.

ACTORS SITTING IN THE BACK, IN ROWS: (look bored, throw paper airplanes, check phones)

ACTOR AT THE FRONT: (authoritatievely) Unicorn unicorn unicorn?

SOMEONE IN THE BACK: (hesitating) Uni… corn?


Immediately afterward, each team vigorously disclaimed that their skits looked nothing like their classroom experiences in college; both Olin and Insper are known for their experimental, hands-on, team project approaches to engineering education. And yet, when communicating “classroom” to an audience, they had perfectly replicated (well, parodied) the traditional lecture setup. Why?

The “lecture” setup, with an authoritative teacher trying (unsuccessfully) to reach a group of passive students, is a deeply ingrained cultural script. We recognize it, and we know others will as well. There are many other portrayals of classrooms one could put on, and they would show far more productive learning setups — but they’re not instantly recognizable as “a classroom.” Because of this, the unsuccessful-lecture setup remains our default shorthand for learning experiences, the same way a wheelchair icon is our default shorthand for “disability” (even if the vast majority of disabled people are not wheelchair users) and a white man in a lab coat with goggles is our default shorthand for “scientist.” Yes, there are many people with invisible disabilities; yes, there are many female scientists of color who don’t wear lab coats on a daily basis… but those are harder to point out, explain, see.

In our post-skit discussion, we talked about how we would need to set up a skit portraying an Olin or Insper classroom. Without (comprehensible) dialogue or huge labels on the costumes — which would also deviate from “reality” — it would be hard to tell who was who and who was doing what. Lectures are performances meant to be put on for an audience; you can drop in and fairly quickly see what’s going on. However, a team discussing the past month of their project has already built a context for themselves that outside observers can’t necessarily penetrate; you’d need some way to point out the web of relationships, fill in past decisions, pick up acronyms. The team’s discussion is already in a shorthand peculiar to them. The faculty dropping into their meeting isn’t running it; they’re more likely to be listening, so it’s hard at a quick glance to tell their “role” apart from that of another student in the classroom. The students may not even be in the classroom at all; they may be in the hallway meeting, in the machine shop fabricating, off-campus testing their prototypes… it’s hard to tell the difference between a team at dinner debating their experimental setup and a group of friends at dinner debating which movie to watch that weekend. (In fact, the same bunch of students may be both groups at once, having both conversations at once.)

When there isn’t a clear “archetype” of an experience, it’s difficult to communicate it; instead of clicking into a pre-arranged “script,” you need to show and explain the details of an unfamiliar context. As observers, it’s difficult to stay aware of our habitual “scripts” — we can prematurely decide a situation is a script we’ve seen before, or try to force a situation into a script that it doesn’t fit, instead of opening our eyes to what’s before us. It’s hard to constantly take part in what is actually happening, as opposed to what our brain is telling us is happening.

This is one of the great challenges of reinventing education. Communication becomes that much harder; we are making up our language as we go along, building shared experiences we will inevitably and initially struggle to communicate to others who have not yet shared them.

Activity design inspired by the Stigler and Hiebert TIMSS video studies of math classrooms around the world; hat-tip to Rehana Patel for the pointer and subsequent discussion on cultural classroom scripts.

QualMIP Week 3: protocol testing and “qual is everywhere”

Part of the QualMIP series, diagnosis  introduced here.

Part 1: Protocol testing

Today’s studio time started with a guest appearance by Elizabeth Doyle, more about who brought a rough protocol draft for her capstone project. Our mission: split into pairs and stress-test the protocol, and trying to misinterpret questions in as many ways as possible. By doing this, we were trying to look at what makes a protocol robust and mature (like last week’s “Life Stories” protocol) vs easily thrown off (which we’d expect from any protocol draft — this is why piloting is so important). Just like code reviews and editing writing are important, testing our qualitative research prototypes is how we iterate and make ethem better.

Part 2: Everything can be a qualitative research project

After a review of what we’ve learned from our experiences so far and a discussion on future project focus, I stepped back and asked the team: how have they observed me using the techniques we’ve been playing with inside our studio time itself — in other words, “if I (Mel) have been treating this engagement as a qualitative research project, what could you say about my study design?”

We discussed my study population and unit of analysis (the three of them, primarily as individuals) and data collection methods (open-ended, opportunistic, blended methodologies; echoing their language a lot, using concrete referents and artifacts to provide commonality) and research question (“how do Olin students experience qualitative fieldwork?”) alongside other things.

The point was to make visible that these techniques aren’t only for formal engagement, they can be used at any time and in any situation. Just like every movement is a dance even if it’s outside a dance studio, every conversation is an interview and every document can be analyzed, and so forth in daily life.

Preparing for next week

Paige, Cesar, and Emily are under instructions to produce at least 100 research questions for their space (“dance meetups”) in any Mel-consumable format, with some sort of organization.

Next week, Mel and Brittany will be on a plane to El Paso, so a scavenger hunt exercise in document analysis will be waiting in the office during our normal studio time.

QualMIP week 2: Interview nonverbals

Part of the QualMIP series, page introduced here.

This is a hurried record that won’t be as eloquent as I would like, but that’s okay. Some record is better than none. The reason: we have visitors from Brazil here this week, and the team opportunistically retooled and bumped a lot of things one week forward so we could teach them some of the things we’ve started exploring in order to give them tools for their own observations of Olin. (I am incredibly proud of them.) So this is… a hurried “wow, they’re coming in half an hour” typing spree on what we did today.

Nonverbals don’t show up on your audio recorder

When conducting qualitative research interviews, researchers often audio-record them for later transcription and analysis. This means that every word you say… is a word you need to transcribe. Sometimes this is great, because you want a verbatim record of something that wouldn’t otherwise show on tape (“okay, you’re laughing now, what’s up?”). Sometimes it’s less great, because your utterances (“uh huh”) are noise rather than signal and add nothing to your analysis. It depends on your goals.

Therefore, it’s useful to think about how to communicate things nonverbally so you can prompt people in ways that don’t — or do — show up in the recording. We played with how to communicate phrases like:

  1. Tell me more!
  2. Why do you say that?
  3. What do you mean by…
  4. Please expand on that.
  5. I don’t understand.
  6. Could you say that again?
  8. Ooh, total empathy here.

Note that we’re also making-visible the disconnect between “what really happened” and what got captured — there’s no such thing as being able to capture everything. In fact, the previous activity is an exercise in manipulating “what happens” so that it either does — or doesn’t — show up.

Interviewing through a window

After discussing nonverbals, we split into three different roles:

  1. An interviewer
  2. An interviewee
  3. Observers

For this exercise, we used a section of the life stories protocol because it was familiar to Olin’s campus (thanks, Jon Adler!) and open-ended while lending itself to taking notes (you could write chapter titles on the whiteboard as you interviewed so observers could easily track where the conversation was).

The interviewer was under instructions to be as nonverbal as possible, as per the previous exercise. The interviewee was instructed to pay attention to what actions of the interviewer made it easier or harder for them to elicit information. Observers watched the interview through a window — we could only see nonverbals. Afterwards, we came back together and the observers did an instant replay of body language and things we could infer about the experience through observation.

This exercise served as a combination interview and observation one, and ideally we’d repeat and rotate through it if we had more time in the semester. We also discussed room and furniture setup and thinking about the effect of various affordances and arrangements in the interview space, and how it might affect you interviewing “down” versus “up” (people who are higher or lower power in status than you, as seen by most of society).

Evening culture skit with Insper students

We went on to plan the evening workshop with Insper students, our visitors from Brazil.

Learning goals:

  • Understand culture and motivation at Olin and Insper
  • Understand power dynamics and its cultural influence

How will we know we’ve won?

  • Participants will be able to articulate observations about culture, power, and motivation differences.


  1. Repeat of the power dynamics warmup from the previous QualMIP, adapted to a larger team.
  2. Divide into two teams of equal size, with one Olin team and one Insper team.
  3. Each team prepares a 1-2 minute skit of what a “normal” classroom looks like for them. Exaggeration and hilarity is encouraged. Each skit is allowed to say one word and one word only, in the vein of Doug Zongker’s “Chicken Chicken Chicken” paper/presentation.
  4. Performances. Laughing.
  5. Break to think alone for a moment.
  6. Pair off (1 Olin student + 1 Insper student in each pair) and interview each other as a first debrief as to what sorts of cultural scripts you saw in operation. This isn’t really a formal interview, it’s more like a slightly extended “pair” in “think pair share.”
  7. Bring back to group discussion and comparison and bringing up of cultural assumptions/norms/etc. The “share” of “think pair share.”

My secret intent here was to have the team engage in the early stages of a (hurried) protocol design process, albeit a heavily interventional one. They came back with all sorts of thoughts about positionality (“how do we introduce ourselves so they jump into the activity with us?”), sensitivity (“I noticed the Insper students were responding to this in the morning…”), a priori frameworks for analysis (“what should we be watching for?”) which we got to (again, hurriedly) pull out in the discussion. Ideally I’d like to have a more extended tour through some of these ideas, but we’ll revisit them in future classes.

Debrief from prior week’s exercise

The team then started filling me in on their fieldwork, using our “debrief the alien” framing for making-strange. (Apparently they visited a metabolism-maintenance station in an institution other than their own, aka the Wellesley cafeteria.) We used the SAID (Situation, Affect, Interpretation, Debrief) framework to talk about the (arbitrary) separation betwee Situation and Interpretation, and the not-one-to-one-or-onto nature of the two.

Next week

Next week we’ll resume with our normal planned activities, discussing memo formats and avoidance techniques before launching into more interview work with a special guest.

Out-of-studio work for this week is nearly nonexistent because of the workshop with Brazilian students. Decompress, memo, and treat yourself well. The exception is Emily, who can’t make the workshop with the Brazilian students and is going to do some sort of follow-up afterwards to see what she can glean of the aftereffects of the event. We’ll compare notes from a distance and it’ll be a good exercise on intersubjectivity. Emily is still supposed to decompress and treat herself well, though.

Part of the QualMIP series, introduced here.