Posts that are engineering edu-ish
I wrote a long reply to a facebook thread on Olin colleague Debbie Chachra’s piece, “Why I Am Not A Maker.” Reposted here with light editing for context.
First, this response is written by someone who built her early career inside the hacker/maker world, working with self-identified hackers/makers and being seen by them as “one of us.” I still identify as such, and it’s an important community — or rather, loose collective of communities — for me.
It is also written by a postmodernist researcher in engineering education, and the response below is my attempt to bring some of those ideas to bear on “making” in an engineering-accessible way. Specifically, part of the debate in the conversation (that I was responding to) is about what “maker” can or should mean (this is a massive oversimplification and not an accurate summary, but it’s the shortest way to provide context for my two-part response, below).
Point 1: Language matters tremendously. The word “Maker” has a shifting, socially constructed meaning — it’s not that the meaning has changed over time, it’s that the meaning is always changing over time, and that it’s always multiple contradictory meanings at once… because so many people use that word in so many contexts and with so many intents.
So when we argue about what the word “maker” means, or what it “really” is or what it “should” be — we’re clutching at solidity that isn’t there. There’s no Platonic Solid of the Definition of Making. We’ve made it, and we continue to make and unmake it, and there is no firm ground.
Point 2: Action matters tremendously. As others have pointed out, language can be reclaimed both by its usage and by taking actions that can then be pointed to by that language. All of us can do this. Language belongs to anyone who speaks (or signs, or writes) it; action belongs to anyone with a will. (Some actions cost some people more than others; some people can take actions others can’t, but everyone can do something, even if it seems to be trivially small.)
In other words: if you think of words as signs pointing to things/events/actions, you can either write different things on the signs, move the signs to point at different (or more or fewer) things, or make more things/events/actions for the signs to point to. And anyone can do one or more of these things in some way.
I see Debbie working to point out the holes in the Big Grand Ol’ Story of Making — the one that holds it up as shiny and wonderful, without acknowledging how it and its claim of “meritocracy” and “everyone can do it” is also dangerously exclusionary. There are people being made invisible by this narrative. Other narratives of making are possible. Debbie’s piece points towards this.
In my mind, there isn’t a “better” narrative of making. Any single narrative that claims to be the only one is going to have flaws. Multiple narratives, multiple viewpoints, multiple contradictory ways of seeing things — there is no single self-consistent system we can make where all is “equal” and “all voices are heard.” That’s why we see people pushing so hard for multiplicity, for getting voices out there; whether they know it or not, they’re working to unmake “The Story Of Making” into thousands of tiny stories.
Part of the QualMIP series, introduced here.
I’m on a research trip this week, so QualMIP gets a scavenger hunt (of sorts; strictly speaking, it doesn’t fit the definition, but they are roaming around campus to look for things) in my absence as a way to start folding document/artifact analysis into the things they’re doing. I’ve included the entirety of their clues document below.
Scavenger hunt start!
Welcome! You’ll find the relevant materials in my office, on the table; there are 4 things labeled with post-it notes (A/B/C/D) for each section. Take them with you as you go about the day.
You’ve got a hard stop at 3pm, so pace yourselves; each section should take about the same amount of time (this accounts for you eating during the first section), but you’ll want to make the last one shorter so you have time to do the reflection questions (maybe 10-15 minutes).
First thing to do: send me your 100 research questions.
Second thing to do: create a shared google doc with public editing and drop the link into the QualMIP chat. Name it whatever you like. You’ll be putting shared notes in this document as you go around; whenever I talk about a doc or document in the instructions later, that’s the document I mean.
Now go to the dining hall, get some food, and use http://www.rot13.com/ to decode the first message. You can leave your other stuff behind in the office — you’ll come back here later — but you’ll want (1) somebody’s laptop and (2) somebody’s smartphone.
## NEGVSNPG N
Lbh’er ybbxvat ng n tyvzcfr bs Byva cnfg — gur qvavat unyy va Snyy 2003 (vapvqragnyyl, guvf jnf bar bs zl qvtvgny cubgbtencul cebwrpgf sbe Cebs. Qbavf-Xryyre).
1. Svaq gur crefcrpgvir gur cubgb jnf gnxra sebz. Gnxr gur fnzr cubgb (be nf pybfr nf lbh pna trg) jvgu fbzrobql’f cubar, naq fraq vg gb gur DhnyZVC tebhc, naq cnfgr vg va gur qbphzrag.
2. Pbzcner gur vzntrf gb gur fprar lbh frr orsber lbh. Jung unf punatrq? (Crbcyr? Pybguvat? Sheavgher? Sbbq? Cyngrf?) Znxr na vairagbel bs punatrf — erzrzore, jr ner ybbxvat sbe bofreinoyr punatrf evtug abj, abg vasreraprf bs punatrf be fcrphyngvbaf nf gb jul guvatf unir punatrq. (Va bgure jbeqf, n gbgny fgenatre gb Byva fubhyq or noyr gb ybbx ng gur fnzr cubgb naq gur fnzr ivrj nf lbh’er ybbxvat ng evtug abj, naq nterr pbzcyrgryl jvgu lbhe vairagbel.) Chg vg va gur DhnyZVC qbphzrag, cersnprq jvgu “vairagbel bs punatrf orgjrra cubgbf naq qvavat unyy.”
3. Svavfu rngvat lbhe yhapu naq zbir ba gb gur frpbaq negvsnpg ol jnyxvat gb gur jbbqra jngresnyy. Bapr lbh trg gurer, qrpbqr gur arkg zrffntr.
## NEGVSNPG O
Fb… gur Ubabe Pbqr.
1. Ybbx hc ng gur jnyy jvgu nyy gur fvtangherf naq cyndhrf ba vg. Jurer ner gur qbphzragf cynprq? Ubj ner gurl neenatrq? Jung qbrf guvf cynprzrag nssbeq — qb gurl nssbeq ernqvat? Gbhpuvat? Jung zvtug gurfr pubvprf gryy lbh nobhg jung lbh’er “fhccbfrq” gb trg sebz frrvat gurfr qbphzragf urer — jung zrffntr vf guvf fraqvat? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC qbp cersnprq jvgu “Ubabe Pbqr cynprzrag gubhtugf.”
2. Ubj ner gur qbphzragf cynprq naq zbhagrq ba gur jnyy? Ubj ner gurl cebgrpgrq (be abg cebgrpgrq)? Jung qbrf gung gryy lbh nobhg gur vzcbegnapr bs gurfr qbphzragf naq/be gur rkcrpgngvbaf sbe gur raivebazragny pbaqvgvbaf gurl’yy raqher? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC qbp cersnprq jvgu “Ubabe Pbqr cebgrpgvba gubhtugf.”
3. Tb onpx gb gur rkcrevrapr lbh unir bs fvtavat bar bs gur qbphzragf hc ba gur jnyy. Ubj jnf guvf jnyy, gur qbphzrag, naq gur phygheny snpgbef vg flzobyvmrf rkcynvarq gb lbh onpx gura? Jung zngpurf, be qbrf abg zngpu, gur guvatf lbh’ir orra bofreivat? Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC pung cersnprq jvgu “Ubabe Pbqr rkcrevrapr gubhtugf.”
4. Jr’er tbvat gb purng n yvggyr: nf zrzoref bs guvf pbzzhavgl, jr xabj gur ahzore bs qbphzragf hc ba guvf jnyy vaperzrag ol bar rirel lrne. Vs lbh qvqa’g xabj gung, jbhyq lbh thrff vg? Jung pyhrf jbhyq lbh hfr? Tvira gur pheerag gvyvat bs qbphzragf ba gur jnyy, jura jvyy jr eha bhg bs ebbz? Qb lbh guvax gur qbphzragf ba gur jnyy jrer nyjnlf neenatrq va gurve pheerag gvyvat? Ubj zvtug lbh svaq bhg (N) jung zvtug unccra jura gur jnyy ehaf bhg bs ebbz va vgf pheerag neenatrzrag naq (O) vs vg jnf neenatrq qvssreragyl va gur cnfg (Qba’g npghnyyl uhag guvf qbja, ohg gryy zr ubj lbh jbhyq tb nobhg vg — n srj qvssrerag jnlf vf rira orggre). Qvfphff, gura glcr fbzr gubhtugf ba guvf vagb gur DhnyZVC pung cersnprq jvgu “Ubabe Pbqr vaperzragvat gubhtugf.”
5. Gur qbphzragf hc ba gur jnyy unir fyvtug qvssreraprf va gurve grkg — gurl’ir orra rqvgrq guebhtu gur lrnef. Vs jr jrer nepunrbybtvfgf, jr zvtug chyy gurz nyy qbja naq ybbx guebhtu gurz yvar ol yvar gb svaq gur qvssreraprf. Ohg evtug abj, jr qba’g unir gung zhpu gvzr. Fb… chyy hc uggc://jjj.byva.rqh/npnqrzvp-yvsr/fghqrag-nssnvef-erfbheprf/fghqrag-yvsr/ubabe-pbqr/ naq qb n pbzcnevfba. Jung unf punatrq? Gnxr jungrire abgrf lbh arrq sbe guvf va gur qbphzrag, cersnprq jvgu “Ubabe pbqr qvssreraprf.”
6. Zvav-fghql qrfvta rkrepvfr gvzr! Vs lbh jrer tbvat gb vairfgvtngr gubfr punatrf — jung gurl zrnag, jub znqr gurz, jul gurl unccrarq, jung qvssrerapr vg znqr — jung birenepuvat erfrnepu dhrfgvba(f) jbhyq lbh nfx, jub jbhyq lbh gnyx gb, naq jung dhrfgvbaf jbhyq lbh nfx gurz? Guvax nobhg 2-3 crbcyr naq orgjrra 3-5 vagreivrj dhrfgvbaf sbe rnpu — gur dhrfgvbaf pna or gur fnzr be qvssrerag sbe rnpu crefba. Jevgr qbja gur erfrnepu dhrfgvba(f), gur crbcyr lbh’q gnetrg gb vagreivrj (rvgure ol anzr be ol trareny qrfpevcgvba/ryvtvovyvgl pevgrevn) naq gur vagreivrj dhrfgvbaf, jvgu n engvbanyr sbe rnpu pubvpr (jul guvf crefba, jung jbhyq lbh ubcr gb nppbzcyvfu? jul guvf dhrfgvba, jung ner lbh gelvat gb trg ng?) Glcr gurz vagb gur qbphzrag, cersnprq ol “Ubabe pbqr vagreivrj cebgbpby.” Erzrzore bhe rkcrevzragf jvgu Ryvmnorgu’f cebgbpby ynfg jrrx; jung ryrzragf qb lbh jnag gb chg vagb lbhe dhrfgvba qrfvta?
7. Purpx lbhe jbex sebz #5. Ybbx ng gur pbzzvg uvfgbel ba uggcf://tvguho.pbz/byva/ubabepbqr — qbrf vg zngpu? Jung qvq lbh zvff, vs nalguvat? Lbh ner rssrpgviryl ybbxvat ng gur fnzr vasbezngvba jvgu gjb qvssrerag vagresnprf… jung qvssrerapr qbrf vg znxr? (Sbe nzhfrzrag naq ybym, frr uggcf://tvguho.pbz/byva/ubabepbqr/chyy/1 — nygubhtu vg qbrf envfr vagrerfgvat dhrfgvbaf nobhg ubj gb ercerfrag npgvbaf va uvfgbel.) Gnxr nal abgrf lbh jnag ba guvf va gur qbphzrag, be srry serr gb fxvc vs gurer’f abguvat lbh jnag gb funer/obhapr bss zr.
8. Jnyx vagb gur NP naq qbja gbjneqf gur znpuvar fubc. Bapr lbh trg gurer, qrpbqr gur arkg zrffntr.
## NEGVSNPG P
Nyy evtug, guvf vf jurer V fgneg rnfvat hc ba gur fpnssbyqvat n ovg. Lbh’er jnezrq hc… xrrc gelvat gb trg nf zhpu vasbezngvba nf lbh pna ol ybbxvat ng lbhe negvsnpgf naq cynlvat qrgrpgvir.
1. Gur yvggyr oyhr pneq lbh’er ubyqvat… jung vf vg? (Jung ner gur qvssrerag jnlf lbh pbhyq rkcnaq ba gung dhrfgvba — rknzcyrf: jubfr vf vg? jub znqr vg? haqre jung pvephzfgnaprf jnf vg perngrq? jung jnf vgf vagrag? jung qbrf vg gryy lbh nobhg culfvpny rdhvczrag, crqntbtvpny nffhzcgvbaf, yrneavat erdhverzragf, rgp. ng gur gvzr?) Lbhe dhrfgvbaf, nal nafjref lbh pna fcrphyngr, naq gur qngn/bofreingvbaf gung jneenag gung fcrphyngvba, gb gur qbphzrag, haqre gur urnqre “oyhr pneq dhrfgvbaf.”
2. Jung shapgvba qbrf gur yvggyr oyhr pneq freir? Jung flfgrz freirf gung shapgvba ng Byva abj? Qrfpevor vg gb zr, naq gura pbzcner/pbagenfg qrfvta genqrbssf orgjrra gurz. Nafjref gb gur qbphzrag, znxr hc na nccebcevngr urnqre. ;-)
3. Lbh znl jnag gb tb onpx gb gur bssvpr abj. Guvaxvat bayl nobhg gur 3 negvsnpgf lbh’ir hfrq naq gur 3 cynprf lbh’ir bofreirq fb sne, jung vf gur fpbcr — gur obhaqf — bs jung lbh pbhyq znxr pynvzf nobhg ng guvf cbvag? (Sbe vafgnapr: lbh pna’g ernyyl fnl zhpu nobhg cneragny creprcgvbaf bs Byva sebz nal bs gur qngn lbh ybbxrq ng gbqnl… jung pbhyq lbh fnl fbzrguvat nobhg? Fbzr bs gurfr jvyy or jrnxre/fgebatre guna bguref. Juvpu, naq jul?) Nafjref gb qbphzrag, znxr hc na nccebcevngr urnqre. Guvf vf na rkrepvfr va yrneavat ubj gb obhaq lbhe fpbcr naq fgngr lbhe pynvzf.
4. Ybbxvat ng gur abgrf lbh’ir gnxra sbe negvsnpgf N guebhtu P, jung vasreraprf pna lbh znxr nobhg Byva va 2003 if 2016? Xrrc va zvaq gur fpbcr lbh’ir ynvq bhg va #3. Jung ulcbgurfrf pbzr hc va guvf fcnpr, naq jung shegure qngn jbhyq lbh arrq gb tngure va beqre gb grfg lbhe pynvzf? Nafjref gb qbphzrag, znxr hc n urnqre.
Bapr lbh ner qbar jvgu guvf, qrpbqr gur arkg zrffntr.
## NEGVSNPG Q
Yrg’f frr ubj lbh qb jvgubhg fpnssbyqvat — ybbx ng guvf, naq znxr lbhefryirf n frevrf bs dhrfgvbaf (nf va gur cevbe negvsnpgf) gb jnyx guebhtu, gura jnyx guebhtu gurz. Erzrzore, jura lbh tvir zr n ulcbgurfvf be vasrerapr, lbh arrq gb nyfb tvir zr qngn/ernfbaf jul lbh’ir pbzr gb gung pbapyhfvba. Nafjref gb qbphzrag, znxr n fgehpgher gung jvyy jbex sbe lbh.
Bapr lbh ner qbar jvgu guvf, qrpbqr gur ynfg zrffntr.
## ERSYRPGVBA DHRFGVBAF
Teno cbfg-vgf sebz gur jvaqbj oruvaq zl qrfx. Rnpu crefba fubhyq nafjre rnpu dhrfgvba ba bar be zber cbfg-vgf. Svaq n perngvir cynpr gb fgvpx ‘rz… jurer V’yy frr ‘rz jura V trg onpx.
0. Jung fhecevfrq lbh nobhg guvf rkrepvfr? (vs nalguvat; “abguvat” vf n svar nafjre)
1. Jung vf qbphzrag/negvsnpg nanylfvf?
2. Jung pna vg qb — be gryy lbh — gung bgure zrgubqf pna’g? (Juvpu zrgubqf?)
3. Jung pna vg ABG qb — be gryy lbh — gung bgure zrgubqf pna? (Juvpu zrgubqf?)
4. Ubj zvtug lbh hfr gur fgengrtvrf lbh rkcrevzragrq jvgu gbqnl… ba lbhe cebwrpg sbe DhnyZVC?
5. Ubj zvtug lbh hfr gur fgengrtvrf lbh rkcrevzragrq jvgu gbqnl… bhgfvqr bs DhnyZVC?
6. Ubj jbhyq lbh erivfr guvf fpniratre uhag sbe gur arkg DhnyZVC ongpu?
Jura lbh’er qbar, ybbx ng gur zhfvp fgnaq gung’f evtug oruvaq zl bssvpr qbbe. Gurer’f n terra furrg bs cncre ba vg. Ybbx oruvaq gur cncre, naq rawbl.
Cvat zr ba gur pung gung lbh’er qbar, naq tb ubzr. :) Fcraq lbhe svryqjbex gvzr guvf jrrx jbexvat gbjneqf lbhe cebwrpg — srry serr gb nfx, rgp. va pung vs lbh’q yvxr srrqonpx ba nalguvat.
Writing out loud for clarity of thought. Here’s what I think I know about my EFAL project (engineering faculty as learners).
My current writeup of EFAL is a postmodern work. This is causing massive problems. While I’m instinctively good at being postmodern, I’ve internalized it so much that I’m struggling to write about postmodernism in a way that lets others join me in the space. Postmodernism is about things like constant flux and change and tension, and my attempts to write that way so far have come out as disorganized, hand-wavy blathering. And while I can write like Lewis Carroll or Douglas Hofstadter, that’s not the way I’ll make an academic argument that others will recognize.
Patti suggested a possible solution last night: write it 2-3 different ways. Write the postmodern way, focused on deconstruction. But also write it the critical theory way, focused on emancipation (which I’m almost doing already in my discussion-like sections). And write it in some interpretivist way, focused on understanding (a familiar mode within my discipline, within which phenomenology, narrative analysis, grounded theory, and stuff like that lie). Or whatever ways are recognizable. Show the “same” thing through different lens; triangulate aggressively.
Immediately I went: that is an excellent suggestion; I am technically already doing this, but I’m not being delibrerate about pointing them out as different lens, and if I specifically addressed three –
And then my next thought was: well, crap. My results have got to get smaller. Again. I thought I’d cut them down — multiple times — but what could I distill out even more, more, more so that I could see it through three lens… and finish writing soon, and not die?
I’m looking at ontology — understanding and challenging ontological conceptions of engineering faculty as learners (in their narratives of curricular design). Ontological means “related to the nature of being,” so that’s basically: what kinds of things are there in the world? What roles are they allowed to play in relation to each other? How do we chunk and connect reality — to borrow a CS term, what’s our object model?
And what I’m doing is pulling out ontologies from underneath those stories, and I’m… really, I’m playing wit hthem. I’m not proposing a “better” ontology. I’m not creating a categorization of ontologies. I’m not saying our current ontologies (really, ontologies) are AWFUL HORRIBLE BAD WRONG. I sort of do parts of all of these things, but to narrow it down to only one of them is… inaccurate. I feel like I’m a puppy gamboling around going well let’s just play with all of this! all of the things!!! – because I’m playing with the notion that we need to have an ontology, tugging at the way we see reality and going “see, see, it’s all melty… it’s all melty… it’ll always be all melty… come and play!”
Clearly I need some better words for this, but I know what I’m going to read to work on that. Whee, I’m going to revise again. Again again. Why.
This afternoon, Julia suggested that I look at engineering faculty ontologies implicit in the Big Official Reports on the State of Engineering Education And So Forth. Which is a great idea, and might solve the problem of the EFAL lit review being in shambles when it comes to contextualizing EFAL’s work in the domain of engineering education. So. That, too. Yes.
I’m glad Robin is willing to let me stay in that space trying to say things, instead of hauling me back to places where I can currently speak in ways she can understand — it’s not every advisor that does that. I know it’s painful for us both right now, and every conversation makes me want to pull my hair out — but I’m glad we’re having them. And I’m glad I finally called Patti last night and admitted I needed postmodern eyeballs on my (attempting-to-be) postmodern work. And glad for all my friends: Ben sitting me down in his office for writing accountability, Alexandra and Stephanie reminding me of all that lies ahead, Sebastian mailing chocolate, Todd letting me rant tantrums at him, Julia… gosh. I can’t list them all. They can’t carry my dissertation, but they can carry me. And so they are.
Part of the QualMIP series, introduced here.
What would happen if you had 15 minutes to create a skit portraying a classroom and could only say the word “unicorn”? That was the exercise facing a mixed group of students from Olin and Insper (a new engineering program in Sao Paolo, Brazil) last week. By taking dialogue away, we wanted to see what other clues the 3 teams used to show us what they were doing.
The results were hilarious — and telling. All three teams had identical staging for their skits, even if they set them up independently. All three skits featured one actor at the front and several actors looking bored in the back, and the dialogue for all of them sounded like this:
ACTOR AT THE FRONT: (sternly) Unicorn. Unicorn, unicorn unicorn.
ACTORS SITTING IN THE BACK, IN ROWS: (look bored, throw paper airplanes, check phones)
ACTOR AT THE FRONT: (authoritatievely) Unicorn unicorn unicorn?
SOMEONE IN THE BACK: (hesitating) Uni… corn?
ACTOR AT THE FRONT: (frustrated) UNICORN UNICORN UNICORN. UNICORN. (points emphatically) UNICORN!
Immediately afterward, each team vigorously disclaimed that their skits looked nothing like their classroom experiences in college; both Olin and Insper are known for their experimental, hands-on, team project approaches to engineering education. And yet, when communicating “classroom” to an audience, they had perfectly replicated (well, parodied) the traditional lecture setup. Why?
The “lecture” setup, with an authoritative teacher trying (unsuccessfully) to reach a group of passive students, is a deeply ingrained cultural script. We recognize it, and we know others will as well. There are many other portrayals of classrooms one could put on, and they would show far more productive learning setups — but they’re not instantly recognizable as “a classroom.” Because of this, the unsuccessful-lecture setup remains our default shorthand for learning experiences, the same way a wheelchair icon is our default shorthand for “disability” (even if the vast majority of disabled people are not wheelchair users) and a white man in a lab coat with goggles is our default shorthand for “scientist.” Yes, there are many people with invisible disabilities; yes, there are many female scientists of color who don’t wear lab coats on a daily basis… but those are harder to point out, explain, see.
In our post-skit discussion, we talked about how we would need to set up a skit portraying an Olin or Insper classroom. Without (comprehensible) dialogue or huge labels on the costumes — which would also deviate from “reality” — it would be hard to tell who was who and who was doing what. Lectures are performances meant to be put on for an audience; you can drop in and fairly quickly see what’s going on. However, a team discussing the past month of their project has already built a context for themselves that outside observers can’t necessarily penetrate; you’d need some way to point out the web of relationships, fill in past decisions, pick up acronyms. The team’s discussion is already in a shorthand peculiar to them. The faculty dropping into their meeting isn’t running it; they’re more likely to be listening, so it’s hard at a quick glance to tell their “role” apart from that of another student in the classroom. The students may not even be in the classroom at all; they may be in the hallway meeting, in the machine shop fabricating, off-campus testing their prototypes… it’s hard to tell the difference between a team at dinner debating their experimental setup and a group of friends at dinner debating which movie to watch that weekend. (In fact, the same bunch of students may be both groups at once, having both conversations at once.)
When there isn’t a clear “archetype” of an experience, it’s difficult to communicate it; instead of clicking into a pre-arranged “script,” you need to show and explain the details of an unfamiliar context. As observers, it’s difficult to stay aware of our habitual “scripts” — we can prematurely decide a situation is a script we’ve seen before, or try to force a situation into a script that it doesn’t fit, instead of opening our eyes to what’s before us. It’s hard to constantly take part in what is actually happening, as opposed to what our brain is telling us is happening.
This is one of the great challenges of reinventing education. Communication becomes that much harder; we are making up our language as we go along, building shared experiences we will inevitably and initially struggle to communicate to others who have not yet shared them.
Activity design inspired by the Stigler and Hiebert TIMSS video studies of math classrooms around the world; hat-tip to Rehana Patel for the pointer and subsequent discussion on cultural classroom scripts.
Part of the QualMIP series, introduced here.
Part 1: Protocol testing
Today’s studio time started with a guest appearance by Elizabeth Doyle, who brought a rough protocol draft for her capstone project. Our mission: split into pairs and stress-test the protocol, trying to misinterpret questions in as many ways as possible. By doing this, we were trying to look at what makes a protocol robust and mature (like last week’s “Life Stories” protocol) vs easily thrown off (which we’d expect from any protocol draft — this is why piloting is so important). Just like code reviews and editing writing are important, testing our qualitative research prototypes is how we iterate and make ethem better.
Part 2: Everything can be a qualitative research project
After a review of what we’ve learned from our experiences so far and a discussion on future project focus, I stepped back and asked the team: how have they observed me using the techniques we’ve been playing with inside our studio time itself — in other words, “if I (Mel) have been treating this engagement as a qualitative research project, what could you say about my study design?”
We discussed my study population and unit of analysis (the three of them, primarily as individuals) and data collection methods (open-ended, opportunistic, blended methodologies; echoing their language a lot, using concrete referents and artifacts to provide commonality) and research question (“how do Olin students experience qualitative fieldwork?”) alongside other things.
The point was to make visible that these techniques aren’t only for formal engagement, they can be used at any time and in any situation. Just like every movement is a dance even if it’s outside a dance studio, every conversation is an interview and every document can be analyzed, and so forth in daily life.
Preparing for next week
Paige, Cesar, and Emily are under instructions to produce at least 100 research questions for their space (“dance meetups”) in any Mel-consumable format, with some sort of organization.
Next week, Mel and Brittany will be on a plane to El Paso, so a scavenger hunt exercise in document analysis will be waiting in the office during our normal studio time.
Part of the QualMIP series, introduced here.
This is a hurried record that won’t be as eloquent as I would like, but that’s okay. Some record is better than none. The reason: we have visitors from Brazil here this week, and the team opportunistically retooled and bumped a lot of things one week forward so we could teach them some of the things we’ve started exploring in order to give them tools for their own observations of Olin. (I am incredibly proud of them.) So this is… a hurried “wow, they’re coming in half an hour” typing spree on what we did today.
Nonverbals don’t show up on your audio recorder
When conducting qualitative research interviews, researchers often audio-record them for later transcription and analysis. This means that every word you say… is a word you need to transcribe. Sometimes this is great, because you want a verbatim record of something that wouldn’t otherwise show on tape (“okay, you’re laughing now, what’s up?”). Sometimes it’s less great, because your utterances (“uh huh”) are noise rather than signal and add nothing to your analysis. It depends on your goals.
Therefore, it’s useful to think about how to communicate things nonverbally so you can prompt people in ways that don’t — or do — show up in the recording. We played with how to communicate phrases like:
- Tell me more!
- Why do you say that?
- What do you mean by…
- Please expand on that.
- I don’t understand.
- Could you say that again?
- THAT’S SO COOL!
- Ooh, total empathy here.
Note that we’re also making-visible the disconnect between “what really happened” and what got captured — there’s no such thing as being able to capture everything. In fact, the previous activity is an exercise in manipulating “what happens” so that it either does — or doesn’t — show up.
Interviewing through a window
After discussing nonverbals, we split into three different roles:
- An interviewer
- An interviewee
For this exercise, we used a section of the life stories protocol because it was familiar to Olin’s campus (thanks, Jon Adler!) and open-ended while lending itself to taking notes (you could write chapter titles on the whiteboard as you interviewed so observers could easily track where the conversation was).
The interviewer was under instructions to be as nonverbal as possible, as per the previous exercise. The interviewee was instructed to pay attention to what actions of the interviewer made it easier or harder for them to elicit information. Observers watched the interview through a window — we could only see nonverbals. Afterwards, we came back together and the observers did an instant replay of body language and things we could infer about the experience through observation.
This exercise served as a combination interview and observation one, and ideally we’d repeat and rotate through it if we had more time in the semester. We also discussed room and furniture setup and thinking about the effect of various affordances and arrangements in the interview space, and how it might affect you interviewing “down” versus “up” (people who are higher or lower power in status than you, as seen by most of society).
Evening culture skit with Insper students
We went on to plan the evening workshop with Insper students, our visitors from Brazil.
- Understand culture and motivation at Olin and Insper
- Understand power dynamics and its cultural influence
How will we know we’ve won?
- Participants will be able to articulate observations about culture, power, and motivation differences.
- Repeat of the power dynamics warmup from the previous QualMIP, adapted to a larger team.
- Divide into two teams of equal size, with one Olin team and one Insper team.
- Each team prepares a 1-2 minute skit of what a “normal” classroom looks like for them. Exaggeration and hilarity is encouraged. Each skit is allowed to say one word and one word only, in the vein of Doug Zongker’s “Chicken Chicken Chicken” paper/presentation.
- Performances. Laughing.
- Break to think alone for a moment.
- Pair off (1 Olin student + 1 Insper student in each pair) and interview each other as a first debrief as to what sorts of cultural scripts you saw in operation. This isn’t really a formal interview, it’s more like a slightly extended “pair” in “think pair share.”
- Bring back to group discussion and comparison and bringing up of cultural assumptions/norms/etc. The “share” of “think pair share.”
My secret intent here was to have the team engage in the early stages of a (hurried) protocol design process, albeit a heavily interventional one. They came back with all sorts of thoughts about positionality (“how do we introduce ourselves so they jump into the activity with us?”), sensitivity (“I noticed the Insper students were responding to this in the morning…”), a priori frameworks for analysis (“what should we be watching for?”) which we got to (again, hurriedly) pull out in the discussion. Ideally I’d like to have a more extended tour through some of these ideas, but we’ll revisit them in future classes.
Debrief from prior week’s exercise
The team then started filling me in on their fieldwork, using our “debrief the alien” framing for making-strange. (Apparently they visited a metabolism-maintenance station in an institution other than their own, aka the Wellesley cafeteria.) We used the SAID (Situation, Affect, Interpretation, Debrief) framework to talk about the (arbitrary) separation betwee Situation and Interpretation, and the not-one-to-one-or-onto nature of the two.
Next week we’ll resume with our normal planned activities, discussing memo formats and avoidance techniques before launching into more interview work with a special guest.
Out-of-studio work for this week is nearly nonexistent because of the workshop with Brazilian students. Decompress, memo, and treat yourself well. The exception is Emily, who can’t make the workshop with the Brazilian students and is going to do some sort of follow-up afterwards to see what she can glean of the aftereffects of the event. We’ll compare notes from a distance and it’ll be a good exercise on intersubjectivity. Emily is still supposed to decompress and treat herself well, though.
Part of the QualMIP series, introduced here.
This is a quick sketch of a far more comprehensive series of thoughts I’ve been swimming in for the past… I don’t know how long. It’s about curriculum design (or redesign). One metanarrative we sometimes say is that “faculty make the curriculum for the students” — in other words, it’s by the faculty and for the students, and those are three distinct and separate things.
This is a true statement. But it’s not the only true statement we could make.
1. Curriculum is also “for the faculty” — we teach things based on our own learning/career/etc. goals, we assign people to instruct classes we think they’ll learn from, we go into teaching and curriculum design experiences as growth opportunities for ourselves as well.
2. Curriculum is also “of the faculty” — our course designs come from our prior experiences, interests, history; we bring ourselves into the room. (The same course taught by 2 professors in different sections… is not the same course at all.)
3. Curriculum is also “by the students” — they give suggestions, shape the course, TA it, send feedback, set up out-of-class study sessions, pass notes around, etc.
4. Curriculum is also “of the students” — they bring in hopes and dreams and aspirations, baggage, preparation and/or lack thereof, relationships, skills, temperaments, vocabulary… the classic experience of teaching 2 sections of the same course in one semester and having the sections be wildly, wildly different? That’s this bit.
(“for” and “of” and “by” all blur together, too, as you may have noticed.)
So basically, curriculum is by, for, and of both faculty and students — and being willing to (at least temporarily, mentally) suspend those boundaries has intriguing implications for the student/faculty relationship/roles — much closer to a junior/senior partnership than the usual high power-distance separation. Which is transformative for students from an empowerment and identity perspective, for faculty, for the curriculum… etc.
If I had to summarize about 50% of what I’m working on now, this would be it.
A friend asked for some pointers towards “engineering identity” materials. Here’s what I shared — this isn’t comprehensive, but rather it’s material I thought was cool.
From what I’ve seen, most engineering identity stuff is related to engineering epistemology (“what is engineering, anyway?”) and/or underrepresented groups (“how does minority X understand themselves as engineers?”.
My favorite “what is engineering, anyway — and who is an engineer?” instrument is simultaneously adorable and disturbing. It’s the Draw-an-engineer test, or the DAET. Basically, you get kids — or anyone, but it’s usually kids — to draw whatever they think an engineer looks like. Afterwards, you analyze the drawings to see what kinds of assumptions they’re making: are all the engineer-drawings of white, apparently abled, conventionally-dressed men wearing safety goggles and standing on a train? (Sometimes they are.)
Then there’s the space of “how do engineering identities come to be?” Kerry Meyers has explored this space. Here’s a short version of some of her early findings. Among other things, she found that language is a huge deal — what you call people matters, and it matters from the very beginning. She also describes engineering identity development as a staged developmental process, not a binary “you have no engineering identity… and… BING! Now you magically do!”
Folks have also started to look at how different underrepresented groups (race, gender, class, disability, etc. etc. etc.) construct their engineering identities compared to the dominant group. Here’s one brief on the gender perspective, but researchers are looking at indigenous students, young black men, and many more.
To my knowledge, very few engineering programs have explicit curricular discussion of students’ engineering identities. Smith and UTEP’s E-LEAD program are two exceptions with required first-year courses that provide devoted space to the topic. I’d love to see more of this sort of exploration, since developing a personal engineering identity can help students persist in bridging their engineering knowledge into their lives and out to a world that needs it.
Part of the QualMIP series, introduced here.
A format that seems to work for now:
- Starting discussion
- Studio time with guided exercises
- Fieldwork kickout 1
- Discussion 1
- Fieldwork kickout 2
- Discussion 2 and wrap-up
One of our big ideas for the semester is sensitizing ourselves as research instruments. Therefore, rule #1 of the semester is that we must take care of ourselves. Sleep, eat, do things that make you happy. This is officially homework.
When was taking my own qualitative methods courses, one thing I noticed is that my (hearing) classmates seemed to focus first and foremost on the dialogue — words, what was said. I want to start out trying to shake you up from your habits a little bit by specifically not allowing you to do that. Today we’re going to focus on everything that isn’t words.
Studio time 1:
1. Sculpture garden: show us what you look like when you’re studying — now take turns picking one person’s posture to be the “sculpture” and fine-tuning and adjusting the postures of the other student(s) so that they match the sculpture. Notice: bodies are different, and that it’s easier to imitate some people than others. (Exercise inspired by Mary Pilat’s workshops on trauma)
2. Posture amplification: take turns showing us what you look like when you’re really interested in something… and really bored by it. Walk around the “model” student — what are the physical indications of interest/disinterest? Can you amplify/mute them — what does it look like when it’s big and obvious? Subtle? Notice: we can’t tell someone’s “true” interior state, even if they tell us what they claim their state is; we can only guess and make our own observations. (Exercise inspired by Sally Wallace’s choreography course)
3. Slideshow: I’m going to close my eyes and count to 10. Before I open my eyes, I want you to arrange yourselves to show me that one of you is dominant over the other two — use position, gesture, height, etc. Rotate who’s dominant. No talking. We’ll do this 10 times. Afterwards, let’s discuss: what indicators did you use? How did you communicate? (Exercise inspired by Rebecca Bryant’s improvisation course)
4. Conversational arrangements: Try out different positions for discussion — facing each other, angled, far, near, close — how do they feel, and how do they affect the quality of the conversation? What affordances do they give you? What can you do from a distance to the conversation — how is your position different?
Go to the dining hall. Don’t sit together, don’t communicate; come back with 3 observations of different types of what you see.
1. Share observations with the group. What did you notice/do? How did you interact? (Because even if I said “no interacting,” you did interact — waves, nods, etc — or if you didn’t, that probably felt weird.)
2. Discussion of how to become conscious of our default interpretations, and how to back up to explain that interpretation so others could see how a reasonable person would reach it. What makes you think the two people you watched were friends? How else could you have interpreted that situation?
3. Positionality. What is it? What was yours? What are the different decisions you might make, and what assumptions about research and the role or research might it imply? New vocabulary: agential cut / Bohrian cut.
Go somewhere together — studio, etc — and observe, but this time as a team. Sensitize to yourself again, but this time use your teammates as instruments as well — what do their reactions tell you? Knowing that your teammates are watching your reactions, what will you do to make it clear (or not) to them what you are doing or thinking about?
1. What did you notice in the thing you were describing? Try to help me guess what it is without using any names or titles — this is a “making strange.” (New term!) Pretend I’m an alien from another planet.
2. What did you notice about each other and how you communicated?
During your 3 hours of fieldwork (including prep) this week, you should get closer to three things:
1. Bounding your project for the semester. What population will I be observing, what will I be looking for, how will I be looking for it… what is interesting?
2. Sensitizing to your own reactions, specifically when you disengage. What makes you disconnect from a situation, what do you tend to do in order to disconnect (shift back? check phone? write? sing inside your head?), and what indicators can you use to flag awareness of your own engagement level? Longer is NOT necessarily better — how long can you reasonably sustain it?
3. Playing with memo formats. Experiment with at least 3 different formats for your private memos this week; Use different mediums, different human languages (digital writing counts as one medium, so writing in notepad vs word does not count as different mediums). Example mediums: audio recordings, webcam video, sketching, handwriting in Spanish, typing in Swedish, livejournal, collage, etc. Remember that memos are one level abstracted from raw data — so video of your interview is not a memo, but a reflection from you on the interview afterwards can be. (Also remember that your private memos only need to be understandable by you as long as the end of the semester. We’re used to outputting for the consumption of others, and this is not necessarily how we best produce for ourselves as things-to-think-with.)
We will begin in “alien from another planet” format during our discussion next week.
Publicly documenting QualMIP (Qualitative Methods In Practice), an Olin group independent study with Paige, Cesar, and Emily — I’ve been waiting to teach this ever since my senior year of undergrad when I took my first qual course, so I am very excited to be working with this group.
We had a short pre-semester meeting tonight to get on the same page, since we’re codesigning/evolving the experience together instead of me dictating everything (I think the word “facilitating” or “hosting” were used). A few important things we’re agreeing on:
Sustainability is key. Two credits this semester means 3 hours in studio (one block during the week) and a maximum of 3 hours of work outside of class. If you’re running up on 3 hours, document and stop. We are learning process, and pacing ourselves is part of that process; it is more important than the finished product we may have predicted. Gauge by process, and learn to scale your product to match the time budget you have.
Focus on fieldwork. Qualitative research methods is a huge topic that one could take a lifetime to master. We want to hone the skills of presence and attention in the moment of data collection, be that observation, interview, artifact analysis (yes, the boundaries blur), etc. This means we won’t be doing readings, won’t be writing papers, and will optimize subject selection for convenience in accessing.
Private and portfolio deliverables. Everyone will be keeping two types of documents/artifacts throughout the semester: private memos (in any format/language) that are only for your future self (and whoever else you choose to share them with) and will never be required to turn in, and portfolio documents, which are assignments to be written from the material in your private memos, and do get turned in and otherwise publicly shared.
One grace week. You can miss studio for one week, on any week, no questions asked. You can also not do fieldwork for one week, on any week, no questions asked. Just give the group a heads-up in advance. Any absences/work-skips beyond that will need to be negotiated with the whole group in terms of how it affects assessment.
Formative, nonstandardized feedback. I’ll be giving formative feedback on how I see you developing your fieldwork style, not comparing you to a standardized rubric/method.
There’s more that I’ve forgotten, I’m sure. In any case, the Week Zero “assignment” is for everyone to get CITI certified (optional for Monday, but you’ll need it later on so you might as well do it now).
The agenda will evolve in response to where student projects go; we know we’ll spend the first 3 weeks settling on a study population, the last 3 weeks in “choose your own adventure” format where people can return to their favorite techniques, and the ones in between doing a survey of as many techniques as we can reasonably get to. Paige asked for the first 3 weeks to be spec’d out
Paige asked for the first 3 weeks to be spec’d out, so here they are:
Week 1: Nonverbals: groupings and gestures (Exercise: attention memos)
Week 2: Agential cuts: slicing and dicing units of analysis (Exercise: life story interviews)
Week 3: Interview protocols and practices (Exercise: more interviews)