Posts that are engineering edu-ish
Processing thoughts from Wednesday’s conversation with Ruth, for my own remembrance later. Probably won’t make sense to others, and that’s fine.
Translational work is important, especially if I’m doing unconventional things. Sometimes it helps to do the orthodox first, to not spend so much energy pushing for newness. Abstract artists tend to have extensive realist art training — the “traditional” fundamentals — before they move into doing abstract art. In qualitative research, the parallel for me is between interpretative and postmodern work.
Stravinsky’s “Rite of Spring” caused riots on first hearing; ears needed to get used to it. I’m no Stravinsky, but I do believe I’m not doing the type of work I want to do… as well as I need to be doing it.
I want to demonstrate that I can research in ways that articulate and further understanding, as well as critique and deconstruct it. Additionally, I should reconstruct the understanding (at least partially, while talking about why reconstruction may be problematic) after deconstructing it… for this specific project, readers need to leave knowing “what have we learned from this?”
I’m struggling in this balance, and I’m trying to push most of my energy into continuing to walk forward, not ponder down philosophical rabbit holes. Part of the challenge is that some of the rabbit holes are the forward I’ve defined. Someday I’ll look back on this and it’ll be clearer. Close, ship, and go.
Someday I’ll look back on this and it’ll be clearer. Right now I’m still pulling language into things I’ve been doing for the past few years — which is ironic, because part of what I found in my data is the art and act of putting language onto things in retrospect.
I’m struggling in this ba
Part of the QualMIP series, introduced here.
Another short post this week because of a time crunch and my inability to write longer coherent things today (thanks, ADHD! Sometimes you just gotta roll with it).
Today was an analysis day — project updates, studio time, collaborative analysis — not a lot of scaffolding, more tracking where people are in their work and how to move forward. This post will likely be most useful for my future self as a way of tracking which topics are good to build more scaffolding around. Here are some:
Subject protection - using pseudonyms, talking with participants about how their data will be used, etc.
Positionality (yet again) - How does your prior exposure to a topic shape the analysis you do with it — and the way you triangulate your analysis with others’ when you’re checking inter-rater reliability? Our group has a mix of people with extensive social dance experience to no social dance experience (with social dance as the chosen context of study this semester). This gives a great variety of perspectives and positionalities that become visible in projects — and need to explicitly be pointed out (what terminology should you assume familiarity with, etc?)
Relationship between methods, results, and discussion - what you did, what you learned, and what difference it makes (“so what?”)
Instrument design as a subjective act – a reminder once again that all surveys, all instruments, all scales, etc. are reflections of human decisions about meaning creation. This does not mean they are all created equal, but it does mean they can and should be tested and interrogated and developed themselves (for instance, the IQ scale has a long history of “what does it mean to be intelligent?” debates behind it).
Validity. One thing I wish I’d prepared — and perhaps will prepare for next week, we’ll see — is a discussion on validity. In the context of qualitative research, what does it mean for a study to be “rigorous” and “valid”? (I have a lot to say about assumptions in this domain…)
It’s pretty cool to see the projects shaping up — intermediate artifacts are popping out. Emily has a nice table shaping up, which spurred a discussion on externalizing one’s process and the iterative nature of instrument design. Cesar should be bringing at least one draft poster in next week so we can start the “so what?” discussion (the “discussion discussion”), and Paige will be going through her emergent analysis process and then working to articulate it. Just a few weeks left to go…
Part of the QualMIP series, introduced here.
You can tell the semester is ending soon, because my posts are getting terser… this week our studio focus was on three things: project design alignment, creation of a data inventory, and discussion of what “done” means for individual projects.
The week will be spent moving towards whatever “done” is for each person’s project — but having alignment and clarity as to what the project is is an important prerequisite to being able to do that. We spent the day workshopping alignment, with the (draft! handwritten! first version!) worksheet below as representative of how we scaffolded that discussion. (I’d love comments on the worksheet, by the way.)
Qualitative Research Design Alignment Worksheet
Sometimes, trying to clarify your problems can be a helpful step in starting to resolve them. I’m currently struggling with the following:
1. A succinct history of how postmodernist philosophy troubles old notions of self/world and self/other relations… in ways that matter to STEM folks (that second part is the problem), and…
2. Potential misconceptions that could steer readers into misinterpreting what I’m actually trying to do. I keep feeling like I’m being asked to use my data to “prove” the ontological framings I’m using are valid, that the ontologies are frameworks that “emerge from the data” and are therefore somehow “real,” but that’s the assumption I’m trying to write against.
The last thing I’m wrestling with is something that’s less of an immediate crisis, but… at the same time, it’s very hard for me to unpack/articulate my process without using it, because it’s so much a part of my process, and so much a part of the paradigm I’m trying to convey (you’re not passive readers; this text is for you to engage with, and I’m deliberately trying to disrupt the ways you’re used to reading things). So, challenge #3 is to articulate…
3. The validity of writing something that deliberately asks the reader to experience/work with the text in a way other than being a passive reader that the writer takes by the hand and leads through a maze. I want to do this for at least some little intertexts because I think that sense of reader/writer positionality is important… and I feel like the responses I’m getting are akin to saying that T.S. Eliot is a lousy writer because he doesn’t just tell us what the Wasteland is “supposed” to be about. I need to frame this technique more, name it, justify it… I am struggling to do that.
My current articulation of what I’m actually trying to do is a sort of “Intro to Postmodernism for Engineers” stepthrough. I’m using different, related, and logically contradictory ontologies on the same qualitative data. The ontologies are basically different combinations of relationships between self/other and self/world, which are big questions that plenty of philosophers have tackled.
If you imagine each ontology as a pair of eyeglasses, I use each pair of eyeglasses to look at the data… and to look at each of the other pairs of eyeglasses. In the end, the dissertation is about the eyeglasses themselves. The data is from faculty talking about curriculum development, so here’s how the ontologies translate:
- The self, the world, and the other are all distinct and separate components! (We, the faculty, are the narrating selves… who are distinct from the curricular worlds we create for other people — namely, students — to experience and learn from.)
- Let’s trouble the boundary/relationship between world and other! (We, the faculty, are the narrating selves… who enter into and learn with and within a curricular world that’s inseparable from the others we interact with within that curricular space. Our colleagues and students have shaped the curriculum we’re “inheriting,” — even with a new course, there are histories and cultural notions of what it means to teach a certain thing — and we learn within a situated experience of teaching these courses with these colleagues and these students in the room.)
- Let’s trouble the boundary/relationship between self and world! (We, the faculty, are the narrating selves… and we and our curricular worlds co-construct one another. We have been (and continue to be) shaped by our learning experiences as students, teachers, and humans existing in the world, and our values and personalities and habits are reflected in the courses we design and what and how we teach them. Other people, such as colleagues and students, can’t encounter our curriculum without encountering us, and vice versa.) (Bonus: since I have multiple narrators telling stories of courses they co-taught, each narrator shows up as both a “self” in this ontology, and an “other” in ontology #2… in stories about the “same” class.)
- Let’s trouble the boundary/relationship between self and other! (We, the faculty, are the narrating selves… but wait, are we? In terms of who impacts curricular design decisions and how, it’s overly simplistic to define a group of faculty and say “yep, these are the people who made the curriculum, that’s it.” Sometimes we talk about ourselves as individuals; sometimes we speak as part of a group of colleagues, sometimes we include students and alumni and industry in the conversation… We constantly blur and move the boundary between “self” and “other” when deciding who to include as part of the “selves” who create curricular experiences.)
The end goal is to call into question our assumptions/usages of ontologies in general, not to prove that any ontology is “correct” or “better,” or that any of the ontologies can show us the “truth.” (Or even “all of them combined,” since there are an infinite number of possible viewpoints.) Instead, I’m trying to show that each ontology hides and reveals different things, and that these stories (and our realities as engineering educators) are complex and not always logically coherent. Lack of logical consistency isn’t a sign that there’s an error to be fixed; the world itself is complex and messy.
These things are all surprisingly hard for me to unpack and convey, and this almost certainly an issue with my weakness in articulating what I’m trying to do.
I’m hoping that talking with postmodernist researchers (especially qualitative ones!) and/or STS (science, technology, and society) folks who do or understand postmodernism — and how to articulate it to a STEM audience — might kick me out of my rut and get some useful insights on how to bridge these worlds, because right now I’m feeling like an incompetent engineer who isn’t thinking “rigorously,” as well as an incompetent philosopher/postmodernist/cultural studies/etc. because I’m not really any of those things; I’m an engineer trying to translate a few ideas to my own realm.
You know, I might as well start posting about how I handle ADHD stuff too — those moments of little practice-sharing to remind my future self of things that work, and maybe other people might find some of this useful as well? Yeah, that.
One of the things I’ve done to get my ADHD brain more tamed (applicable to non-ADHD folks too!) is to freewrite-out-loud at the beginning of a writing sprint, letting my thoughts meander while leaving traces that eventually arrive at pretty reasonable small concrete tasks. This is the self-instructional freewriting from today, unedited.
I need to get the data into the results chapters; they should be the backbone of those chapters. (I feel like I’ve done this at least a half dozen times before. Guh repeated drafting process. Guh guh guh.)
I’m worried about the open licensing for Gary and Alan’s stuff, but should not think too much about that right now. Tomorrow, Stan can start to help me pull out of that thinking hole. Right now, the important thing is the data.
I’ve got a list of stories on these post-its; I can try to type them up and see where they cluster. The largest clusters are D&D and UOCD, and there are individual conversation/disagreement stories inside them. I think these are going to be collections that get edited into dialogues — I already have some really worked-over pieces for UOCD and D&D, and the Alan/Mark disagreement; I have indexes by page number in this document beside me.
I need to find a name for what I’m doing methodologically; at some point I should probably just ask Patti to help, or reach out to methodologists (which I should get more in touch with, honestly). I wonder if I can do an intro/letters sprint at some point, and be brave about reaching out to people I don’t know who… do really awesome work… and… yeah.
Back to the topic, though. What’s my intermediate artifacts here? I’ve got about an hour on this sprint clock before I need to get home and change for the exercise that’s going to keep me in gear.
Intermediate artifacts… well, first I want…
1. A grouped list of story clusters (I know I have multiple extended versions of this, they were for different purposes, do it AGAIN.)
2. Look up the status of each of those story clusters; which ones have existing drafts? And then, from the index: which transcripts and which page numbers do I pull from?
3. Turn page numbers into line numbers and pull the data. No, Mel. You are not allowed to write another Python script for this.
…stop there for now — the next step will be pulling out these stories and I know we’ll insert them into the chapters soon… but remember how much easier the chapter drafting was when you had the stories whittled out already? We’re doing this again. So focus in on these three steps. Let’s do it.
Because people have been asking how the dissertation writing is going, and giving myself permission to post the informally worded, slapdash quickie explanation versions I give friends is somehow very liberating and helpful and I can get on with my life and so forth. This was (mostly) an email to a friend, so my language usage is totally messy and slapdash where I should be careful about terms — that’s entirely fine, fine, fine for now.
Thanks for asking. The writing is progressing, which is — that’s good, right?
It’s a tricky results section to write, because I’m actually kind of writing four results section and then a fifth meta-results-discussion section. Well, more like rewriting, and rewriting… it hurts to look at the folder with all of my aborted drafts. I’m writing four separate lens through which to look at the same data:
1. Curriculum is a thing that faculty make for the benefit of their students (they’re three separate things: faculty, curriculum, students — corresponding to self, world, and other.)
2. Faculty are learners who step into curricular experiences (broadly defined, not just courses) that have a long history of being shaped by others (faculty colleagues and students) Basically, instead of the three-separate-things perspective above, this separates the faculty “self” but lumps everything else together — the curriculum/world and the “others” in that world — as a soup the faculty learn with/in.
3. Curriculum and faculty are inseparably intertwined; past curricular experiences deeply influence one’s identity as a scholar and teacher, and one’s identity and values can’t help but spill out into the curriculum one designs and teaches. In other words, this one tangles up the faculty “self” and the curricular “world” (and separates out, for the time being, “others” like students and colleagues and external evaluators and so forth).
4. Faculty and colleagues and students are inseparably intertwined; they’re not uniform/same/ASSIMILATED BY THE BORG, but there is a sense of partnership and co-journeying and co-creation of the curriculum. This ontology separates all the “agential” (people!) parts of reality from all the non-agential ones (an inanimate “curriculum” as boundary object)
You’ll note that these are basically rearrangements and regroupings of the components in the first ontology (self, world, and other) — all possible logical pairings/clumpings of the trio, except for the one where everything gloms into everything else (that’s an ontology for the discussion section).
Each of these lens comes from my data, and each could be a dissertation in its own right. The point I’m trying to make is that these four lens coexist AND logically conflict and can’t be stitched into a calm, untroubled Frankenstein(‘s monster) — that the reality is super messy and full of tensions.
Simple example: is the curriculum for the benefit of the students, or the growth and interest of faculty? Oftentimes, it’s an “and” and not an “or,” but when they conflict… which lens do we make decisions from? For instance, if a professor can teach one more class without overloading, do we let her teach the advanced elective she’d be really badass at because it comes from her research (which a small handful of students would take), or assign her to teach an intro section for a subject she barely knows because we need to cover another faculty’s sabbatical (and lots of students are super-interested in the topic and need it for graduation)? No real right or wrong answer; depends on priorities, depends on the lens you use to view it… (this is not the greatest example but I’m rushing to explain while trying to figure out my next cup of coffee).
So the end result is really all four ontologies played and diffracted against each other, showing how each makes different things visible/invisible; it’s not that one is inherently “the truth” or “better” (all models are broken) but all of them (and more!) are present at the same time, and… hrnghh. It’s really hard to describe each ontology well and “prove” it’s there, and yet not spiral into the trap of writing FOUR SEPARATE DISSERTATIONS because ugh.
I keep pulling myself back from writing an entire dissertation for each sub-result in non-helpful ways; my prior attempts have all used a technique I jokingly refer to as “data vivisection” — taking a small excerpt and then picking it apart into the tiniest atoms possible. As in — I was diagramming sentences, obsessively tracing a pronoun’s referents throughout a paragraph — I kid you not. Tiny, tiny microscope.
And then, two weeks ago, Robin asked if that was actually the process I went through to arrive at my results, and my response was an immediate no. And then a headdesk. I see all the stories at once, but I’ve been living with them for two years — and I should be trying to convey that glorious big web to others, not….
Okay, back to it before my Ritalin wears out for the morning. And then postdoc things, because there’s a slidedeck draft I want to finish before 3pm. See you online.
(and rewriting, and rewriting… it hurts to look at the folder with all of my aborted drafts)
Part of the QualMIP series, introduced here.
Sparse notes again because of the busy time, but today each person:
- Gave an overview of the data collection/analysis they did over the past two weeks
- Had someone else in the group write an “abstract” consisting of 3-6 points they saw throughout our reported data. After a time of trying to diverge and divest ourselves of as many filters as possible, we are starting to converge and filter once again — but this time, much more consciously.
- Figured out what research paradigm they seemed to be working in (everyone is interpretivist for this project; I re-presented a version of the main paradigms for this discussion).
- Honed down on their unit of analysis and research question (by me giving examples and then everybody iterating on theirs with studio critique).
This is the last week before our Data Collection Cutoff. A constant part of our iterative focus is trying to make sure that our data and questions coevolve in such a way that the data can answer the questions. (I’m also thinking on how this dynamic scales up the next time I teach this as a course; I suspect more scaffolding and possibly even some readings will appear.)
We also looked at the classic paper “Body Ritual Among the Nacirema” as an example of… well, I’ll leave it to the reader to decide what one could use it as an example of — quite a few things, as it turns out. (It wasn’t assigned reading; we read parts of it aloud in studio, so my “no readings” rule stands unbroken.)
I’ll say it again. The infrastructure that supports a dissertation is ridiculous.
My infrastructure has a lot of “normal” things. I’ve got a place to sleep, food to eat, an advisor who both pushes me and gives me room. (I am incredibly blessed to be working with Robin Adams, and can only hope to pay this forward.) I have friends and colleagues who are feeding me, encouraging me on the days I’ve broken down in tears (yep, this project has done that). I have library access. We recognize these things as being in support of dissertations.
I also have a lot of other things. I have a car that works. I have access to paved highways that can get me to my office in a reasonable amount of time. I have access to gasoline — and food — at prices that I can afford in the economy I live in, with the job I have. I can fall asleep at night without worrying that my building will be bombed or that a family member will be shot.
I largely take these things, and more, for granted. I am not caring for sick relatives; I do not need to tend to children, and I’m a citizen of the country that I live and work in. I have so much privilege. So much. And it helps me progress on my dissertation because I don’t have to spend energy on these things… and can use that energy to write instead.
There’s also the infrastructure that I’m more acutely aware of — because it’s new to me, because it feels so much more fragile, so easily taken away. Most people aren’t affected by that infrastructure’s presence or its lack, so unnoticed failure is my job to flag, my contingency to plan for. Actually, a lot of people don’t think that I need it. I… used to not think I needed it.
But I do need the Ritalin that keeps my wandering mind able to look at something for more than a few minutes at a time, at will. I have willpower; I also have ADHD. And I can use that willpower to pour my considerable energy against itself, struggling and wrestling to even get myself to sit down, battle guilt, open a page — or I can use it to think, write, teach, serve — graduate.
And I do need the PT and massage that keep my hands from locking up, my neck and shoulders from screaming. I have the ability to stretch and exercise; I also have joint structures and use patterns that give rise to RSI. Even when I do self-care, if I do the typing/thinking/writing work I do, it keeps on coming; I could push through the pain until my muscles literally will not move on my command — I’ve been there and don’t care to go back — so yes, I need other people. People who can treat and unlock what keeps creeping up on me, people who can keep teaching me how to understand and work with the body I have, because this is the body I write with, the body that will walk acros the stage and graduate.
And I do need the captioning and subtitling and interpreting that allow me to understand things without lipreading, and the CI and hearing aid that make lipreading easier when I have to do it. I’m an excellent reader, lipreader, and bluffer; I’m also deaf. I can use my considerable powers of rapid mental analysis to guess at syllables for short periods of time — or I can use them to come up with knowledge nobody has ever thought before, knowledge that makes a contribution to the world. One of these things allows me to pretend I’m hearing when I’m not. The other is what I need to graduate.
I need this infrastructure just as much as I need a way to commute to work, just as much as I need to not scramble for clean water to drink, just as much as the other things that quietly and seamlessly support us. Will I die without these things — my ADHD meds, a car, subtitles, running water? Nope. Will it hit me in a way that makes it significantly harder for me to work, move forward, graduate? Yes.
And I’m painfully aware that some of my infrastructure is a localized and temporary patch. If running water stops, everyone notices. If I can’t get a talk interpreted… few people do. These things are sometimes ignored and forgotten because most people aren’t affected by their presence or their lack.
I just got off a two-hour phone call with my advisor. It was hard — for all the right reasons. It was amazing, it was productive, it was wrestling with difficult ideas. And for that to happen, I needed my advisor. I needed the invention of the telephone, the network that allows me to be in Boston sending audio to her tiny handheld device in Indiana. I needed the shared document editing and the wifi connection over which I sent my slide deck. And I needed the relay service interpreter who made it possible for me to understand how Robin was responding to my slides.
Take away any one of these things, and I’d have to find a workaround. Some of the workarounds would be more costly than others, force a rippling-out of more tradeoffs. Some of them live in the space that still makes it possible for me to graduate. Some don’t.
It’s not only about talent or willpower. If the world you live in doesn’t support your ability to use that talent and willpower towards something good — you might as well not have it. I want to graduate; I want to do good things. And in order to do those things, I rely on so much infrastructure… and sometimes it feels so fragile, and I stand in fear and gratitude for all the balances that bring us where we are.
Part of the QualMIP series, introduced here.
Sparse notes today, because we worked with data that’s non-public… and I have about 5 minutes to write this blog post before my next meeting.
Today we went through a close reading exercise; we took my data from a prior project and read it — out loud — in small increments, with discussion in-between. Each person in the group took a small sample of data from a different respondent, effectively “adopting” them for the duration of the conversation — with the disclaimer that we were working from a limited dataset from people we didn’t know, so our guesses were exactly that: guesses.
In between rounds of reading their words, we talked a bit about what “putting data in conversation” meant. It can mean many things; do you see commonalities between “your” data and “other people’s” data? How do you think “your person” (the person who wrote the response you’re reading out loud) would reply to notes/thoughts from or about the other data/observations? If you were going back to do a member-check, what would you want to ask — and how does that help you think about the fieldwork you are doing now, where you still (might) have that opportunity?
In all these responses, we kept on trying to go back to specific phrasings and sections of the data to back up our guesses, working to keep awareness of possible biases we might be bringing to the conversation.
Honestly, right now — it’s hard to sum up in a blog post writing “about” the practice… we’re in the middle of doing qualitative analysis/fieldwork, not talking about it. Over the next two weeks, we’ll be responding to each other’s data (and doing a fair amount of self-care). I’ll be modeling half-hour bounded response sprints for each person’s data on Thursday, because one of the hardest parts of the semester is learning just how much work goes into qualitative research… how tiny and bounded projects need to be to actually get done.
Part of the QualMIP series, introduced here.
This is public documentation of a message I recently sent to the group; there have been lots of changes/info floating around lately, so I brought them all into one place for clarity.
This email contains 5 parts:
- What you need to bring to Monday
- What I need to bring to Monday
- What we will be doing Monday
- What you’ll need to do between Monday and returning from spring break
- What I expect you to bring back to studio after spring break
What you need to bring to Monday:
- A data collection schedule for the rest of the project; your data collection time (observations, interviews, time spent with artifacts) should be solidly on each of your individual calendars. We’ll be creating a shared one on Monday.
- An annotated/scaffolded selection of your fieldnotes to share. Your scaffolding should include specifications of what you’d like your teammates to give back to you — think of it as giving them a small homework assignment. Your teammates will be working with it for the next 2 weeks, so point out hard parts, ask them good questions; put them to work for you. (I’ll be modeling this process on Monday, so we’ll leave time at the end of Monday to update your annotations and print them out — but you should have your data and your first attempt at scaffolding/annotating them before arriving, even if that scaffolding is a piece of paper with 2 sentences of context and 3 questions for the group.)
What I need to bring to Monday:
- My data, scaffolded for you to engage with it.
- Questions about your data collection schedule and your process and evolving focus.
- Encouragement. :)
What we’re doing on Monday:
12:00-1:00: (in office) Mel models data-sharing
1:00-1:30: (at lunch) — brain break time
1:30-2:00: (at lunch) — reflection/comparison discussion; what was I doing as I modeled? What can you do with your own data?
2:00-2:30: (on your own) — artifact updating/printing
2:30-3:00: (in office) Project forward planning, including another round of clarification on your research questions/data/methods, master schedule creation, and sending a participation opportunity list to potential collaborators from another professor’s research group.
What you’ll do between Monday and when we get back from spring break:
- Respond to your teammates’ data, using the prompts they wrote to guide you. Ask them for clarification if necessary; don’t feel like you need to engage with this all on your own.
- Collect/analyze more of your own data. Remember to memo! Leave traces that you think will help you later. Remember that your teammates’ data/prompts may inspire some of your own thinking.
What you’ll bring back from spring break:
- Responses to your teammates’ data, in whatever format they’ve requested.
- All your data, in digital or physical form. If digital, write what it is on a post-it so we have a physical representation (if you come a few minutes early, our office has plenty of post-its). We’ll be creating a master data inventory/taxonomy during the post-spring-break studio.