Stacey asked me for a refresher on Test Driven Learning for Hacker School, so here we go.
Test Driven Learning is a software engineer’s articulation of Wiggin & McTighe’s Understanding by Design framework after being strongly influenced by Ruth Streveler’s ”Curriculum, Assessment, and Pedagogy” course at Purdue.
Write the test (“how will you know if it’s working, exactly?”)
Make the code pass the test.
Test Driven Learning (TDL) simply says “it’s the same thing… for your brainnnnn!”
Decide on the goal (“learning objective”).
Design the assessment (“how will you know if you’ve learned it, exactly?”)
Go through the experiences/etc. you need to pass your assessment.
That’s it. Really.
Step 2 is the part most people flub. With software tests, you have a compiler/interpreter forcing you to be precise. With learning assessments, you don’t — but you need exactly the same level of precision and external execution. If you asked a group of external people (with appropriate expertise) whether you’d passed the assessment you set for yourself, there should be no disagreement. If there’s disagreement, your assessment needs a redesign.
A good assessment is a goal that helps you stretch and reach it; sometimes it encourages you to do more. But sometimes it also gives you permission to stop doing stuff – you’ve written the code, you’ve delivered the talk, they met the criteria you set — and now you’re done. You can absolutely set a new goal up and keep on learning. However, you’re no longer allowed to say you Haven’t Learned X, because you’ve just proven that you have.
Here are some rough-draft quality TDL assessments you might start with, and a bit of how you might improve them.
I will learn Python. (What does that even mean? How will you know you’ve learned it?) I will complete and pass any 50 CodingBat exercises in Python. (But I could do that by solving 50 really easy problems.) Only 10 of those 50 problems can be warm-ups, and at least 20 of them must be Medium difficulty or greater. (Does it matter if you get help with the problems?) Nope, I can get as much help as I want from anyone, as long as I could explain the final solution to another programmer.
I will get better at testing. (What do you mean by “testing”?) I write a lot of code, but I’ve never written tests for any of it. I hear the nose framework is nice. (What do you mean by “better”?) Well, I’ve never written a test at all, so even going from 0 to 1 would be an improvement. I could use nose to write tests for 3 different pieces of working code I’ve already written. (Do these need to be big or exhaustive tests?) Nope, I’m just trying to learn what writing tests is like, not get full test coverage on my code… at least not yet. Even if I write a 3-line test that checks out one minor function, it counts as one of the 3 tests. (What does it mean for a test to be “done”?) When someone else can check out and successfully run my code and my test suite on their computer without needing to modify either bit of code, it’s done.
I will understand how databases work. (By “understand,” do you mean the mathematical theory behind their design? Or how to actually implement and use one?) Oh geez, the latter. I don’t care about the math so long as I know how to interface with a database. Any sort of database. (So you need to make a demo.) Yeah, but that’s not enough; I can blindly type in code from a tutorial, but that doesn’t mean I’d be able to field questions on it. (What could you do about that?) I will give a presentation to fellow Hacker Schoolers demonstrating a small database interaction in code I have written. That’s an easy binary to check; either I’ve given the presentation or I haven’t.
Thoughts, questions, ideas? Got your own example TDL assessment (at any stage of revision), or ways to improve the ones above? Holler in the comments.
Aha! This is the best version so far of my criteria for Radically Transparent Research (website really needs rewriting; it’s currently a mess of writing from my first year of grad school), which is basically “a methodology for producing Free Scholarship.” I know, I know, there all tons of Open Research movements and projects out there; I’m trying to write an examination right now and will loop back and check in with them again after I pass it, ok?
The work is public and freely accessible.
The artifacts (data, analysis, etc.) used to create the work is also public and freely accessible so that it can be studied and peer-reviewed by communities of practitioners.
The work and its artifacts can be freely modified and distributed so others in these communities can benefit from and build atop it.
“Chua’s 3 criteria for RTR” (or whatever less-silly name I can tack on that later) comes directly from a Free Culture + Academia mashup. From academic-land (specifically, the scholarship of teaching and learning), we have Lee Shulman’s 3 criteria for scholarship (paraphrased):
It is public
It is peer-reviewed by the practitioner’s community
It can be used by that community as a stepping-stone towards future work
The freedom to study how the program works, and change it so it does your computing as you wish. Access to the source code is a precondition for this.
The freedom to redistribute copies so you can help your neighbor.
The freedom to distribute copies of your modified versions to others. By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.
See the connections between Shulman and Stallman? I don’t imagine this is the final or best statement ever (and look forward to seeing what future version comes up), but it sounds pretty good to me right now.
Our “Gender/Race/Class in engineering education” class has an “open topic” period that I’ve volunteered to help design… which means I’m going to Ask The Internet for help. (Hi!)
Based on our class discussion just now, we are interested in tackling this question: How do we interrupt the discourse that perpetuates inequity in engineering education? (Subquestions: who has access to this discourse as a listener? A speaker? What is that access based on — gender, race, class… age? geography? language? disability? intersections of any subset of that? What strategies do we have for doing this dialogue-interrupting work in professional and personal contexts?)
The course will be Monday, November 18, which is 2 weeks from now. We’re mostly PhD students in engineering education (technical backgrounds, social science research interests, lots of future engineering professors who care deeply about teaching). We have 3 hours in class, plus the ability to ask people to read a reasonable amount (<100 pages, English) before class. I’d love to hear thoughts, especially half-baked ones, on:
“learning objective” suggestions — in other words, what do we want to learn during the course of the 3 hours? (Can be fact-based, skill-based, emotion-based, perspective-expanding-based…)
“assessment” suggestions — given those learning objectives, how will we tell (at the end of the 3 hours) whether we’ve learned them, and how well? Does not need to be a test; could be questions for reflection on our own, etc.
Reading suggestions — scholarly or not. (For instance, Alice Pawley has offered to let us read her CAREER proposal on feminist engineering — a short, highly competitive grant for junior scholars whose committee was probably not used to getting “feminist” proposals.)
Activity suggestions — discussions, games to play, short bits of theatre to act out and/or improvise upon, provocative question prompts, etc…
Potential inspiration: our guiding question/framing about “interrupting discourse” came from a discussion on “how do we talk to people about this?” and an interest in intersectionality, especially with disability/access. I’m personally curious about the history of opening these dialogues in STEM: who (tenured? white? male? western?) started the conversations about women in physics, minority races in computing, wheelchair-accessible chemistry labs, etc — and when, and how, and what were the responses?
Comment away! I will post readings (or reading notes, if readings are not freely available), discussion questions/guidelines, and a story of what happened in the class once we run it — basically, whatever I can do to make the experience we’re creating here available and reusable by more people.
Once again I have the (incredible) opportunity to be at Hacker School playing around with my “edupsych for hackers” material… I’ve never revised and re-delivered a talk so often, and it’s good to be forced to see how this material improves with age and experience.
TL;DR – if my work in open source has helped you in some way, please donate to the Ada Initiative, which supports women in open technology and culture. Not convinced yet? Here’s why I donated.
There’s a world out there to patch. I love the universe of open technology and culture where I’ve built much of my career and friendships. It’s a wonderful world that can be wide and welcoming — but it also has horrific bug reports of sexual abuse and gender discrimination, along with many more that haven’t been reported out of fear and shame. I’ve lived a few not-so-good stories myself; some I’ve told, some I haven’t. What saddens me most, though, isn’t the bad stories that have happened; it’s the good ones that never will — stories of women and men working together to hack the universe in marvelous ways. If we want to see these stories happen, we’ve got to make a world where they can happen, a world where it’s safe for them to happen. Don’t WONTFIX that ticket; do something. When you care about something, you want to make it better.
We change the world with millions of tiny patches. I’m a grad student; money is tight, and my $64 contribution represents half a month of groceries. I was initially ashamed of my “tiny” contribution, even if it’s a nontrivial one for me. Then I remembered: our world of open technology and culture is built one patch, one line, one edit at a time — and that’s precisely why it’s powerful. It brings billions of tiny, ordinary moments together to transform the world. If we teach it for our code, we can preach it for our giving. If you’d buy me a drink, or treat an open source newcomer to dinner, send that $3-$20 to the Ada Initiative tonight.
Someone’s got to integrate these patches into a whole… and it’d be nice if they didn’t burn out in the process. Honestly? I support the Ada Initiative because it does this work so I don’t have to. I’m young and energetic, but I’m often wiped out just being a woman in open technology and culture. It’s not just physical and mental exhaustion; it’s emotional and psychological, which is worse. And being an activist is harder still. Do I agree with everything the Ada Initiative says or does? Nope. But it’s a job I want done, and I don’t want the job. This is why we hire maintainers for Free Software; we give them the gift of bandwidth so they can help us contribute more for a project with less effort by supporting and connecting our patches with the bigger picture. Val and Mary are good maintainers for feminism in our open universe — and I’d like more. After all, it’s a big world out there that we’ve got to work on.
The last day of their fund drive is tomorrow. (I’m coming late to the game; summer travel + school year start + RSI = no internet for Mel.) But it wasn’t too late for me to throw in my $64 patch this morning — and it’s not too late for you to contribute your patch today. If my work in open technology and culture has touched, helped, or inspired you in some way, please help me pay it forward and create a supportive, welcoming environment for everyone in the open world.
EduPsych for Python Hackers 2.0 is about to go live in Toronto in 45 minutes, which means it’s time to upload slides! Questions, comments, etc. welcome as usual. This is an expanded, revised version of the previous EduPsych-For-Hackers talks I’ve given, so if you’ve seen both, I’m curious what you think of the changes.
Also: my parents will be in the audience for the first time (since I started speaking over half a decade ago), so they get a special call-out on slide 29.
These are rough, incomplete notes from my getting started in open source session at Hacker School, cribbed from chat notes taken by attendees (thanks, folks!)
We started with a replay of the 5 minute exercise wherein participants dump me in the middle of an open source project I’m clueless about, and watch me think-aloud as I desperately try to figure out what’s going on — basically, “how does an experienced hacker evaluate an open source community?” This time I had 10 minutes, so I got pretty far checking with out Ogre3D (which looks great).
Our first big goal for the session was lurking. You can find projects on a topic by searching the internet for “[topic] open source” (or “[topic] Free Software,” or so forth). When you have a few potentials, ask yourself:
Is this project alive? Are code commits recent? Are mailing list messages recent and responded-to in a timely, helpful manner? Are people using this software? (Do you want to use this software? Can you figure out how?)
Is this a community I want to be part of? (Do they treat each other well?) The people are more important than the code; they’re the ones who make the code, and with release cycles that average 6 months, the code moves so fast that your relationships are what will really orient you.
Where do they hang out and do their work? (What chatroom — usually in IRC — do they use? Do they have a bugtracker or some other giant shared to-do list for the project?) Once you find out where you can overhear things, you can figure out who you’re overhearing, and then start contacting them directly: “I’ve seen you answering questions on X; can you help me navigate X?”
Most projects have communication methods for code and not-code, and for asynchronous and synchronous work. Try to lurk all four. The table below may help.
Synchronous Code: git commits (announced by a bot in chat, sent to a feed, etc)
Synchronous not-code: chat (typically IRC)
Asynchronous Code: issue/ticket/bug tracker
Asynchronous not-code: mailing lists or forum, AND wiki
Our second big goal for the session was introducing yourself. This usually happens by sending an email introduction to the developers mailing list, then referencing that email (find the URL of your message in the mailing list archives) during initial chat conversations with people. Maggie brought up “submitting a pull request as your intro letter,” which is a great idea. What this means is that your introduction email should explain how you are:
already in the middle of doing a specific helpful task
and what you’re asking for is help doing that specific helpful task.
This sounds intimidating until you realize “something useful to help” can be very, very small. For example, Rebecca emailed tent saying that she’d been working through their documentation and had ideas for how to improve the clarity of the particular docs at a certain URL (specific helpful task!) and was wondering where to submit her changes (help me do it!). Jade emailed GIMP offering to test patches (specific helpful task!) and asked which branch and patches would be most helpful to verify (help me do it!). None of these tasks involve deep knowledge of the code; that comes later. They involved “writing in English” (not a problem for most Hacker Schoolers) and “compiling C” (not an impossible thing to learn, especially when you’re surrounded by programmers eager to teach).
It’s helpful to pair with someone and peer-pressure (positively!) each other to ship your intro emails.
Here’s the transcript of the seminar “Psst: wanna eavesdrop on my research?” [materials] I delivered on Thursday about applying Free Culture / Open Source practices to qualitative research (for the engineering education department at Purdue, hence the disciplinary focus). I’ve edited in some context for readers who weren’t there, and anonymized audience comments (except Jake Wheadon’s interview — thanks, Jake!) and the transcript cuts out before the last 2 audience questions, but otherwise this is what happened; click on any slide’s photo to enlarge it.
This will be a slightly strange seminar; I’ve had at least half a dozen people e-mail me and say they can’t be here but would like to catch up later on. So we’re interacting here and it’s being recorded by Boilercast and transcribed by Terry over there.
The title is long and fancy and we’re going to ignore that.
I’m Mel. I think you all know me. I’m a Ph.D. student here in engineering education and one of the things I do is qualitative research, because it’s fun. I also come from the hacker world, the open source, open content world where there’s this radical transparency culture that defaults to open and share everything about what they’re doing.
This is Terry. She is a CART provider and the one responsible for typing super, super fast on our shared transcript document. The URL for the live transcript is on the slide for those of you who are following along on Boilercast. You should know that all of the recordings and the documentation and so forth we’re producing in the seminar will be open data. That means a couple of things.
First of all, the document that Terry is transcribing in is a collaborative text editor and you can type and annotate and fix spelling or whatever you want. It will be the canonical record of our discussion. The second is, as we’re recording this, no names will be taken down. So it’s not going to say I said this and you said that and this person said that other thing. It’s just going to be “person in room” said words. If you want to be off the record, if you don’t want Terry to type down what you’re saying, say that before you speak and she will stop typing for a moment, or if you see something in the document you can go and delete the stuff you said if you don’t want it on the record. The document will be available for editing right after the seminar as well. I probably won’t read and post the final version until sometime after dinner. So you can also take out stuff you said afterward if you don’t want to be captured here. The ground rules are also in the document.
This is what we’re going to be doing today. It’s a bit of an adventure. I’ve been playing are something called radically transparent research. There’s a website and it’s out of date and once I finish my papers I’ll fix it. What radically transparent research refers to is this: what if we did engineering education research, or any kind of qualitative research, as if it were an open project? Make the data open, the analysis open in terms of both being publicly available and open to anyone who wants to participate, and not having a delineation saying “these are the official people” on the project and “these are not.”
What would it look like if we defaulted to open instead of defaulting to closed? So what we’re going to do is we’re going to do a little RTR — radically transparent research — project right here in this room. We’ll be collecting data, going through the licensing process, seeing what analysis looks like, and dissemination — we’ll get back to that. We’ll see a few examples of other projects that RTR is being used in and then we’re going to loop back and and do an instant replay of “okay what the heck just happened?” I’m hoping as we go through the steps they’ll seem fairly logical, but then when we go back and compare them to the normal way of doing qualitative research they’ll start seeming really weird and the implications of the pieces lining up will start piling and piling and piling.
(Note: this was a 45-minute talk, so for the sanity of feed readers, I’m going to say “click to read more” here.)
I had the privilege of spending several days last week at Georgia Gwinnett College with professors Nannette Napier and Evelyn Brannock, alumni of our 2011 Professors’ Open Source Summer Experience (POSSE) cohort. They work you hard, these Georgia profs — within the first 24 hours of my landing in Atlanta, I’d spoken at 3 classes, had lunch with Science and Technology faculty and a meeting with the dean, gone on a library tour, hosted a Google Summer of Code application prep session, and delivered an all-college Tech Talk on humanitarian free/libre/open source contribution as a career stepping-stone. [slides]
In addition to a stroll around the CNN Center and Centennial Park, Evelyn and Nannette introduced me to tomato aspic and fried chicken livers at The Colonnade. The former is a virgin bloody mary in gelatin format and is… edible. The latter is a very, very good idea (om nom NOM). Then there was Magic Bread, a basket of sweet, malty, butter-laden pillows of goodness. They are supposedly called “yeast rolls.” I like my naming better.
On my last day there, I spoke in front of a group of K-12 (mostly middle-school) teachers in for a Saturday workshop on computing while middle school girls played with Scratch and Lego Mindstorms next door. Of all the moments I had in Georgia, this surprised me the most; the workshop was jam-packed, fast-paced, INSPIRE CHILDREN go go GO! and normally I’d jump right in and surf that wave of hyper, even ratchet the speed up a notch — but this time, the wave rushed through me and then past me — and I was aware of it but not swept into it; a strange experience for me. I heard how fast people were talking. How little they were breathing. These were teachers with no technical background who’d been suddenly asked to teach computing. I wondered what that pressure felt like, having to teach kids something you didn’t know and had no time to learn. What could I give them, if what they really needed was more time –
And suddenly I was being introduced (fast! high-energy! She has INSPIRED CHILDREN go go GO!) and I was up front, and pulled a chair and sat and looked at them and breathed –
I remember only vaguely what I said; this talk came from a different place; a quiet place, a gently rooted one. I told them stories of the 15-year-old girl I used to be. I spoke about uncertainty and shyness, needing to watch and be safe, having my fears and hesitations accepted as a valid thing and held so I could address them in my own time. About creating space and letting go and knowing you’re fine even if the world is yelling faster! you’re behind! and how wrenchingly difficult it is to stand there and create a peace amidst the pressure. About the patience needed to let playfulness and ownership emerge and intertwine. About giving yourselves permission to show those around you — especially the kids — how a grown-up learns new things in an unfamiliar space. I sat up there, speaking slowly, looking into eyes. I had the strength of gentleness and all the time in the world, and I could share it. The room’s energy relaxed and softened, and it felt — in a very small way — like hugging a tiny corner of the universe.
Yeah. Like I said, it was the strangest thing. I’ve felt this quiet power once before, during my last brief remarks on a 2010 panel at UIUC. I don’t know what this space is or when I will be back, but I suspect that silence (one of my biggest fears) is guardian of its doors. I’d like to stand here more, though. It feels… right. It’s very difficult, and I’m not sure if I would say I’m happy when I’m there, but I am more… me. So I will continue learning towards that.
Thanks, Georgia. Thanks, Nannette and Evelyn and GGC. Thanks, tiny corner of the universe — you hugged me back.
(Disclaimer: I’m transcribing my own talk about a week after having given it, but I am deaf, so I’m typing this out through a combination of residual hearing, remembering what I said last Thursday, lipreading myself in the video, and reading slide content. It’s probably 98% accurate; patches welcome on Universal Subtitles.)