Tuesday, Jul. 21, 2015
Teachers College and the Educational Testing Service (ETS) held the first ever joint forum on Teaching, Learning and the Assessment of English Language Learners this past April. The forum hosted scholars and research scientists, both professional and developing professionals alike, to discuss scenario based language assessment in three separate talks. Two Teachers College doctoral students who have been heavily involved at both institutions are Fred Tsutagawa and Saerhim Oh. Both were offered ETS fellowships three years ago, and have gone on to redesign placement tests for the Community Language Program (CLP) for Teachers College, were present at roundtable events, and have grown close with the orchestrator of the TC/ETS relationship, Dr. James Purpura. Here, they discuss their experience in the fellowship and their future goals.
KATO: Can you tell me your courses of study and about what led you to assessment?
TSUTAGAWA: There isn’t really much to say -- I wasn’t a very good student in my undergrad, so I’m one of the older students now in my program. It took me a while to figure out what career path I wanted to do -- it took me a while to figure out that I even wanted to pursue a doctorate. But once I did, I’ve been in school a lot since then. I’m in the assessment track at Teachers College. I’m a current Ed.D. student studying under Dr. James Purpura, studying second language assessment. My specific area is Pragmatics Assessment. I really got interested in that when I was working on my second Masters degree at York University up in Toronto, Canada. So, for my thesis paper, I got into humor. I did my own empirical study, collected my own data, you know, examining explicit humor instruction. And actually through that process is where I got interested in Pragmatics, because humor falls into the Pragmatics field.
KATO: Could you explain what pragmatics is?
TSUTAGAWA: So, for example, in social usages of language, we adjust how we speak and what we say based on who we’re talking to. If you’re talking to an ambassador at the UN, you would obviously use very formal, more polite language, but if you’re talking to your friend, you would use very different colloquial language, that kind of thing. And we adjust language based on the degree of severity of a request or an apology or something like that. What I was saying before is humor falls into the big framework of Pragmatics. That’s what I studied during my second M.A. at York University.
KATO: And what did you study before that?
TSUTAGAWA: Oh, yeah, well I’ve just been a journeyman. [laughs] Like I said, it took me a long time to figure out what I wanted to do. But in general, I just like teaching. I did have a previous M.A. in TESOL I got in 2003, but I didn’t use it. I was long-term subbing in the Long Beach Unified School District in California. I was mainly teaching Science and Math because that’s where the need was. That burnt me out, so that’s when I realized I wanted to get a doctorate because if you’re going to make any kind of change, you need to have credentials. I don’t think I’ll get back into public teaching, but it’s an option if I do get a doctorate, I suppose.
KATO: Thanks for explaining that. Saerhim, could you introduce yourself and explain your course of study and your concentration?
OH: I’m also in the Language Assessment Track, same as Fred. And I’m focusing on assessing writing for Second Language Learners, especially the use of linguistic resources in language tests. So dictionaries, spell check and things like that when students take tests. I look at questions and try to determine whether they affect students’ performance.
KATO: What did you study before you came to TC?
OH: Before TC, I also studied the same field for my Masters, in the University of Hawaii. Second Language Studies and my concentration was also in Language Assessment. But before that, I went to undergrad in Korea, but my major was in Computer Science.
KATO: Did you go straight to TC after your Masters?
OH: No, after my Masters I went back to Korea to work as a researcher and a teacher instructor teaching EFL in Korea at a university in Korea.
TSUTAGAWA: You were writing textbooks too, during that time right, Saerhim?
OH: Yeah, I was writing middle school textbooks. Grammar and Reading textbooks in Korea for middle schoolers.
KATO: Why did you decide to go for your Ed.D.?
OH: Well, I always wanted to do my doctoral study -- I think that was something I had in mind when I did my Masters. But I thought that I would need some teaching experience. That’s why I thought I should go back to Korea to get a couple of years of teaching experience before coming back to the States.
KATO: You mentioned before, Fred, that Dr. Purpura was one of the people, or the person, to garner the relationship between TC and ETS. Can you tell me what it’s been like for both of you to work with him and with ETS?
TSUTAGAWA: I was working with Jim as his research assistant when he was getting into his Learning Oriented Assessment. He’s just a really great human being. You can relate to him really easily, he really cares about you as a person, and if you’re struggling, then he’s really compassionate. That’s what I like about Jim. He’s super knowledgable and very experienced, obviously, as a teacher. And he’s well accomplished as an academic as well, but he doesn’t come off that way. He asks us to call him by his first name, Jim. He said don’t call him “Dr. Purpura.” But I still was calling him “Dr. Purpura” until I was in the Ed.D. program. I guess that’s me though. I couldn’t get past it.
KATO: And what projects have you been working on with him?
OH: Well, let’s see. Some of the students are helping come up with a new placement test for the Community English Program [CEP, which is part of the CLP]. Because we’ve been using the tests for, I think, more than ten years. It’s a very old test, so we’re in the process of revising it. And Jim is on the lead of helping make the test much more interesting. What we’re doing is going beyond the traditional type of test, and making the test scenario based. And then the result will be used to place them into an appropriate level.
KATO: And is that part of the internship course that you’re in?
OH: Yes, that is. So that’s the main thing that the students taking the internship work on. The internship class is highly connected with the CEP. Even the director of the CEP comes to sit in the class so she knows what’s going on and everything. We help administer the test, scoring the speaking and the writing, analyzing the data. That’s one part of it, and a big part of the class has to do with revising the existing test. So, we wrote a literature review and right now we’re making the items and designing the test.
KATO: Wow, so you guys are quite involved.
TSUTAGAWA: Yeah, I think Jim has it so a lot of us in the internship class are doctoral students, but it’s been open to Masters students and Ed.M. students as well. There have been times when Jim has asked us to take on a mentor role for the M.A./Ed.M. students and things like that. It’s a neat class.
OH: I think it’s a very unique class.
TSUTAGAWA: Oh, it’s totally unique!
OH: We don’t have a set syllabus where the professor comes in and these are the things that you have to read and things like that. It evolves as we go on.
KATO: And how long did that course last?
OH: It’s a yearlong course. Fall and Spring. Jim’s been offering it for two years.
TSUTAGAWA: So it’s three credits for the entire year.
KATO: Oh, that’s really nice! So he’s teaching the internship course, and he’s also working on ETS. But is he also working on the project that you both are working on in the fellowship?
OH: Not quite, he has his relationship with ETS.
TSUTAGAWA: It’s called a Committee of Examiners, the COE. ETS likes to consult with experts in the field that are not associated with ETS. They’re independent experts in the field of assessment, so they sit on these committees for new projects and new things that they’re working on to get their opinion, bounce ideas off, things like that. He’s in his last year of that.
KATO: What’s been the most interesting part of being in the fellowship for both of you?
OH: For me, being involved in real projects. These are projects that have tons of data, and it has a lot of interesting and very innovative things going on. The most interesting part is being involved in the projects and also working with other people. It’s very stressful to do group work in school sometimes, but I don’t think I’ve felt that way doing team work working with people at ETS.
TSUTAGAWA: Yeah, the people at ETS are pretty top notch. Everyone works really hard and they want to do things the right way. In assessment we have validation arguments and we have a certain way of doing things -- of thinking about constructs and prototyping and piloting, and doing things in the right sequence. Everyone wants to do things in the right way, so it’s really neat to work with people that are committed to that. They really want to put out the best research that they can.
KATO: How has working at ETS met your expectations or been different from what you were expecting?
TSUTAGAWA: I don’t think I had much expectation going in. I personally feel honored to be asked to work on certain projects that they’re working on. I’m like, “Oh, I’m just an intern.” That’s my attitude, but they’re asking us to do [specific] things. It’s exceeded anything I ever thought I would do during an internship.
OH: Same for me. Giving interns a big role in projects, I think that says a lot.
KATO: Going back to your future plans. I think, Fred, you said you wanted to affect policy, but then you also mentioned wanting to go back into academia eventually? What are your future plans?
TSUTAGAWA: I think for the time being, I want to work on this Pragmatics Assessment thing. Pragmatics Assessment is still one area of Assessment that is not really unified, so I’m hoping my dissertation can help in that area. That’s really where my main passion is. I want to work on this main area of Pragmatics Assessment and make assessments that are useful and hopefully helpful for people. I’m Japanese-American, so one of the reasons I want to be a tenured professor in Japan is I would like to help education policy -- English education policy -- because I know it’s not ideal, right? It’s not perfect. I’d be a foreigner, I’d be an outsider, but if I can be a professional and develop my career and be well published, then maybe I can be at that point where I could consult for them and give them advice and maybe do some positive change. That’s what I hope will happen for my career.
KATO: I think it will happen for you. Saerhim, what are your future plans and goals?
OH: Well. A short plan would be to graduate, first of all! [laughs]
TSUTAGAWA: Yeah, seriously! [laughs]
OH: I don’t think I have too much in detail but because I am interested in Writing Assessment. I do want to be involved in innovating ways of assessing writing for Second Language Learners. And I myself grew up as a Second Language Learner in a different context. I came to the United States at the age of six. I went back to Korea, I didn’t know much Korean. I wasn’t able to write a lot in Korean. So, that’s why I think I’m mostly interested in Writing. That’s what I think goes beyond just spoken language. If I can contribute something in the field of Second Language Writing, and especially in Assessment. Yeah.
KATO: Thanks to both of you.
Nori Kato is a Staff Writer and Office Assistant for the Department of Arts and Humanities. She is also a second year M.A. student in the International Educational Development program at Teachers College.