REFLECTIVE MEMO

UNIFIED PROPULSION

Spring Term 2005, Ian Waitz

(Previous Reflective Memo is here)

Learning Objectives

1. What are the learning objectives (expressed as measurable outcomes) for this subject?

The subject learning objectives are contained on the course web page. Note that these objectives represent a wide range of new topics relative to the time available for the students to learn the material. The objectives thus correspond to lower levels of cognitive ability (explain, describe, estimate, apply).

2. To what extent were you able to integrate the CDIO skills specified for this subject in the Curriculum Plan of 2002?

I did not specifically implement any CDIO syllabus items since these are largely covered in the systems portion of Unified. However, some of the in-class concept questions and homeworks required 2.1.2 Modeling and 2.1.3 Estimation and Qualitative Analysis. I would say both skills were taught at the "Introduce" level.

Teaching and Assessment Methods

3. What teaching and assessment methods did you use and what evidence indicates these methods were successful or not?
a) Prepared lecture notes were available on the web for all of the material: http://web.mit.edu/16.unified/www/SPRING/propulsion/index.html. These notes have envolved over several years starting with a set of handwritten lecture notes. Each year I try to augment them when I find specific areas of difficulty from mud responses, etc. I am quite happy with them at this point. In the end-of-term evaluations 88% of the respondents rated the web page (for all of Unified) Generally Effective or Very Effective. 81% of the respondents rated the prepared lecture notes (for all of Unified) Generally Effective or Very Effective. These assessments are down somewhat from last year. My materials have not changed (however, this may be due to changes in other faculty member's materials).

b) I used 16 concept questions over the 9 lectures with responses taken on the PRS system. The performance and answers to these were provided on the web page. I continue to find these very useful for engaging the class in the material while I am lecturing. 98% of the respondents on the SEF rated the concept questions in PRS (for all of Unified) as Very Effective or Generally Effective. Also several positive comments were made about the PRS/concept questions in the written comments from the end-of-term evaluations. 93% of the students who responded in the SEF Agreed or Strongly Agreed that "Professor Waitz effectively uses active learning techniques" 87% of the respondents to the SEF Agreed or Strongly Agreed that I encourage them to take an active part in their own learning. In general my teaching reviews were good (see below), so I think the students found my lectures and homework assignments to be helpful to them. One aspect of my teaching that the students occasionally note is that my lectures, homework questions and quiz questions are closely aligned with the stated learning objectives. This is something that I work hard on and of which I am proud. It is nice when the students notice.

c) I used mud cards for each lecture. However, following the recommended action from last year, I no longer responded to the mud questions on the subject web page after each lecture. This significantly reduced the time I spent on the subject. Further, many of the students said they used the responses from prior year mud questions (all my responses are still available on the web page) as a "frequently asked questions" database that was useful in preparing for the quiz. (From last year: "However Steve Hall found (through looking at web page hits) that very few students were reading the on-line mud responses. Since this takes a great deal of time (and I already have 3 years of questions and answers posted), I will discontinue responding to these questions on-line.")

d) My other teaching reviews were also good (overall rating of 4.51 out of 5) and there were several specific positive comments in the essay responses to the SEF. Details follow.

e) For 4 of the lectures, the students and I wheeled the CFM56 engine into the classroom so I could point to specific features as I was presenting material. As in the past, the comments from the students on this were very positive. It was particularly helpful for the section on ideal cycle analysis and the section on velocity triangles. On several occasions I found myself having conversations with students both in class and after class that would not have happened if not motivated by the presence of the engine (e.g. "Why are all the blades loose?", "How do they light the combustor?", and "How much does one of these cost?").

f) Based on Steve Hall's suggestion in a prior year (and data collection -- see ST2004 reflective memo) and good experience with the technique, I continued to use a method of getting the students actively involved in the recitation sections. Instead of doing any lecturing, I brought in old exam questions and had them all break-up into groups of 2-4 and work the questions at the board while I roamed around and provided advice. In most cases we made it through 2-3 questions.

g) Based on a recommendation from last year, I did not reuse any prior homework assignments (the students would use the old solutions and did not learn the material as well). Instead, I generated a new set. This took a good bit of time, but I think it had a positive effect on student learning (more on this below). I still provided the solutions to prior years' homeworks for the students to use as study aids and examples.

h) I coordinated more closely with Professor Drela on the material in the Fluids and Propulsion lectures. We shifted the timing of some lectures (in particular for the compressible flows lectures) and we adopted some common nomenclature. I also coordinated with Professor Drela more closely on the Dragonfly systems problems. I had two Propulsion homework problems devoted to the Dragonfly (P4 and P8). One challenge in Unified (with only 9 Propulsion lectures) is that each year my lectures occur at a different time--sometimes before material in fluids, sometimes after. So each year it is necessary to re-evaluate some details of the learning objectives and to re-coordinate with Professor Drela.

i) I mentored Sean Bradshaw (Graduate Teaching Fellow) throughout the year. This culminated in him delivering lecture P6 to the Unified class. He did an excellent job for his first time and the class was very supportive. He also taught one of my recitations. He was widely praised for his work with the students in Unified.

Student Learning

4. How well did the students perform on each subject learning objective? (Where possible, make reference to specific data to support your conclusion.)

The use of the PRS system in-class and the collection of the mud cards gave me a large amount of data on the class performance (formative and summative). Also, as in the past in Unified, we collected data on time spent on various activities (its primary use is to make sure the class stays within bounds).

The performance on the learning objectives overall was good with about half the class scoring above the Joe B level (middle B=75 for propulsion) on the propulsion section of Unified. Class median was 74.5 with an average of 73.9. This performance is typical of past performance with the exception of last year. Last year the class average was about 2 points lower than typical and I believe this was due to having all the homework solutions available online-- something I did not repeat this year. I will address the performance on each of the learning objectives in greater detail below. Before I do, I will provide some of the summary data. The following figure shows the performance on the Propulsion Quiz. Overall I was pleased with the class performance, with the exception of 9 students who scored below 60 points.

The homework scores (below) were all pretty good on average:

Even though all the homework problems were new, most of them were completed by the students within the expected time bounds (it is always a challenge to get this right when designing the problems from scratch). A little more time was spent on P6 and P7 which required some work with a spreadsheet (and the standard deviation was higher too, reflecting variability in student capabilities with spreadsheets):

Specific comments on the learning objectives follow:

A. To be able to explain at a level understandable by a high school senior or non-technical person what the various terms are in the integral momentum equation and how jet propulsion works (Homeworks P1, P2, P3 and Quiz Problem 1). The students did fairly well on the homeworks at the level of this learning objective and achieved a median score of 18 out of 25 (and average of 16.5/25) on the first problem of the quiz. Although they were able to describe at a high-level how jet propulsion works, they were not as comfortable with the integral momentum equation as I would desire (see discussion for next learning objective). Their main deficiency was in describing in physical terms what the various components in the integral momentum equation are.

B. To be able to apply control volume analysis and the integral momentum equation to estimate the forces produced by aerospace propulsion systems (Homeworks P1, P2, P3 and Quiz Problem 1). This continues to be an area of concern. The students make many mistakes in applying the integral momentum theorem (sign errors, incorrect evaluation of dot-products, and missing or extra terms). It gets covered in Fluids also, and 3 out of 9 homeworks were devoted to it. The performance on the homeworks is lower than desired (particularly P2). These are almost carbon copies of prior year homework problems. The median score of 18 of 25 (and average of 16.5) on the quiz suggests that about half the students are below Joe B on this learning objective. I think they need more practice to be comfortable with applying the integral momentum equation (about double the number of homework problems).

C. To be able to describe the principal figures of merit for aircraft engine and rocket motor performance and explain how they are related to vehicle performance. (Homeworks P5 and P6 and Quiz Problems 2 and 3) I believe most of the students achieved this learning objective. The students did well on the homework problems. The median performance on the quiz for these two problems was 18/25 and 11/15 (average performance was 16.7 and 10.6) for the aircraft gas turbine and rocket questions, respectively. One common deficiency was in limited system-level thinking (e.g. not thinking about all the possible impacts of a change in aircraft engine or rocket motor performance). A second common deficiency was in not writing the equations to support the relationships they were describing.

D. Given weight, geometry, and aerodynamic and propulsion system performance information, to be able to estimate the power required for flight, the range, the endurance, and the time-to-climb for an aircraft. (Homeworks P4 and P5, and Quiz Problem 2). I believe most of the students achieved this learning objective. The students performed well on the homework problems. These are applications of straightforward dynamics concepts and they seem to do well with the mechanics of solving the problems. This was only addressed partially in the quiz through checking that they could write down the major equations connecting these parameters to mission performance in Quiz Problem 2.

E. Given mass fractions, and propulsion system performance information, to be able to estimate the range and velocity of single-stage rockets. (Homework P6, Quiz Problem 3). I believe most of the students achieved this learning objective. However, some of the students struggled on P6 -- it required them to use fluids, thermo and propulsion together. It also required the use of a spreadsheet. This is reflected in the low score for Homework P6 (65% on average). Also the performance on Problem 3 on the quiz suggests that some of the students were unable to apply first principles and apply the rocket equation correctly (there were several students who did not recognize that a very simplified form of the rocket equation was applicable to solve the problem).

F. To be able to describe the principal design parameters and constraints that set the performance of gas turbine engines, and to apply ideal-cycle analysis to a gas turbine engine to relate thrust and fuel burn to component-level performance parameters and flight conditions. (Homework P7). I think most of the students did not achieve this learning objective. The one problem I gave them with this as a focus involved a bit of crunching in Excel/Matlab. Since they only had about an hour for the homework assignment, I think they spent most of the time worrying about Excel/Matlab and very little time thinking about the results. Nonetheless, the average on the homework was about 8/10.

G. To be able to explain at a level understandable by a high school senior or non-technical person the energy exchange processes that underlie the workings of multistage compressor or turbine, and to be able to use velocity triangles and the Euler Turbine Equation to estimate the performance of a compressor or turbine stage. (Homeworks P8 and P9, Quiz Problem 4) Most of the students achieved this learning objective. The performance on the quiz was very good (a median of 30 out of 35 points, an average of 28 points). The performance on the homeworks was low (61%) for P8 where they were asked to apply the concepts to something new (the Dragonfly propeller), but better (81%) for P9 which was a fairly standard application of the concepts. This is an area where I have provided many different learning aids to the students. A video of the lecture is available thanks to OCW.

Continuous Improvement

5. What actions did you take this semester to improve the subject as a result of previous reflections or input from students or colleagues?

a) I stopped responding to each muddiest part of the lecture card on the web after each lecture

b) I developed a new set of homework problems.

c) I colloborated more closely with Professor Drela than in past years. This led to two homework problems (of 9) specifically devoted to the spring design project, and to a reconfiguration of our lecture material.

6. What did you learn about your teaching and assessment methods this semester?
Since the student performance improved this year (back to historical levels), and the only major change was the new homeworks, I think it supports my hypothesis about the poor performance last year (students looking at the solutions rather than doing the homeworks themselves).

I also learned how important coordinating with the fluids instructor is EACH year (with the schedule shifting each year). Most years in the past, I have not done a good job on this.

7. What actions do you recommend to improve this subject in the future?
a) I recommend this material receive two additional lectures (11 instead of 9). I believe that some of the reason for the lackluster performance on some of the learning objectives is that I am stuffing too much into 9 lectures. I have carried this recommendation forward for several years.

b) Develop a new set of homework problems again.

Information Sharing

8. To whom have you forwarded this reflective memo?

a) Unified teaching staff

b) 16.50 instructor

c) 16.05 instructor

d) Doris Brodeur and Diane Soderholm

Appendix: Subject Syllabus

• All course materials (syllabus, notes, homework, solutions, quiz) can be found on the Unified web page.