Assessment of Student Academic Achievement
Mechanical Engineering-Engineering Mechanics Department
Michigan Technological University
Undergraduate Program
The faculty of the ME-EM Department has always been concerned with the quality of its instructional programs. What this plan proposes is to formalize and quantify the evaluation of the undergraduate program. With the implementation of the proposed plan, measures of student academic achievement will become available. These measures will be used to evaluate the effect of curricular changes, suggest new directions for curriculum development and suggest educational environment changes which will maximize student development. The ME-EM faculty believes that this assessment program will become an extremely valuable tool enabling the continuous improvement in the value and quality of our undergraduate program.
Goals
In developing the goals to be met by our students consideration was given to the ME-EM Department mission statement, the University mission statement, and the ABET 2000 goals. Each of these is detailed below.
1) The ME-EM Department¹s mission statement states:
³The ME-EM Department is nationally recognized as having one of
the best undergraduate programs in the country. A healthy balance between
undergraduate and graduate teaching and research is the cornerstone of the
department. A challenging curriculum, a supportive relationship between
students and faculty, and an environment that welcomes diversity are the
hallmarks of the department as recognized by students, alumni and
industry.²
2) The University's mission statement states:
³Michigan Technological University will be a nationally and
internationally recognized leader in meeting challenges of the future
through undergraduate and graduate education and research in sciences and
engineering. At the undergraduate level, we will have comprehensive,
forward-looking curricula in sciences and engineering that educate
technically-competent, intellectually-vital graduates who are at the same
time effective communicators and aware of the social, economic, and
cultural contexts of their work. . . .²
3) The recently published ABET Criteria 2000 goals are:
Engineering graduates should have:
A. An appropriate knowledge base in the areas of mathematics, science,
and the engineering sciences.
B. An ability to design and conduct experiments, analyze and interpret
data.
C. An ability to identify, formulate and solve engineering problems.
D. An ability to use modern engineering tools such as CAD and CAE.
E. An ability to design a system, component, or process to meet desired
needs.
F. An understanding of professional and ethical responsibility.
G. An ability to function as a member of a team.
H. An ability to communicate effectively.
I. The broad education necessary to understand the impact of engineering
solutions in a global societal context.
J. A knowledge of contemporary issues.
K. A recognition of the need for an ability to engage in life-long learning.
After considering the above goals the faculty formulated the following set
of four encompassing goals.
1. A graduate of the ME-EM Department should possess a solid foundation in and be able to apply the principles of engineering science and mathematics to the solution of engineering problems.
2. A graduate of the ME-EM Department should be proficient in the use of techniques, procedures, and knowledge bases available to mechanical engineers and to be able to use them to solve structured and unstructured problems in mechanical engineering.
3. A graduate of the ME-EM Department should have demonstrated the ability to synthesize and develop useful products or processes.
4. A graduate of the ME-EM Department should be able to work well with others and accurately communicate his/her work.
Assessment of Goals
In order to quantify the students' success in achieving these goals, the following assessment devices will be employed:
Pre-Graduation
1. Material presented in the department¹s core curriculum will be monitored/evaluated during the Junior ans Senior yeas of student programs. The ME-EM department is structured around groups of faculty in four technical areas. Each of these areas, Design & Dynamic Systems, Energy Thermofluids, Manufacturing, and Solid Mechanics, has defined learning objectives for our mechanical engineering graduates. Samples of student work will be collected in selected courses. Following this compilation, a team of faculty will evaluate student performance on the objectives defined by each of the technical areas. This evaluation will take place yearly. A list of the learning objectives, by technical area, is presented in appendix A.
2. Student performance on the Fundamentals of Engineering Exam will be monitored. The Fundamentals of Engineering Exam is a nationally-normed examination which reports performance in 10 subject areas basic to engineering. Five of these subject areas Dynamics, Fluid Mechanics, Mechanics of Materials, Statics, and Thermodynamics evaluate performance on learning objectives considered fundamental to mechanical engineering. Monitoring student performance on this exam will allow the curriculum committee to compare MTU/ME-EM student performance, on an examination testing fundamental engineering knowledge, against state and national norms. (This exam is not currently required of our graduates. Historically 1/3 of our ME graduates take this exam.)
3. Student performance on the ME-EM capstone design project will be monitored. Assessment of student performance on the capstone design project will be selectively evaluated by teams of faculty. Each faculty team will be made up of faculty who did not advise the students submitting the selected reports. This assessment will include the evaluation of indicators such as written and oral project presentations, use and coalescence of engineering theory, and creativity. Student design reports will be shared with the industrial sponsors and their comments will be solicited. Whenever possible industrial representatives will attend design presentations.
4. The movement of ME-EM graduates into graduate and professional school will be monitored. Students will be asked to submit a form asking whether or not they have applied for or been accepted to graduate school when they submit their degree schedule.
Post-Graduation
1. Information on the flow of ME-EM graduates into the workplace will be evaluated. Statistical information on the number of job offers each student receives and average starting salary of graduates will be requested from the University Career Center.
2. Information on the success of ME-EM graduates in the workplace will be collected. Surveys will be sent to a statistically significant sample of our graduates 5 years after graduation. In these surveys input will be sought on the relevance and value of the education each alumnus received at MTU. Suggestions for curriculum changes will also be solicited.
The ME-EM curriculum committee believes that these assessment tools will provide sufficient information to track the department's progress toward providing the best possible education for its students. All of these assessment tools are directly applicable to the evaluation of goal 1. Clearly course work performance, performance on the Fundamentals of Engineering exam, and post-graduation performance are all influenced by the amount of fundamental scientific and mathematical knowledge students acquire at MTU. Departmental progress toward goal 2 can be measured by the evaluation of the capstone design experience, course work and the post-graduation survey.. Evaluation of the department's success in achieving goals 3 and 4 will be measured by evaluation of capstone designs, observation of our students ability to find employment, and results from downstream alumni surveys.
The ME-EM faculty believes that the above-stated assessment procedure will not deny equal access of any qualified student to our program.
Implementation
The curriculum committee of the ME-EM Department will be the group responsible for collecting and analyzing the data gathered using the assessment tools. Following this analysis, the committee will present its findings with recommendations for curricular improvement at a faculty meeting during the winter quarter of each academic year. At the same time, a written report documenting the assessment results will be prepared by the committee for distribution to the student population. Student distribution will be accomplished using the active student societies within the department (ASME, SME, SAE, Pi Tau Sigma). Each year copies of this written report will be provided to the president of each student society.
Time line for implementation
During the 1996-97 academic year assessment will begin using:
1. The Fundamentals of Engineering Exam
2. Student performance in selected courses
During the 1997-98 academic year assessment will begin using:
1. Information on the flow of students into the workplace.
2. Data concerning the movement of graduates into graduate and
professional schools.
3. Student performance in the capstone design.
During the 1998-99 academic year assessment will begin using:
1. Post-graduation surveys.
M.S. and Ph.D. Programs
INTRODUCTION
The Department of Mechanical Engineering Engineering Mechanics has graduate programs leading to an M.S. degree in Mechanical Engineering, an M.S. degree in Engineering Mechanics, an M.S. degree in Manufacturing Systems Engineering, and a Ph.D. in Mechanical Engineering Engineering Mechanics.
Students entering the M.S. degree program must have a B.S. degree in one of the branches of Engineering or Physical Sciences or Mathematics, and for the Ph.D. degree they must have, in addition, a year of graduate studies in Mechanical Engineering or Engineering Mechanics. These formal requirements may be waived if an equivalent proficiency acquired in some other manner can be demonstrated. For the M.S. degree students can select either the thesis, project or course work option. The Department has no required uniform "core curriculum" at the graduate level. However, for the M.S. degree students are required to take two (2) mathematics courses (six credits) for the Mechanical Engineering and the Manufacturing Systems Engineering programs and three (3) mathematics courses (nine credits) for the Engineering Mechanics program. These must be selected from a Department approved list of mathematics courses. A student works with their graduate advisor and advisory committee to define a set of courses appropriate to the goals of the research program. The set of courses must be approved by student's graduate advisor, advisory committee, Director of Graduate Studies and the Graduate Dean. Upon completion of a graduate research program, M.S. degree students on the project or thesis option and Ph.D. degree students must write a project/thesis/dissertation on their research. The student must also publicly present a summary and defense of the research and conclusions.
Research at the Ph.D. level is characterized by major contribution to
the field of study normally resulting in publication in the archival
literature. Research at the M.S. level may also result in archival
publications but it is primarily characterized by a literature review,
planning, performing, analyzing, and presenting a body of research for
which (s)he is responsible.
This document describes goals for the graduate programs and a plan for
assessment of graduate student progress toward these goals This assessment
plan was approved by the graduate faculty of the Department on May 14,
1996.
PROGRAM GOALS AND OBJECTIVES
The mission of the Department graduate programs is to prepare students for work in design and research and development of advanced engineering and technology and, in the case of the Ph.D. program, to become research engineers or to enter the academic community. In addition to meeting the requirements of a variety of advanced courses, the students enrolled in the Department's graduate programs will achieve a significant level of specialization in the topic area of their theses (for M.S. students) or dissertations (for Ph.D. students).
The goals of the Department graduate program to be assessed are that graduates of these programs must demonstrate that they are:
ò technically competent
ò effective communicators written and oral
ò critical thinkers, creative and innovative
ò life long learners.
Once our graduates have achieved these goals, we expect them to make significant contributions to, and to be leaders in, their profession and society.
MEANS OF ASSESSMENT
The following measures of assessment will be used for determining a graduate student's success in meeting the stated goals:
A. Evaluation of the Thesis/Dissertation
All PhD candidates and those MS candidates on the thesis or project
degree option must submit a written document detailing their research work.
This document is thoroughly reviewed by the student's advisory committee
which must give its final approval of the document.
A form will be developed by the graduate committee, with input from the graduate faculty, for the evaluation of this written document. The evaluation instrument will address the following topics:
1. The effective written communication of the material
2. The technical level of the work.
3. The innovative/creativity of the work.
This form will be completed by each member of the student's advisory committee.
B. Evaluation of the Thesis/Dissertation Defense or Final Oral Report
All PhD and MS candidates must conduct an oral presentation at the conclusion of their degree program. For the PhD candidates and the MS candidates on the thesis or project option this oral presentation is a defense of their research work. For the MS candidate on the course work option it is an oral examination of his/her courses. The presentation is open to the public and is evaluated by the student's advisory committee.
The form developed for Item D will be used for the evaluation of the oral presentation. (The portion of the form pertaining to technical content will not be used for this evaluation.) The evaluation instrument will address the following topic:
1. The effective oral communication of the material
This form will be completed by each member of the student's advisory committee and possibly by selected members of the audience.
C. Publications/Proposals
During the course of a graduate student's studies and also immediately after their completion, the student may have opportunities to participate in the submission of research proposals and technical publications. These documents may be totally or partially written by the graduate student and will be directly related to their research work.
The department will keep a record of such activity by its graduate students as it does of its faculty. Lists of publications and proposals submitted will be maintained. Publications accepted and proposals funded will be evidence of effective written communication, technical competence, and innovative, critical and creative thinking.
D. Presentations at Professional Meetings
On occasions, typically near the end, or after the completion of their degree program, a student has the opportunity to give a technical presentation at a professional meeting based on their research. On most occasions the student's advisor or a member of the student's advisory committee is present at the meeting and attends the presentation.
A form will be developed by the graduate committee, with input from the graduate faculty, for the evaluation of this technical presentation. The evaluation instrument will address the following topics:
1. The effective oral communication of the material
2. The technical level of the work.
This form will be completed by each of the student's advisory committee
members who are in attendance at the presentation and possibly by selected
members of the audience. If evaluations are made by the professional
society sponsoring the conference, efforts will be made to obtain those
results.
E. Qualification Examination
All PhD students are required to pass a qualification examination within 12 months after their admittance to the PhD program. This examination contains both an oral and a written component. The written component consists of three two hour examinations; one in the student's major subject, one in the student's minor subject and one in mathematics. The oral component is a minimum hour long examination in the student's major subject which contains "open ended components." The exam is designed to test for a mature understanding of undergraduate material, except for the mathematics portion which covers graduate level material.
Currently, the written examinations questions are kept on file by the Director of Graduate studies. This practice will continue. Anonymous records will be kept of student performance. This will be a measure of technical competence and critical thinking. The form developed under item B will be used to evaluate the oral portion of the exam. This evaluation will be kept in the student's records.
One function of the qualification examination is as a diagnostic tool. Therefore, the results obtained from this instrument may be used not only as an evaluation of the student in the area os technical competence, critical thinking and oral communication, but also to show improvement as the student continues in their program. In the latter circumstance, these results will be compared to the results obtained in items A, B and C to show the effectiveness of the program in reaching the stated goals.
G. Exit Interview Form
There is currently an exit interview form which all graduate students must complete prior to their leaving the university. This includes information such as future employment plans, and comments or suggestions about the graduate program.
The graduate committee, with input from the graduate faculty, will consider revising this form to make it more useful to the assessment program. It will be used to as a measure of technical competence and also as an overall evaluation of the graduate program. The information obtained from this form will be compiled and when necessary will be kept confidential.
G. Alumni Survey
A practical means of obtaining information about the effectiveness of the graduate program is from the graduates themselves. Therefore, in addition to the exit interview, an alumni survey will be conducted.
The survey will be developed by the graduate committee with input from the graduate faculty. It will be conducted on a regular basis, at yet to be determined time intervals. An efficient means of administering the survey, i.e. mail, electronic mail, world wide web, will be determined as will an appropriate sample size. The survey will address the following topics:
1. Evidence of life long learning.
2. Evidence of technical competence.
3. Evidence of innovative and creative activity.
The information obtained from this survey will be kept anonymous.
H. Company Survey
An additional means of determining the effectiveness of a graduate program is by obtaining information from the people who work with our graduates. Therefore, a survey will be conducted of the employers of our graduates. This may be done in cooperation with the career center and/or our graduates.
A survey form will be developed by the graduate committee with input from the graduate faculty. This survey will be conducted on a regular basis at five year intervals. All responses will be kept anonymous with respondents notcommenting on individual employees but rather our graduates as a whole. This evaluation instrument will assess the following:
1. Evidence of life long learning.
2. Evidence of technical competence.
3. Evidence of critical thinking and, innovative and creative activity.
4. Evidence of effective communication both written and oral.
I. Patents/Entrepreneurial Activity
On occasion a patentable item is developed during the course of a
research project. This may occur during the course of a student's studies
or after they have completed their degree. There may also exist some
opportunity for entrepreneurial activity, or a graduate of the program may
conduct such activity during the course of their careers.
Information on patents and entrepreneurial activity will be documented by the department. This information will be obtained through the graduate faculty and/or through the alumni survey.
One external measure which will be used to assesss the quality and success of the PhD program is the releative ranking in the National Research Council publication on PhD programs.
FEEDBACK TO STUDENTS AND FACULTY
The Department Graduate Committee will have the responsibility for overseeing the assessment of the graduate programs. This will include the following:
ò working with the graduate faculty to develop the stated
forms and survey instruments.
ò informing the graduate students of the purpose and the
nature of the assessment plan.
ò compiling the data obtained from Section III and informing
the graduate faculty and graduate students of these
results.
The results obtained from this assessment program will be used to guide the evolution of the curriculum, teaching, advising and research policies of the Department's graduate program on a bi annual basis.
IMPLEMENTATION PLAN
Measures A, B, E, and F of the assessment plan are currently a part of our program. During the 1996 97 academic year the Graduate Committee will develop the needed evaluation forms and procedures for their use. Measure C and parts of Measure I will also be implemented during the 1996 97 academic year. Details pertaining to measures G, H, and the remaining portion of I will be determined during the 1997 98 academic year and implemented during the following year.
EVALUATION OF THE ASSESSMENT PLAN
The Department Graduate Committee will formally evaluate the assessment plan every other year. In addition this Committee will seek input from the graduate faculty and graduate students on the assessment plan and review it on a continual basis.
Appendix A
Core Requirement Learning Objectives for Mechanical Engineering Graduates
Energy Thermo-Fluids
Graduates should be able to:
1. Identify the state of a pure, simple compressible substance.
2. Apply the Laws of Thermodynamics for a given process
and/or cycle.
3. Understand common engineering cycles.
4. Identify the flow and heat transfer regime for given
applications.
5. Calculate a heat transfer rate.
6. Understand Linear and Angular Momentum Theorems
7. Calculate the minor losses in piping systems.
Design & Dynamic Systems
Graduates should be able to:
1. Be able to recognize specific type of engineering situations (Spatial,
Strength, Synthesis, Design, etc.)
2. Apply the appropriate engineering skills (Design and/or Analysis) to
realize a complete solution to an engineering problem.
3. Produce the necessary graphical, spacial, or analytical representation
of an engineering situation.
3. Utilize equilibrium and kinematic relationships to understand motion
and force (internal and external) conditions in mechanical systems.
Manufacturing
Graduates should be able to:
1. Identify and classify various manufacturing processes.
2. Identify and name various manufacturing equipment.
3. Suggest a manufacturing process or processes which are suitable for the
manufacture of a given part made from a specified material providing
supporting justifications.
4. Associate common defect with the appropriate
manufacturing process.
Solid Mechanics
Graduates should be able to:
1. Calculate the force resultants acting external and internal to machine
components.
2. Calculate the stress distribution acting internal to structural
components.
3. Use an appropriate failure criterion to identify whether a machine
component is adequately designed to resist the applied loading.
4. Evaluate the elastic deflection of a structural member.