Areas of Assessment: Current and Projected Activities
To guide the collection and review of relevant information, the Assessment Review Committee has defined nine critical areas of assessment. The first three areas focus on academic programs (the undergraduate major, general education, and graduate studies). Areas four and five focus on off-campus learning and the general campus learning environment. Areas six through nine cut across the first five areas, suggesting specific cohorts of students about whom assessment information is to be collected. For each area, the Committee has identified units responsible for collecting assessment information and summarized current and planned assessment activities.
Area One: Student achievement in the undergraduate major
Colleges and academic departments have primary responsibility for assessing their majors. Academic departments are encouraged to use a variety of assessment methods. Twenty out of thirty-two departments require a nationally normed test in the major. All education students take CBASE when applying for the program and the NTE subject matter test when exiting from the program. All business students take the MFAT Business exam. Certain programs (Accounting, Recreation, Athletic Training, Medical Technology, and Nursing) look at performance on certification exams. Four departments use locally developed objective tests of knowledge mastery. In 1993 six departments used performance based assessments such as portfolios in capstone courses or juried art shows and more are planning to incorporate this approach into their existing assessment plans. Four departments conduct senior exit interviews, and other departments, noting their usefulness, are considering doing likewise. All departments track special student achievements (awards in national and state competitions, student papers presented at national and state meetings, student publications, etc.). As of 1993, four departments conscientiously tracked job placement rates and post-graduate study acceptance rates as part of their assessment plans. Twelve departments conduct their own follow-up on departmental alumni.
A review of departmental assessment activities was conducted in Fall 1993. The Academic Associate to the Provost, serving in the absence of the Director of Assessment (ACE Fellowship leave, 1993-94), interviewed all departmental chairpersons. The results of the interview were reported to the Assessment Review Committee. (Appendix 4, Inventory of Departmental Assessment Activities, 1993-94).
Consequently, the Committee approved the following six objectives for continued development of assessment in the major.
- Within the next academic year departments will begin to report assessment information and activities in Annual Departmental Reports as part of the regular planning and departmental review process.
- In 1994-95, all departments will be asked to submit revised assessment plan documents to the Assessment Review Committee for approval (Appendix 5. Components of Revised Departmental Assessment Plans).
- In Spring 1995, the Assessment Review Committee will implement an annual graduating senior survey.
- The Assistant Provost, Director of Career Planning and Placement, and Assessment Review Committee will seek ways to improve the data-base on job placement and graduate school acceptance rates.
- The Assistant Provost and representatives on the Assessment Review Committee will help departments develop methods for summarizing and reporting results of internship evaluations.
- Use of performance-based assessments such as portfolios and senior capstone course projects will be encouraged.
The School of University Studies and faculty teaching University Studies courses have primary responsibility for assessing the University's general education program. The objectives of University Studies provide a focus for assessment in general education. Sources and techniques used to gather assessment information include the following:
Writing Proficiency Exams All entering freshmen take a holistically scored essay exam. Students are tested once again upon exiting the capstone English composition course with a two part essay exam. A third test of writing proficiency, also consisting of two parts, is required of all students after they complete seventy-five credit hours. Students must demonstrate competency on this test or, if they fail, on an approved portfolio option, in order to graduate. Part One calls for a personal essay. Part Two calls for a source-based analytic essay on a related topic. The requirement has been in place since 1985.
Nationally Normed Exams of General Education Annually the short form of Academic Profile is administered to a convenience sample of freshmen in Fall and juniors and seniors in Spring. Comparison of freshmen with upper-division scores provides evidence of improvement in basic general education skills from the freshmen through the upper-division years. Comparisons with norming groups indicate that students at Southeast are much like students at comparative institutions. The test, however, has not provided information useful for program improvement, and the School of University Studies intends to explore alternative approaches to collecting nationally normed data on general education.
Aggregated Student Course Evaluations In Spring 1994, all faculty used the IDEA student evaluation form. In GS101 (University Studies freshmen seminar) ratings were requested for each of the nine general education objectives. It is planned to use objective-based ratings in all courses to determine student perceptions of their progress on each of these objectives by course and course cluster in the University Studies curriculum.
Surveys Items related to University Studies objectives are included on the Graduate Follow-Up Survey and enrolled student surveys. A special survey form devoted to oral communication skills has been developed. A Graduating Senior Survey was piloted in Spring 1994 and will include items related to University Studies when it is administered again in Spring 1995.
Objective-Oriented Scoring of Writing Exams Rubrics specific to selected University Studies objectives will be used to assess student performance on Part Two of the Writing Proficiency Exam. Rubrics have been developed for critical thinking, reasoning, and analysis (similar to rubrics for essays on the GMAT and the ETS Tasks in Critical Thinking). A team of instructors in Ethics and Business have written a rubric for the ethical analysis of cases in business ethics which can be generalized to the University Studies valuing objective.
Samples of Student Work The University Studies Planning and Assessment Committee will set up a five year plan to analyze samples of student work in the upperdivision, interdisciplinary component of University Studies relative to the nine objectives.
Analysis of "Found" Data Already existing information on student behaviors relevant to University Studies objectives needs have been identified. A new student tracking file will enable analysis of course-taking patterns within University Studies by major, GPA, ACT, and other variables. Academic departments and units within Student Affairs can provide information on student involvement in community service (a measure of responsible functioning in the natural, social, and political environment). Other possible sources of data include statistics on use of the library, the Writing Center, the University Museum, and attendance at cultural events.
The University Studies Council approved a University Studies assessment plan in Fall 1993. Some of the elements of the plan are indicated above. Specifically, the following recommendations were made:
- Create a standing University Studies Planning and Assessment Committee to oversee and develop assessment in University Studies,
- Redefine existing administrative positions in University Studies to create a new position of Director of Assessment and Planning for University Studies
- Expand the use of aggregated analysis of student course ratings of progress on general education objectives to all University Studies courses
- Explore the use of writing proficiency exams for assessment of critical thinking
- Initiate a four-year cycle of assessment of University Studies objectives based on external evaluation of student performance in upper-division University Studies courses.
Area Three: Student achievement in graduate programs.
Primary responsibility for assessment of graduate programs belongs to the academic departments in which they are housed, supported by the School of Graduate Studies and Extended Learning. Some graduate programs have access to external assessments. For example, graduate students in school administration participate in the Missouri administrators assessment process, and communication disorders students take a licensure exam. GRE scores are required for admission to graduate programs in education. All graduate students must be formally advanced to degree candidacy on the basis of GPA midway to completion of their program, and all students must complete either a research project and comprehensive written exam or a thesis and oral exam, which are evaluated by a committee of graduate faculty.
Planned developments in assessment of graduate programs include the following:
- Beginning in the Spring 1995, all students completing graduate degrees will be asked to participate in an exit assessment of student satisfaction and future plans for employment or education.
- All graduate programs will be included in the 1994-95 call for revised departmental assessment plans.
Responsibility for assessing off-campus and transfer students is distributed across the institution. The Missouri Student Achievement Study provides data on students transferring into and out of the University within the state of Missouri. The Writing Outcomes Program has done comparative studies of transfer versus non-transfer student performance on the Writing Proficiency Test. Most off-campus students also attend courses on campus and are assessed by departments in their major. Students in the University's 2+2 Nursing program are assessed along with all other nursing students. The IDEA student course rating form provides aggregated data about performance of off-campus courses. The Bootheel Educational Center analyzes enrollment patterns in its Annual Report and has conducted surveys of students and business and industry in its region. Needs assessments at the BEC are conducted annually by the BEC staff.
Planned developments in the assessment of off-campus and transfer programs include the following:
- A student tracking file integrated with the student record system (developed by the Student Information System Task Force, Spring-Fall 1994) will provide improved access to information about off-campus and transfer students.
- Success of University\community college articulation agreements will be monitored through surveys and focus groups.
Responsibility for assessing the quality of the campus learning environment is shared among divisions, with support and guidance from the Office of the Provost and the Office of Institutional Research. The alumni surveys and enrolled student surveys primarily address questions about student satisfaction. The quality improvement program within the Division of Finance and Administration seeks information related to student needs. The Division of Student Affairs maintains an ongoing concern with the quality of student life (Appendix 6, Campus Learning Environment Inventory of Assessment Activities). It is anticipated that the Graduating Senior Survey and periodic enrolled student surveys will target specific aspects of the campus learning environment.
Area Six: Information about knowledge, skills, aptitudes, attitudes, values, and achievements of entering and prospective students.
Freshmen baseline information cuts across areas one, two, four, and five. Data on ACTS, high school GPA's, high school rank, completion of core curriculum, etc., are reported in the Missouri Student Achievement Study, and shared with faculty and administrators. Data on placement tests, such as the Writing Placement test, are kept in a test file from which they can be accessed for assessment purposes. The CIRP provides demographic and attitudinal information about entering freshmen. The short form of Academic Profile, a test of general education, has been taken every fall by a sample of beginning freshmen. IDEA instructional evaluation in GS101 provides additional information about perceptions of freshmen.
Area Seven: Information about enrollment and retention.
Enrollment and retention information cuts across areas one, two, four, and five. The Assistant Vice President for Enrollment Management produces periodic reports on enrollment and course-taking patterns. The Office of Institutional Research produces reports on retention and persistence rates. The College Student Inventory is being piloted as an instrument for identifying students at risk. Project Recovery assesses the effect of intervention efforts in improving retention. The newly formed Enrollment Development Task Force will continue to seek information related to enrollment and retention.
Area Eight. Information about the knowledge, skills, aptitudes, perceptions, prospects, and achievements of graduating seniors.
All academic departments collect assessment information (for example, exit interviews, scores on nationally normed or locally developed tests, special achievements, medical school acceptance rates) about their graduating seniors. The Graduating Senior Survey will provide additional information about the prospects and perceptions of graduating seniors.
Area Nine: Information about employment, post-graduate education, achievements, and perceptions of graduates.
The alumni follow-up survey has been conducted since 1983. Several departments maintain contact with their graduates through departmental newsletters. The University participates in periodic CBHE surveys of alumni and employers. Future plans include revision of current institutional survey procedures and increased emphasis on departmental follow-up on graduates.