MIDAMERICA NAZARENE UNIVERSITY

ASSESSMENT PLAN

SCOPE OF THE PLAN

“MidAmerica Nazarene University, a liberal arts institution of the Church of the Nazarene, is a Christian community in the Wesleyan-Holiness tradition that seeks to transform the individual through intellectual, spiritual and personal development for a life of service to God, the church, the nation, and the world.”

In light of this mission, the primary focus of the assessment effort is:  

Ø      Developing intended learning, co-curricular and administrative outcomes.

Ø      Aligning curriculum with learning outcomes

Ø      Developing appropriate measures to assess learning outcomes

Ø      Collecting pertinent data using assessment measures

Ø      Using collected data to improve learning

Ø      Sharing results of data collection with key stakeholders

Ø      Using results of data collection and analysis in planning, budgeting, and program modification.  See chart below.

Student Learning Assessment works in conjunction with Program review to continuously improve the effectiveness of MidAmerica Nazarene University.  Program review is undertaken every five years to encourage self-study and planning within units.  The purpose of program review is unit-level improvement through a systematic, cyclical review process.   The structure of the review is based upon the Higher Learning Commission institutional self study questions but the questions are re-written so that they are appropriate for department level analysis.  The process assists each office in evaluating the strengths and weakness of each of its department with relationship to program relevancy and provides a basis for assessing current issues/needs and making recommendations for future directions.  The systematic procedures of the review process also identify strengths and weaknesses of programs.  The review results in recommendations for program development and, where appropriate, modification.  The goal of this progression is to promote and maintain quality and efficiently administered offices and programs.    Program review serves as a planning document for the office under review.  The unit and the University use the results of program review as a guide for decisions concerning resource allocations, staffing, admissions standards, curriculum content, and other related matters. Recommendations based on program review are considered in relationship to availability of resources, institutional mission, and master planning goals.

The University has developed an Assessment Records Book (Nichols & Nichols) which is part of the assessment plan.  It is a formal method to rigorously and systematically document student learning in a consistent and integrated format.  The Assessment Record Book consists of three types of forms which are completed by every office on an annual basis:

Ø      Form A is a title page that identifies the assessment period covered and the unit’s name and degree-granting program or sub-units.

Ø      Form B links the academic program objective and/or educational-support unit mission/objectives to MidAmerica Nazarene University’s institutional mission and, if applicable, to university-wide general education outcomes.  Form B also lists the intended educational (student) outcomes or administrative objectives to be evaluated during the assessment period.

Ø      Form C constitutes an assessment report concerning each educational or administrative outcome; its means of assessment and criteria for success; a summary of the assessment data collected; and a description of how this assessment information was utilized to improve student learning, departmental operations, or services to students.  

Completion of these forms demonstrates the unit’s movement from the planning stage to the implementation and reporting stages of the assessment cycle.

The Assessment Record Book answers the need for documentation of the use of assessment results by creation of tightly focused, centralized and concise historical record of the assessment process that can be verified and validated by the Higher Learning Commission.

OUTCOMES ASSESSMENT SYSTEM – OASYS

The Assessment System is utilized for the assessment of and improvement in student academic achievement and educational support services at MidAmerica Nazarene University.  The system is comprised of four major processes (Planning, Implementation, Assessment and Reporting).   Each process feeds into the next process and is used as a continuous cycle to maintain strengths, correct weaknesses, and continuously improve the curriculum and co-curriculum.  

Planning/ Revision Process

Each unit will plan, review and revise intended educational (student) outcomes established during the previous cycle of OASYS.  These outcomes should be congruent with the mission of the University and the mission of their office.  For achieved outcomes, units decide if the desired student learning level is satisfactory or not and revise the criteria for success if needed.  For unachieved outcomes, each unit plans program improvements (which must be documented in all university minutes and annual assessment reports), adjusts the outcome, adjust the criteria for success, or a combination of these.  At this stage, units also review the methods used to measure the outcome and ask, “Did this method measure what it was intended to measure?  If not, is there a different method which should be used?  Did the method indicate a problem in another area which needs attention?  Then units plan improvements based on assessment conclusions.

Implementation Process

Each unit implements improvements and interventions identified during the planning process to meet the intended educational (student) outcomes.   The unit tracks program improvement changes through existing or new means of assessment designed to measure intended outcomes.  Multiple assessment means should be used by each unit for each outcome.  Possible direct means of assessment include: standardized tests; course embedded tests, projects and papers; portfolios, and capstone coursework.   Examples of indirect means of assessment include: student assessment of performance, behaviors and attitudes; peer analysis of student performance (internal customer); analysis by business, graduate schools, hospitals, churches, etc. of student and former student performance (external customer).   A criterion for success is set for each means of assessment.  These criteria are most often stated in terms of percentages, percentiles, averages, or other quantitative measures.

Assessment Process

During the assessment process, units collect information to measure each intended outcome using the selected assessment methods.  

Following the data collection, each unit reviews the results of measurements against criteria for success to see if each intended outcome was achieved.  The unit then identifies outcomes which were achieved or exceeded and strengths to be maintained.  When outcomes which were not achieved are identified by units, then weaknesses to be corrected are specified so that action plans can be created to make improvements in the planning stage.

The areas of strengths and opportunities for improvement identified during the assessment process are used as input into the next cycle of OASYS.  If improvements are achieved, the change is standardized.  If improvements are not achieved then the approach is modified. 

Reporting Process

Data collection, analysis and any subsequent program improvements must be fully documented.  Data collection is documented when each unit reports data mastery sheets, grades by assignment and student artifacts; and in the unit’s annual assessment report.    Program improvements are documented when each unit reports its assessment day minutes and turns in its annual assessment report.  Units provide constituents with appropriate feedback regarding student learning.  The areas of strengths and opportunities for improvement reported are used as input into the next cycle of OASYS.

Each unit stores and maintains data mastery summaries, grades by assignment and student artifacts and sends electronic copies to the Associate Academic Dean, Program Review and Assessment and to the Chair of the General Education committee.  Each unit stores annual assessment reports and sends electronic copies to the Associate Dean for inclusion in the Assessment Records Book.

OUTCOMES ASSESSMENT TIMELINE

The units responsible for assessing student learning, co-curricular and administrative outcomes are academic divisions/programs, the general education committee and administrative support units.   The general education committee, in consultation with each undergraduate division, is responsible for implementing this system of assessment with regard to general core requirements.  Inherent in this plan is the requirement that these processes contribute to self-regulation and ongoing program revision through effective feedback and accountability mechanisms. 

ADMINISTRATIVE STRUCTURE OF ASSESSMENT

Associate Academic Dean of Accreditation, Program Review and Assessment

In recent years, realizing the growing importance of the national conversation on accountability in higher education, the University has taken steps to appoint an Associate Academic Dean.   

Assessment Committees

A major component of OASYS is the Outcomes Assessment Facilitation Team (OAFT) and the Nonacademic Assessment Team (NAT).   These committees facilitate each process in the system.  Each committee and its chair is appointed by the Associate Academic Dean.   The members of OAFT will be representative of the academic units responsible for implementing OASYS.   The members of NAT will be representative of the non-academic units responsible for implementing OASYS.  OAFT and NAT will develop and present assessment in-service  training for academic units.   These teams will provide resources, information, and feedback to each office as they work through OASYS.  These committees will provide a forum for units to share the results of improvement efforts, to solicit advice from peers, and to share lessons learned.   The committees will help units follow the time line and meet deadlines. 

The committees will establish an accountability plan for units in reporting results of assessment efforts.  The committees will provide a standardized reporting format to units so that information is reviewed on a regular and systematic basis.  The committees will ensure that information will be utilized to encourage improvement and not for punitive measure.  The committees will encourage dialogue which reinforces commitment to the improvement and assessment; which helps to reshape vision and goals; which supports continued learning about student learning assessment; and which supports planned improvement and change.

Each academic office must appoint a representative to OAFT who is eligible to receive a one course load reduction for this responsibility.  Each non-academic office is responsible for appointing a representative to NAT.  The membership of these committees is confirmed by the Associate Academic Dean and the faculty during fall Faculty Orientation.

The Associate Dean receives annual assessment reports from each office on behalf of the Outcomes Assessment Facilitation Team and provides annual feedback to each unit head regarding their progress in fully implementing an assessment program and using results to make program improvements.  

Unit Assessment Representative’s

Under supervision of the unit head, the representative provides administrative coordination and support for the educational outcomes assessment activities of the unit.

Duties and Responsibilities:

1. Serves as the unit’s representative on the Outcomes Assessment Facilitation Team or on the Non-academic Assessment Team.
2. Assists colleagues in planning, updating and implementing all outcomes assessment plans for the unit.  This includes all programs, majors or degrees, the general education core and teacher education.
3. Monitors the unit’s syllabi and/or data mastery sheets to make certain that outcomes align properly with the University mission, the unit’s mission, any course objectives, means of assessment, etc.
4. Coordinates the collection of student learning assessment data with the unit’s faculty or staff.
5. Ensures that student learning assessment data (including data mastery summaries, grades by assignment, examples of rubrics and examples of student work) are stored in a central location for the office, and that electronic copies are sent to the Associate Academic Dean for Accreditation, Program Review and Assessment and to the Chair of the General Education committee.
6. Leads faculty through the analysis of outcomes data and use of results (completion of feedback loop/ program improvements).
7. Ensures that all outcomes related program improvements are documented in meeting minutes and that copies are sent to the Associate Dean.
8. Serves as liaison between administration and faculty on a range of student learning assessment issues; also serves as liaison with alumni and employers of graduates.
9. Prepares annual outcomes assessment reports (Forms A, B, and C) by June 15^th for the unit head to include in their annual report to the Associate Dean.
10. Performs miscellaneous outcomes related duties as assigned by their unit head.

NEW EMPLOYEE ORIENTATION

Basic Terminology 

As a new faculty or staff member at MNU, the course(s) which you will teach or the educational support functions which you will perform may play an essential role in your unit’s student learning assessment plan.   It is therefore vital that you clearly communicate with your unit head in order to discover their expectations regarding assessment before you begin to create your syllabi or begin new programming.  

Intended Educational (Student) Outcome:  This term describes what the unit intends for a student to be able to think, know, or do when they’ve completed a given curricular or co-curricular program.

Means of Assessment:  The strategies with which information will be collected in order to validate each intended outcome.  These may include such techniques as comprehensive exams; standardized exams; student portfolios, senior projects or theses evaluated by a committee; exit interviews, alumni surveys, graduating student surveys, employer surveys; results of licensing exams; evaluations by practitioners; or student satisfaction levels.  Units should consider including value-added assessments and external reviews whenever possible.  Multiple measures (including alternate strategies) are suggested for each outcome.  Your office may have embedded a planned measure into your course or co-curricular responsibility to demonstrate student learning.  If so, this means of assessment should not be changed without first consulting with your direct supervisor.

Rubrics: A set of authoritative rules to give direction to the scoring of assessment tasks or activities.  To be useful, a scoring rubric must be derived from careful analysis of existing performance of varying quality.  Your unit may have developed a rubric by which data is to be collected about student learning.  If so, this rubric should not be changed without first consulting with your direct supervisor.

Criteria for Success: The benchmark that the unit sets and against which program performance is judged by faculty or staff within the unit.  These criteria are most often stated in terms of percentages, percentiles, averages, or other quantitative measures.  Criteria should neither be set unrealistically high nor so modestly low that anyone can meet them, for without criteria based on what a program or service “ought” to be able to expect and that can be measured against actual performance, it would be difficult to make use of the data to improve the program or service.

Syllabi and/or Data Mastery Sheets:  If your course assignment, co-curricular activity or administrative responsibility measures one of your unit’s outcomes this must be documented in a syllabus, data mastery sheet or both. Constituents must be able to see the alignment between the University mission, the unit’s mission, the intended educational (student) outcome, the means of assessment and the criteria for success.    

Data collection: Your office will ask you to turn in artifacts of student learning with appropriate rubics attached.   In addition you will be asked to summarize the data by completing a data mastery sheet to help you office analyze whether it has met its stated criteria for success for each outcome.  Outcomes data is due each semester on the same day as final grades are submitted.

Use of data:  You will be asked to participate in assessment day meetings where your unit analyzes outcomes data against its criteria for success.   As a result, program improvement interventions will be agreed upon and implemented which could impact your course(s) or co-curricular responsibilities.  (For example, a division could decide to add an assignment to a professor’s course in order to advance student learning.)

Sources:

James O. Nichols’ The Departmental Guide and Record Book for Student Outcomes Assessment and Institutional Effectiveness (Agathon Press, 1995).

Assessment Plans: Definition of Terms, http://web.odu.edu/