Component C: The organization's ongoing evaluation and assessment processes provide reliable evidence of institutional effectiveness that clearly informs strategies for continuous improvement.
Prior to 2006-2007, the VCSU assessment system was a collection of spreadsheets and databases managed by a few key individuals in various areas. Since that time, it has been evolving into an integrated and seamless entity with faculty, students, and staff involved as contributors and consumers of informative data. The institution uses data from student, faculty, staff, and employer assessments and evaluations to manage and improve operations and programs. Effective data analysis allows knowledge-based decisions about program strengths and weaknesses. At times, aggregated data indicate successful outcomes and promote staying on course, other times the data necessitates change. The VCSU teacher education program has provided institutional leadership in the academic assessment area and a sound model to emulate, as the last NCATE report demonstrates.
As core components of University shared governance, campus committees meet regularly to plan, implement, evaluate, and reflect on decisions related to academic and administrative issues, budgets, staffing, and other matters that are pertinent to their duties and authority. Most have a broad representation that includes faculty, staff, student, and administrative membership. Each decision-making body is open to input from its constituency, and all disseminate information throughout the campus through their connection to the group and by regularly publishing meeting minutes.
The University uses a wealth of data and information services to guide planning, evaluation, and decision-making. Such data often are specialized to suit the needs of a given unit. However, governance structures and a wide commitment to collaboration promote the use of data and information across the University toward evaluating performance, informing sound planning and budgetary processes, formulating strategies for improvement, continued integration of technologies into the learning environment as well as support systems, and implementing positive change. Thus, joint efforts and shared communication contribute to the vitality of the organization in support of both mission and vision.
Underlying the processes is a long-standing commitment to continuous improvement that drives organizational reform, innovation, and advancement, each anchored within the context of vision, mission, purposes, values, institutional abilities, and strategic priorities. The four goals of the 2010-2015 Strategic Plan, provide a framework for specific goals among various constituents and offices. The following examples provide specific instances to illustrate the ways ongoing assessment process are used to inform strategies for continuous improvement.
Goal 1: Quality and Innovation
The first goal of the VCSU 2015 strategic plan emphasizes high-quality educational programs and experiences. While some programs are only beginning to gather learner outcome data and use it for classroom and program improvement, many instructional units are productively gathering data and using it to inform academic decisions. Three examples-from three different types of academic structures-provide examples of these uses of assessment.
The VCSU Teacher Education Program has provided institutional leadership in academic assessment and provided a strong model for other programs to emulate. Excerpts from the 2008 NCATE report identify the types and uses of data in the program:
- Candidates are evaluated at various transition points on dispositions and field experience successes. Final evaluations provide data for future growth, but formative data such as the student teaching progress evaluations after 3 weeks and 7 weeks are helpful for continuous improvement. Candidates assess their confidence level of the three domains, the INTASC standards, the University Abilities and the conceptual framework at three transition points during their undergraduate studies.
- Advanced candidates [graduate students] are assessed at key transition points on their perception and confidence level of the NBPTS and VCSU core values. The assessment is used by candidates to improve performance and help them realize how their learning connects with the University's model for teacher preparation as well as NBPTS expectations. The assessment is aggregated and used by the unit to identify the progression of candidates through the program.
- Candidates in both initial and advanced programs can monitor their progress toward program completion by accessing their own candidate summary on the unit's Central Assessment System.
This assessment process allows for program improvement and quality assurance for a program delivered to three different student groups (on VCSU campus, online, and on NDSU campus), as the report indicates:
- Assessment data collected on candidate performance in specified preparation programs on the main campus, at off-campus sites, and in distant learning programs suggests that program effectiveness for intended learner outcomes is essentially the same across all three areas of the initial teacher licensure programs. For example, elementary education majors on campus and at the off-campus site experience the same curriculum and same assessments.
- Cooperating teacher evaluations of student teachers, PPST, PRAXIS II tests indicate quality outcomes at NDSU (off-campus) as well as VCSU (main campus). The off-campus program is taught primarily by the same faculty members with the same curriculum. The disaggregated data confirms the University's commitment to providing an off-campus program with the same quality as the main campus. Should outcome data demonstrate a deficiency, the unit would need to make adjustments in the curriculum or delivery to ensure quality at both locations. Assessment data at key transition points in the program do not reveal any significant difference in quality performance at the present time.
- Unit faculty use data to make knowledge-based decisions about program strengths and weaknesses. At times aggregated data indicates successful outcomes and promotes staying on course, other times the data necessitates change. The unit not only makes decisions based on assessments, but the unit improves its assessments. "Development and Reflection" has been added to the elementary education portfolio to increase candidate reflection on their growth in the teacher preparation experience. Field experience forms have been made more consistent to indicate candidate growth through transition points in the program. The student teaching data in categories like responsibility and dependability produced almost identical scores on candidates, so the two attributes were combined and additional questions on diversity and the belief that all candidates can learn were added.
The report also describes the process used to initiate change, and lists specific changes informed by this data analysis:
- The Assessment Coordinator aggregates data every semester and shares the information with the unit, Teacher Education committee, and stakeholders. The Assessment Coordinator, Dean and various unit faculty members attend focused meetings every semester with area schools and are open to sharing data relevant to the discussions. When university faculty, cooperating teachers, students, or other members of the professional community express concern or desire for change, the unit analyzes its current data or creates an assessment tool to gather new research in order to make a knowledge based decision.
The following data-driven changes have occurred over the past three years:
- Discussion toward requiring the Global Awareness Ability
- Efforts to help candidates with Praxis II preparation
- Added one credit to social studies methods to improve candidate global awareness
- Faculty integration of technology and student evaluation feedback has allowed the unit to reduce one credit in the Educational Technology class
- Student teaching increased from 10 weeks to 12 weeks
- Dispositions changed to include fairness and exhibiting high expectations toward the learning of each student
- Added a development and reflection component to candidate's portfolio
- Classroom management component emphasized in more method courses
Business and Information Technology Program Revisions
Faculty in the Business Division have mapped all of their courses to the Department of Labor's Knowledge Skills and Abilities Profile for 62 occupations a graduate with a business administration major might fill.
This review has allowed them to test their curriculum against industry expectations, to make certain proper coverage is provided for each area. In addition, they have assessed the divisional abilities against the profile. While the University Abilities match the knowledge and skills required in the workplace, the faculty has determined that they may want to revise their emphasis on global awareness since this is not a primary area in the Department of Labor profile. Discussion of the applications of this analysis are ongoing, as the Division prepares updated materials for the 2012-14 Catalog.
Chemistry Classroom and Program Assessment
The Chemistry program utilizes a variety of assessments to ensure quality and innovation, but one tool has remained in place since 1983, providing longitudinal data which help faculty understand the changes in the students and the program over time. The nationally standardized examinations published by the American Chemical Society are employed to assess both the chemistry curriculum and student achievement. A summary of the results dating from 1983 is available in the Resource Room. These examinations are given as the final test (they count about 10% of the student's grade) at the completion of the following courses:
- Ten credit hour Chem 121 and 122 General Chemistry sequence
- Ten credit hour Chem 341 and 342 Organic Chemistry sequence
- Eight credit hour Chem 330 and 331 Quantitative Analysis sequence
- Five credit hour Chem 116 Introduction to Organic and Biochemistry course
In addition to meeting a number of pedagogical purposes, such as review and integration of material and practice in taking standardized exams, this very traditional assessment tool offers the following benefits:
- Students can compare their individual performance to a national standard. The perception that the quality of education at a small school may not prepare one adequately for graduate work is quickly cleared up, and students have a realistic appraisal of how they may perform in a larger arena. For example, the University of North Dakota uses these same ACS tests for qualifying examinations for their Ph.D. program. Students must score in the 60th percentile in their chosen major area of chemistry and 40th percentile in all other areas. Many VCSU students meet the standards.
- The benchmarks set by using such instruments of assessment over time also prove useful in a number of ways. Since the sample number of students is small, caution is advised when drawing conclusions from the data; however, if dramatic changes in the structure of a course are made, decreases or increases in the average percentiles can provide solid feedback about the impact of these changes. For example, three years ago a radically new textbook was adopted in the Chemistry 121/122 course. This text is 200 to 300 pages shorter than the standard text for this course. It was purposely designed to answer the current criticism in the field that textbooks keep growing in size and the concomitant demands upon student learning that grow with them. Although there was a slight dip in performance after the first year of using the text, the second year resulted in a record performance of achievement.
- The results of these exams provide a comparison of performance over time that allows course grade assignments to be made using a larger data base.
Goal 2: Enrollment and Academic Success
The second goal of the Strategic Plan emphasizes efforts to improved enrollment and student academic success. The use of various tools for program review and unit strategic planning are described in Chapter 6, as is the development of the Learning Center to serve student academic needs, and the development of a successful enrollment plan. Student Affairs makes use of a number of survey tools to improve program offerings, to make better decisions in planning enrollment strategies, and make the University more attractive to prospective and enrolled students alike. For example,
- The 2008 Noel/Levitz Student Satisfaction Inventory (SSI) showed 10 items that students rated as most important. Of those 10 items, two were related to academic advising services. Advisor is Knowledgeable and Advisor is Approachable were very highly rated. As a result, the Student Affairs Division made a special budget request to receive funds to sponsor a consultant to conduct advisor training for faculty at the start of the 2008 school year. In addition, the Director of Student Academic Services began the process of meeting individually with new faculty members to acquaint them with the campus advising process. As a result of these efforts, the 2010 administration of the Noel/Levitz SSI showed significant improvements in student levels of satisfaction with academic advising. Out of a total of 73 programs and services rated by students, two items related to academic advising appeared in the list of top ten highest rated items. Both My academic advisor is knowledgeable about requirements in my major and My advisor is approachable were very highly rated and were significantly higher than national averages and the average of the North Dakota University System.
- On the same 2008 Student Satisfaction Inventory, student safety and security were brought into question with lower scores than the 2006 administration of the same survey. As a result, an increased emphasis on campus safety and security took place on many levels. Members of the student body were invited to do a campus safety walk through with university officials. Student Senate formed a student campus security committee, numerous lighting upgrades were made, the topic of student safety was added to both student and parent orientation sessions and the number of security cameras on campus was doubled. As a result, the 2010 Noel/Levitz SSI showed significant improvements in scores related to student safety. Specifically, the largest of any increase of the 73 items was, "This campus is safe and secure for all students." On a scale of 1 to 7, the 2008 score of 5.68 increased to 6.22 in 2010. In addition, this same item was the third-highest rated item on campus.
- All new students are surveyed at the completion of opening school orientation activities. One of the activities assessed is the computer training sessions for all new students. Survey information showed that students are coming to VCSU with more and more computer skills and experience. The Information Technology staff was able to reduce the number of hours and number of trainers needed for computer training.
- By monitoring the rate by which new students sign-up for summer orientation dates, Enrollment Services determined that the earlier sessions (early June) were the first to fill. In 2010, VCSU opened two orientation/registration dates in April; these April dates fill almost immediately, and academic divisions are able to make fall planning decisions earlier because about half the freshmen are enrolled before Spring semester ends.
Goal 3: Fiscal Strategies
The third goal of the strategic plan focuses on strategies for reallocation or development of new financial resources. Since the implementation of the Growth Plan in 2007, every budget request submitted to the Cabinet is directed to include rationale connecting the request to the strategic plan and/or other planning and assessment tools. Departmental budget reviews also demonstrate the thoughtful use of assessment data to make informed decisions.
Information Technology 2008 Budget Review
In 2008, VCSU users discussed a possible reduction in the student technology fee. A comprehensive study of VCSU IT budget trends over the previous five years and a comparison of other institutions using the EDUCAUSE Core Data Survey were used to determine that VCSU should maintain the current level of technology fee. The data gathered was organized, analyzed, and published. The resulting report has been a continued reference and it has guided many additional decisions, including the following:
- The development of a strategy to leverage excess revenues that drastically reduced the cost of leasing laptops,
- The amount of IT executive emphasis to place on management and development of IT staff, as opposed to project development and management, and
- The amount of technology fee revenue versus state appropriations that should be used to fund IT staff salaries.
Goal 4: Facility Support of VCSU's Enrollment Initiatives, Program Development, and Student Needs
The fourth goal of the strategic plan recognizes the important role facilities play in attracting and retaining students, in addition to supporting programs and maintaining a safe environment.
The 2009-2010 Master Plan
In Fall 2009, VCSU began a new campus master planning process to analyze current physical campus situation and assess future needs. The planning process, conducted at VCSU by JLG Architects, was funded by the 2009 State Legislature to support VCSU's request for expansion of the Rhoades Science Center This review analyzed all phases of the campus physical infrastructure, including academic buildings, athletic facilities, student residence halls, Vangstad Auditorium, landscaping, energy usage, and classroom utilization.
The Master Plan process began with architects visiting all the departments and programs throughout campus. This gave architects an opportunity to interview employees about the current space and structures within which they operate, and explore opportunities for how these areas might be maximized to best serve the needs of current and future students.
Central concerns from the outset were the safety and capacity of the Science areas, renovation of Vangstad Hall, interest in developing capacity for growing athletic programs, a second residence hall renovation, and improved space utilization for all programs.
Recommendations and immediate actions from this planning process include
- Snoeyenbos Residence Hall renovation, completed August 2011
- Addition to Rhoades Science Center, which will provide safety improvements for laboratories, add capacity for biology-based science majors, and strengthen the image and accommodations for STEM-based programs - identified as an immediate need and recommended by the SBHE as its #2 priority to the Legislature in Spring 2011
- Predesign for Vangstad Hall renovation, presented to the SBHE as VCSU's #2 priority, and not included on their list forwarded to the Legislature for funding
- A classroom utilization study which assisted some reorganization of classroom space in 2010-11, to accommodate the enrollment growth in Fall 2010 and the simultaneous development of articulations requiring additional IVN classroom space
- A year-long discussion of student fees in Student Senate, resulting in Spring 2011 with the hire of an architect to prepare some predesign plans of the student center for review
- Ellig Field artificial turf installation, completed Summer 2011
VCSU is committed to the use of thoughtful, planned assessment processes to maintain institutional effectiveness, gathering data from internal and external sources to inform planning and budgetary decisions. Together, effective assessment and evaluation processes at all levels represent a critical cornerstone for continuous improvement and student learning.
Evaluation of Core Component 2c
Formerly managed by a few key individuals in various areas, assessment at VCSU has evolved from a collection of spreadsheets and databases into an integrated, effective, and seamless entity with faculty, students, and staff involved as contributors and consumers of informative data. The institution uses data from student, faculty, staff, and employer assessments and evaluations to manage and improve operations and programs.
Strengths: An academic assessment system is set up and in use by the School of Education; VCSU has a published assessment plan and has been gathering data from portfolios and student projects for a decade; every course syllabus includes an Ability project that may be collected and used for ability assessment. The recently completed Master Plan has identified specific facility upgrades and provided data and recommendations to guide planning.
New initiatives: Processes have been developed that encourage data-gathering and reflection, including Annual Program Updates for academic areas, and required rationales for all budget requests. The Academic Assessment Committee is in its first year, in the early stages of understanding its role in assuring academic quality and innovation.
Challenges: VCSU does not have an Institutional Researcher or an Assessment Office, making reliable and consistent assessment efforts challenging.