Higher Learning Commission

Academic Quality: Improvement through Assessment

Mayville State University

Overview of the Quality Initiative

The 2005‐2006 Higher Learning Commission (HLC) accreditation team expressed concern that Mayville State University’s assessment plan lacked sufficient data to judge its effectiveness. This concerned was well founded. The Institutional Improvement and Assessment Committee (IIAC) was better suited to concentrate on non‐academic matters as it lacked the resources to tackle both non‐academic and academic concerns. Recognizing this, the University President created the Director of Academic Assessment position to concentrate on quality improvement through planning, implementation, interpretation and use of results for activities broadly focused on student learning.

What followed was complete revamping of assessment practices to improve student learning for academic majors, stand‐alone minors (those not a subset of majors) and general education. This was designed as a six year process with the ultimate goal of improving educational quality for our students. The Director appointed one representative from each of five academic divisions to form the Academic Quality Improvement Group (AQIG) to lead the process for majors and minors assessment. Four other representatives were to work with general education. The Director of Instructional Technology became a valued member of both groups.

The process consisted of the following components:

  • Training on assessment
  • Training on software
  • Documenting assessment through course‐embedded activities during the year
  • Documenting non‐course embedded activities during the year
  • Developing an annual assessment report
  • Evaluating the reports and reporting the results

Mayville State sets aside two “Assessment Days” during each academic year. These days were originally set aside for student participation in institutional research activities. As the instruments used in these activities have increasingly been made available online, it has become less important to set aside a common time for these events. Recent Assessment Days have been under division control. Divisions typically use the morning of each Day assessing students in non‐course embedded activities and work with the administrative assessment activities during the afternoon. Preparing the annual report is frequently the primary administrative activity during the Fall Day; it is expected that the future Spring Assessment Day will involve sharing assessment activities and results via short presentations. The Assessment Director and the Director of Instructional Technology also schedule short workshops for Assessment Days. Outside experts are occasionally invited to conduct the workshops.

The North Dakota University System chose to become involved with the American Association of Colleges and Universities’ (AAC&U) Liberal Education – America’s Promise (LEAP) initiative to guide the state’s general education curricula. The NDUS created a General Education Council to spearhead this initiative. MSU willingly supports this work and has two representatives on the Council. General education assessment activities are conducted in accordance with the LEAP Essential Learning Outcomes (ELOs), and instructors are encouraged to use the initiative’s High Impact Practices in their development of assessment activities. At present the Council is considering a proposal to change the NDUS General Education Requirement Transfer Agreement (GERTA) policy to ground it in the ELOs rather than specific course categories. MSU currently requires that each academic major choose one upper division course to emphasize the ELOs. No transfer credit can substitute for this course. This helps ensure that each graduate has at least some exposure to the LEAP outcomes.

The primary focus of this proposal will be on years four and five (2013‐2015) of this six‐‐ year plan. The benchmark achievements in this period will be implementing assessments for LEAP ELO #1 (Knowledge of Human Cultures and the Physical and Natural World) and ELO #2 (Intellectual and Practical Skills for general education, the free standing minors assessments, and the second two‐year cycle for academic majors assessment).

Relevance and Significance

The proposed initiative is relevant and significant for the institution because it aligns with the institution’s mission and values, it is connected with the institution’s planning process, and it is timely for a variety of reasons.

  • Alignment with Mission and Vision

    Mission: The mission of Mayville State University is to educate and guide students, as individuals, so that they may realize their full career potential and enhance their lives. We do this in an environment that reflects our tradition of personal service, commitment to innovative technology‐enriched education and dynamic learning relationships with community, employers and society.

    As a member of the North Dakota University System, Mayville State University offers undergraduate degrees. Vision: To be known for continuing academic excellence in a cooperative, enjoyable learning environment that anticipates and responds to individual needs

    Vision: To be known for continuing academic excellence in a cooperative, enjoyable learning environment that anticipates and responds to individual needs.
    • Each major includes at least one outcome that is technology focused and another that emphasizes experiential learning. The experiential outcome in most cases is either an internship or a service learning project. Technology and experiential enriched education contribute to the enhancement of one’s life and the fulfillment of one’s career potential.
    • The LEAP outcomes to which MSU is committed include civic engagement and teamwork. One new general education course certified through the new assessment process is CIS 385, Multimedia Tools, which promotes technology and LEAP’s lifelong learning outcome by allowing students to explore various software tools of their choosing and learn one tool in depth. Independent exploration of software will be important to anyone’s 21st century practical college education.

  • Connection with the Institution’s Planning Process

    • Information is presented to the President’s Cabinet and the Strategic Planning Committee for consideration in institutional planning.
    • The institution recognized the importance of assessment and granted partial release time to its director and small stipends to each of its division coordinators.
    • The institution authorized the purchase of TaskStream software to help coordinate the assessment process.
    • The institution consistently sends 5‐6 representatives to the HLC annual conferences. It has also provided financial support for coordinators to attend various assessment workshops.
    • Academic plans navigate the governance structure of the curriculum committee, faculty senate and faculty association. All important assessment requests have been approved by these bodies, though certainly not without considerable discussion (and consternation on the part of some).

  • Evidence of significance and relevance at this time

    • The 2005‐2006 HLC accreditation team expressed a concern over assessment.
    • Quality academic programming is one means to recruit and retain quality students.
    • The expectation of increased emphasis on locally developed alumni surveys using new methods to improve response rates is one way to maintain better contact with alumni.
    • There has not been a thorough examination of the general education course offerings and their academic quality for many years.
    • General education assessment timing is meaningful because a new statewide transfer agreement based on LEAP is under consideration.

Intended Impact

The foremost purpose of any assessment initiative is to improve student learning. This work is a deliberate attempt to improve academic quality on our campus. All classes will contribute toward the learning outcomes for a particular major, minor, or general education program. There are specific components of the quality initiative which will accomplish these goals:

  • The action plans for course embedded activities
  • The consideration of annual report results by the President’s Cabinet and the Strategic Planning Committee for institutional planning
  • The sharing of learning during spring Assessment Days
  • The course certification process to keep general education offerings current
  • The encouragement of high impact practices to fulfill LEAP outcomes
  • The sharing of general education improvements with institutions across the state via the North Dakota General Education Council
  • The consistency of offerings for on‐site and distance coursework as well as with full‐time and adjunct faculty
  • The retention of students through improved academic quality, especially with online students (particularly when information is shared via common syllabi).
  • The ease of access for students to find best fit institutions if the state succeeds with its proposed new general education transfer process
  • The newly required syllabi statements that explain course improvements based on assessment results.

Purposes and Goals

The overarching purpose for any assessment initiative is to develop processes whose goal is to improve the academic quality of an institution. For MSU this means developing a plan to improve student learning in the academic major, stand‐alone academic minor, and general education programs. Improvement in academic quality is based primarily on annual report performance. The annual report is completed by each division and encompasses assessment efforts from all majors housed within that division. The report is a synthesis and summary from ongoing assessment work documented in TaskStream during the year. Each report presently includes three components: Important Findings, Course‐Embedded Activities, and Non‐Course‐Embedded Resources. Reports are evaluated by multiple AQIG members who are not faculty in the division whose report is under review. The process uses a rubric unique to one component of the report and is supported by a completed worksheet that, among other uses, defines the concepts referenced in the rubric.

A fourth component to the report will confirm the extent to which the prior year’s actions were implemented and their impact on academic quality. A new rubric and associated worksheet are presently under development. This component will be completed and employed beginning in year four (2013‐2014.)

Stand‐alone academic minors are those which are not subsets of corresponding majors. Faculty in these programs spent year three (2012‐2013) developing student learning outcomes (SLOs), activities, and rubrics for these minors. The programs are to be assessed for the first time in year four (2013‐2014) using the same processes as the academic majors. Minors generally have fewer SLOs than do majors.

The general education program uses LEAP Essential Learning Outcomes (ELOs) for assessment, compliant with the North Dakota General Education Council recommendation. The programs are to be assessed for the first time in year four (2013‐ 2014) using a process similar to that for the academic majors and stand‐alone minors. The MSU General Education Subcommittee (GES) implemented a certification process in 2012‐2013 to qualify courses for general education credit beginning in 2014‐2015.

Table 1 explains the MSU assessment process.

Table 1: MSU Assessment Process

Deliverable Content Purpose Responsible
Parties
Due
Date
Annual Report
Component #1
Important Findings
  • Synthesized findings from SLOs for all majors, minors, and specializations housed within the division
  • Synthesized findings from each general education category for assessment during a particular year
  • Resources used in determining findings
  • Action plans to address the findings
To summarize findings for
division and general education
programs and use them for
improving academic quality
  • Divisions
  • GES
October of
succeeding
year
Annual Report
Component #2
Course‐embedded
Activities
  • Percentage of assigned course activities completed
  • Percentage of completed course activities for which action plans are developed
  • Extent to which distance courses are considered
To summarize and monitor
division and general education
program assessment efforts at
the course level
  • Divisions
  • GES
October of
succeeding
year
Annual Report
Component #3
Non‐Courseembedded
Resources
Specific, acceptable noncourse resources used in generating important findings To encourage divisions and the GES to use resources not exclusively generated from course activities
  • Divisions
  • GES
October of succeeding year
Annual Report Component #4 Action Confirmation
  • Confirmation on the extent to which planned actions from prior annual reports have been implemented
  • Extent to which results from actions have informed the assessment process
To ensure that planned actions to improve student learning are implemented and that the results are considered in efforts to improve student learning
  • Divisions
  • GES
October of succeeding year
Annual Report
Presentation
Description of most innovative findings and actions documented in the annual report To share innovations and learn from others in an effort to promote a culture of assessment
  • Divisions
  • GES
March of
succeeding
year
General Education
Certification
Request
Explanation of how each course offered for general education credit meets all four LEAP ELOs
  • To provide a focus for general education to promote it as a program
  • To encourage an updating of general education course offerings
  • Divisions
  • Individual faculty
April of
succeeding
year

 

Evaluation of Progress and Accomplishments

Table 2 explains the MSU assessment results evaluation process. Every deliverable offers the opportunity to suggest process improvements and to contextualize the content.

Table 2: MSU Assessment Results Evaluation

Initiative
Deliverable
Evaluation
Deliverable
Evaluation
Completed by:
Evaluation
Reported to:
Progress
Definition
Annual Report Scored rubrics for each of
four components supported
by a completed worksheet
for each
  • AQIG members unaffiliated with division report being scored
  • General Education report by any AQIG member
  • Divisions
  • GES
  • President’s Cabinet
  • Strategic Planning Committee

Improving rubric
scores for each
division and
general
education each
year

Annual Report
Presentation
Faculty participants survey to
include identifying the most
intriguing findings or action
plans and their adaptability
to participants’ own division
or the GES
The AQIG
  • Divisions
  • GES
  • President’s Cabinet
  • Strategic Planning Committee

Survey indication
of event
satisfaction

General Education
Certification
Request
Scored rubric for each
request
GES members not affiliated
with division in which course
is housed
  • Divisions
  • Individual faculty member
  • GES
  • Faculty Association
  • 100% of courses applying are successful
  • New courses become certified
Summary of North
Dakota State
General Education
Council
Achievements
Assessment Director submits
Council actionable items to
Faculty Association for vote.
Faculty Association minutes
reflect approval of Council
policies requiring campus
action
  • Divisions
  • GES
  • Faculty Association

NDUS policy changes to define general education transfer agreements by LEAP ELOs

Revamped Syllabi
Format Usage
Report
Percentage of courses using
syllabus format that includes
statement on course
improvements resulting from
assessment findings
AQIG
  • Divisions
  • GES
  • Faculty Association

90% of syllabi
use new format
by 2014‐2015

 

Internal and External Support

The initiative enjoys a high level of support from its primary stakeholders: senior administration, faculty and students.

The President’s Cabinet membership includes all three institutional vice presidents. The President is the former Vice President for Academic Affairs who led assessment efforts before creating the Director position. His concern has always been with “closing the loop.” The Vice President for Academic Affairs meets with the Director for biweekly updates and to offer advice or influence (the Director has no line authority) on assessment matters. The Vice President for Business Affairs funds several representatives to attend HLC annual meetings and assessment workshops. He also approved financial support for TaskStream software. The Director occasionally meets with the Vice President for Student Affairs and Institutional Research as part of the Institutional Improvement and Assessment Committee (IIAC), but this latter group has been concentrating more on non‐academic matters.

Faculty have supported this initiative in their passage of all assessment‐related proposals through the governance structure during the first three years of the project.

Students shape the process by exercising votes on curricular matters and by participating in focus groups to influence general education and technology‐related academic programming decisions.

The State General Education Council supports this project as an important piece of their plan to base transfer credit on LEAP essential learning outcomes rather the course categories.

Groups and Individuals Involved in Implementation

The initiative leaders are the Director of Academic Assessment, The Director of Instructional Technology, The Academic Quality Improvement Group (AQIG) and the General Education Subcommittee (GES) of the Curriculum Committee.

The Director of Academic Assessment coordinates the entire process and chairs the AQIG and the GES. As the AQIG leader, he supervises divisions’ creation of supporting activities and performance measuring tools for all student learning outcomes. He monitors the curriculum maps divisions create and troubleshoots unanticipated assessment issues that divisions are unable or unwilling to confront. He is also charged with developing the annual reporting templates and the measurement tools and processes to evaluate reports completed with the templates. Furthermore, he reports these results to the Presidents’ Cabinet and the Strategic Planning Committee for their use in institutional planning. Similarly, The Director prepares surveys and conducts focus groups on matters related to student learning and interprets the results of these instruments and activities. He conducts workshops during campus Assessment Days and updates faculty on assessment progress during semester pre‐service events. Furthermore, the Director provides the theoretical framework for assessment software design and prepares the detailed templates for assessing non‐course embedded activities. As the GES Chair, the Director develops templates for course certification and processes by which these completed templates are used. He participates as a member of the State General Education Council and actively supports their work to convert campus requirements from course categories to LEAP outcomes. He also monitors general education processes much in the same way as division work in the AQIG.

The Director of Instructional Technology serves as second‐in‐command for both the AQIG and the GES. Her primary responsibility is developing, maintaining, and training faculty on using TaskStream assessment software. To this end, she consults with the corporate TaskStream liaison to create an application that best supports MSU and updates it when new components of the assessment process become active. Individual faculty are primarily responsible for conducting their own assessments, and she therefore spends countless hours in both planned and unplanned training sessions. She is presently developing interactive materials to teach the software to distance faculty. This Director has a solid instructional design background, thus most Assessment Director responsibilities are carried out with her invaluable assistance.

One representative from each of the five academic divisions (Business and Computer Information Systems; Education and Psychology; Health, Physical Education and Recreation; Liberal Arts; Science and Mathematics) comprise the AQIG. These representatives facilitate work within the divisions at the granular level that mirrors the work of the Assessment Director. They also provide input to the Director in developing documents and processes and help with the annual report evaluations. The AQIG sets aside biweekly time slots to discuss assessment matters.

The GES is appointed by the Curriculum Committee. It consists of four faculty representatives, one of whom must be from Liberal Arts. The GES has no formal power to change academic policies or procedures but is a positive catalyst for recommending such actions to the Curriculum Committee. It meets as needed.

The Curriculum Committee is the enforcement arm for proposals recommended by the AQIG or the GES. It studies and recommends revisions to curricular structure, general education requirements, requirements of the majors and minors in the various disciplines, program additions, program revisions and deletions, grading, academic calendar, graduation requirements, and other curricular and instructional issues. The voting membership on the Curriculum Committee shall consist of the Vice President for Academic Affairs and the chair of each of the divisions. The Student Body is represented by two students, nominated and elected by the Student Senate. The two students share one vote. The Registrar, Director of Extended Learning, GES Chair and the Director of the Library serve as non‐voting members. The Curriculum Committee meets weekly.

Committed Resources

Human resources committed are the five division representatives to the AQIG, the four to the GES, and two directors. Forty percent of the total university commitments by the Directors of Academic Assessment and Instructional Technology are allocated to this project. Each division representative is a full‐time faculty member who receives an additional $2250 annually for their service to the initiative. Both groups are expected to continue their work after the project ceases. The GES existed in adherence to university policy prior to the project but has seen its work greatly expand.

The university committed $30,000 for a three‐year contract for TaskStream software beginning in 2011. The software subscription is expected to be renewed indefinitely.

Like any new initiative, resources will be monitored to ensure fair and appropriate workload. The time allocations to the aforementioned directors may be reduced to 25% during the final three project years. The other AQIG members receive small stipends for their commitment; those serving in the general education subcommittee do so as part of their normal committee assignments. Commitments by those other than the directors are for one year at a time. There is hope that any replacements will provide energy and refreshment to the project and the exiting members constitute a critical mass that supports the proposal indirectly.

Faculty seem generally supportive of this initiative. There is a concern that assessment work will fall to the wayside as additional responsibilities are assigned to faculty without a corresponding reduction in teaching load. However, the President and Academic Vice President have been and are expected to remain staunchly supportive of the work in this project and have stated their priority to ensure it is sustained and integrated into the ongoing work of the institution.

Primary Activities and Implementation Timeline


2010-2011

Action Responsible Party Notes
Choose AQIG members Director One representative from each of 5 divisions comprise AQIG
Survey Faculty Director, with AQIG assistance Identify most important concepts students should learn in majors, general education, and technology
Construct SLOs Divisions, with AQIG assistance Develop for all majors housed in division
Identify SLO acceptable evidence,
student targets, and measurement
tools (rubrics); build 2‐year
curriculum maps
Divisions, with AQIG assistance Develop for all majors housed in division
Construct Moodle shells Director, Director of
Instructional Technology
Develop for each division, develop separate
resource shell for all divisions
Conduct multiple brown bag sessions Director Answer basic assessment questions
Select assessment software Director, AQIG subcommittee TaskStream chosen
Attend HLC‐sponsored assessment
workshop
AQIG subcommittee Design assessment process over the six‐year
project
Attend HLC annual meeting AQIG Assessment sessions are primary focus
Attend fall and spring North Dakota General Education Council Summits Director, GES representatives Consideration of LEAP ELOs begins



2011-2012

Action Responsible Party Notes
Train faculty on TaskStream Director of Instructional
Technology with AQIG
assistance
All on‐site faculty participate in two‐hour sessions
in groups of 4‐8 over three week period
Conduct assessments in majors Divisions, with AQIG assistance Course‐embedded activities and non‐courseembedded
resources employed
Develop annual report framework
and rubrics with supporting
worksheets to evaluate division
assessment performance
Director, with AQIG assistance Report and evaluation used next year to assess
2011‐2012 performance
Assessment workshop hosted by
Director of Assessment from
University of North Dakota
Director Fall 2011 Assessment Day activity
Attend HLC annual meeting AQIG Assessment sessions are primary focus
Attend fall and spring North Dakota
General Education Council Summits
Director, GES representatives Consideration of LEAP ELOs continues



2012-2013

Action Responsible Party Notes
TaskStream training continues Director of Instructional
Technology, with AQIG
assistance
Participants generally are new faculty or those
needing refreshment
General Education workshop hosted
by Director of Assessment from
University of North Dakota
Director Fall 2012 pre‐service activity
Assessment in majors continues Divisions, with AQIG assistance Course‐embedded activities and non‐courseembedded
resources employed
Annual reports for 2011‐2012
completed
Director, with AQIG assistance Consists of Important Findings, Course‐embedded
activities and Non‐course embedded resources
components
Annual reports for 2011‐2012
evaluated
AQIG Overall and individual component scores
provided; reported to divisions, Presidents’
Cabinet
General Education course
certification process begins
Director and GES Courses offered as general education complete
request validated to support all four LEAP ELOs
Plans for assessing stand‐alone
minors begin
Divisions and AQIG Process mirrors that of planning for majors’
assessment
Development of interactive, online
assessment training materials
completed
Director of Instructional
Technology
Primarily for distance faculty; to be implemented
in 2013‐2014
Attend HLC annual meeting AQIG Assessment sessions are primary focus
Attend fall and spring North Dakota
General Education Council Summits
Director, GES representatives Council approves “LEAP State” status



2013-2014

Action Responsible Party Notes
TaskStream training continues Director of Instructional
Technology, with AQIG
assistance
Participants generally are new faculty or those
needing refreshment
Interactive, online assessment
training begins
Director of Instructional
Technology
Includes TaskStream training; primarily for
distance faculty
Assessment in majors continues Divisions, with AQIG assistance Course‐embedded activities and non‐course embedded
resources employed
Assessment in stand‐alone minors begins Divisions, with AQIG assistance Process mirrors that of majors
General Education assessment begins General Education Faculty with
GES assistance
LEAP ELO #1 assessed in all general education
courses
Annual reports for 2012‐2013 completed Director, with AQIG assistance Consists of Important Findings, Course‐embedded activities, Non‐course embedded resources, and Action Confirmation components
Annual reports for 2012‐2013 evaluated AQIG Overall and individual component scores
provided; reported to divisions, Presidents’
Cabinet and Strategic Planning Committee
General Education course certification process continues Director and GES Courses offered as general education complete a
request validated to support all four LEAP ELOs,
mostly new or preciously denied courses
considered
Attend HLC annual meeting AQIG Assessment sessions are primary focus
Attend fall and spring North Dakota General Education Council Summits Director, GES representatives Council works to base transfer credit on LEAP
ELOs



2014-2015

Action Responsible Party Notes
TaskStream training continues Director of Instructional
Technology, with AQIG
assistance
Participants are generally new faculty or those
needing refreshment
Interactive, online assessment training continues Director of Instructional
Technology
Includes TaskStream training; primarily for
distance faculty
Assessment in majors continues Divisions, with AQIG assistance Course‐embedded activities and non‐courseembedded resources employed
Assessment in stand‐alone minors continues Divisions, with AQIG assistance Process mirrors that of majors
General Education assessment
continues
GES assistance LEAP ELO #3 assessed in all general education
courses
Annual reports for 2013‐2014
completed
Director, with AQIG assistance; GES Consists of Important Findings, Course‐embedded
activities, Non‐course embedded resources, and
Action Confirmation components; stand‐alone
minors and general education now included
Annual reports for 2013‐2014
evaluated
AQIG Overall and individual component scores
provided; reported to divisions, GES, Presidents’
Cabinet and Strategic Planning Committee
General Education course
certification process continues
Director and GES Courses offered as general education complete
request validated to support all four LEAP ELOs;
mostly for new or previously denied courses
Attend HLC annual meeting AQIG Accreditation sessions are primary focus
Attend fall and spring North Dakota General Education Council Summits Director, GES representatives Agenda unknown at present



2015-2016

Action Responsible Party Notes
TaskStream training continues Director of Instructional
Technology, with AQIG
assistance
Participants are generally new faculty or those
needing refreshment
Interactive, online assessment
training continues
Director of Instructional
Technology
Includes TaskStream training; primarily for
distance faculty
Assessment in majors continues Divisions, with AQIG assistance Course‐embedded activities and non‐course embedded resources employed
Assessment in stand‐alone minors
continues
Divisions, with AQIG assistance Process mirrors that of majors
General Education assessment
continues
General Education Faculty with GES assistance LEAP ELO #4 assessed in all general education
courses
Annual reports for 2014‐2015
completed
Director, with AQIG assistance;
GES
Consists of Important Findings, Course‐embedded
activities, Non‐course embedded resources, and
Action Confirmation components; stand‐alone
minors and general education now included
Annual reports for 2014‐2015
evaluated
AQIG Overall and individual component scores
provided; reported to divisions, GES, Presidents’
Cabinet and Strategic Planning Committee
General Education course
certification process continues
Director and GES Courses offered as general education complete request validated to support all four LEAP ELOs; mostly for new or preciously denied courses
Attend HLC annual meeting AQIG Accreditation sessions are primary focus
Attend fall and spring North Dakota General Education Council Summits Director, GES representatives Agenda unknown at present
Assessment process reflection All parties mentioned Plans for significant restructure possible
HLC accreditation site visit All stakeholders Consider HLC suggestions for improvement

 

Institution Contact

Mark Skean, Director of Academic Assessment

Browse Related Proposals

(by Keyword)

Assessment

Quality Improvement

2014 Quality Initiative Proposals

Copyright © 2017 - Higher Learning Commission

NOTE: The papers included in this collection offer the viewpoints of their authors. HLC highly recommends them for study and for the advice they contain, but none represent official HLC directions, rules or policies.


Higher Learning Commission • 230 South LaSalle Street, Suite 7-500 • Chicago, IL 60604 • info@hlcommission.org • 800.621.7440

Home | About HLC | Contact Us | Privacy Policy

YouTube graybkgrdLinkedIn graybkgdTwitter graybkgd