Higher Learning Commission

Continuous Academic Program Improvement through Comprehensive and Systematic Analysis of Graduates’ Initial Success

Iowa State University

Overview of the Quality Initiative

Iowa State University’s proposed Quality Initiative seeks to build upon prior and current continuous academic program improvement (CAPI) efforts, as well as ongoing efforts to enhance student success, by developing and implementing institutional processes to systematically collect and analyze information about initial success of graduates. The initial phase of the initiative will be the comprehensive collection and use of post-graduation first destination information for at least 90% of all graduates. “First destination” for graduates represents what students do immediately following (within six months of) graduation. Examples of first destinations include enrollment at an institution of higher education for a professional or advanced degree; employment within a corporation, government entity, or non-profit organization; or self-employment as an entrepreneur. The first destination and initial student success information will be systematically utilized to inform curricular and cocurricular planning and improvement at the program and institutional levels. We believe that for programs to better prepare graduates for successful careers and lives, the programs must first have a better understanding of what their graduates do immediately following graduation. A second phase of the initiative will systematically collect additional information, to augment information already available, regarding graduates’ perceptions of their initial success and their preparedness to be successful by the educational experience at ISU. To the extent possible, information that independently documents graduate initial success will also be analyzed (success in initial employment over the first several years or success in graduate/professional education for example).

Iowa State University is leveraging the Quality Initiative to advance the institution’s program-level student outcomes assessment efforts, as well as efforts to enhance the holistic (academic, co-curricular, personal development) experience of ISU students, while also addressing an issue of institutional inefficiency raised by the University Career Services Council. Currently the collection of post-graduation first destination information and graduate initial success information at Iowa State is decentralized and in some cases redundant. Post-graduation student information (such as employer, graduate school attending, employment in field related to degree program, job titles/responsibilities, whether ISU degree program prepared students well for a desired first destination, etc.) is collected using different processes and data elements by eight college-based career services offices, academic colleges, numerous academic departments, faculty members associated with grant funded projects, the ISU Alumni Association and the ISU Foundation. This decentralization and collection by various stakeholders creates substantial duplication of effort, inconsistencies within information collected/reported/used, and lower response rates due to student survey fatigue and confusion. In addition to variation in the collection of information, the use of post-graduation first destination and initial success information to inform program quality enhancement and student outcomes improvement varies greatly across the institution. In some degree programs the information is regularly provided to faculty members who analyze the data as part of systematic CAPI processes. However, in many departments this potentially valuable student outcome information is not shared or used systematically to inform program quality improvements. This type of student outcome information has also not been systematically used to improve the co-curricular experiences provided by the Division of Student Affairs and other units across campus.

Iowa State’s Quality Initiative will: 1) gain stakeholder consensus on a common set of post-graduation first destination and initial success data elements to collect institutionally, 2) develop and implement a common institutional process for collection and analysis of post-graduation first destination and initial success information, 3) develop appropriate reporting and feedback loops to insure that colleges, departments, units, and faculty members have access to the needed post-graduation first-destination and initial success information to inform systematic CAPI processes, and 4) develop an institutional plan for aggregating post-graduation first destination and initial success information and the use of that information across the institution to improve program quality and student outcomes. In addition, it is expected that the aggregated and program level initial success information will provide useful augmentation to ongoing accountability documentation for a variety of stakeholders.

Due to the time-lag of collecting information (six months post-graduation), when the Quality Initiative Final Report is submitted in April 2015, the Quality Initiative will still be gathering input on the success of the new processes, impact on the number of departments using post-graduation first destination and initial success information to revise/inform curriculum, etc. However, preliminary data from the first semester of implementation, as well as analysis of less comprehensive previously collected initial success data and formative assessment data about the Quality Initiative process will be available for the final report. The proposed timeline includes the collection of assessment data and refinement of the Quality Initiative process after the submission of the final report to the Higher Learning Commission. It is expected that ongoing further refinements to collection and analysis of data pertaining to graduate success will continue to be used to inform systematic CAPI processes.

Scope and Significance

Iowa State University’s mission statement emphasizes that “The University must maintain a strong focus on student success and provide exceptional undergraduate, graduate, professional, and outreach programs that prepare students and citizens for leadership and success.” For students to achieve leadership and success after graduation, it is important that the university knows what students do after they graduate and that this information is taken into consideration in the development of curricular and co-curricular activities. The Quality Initiative will provide information that will allow faculty across all degree programs to develop curriculum that will better prepare students for leadership and success after graduation. The Quality Initiative will create greater understanding of what graduates of Iowa State’s degree programs do after graduation and how their postgraduation destinations and initial success are related to their holistic educational and personal development experience at ISU. This information can serve as important data in program-level student outcomes assessment, curriculum development, and systematic CAPI processes. If, for example, the Art and Design department learned that a significant percentage of its graduates attempted to start their own design studios after graduation, the department might consider adding coursework, learning outcomes, or an optional emphasis within the curricula, related to basic business administration skills. Or, if the Construction Engineering program discovered that the first destination for most of their graduates was in positions with the job title of ‘Assistant Project Manager’, the program may want to increase opportunities within the curriculum for students to gain expertise in the use of common commercial project management software packages. Or, if a program realizes that nearly all of their Ph.D. graduates pursue one of two career paths (faculty positions or employment in a national laboratory), the program may implement differential professional development opportunities for its students based on these two primary career paths.

Iowa State’s Quality Initiative also directly supports Goal 6 and Goal 8 of the 2010-2016 Strategic Plan for the Board of Regents, State of Iowa.

  • Strategic plan Goal 6 is that Iowa’s public universities and special schools will demonstrate that their student outcomes assessment programs help students achieve identified learning goals. To meet this goal, the board has set a target that Iowa State will develop student outcomes assessment plans for each academic program offered, and will establish targets for collecting and using assessment results. Currently, all of our degree programs have established some form of student learning outcomes. The Quality Initiative will provide colleges and programs with better data on how their graduates have leveraged their Iowa State experience, which can be used to encourage faculty reflection and discussion on refined student learning outcomes and program assessment.
  • Strategic plan Goal 8 is that Iowa public universities and special schools shall be increasingly efficient and productive. To achieve this, Iowa State is encouraged to develop efficiencies and cost-savings from collaborative initiatives within and between public institutions. The Quality Initiative effort to streamline the collection of post-graduation first destination information and reduce duplication of efforts within the institution is directly aligned with this goal.

The Quality Initiative also closely aligns with the mission of a new presidential initiative to insure that Iowa State University maintains a high quality undergraduate experience (both inside the classroom and out) during a time of significant enrollment growth. In November, 2012, President Leath formed a new Student Experience Enhancement Council (SEEC). The mission of SEEC is to find ways to maintain and enhance the experience for all Iowa State students. The Quality Initiative, through the systematic and comprehensive collection of information on graduate initial success and first destinations, will provide valuable information for consideration by SEEC and the president as they prioritize institutional efforts to meet student needs during this period of institutional growth.

The Quality Initiative will include the engagement of faculty and staff from across all academic colleges, several administrative units, the Division of Student Affairs, and representatives from student government. Since a goal of the project is to create a comprehensive, institutional system, the project has the potential to have long-term programmatic impact for all current and future students (undergraduate, graduate, and professional). The Quality Initiative will focus on the first destination and initial success of graduates, within six months of graduation. There would be a potential in future years to expand the initiative to look at longer-term destinations and career paths of graduates. However, this proposal will focus only on those first destinations and initial success.

Purpose

The overarching purpose of the Quality Initiative is to provide a substantial new element as part of broader efforts to significantly enhance the quality of the student experience at Iowa State with the goal of better preparing graduates for successful careers and lives. Our approach is a strong collaborative effort involving academic affairs and student affairs to address all elements of the student experience – curricular, co-curricular, and personal development. We seek to better understand how these interrelated aspects of the student experience contribute to perceived and actual success of graduates.

Specific goals of the Quality Initiative are to:

  • Encourage degree programs to refine the definition and measurement of student learning outcomes with a focus on systematic continuous academic program improvement
  • Provide a focus on student success and value of the educational experience that students, parents, state legislators, and the general public can understand
  • Leverage and improve data collected to meet existing Board of Regent and Federal reporting needs
  • Graduate students who are better prepared for successful careers and lives

Deliverables from the Quality Initiative efforts will include:

  • A common set of post-graduation first destination and initial success data elements to collect
    institutionally
  • A common institutional process and tools for the collection of post-graduation first destination and initial success information
  • Reports and a feedback system to share with colleges, departments, units, and faculty members postgraduation first-destination and initial success information for the purposes of student outcomes assessment and systematic program improvement
  • An institutional plan for aggregating post-graduation first destination and initial success information and the use of that information to improve program quality and student success

Assessment of the Quality Initiative Process

A full-time academic data analyst position in the Office of the Senior Vice President and Provost will provide expertise and leadership for the development and implementation of the assessment for the Quality Initiative as part of his/her broader responsibilities for institutional CAPI processes. This position will monitor the progress of Quality Initiative in relation to the proposal timeline. Formative assessment will include monthly reflections and agenda discussions of the Quality Initiative Committee on progress, issues, potential modifications to initiative scope/timeline, etc. Stakeholders (all academic departments, units with vested interest and use of the postgraduation and initial student success data, etc.) will also provide formative assessment via two surveys (completed in March 2015 and March 2016).

Assessment of Quality Initiative Impact

During the collection of current practices (July – October 2013), departments will provide benchmark data on current level of use of post-graduation first destination and initial success information in program improvement and student outcomes success measures. This benchmark data will be compared to feedback received in stakeholder surveys conducted in Spring 2015 and Spring 2016. Examples of metrics to be measured include:

  • Student response rate to surveys/data collection
  • Student ‘success’ based on placement rates
  • Faculty/staff time spent collecting post-graduation first destination and initial success data
  • Number of departments who are using post-graduation first destination and initial success information in student outcome success improvement plans and CAPI processes
  • Impact of the use of first destination and initial success information on student outcomes
  • Exemplars of changes made to curricula/programs based on post-graduation first destination and initial success information

Evidence of Commitment to and Capacity for Accomplishing the Initiative

The Office of the Senior Vice President and Provost will take the leadership role for the Quality Initiative. The President and Senior Vice President and Provost have committed to support the following:

  • Information Technology (IT) technical support staff for the survey, database, and systems interface work
  • Staff with assessment expertise to develop survey/data collection development and assessment of the Quality Initiative
  • Prioritization of the Quality Initiative within the IT workflow queue to allow the project development to meet the Quality Initiative timeline
  • The Office of the Senior Vice President and Provost will provide half-time release for a faculty member to serve a two-year term as a Faculty Fellow, to co-chair the Quality Initiative Committee and provide additional leadership with HLC Open Pathway accreditation efforts

Dave Holger, Associate Provost for Academic Programs and Dean of the Graduate College, will serve as the senior leadership contact for the Quality Initiative Committee.

Karen Zunkel, Office of the Senior Vice President and Provost, along with a half-time Faculty Fellow (to be named) will co-chair the Quality Initiative Committee.

The Quality Initiative Committee, appointed by the Senior Vice President and Provost, includes membership from the following groups:

  • University Career Services Council
  • Department Chairs Cabinet
  • Undergraduate Programs Council
  • Associate Deans for Graduate Programs
  • Faculty Senate Student Outcomes Assessment Committee
  • Division of Student Affairs
  • Student representation from the Government of the Student Body, and Graduate and Professional Student Senate
  • Faculty member at large
  • Office of the Senior Vice President and Provost

Since the Quality Initiative enhances Iowa State’s ability to meet Board of Regent expectations related to program-level student outcomes assessment and Federal Compliance reporting requirements for post-graduation status, after the committee has completed its work on the Quality Initiative, the institution plans to merge the ongoing support for these efforts into existing structures. The Office of the Senior Vice President and Provost, with the support of the Faculty Senate Outcomes Assessment Committee, has the responsibility for oversight of program level student outcomes assessment. The college-based career services directors have responsibility for collection of post-graduation information for federal compliance reporting. Therefore, these groups will assume responsibility for insuring the sustainability and ongoing continuous improvement efforts for the Quality Initiative after the initial development and implementation.

Timeline for the Quality Initiative

June - July 2013: Development of process assessment instruments by academic data analyst

July - October 2013: Quality Initiative Committee will develop and administer survey to all colleges, departments, units to collect information on existing post-graduation information gathered (questions asked) and processes used. Collect benchmark data on number of departments using post-graduation first destination and initial success information to inform curricula improvement, student response rates to existing collection systems, staff time utilized to collect current data, etc.

October - December 2013: Quality Initiative Committee will analyze survey results and negotiate institutionally agreed upon data elements to be gathered from graduates for a single, comprehensive university process that will inform program level student outcomes assessment, identify curricular improvement opportunities, and meet the needs of various stakeholders within the university community. Committee representatives will gather input from the constituencies they represent throughout this process. If needed, the committee will also explore the feasibility of incorporating flexibility within the system to allow for additional questions/customization to meet unique stakeholder needs.

December 2013 - March 2014: Develop a common methodology, process, and technology tools for collection and reporting of post-graduation, first destination and initial success information, including

  • Development of institutional survey/collection protocol for gathering data elements related to student first
    destination and initial success
  • Integration of information collection and reporting with existing systems/processes (such as ISUCMS – on-line career management service, the university’s student information system, graduation processing,federal student post-graduation compliance reporting, etc.)
  • Development of appropriate survey and data management tools to process the data
  • Development of customized reporting options for various stakeholders

April – November 2014: First semester of implementation of new process/system with students graduating spring semester 2014. (Collection of information will occur after each fall/spring semester and summer term.)

December 2014: Initial reports/information distributed to colleges and departments for Spring 2014 graduates. This initial one-semester report will be created for purposes of the Quality Initiative final report and for formative assessment analysis of the Quality Initiative. The typical annual reporting cycle (to include information for Fall, Spring, and Summer) will be collected and reported to colleges and departments in Spring Semester each year. The annual report will be distributed in late March or early April of each year, to allow for the collection and analysis of six-month post-graduation information from a summer term ending in August.)

March 2015: First stakeholder survey. Formative assessment from stakeholders to inform processes, information gathered, potential impact for outcomes/program, staff time, etc.

April 2015: Quality Initiative analysis of impact: Comparison to benchmark data, identification of changes needed for data elements collected, process improvement, etc.

April 2015: Annual report distributed to colleges and departments, with previous year information (Fall 2013, Spring 2014, and Summer 2014).

May 1, 2015: Quality Initiative Final Report due to Higher Learning Commission

Summer 2015: Analysis and implementation of changes based on stakeholder feedback and assessment data. Use of post-graduation first destination information compiled as a part of the Board of Regents annual student outcomes assessment report.

March 2016: Second stakeholder survey and comparison to benchmark data. Annual report distributed to colleges and departments. Initiate discussion on second phase of initiative, to add

Summer 2016: Analysis and implementation of changes based on stakeholder feedback and assessment data. Use of post-graduation first destination information compiled as a part of the Board of Regents annual student outcomes assessment report.

Fall 2016: Begin discussions on the feasibility, scope, and timeline of a potential second phase of the Quality Initiative, to include the use of graduate information data beyond first-destination and initial student success information.

Spring 2017 and beyond: Each spring semester results of previous year data collection/analysis will be sent to colleges and departments for us in their continuous improvement plans. College career services directors will monitor and refine process, reporting structures, review progress against benchmarks, etc. as a part of their continuous process improvement.

Summer 2017 and beyond: The Office of the Senior Vice President and Provost will collect/monitor use of postgraduation data, and review progress against benchmarks as a part of the annual student outcomes assessment report to the Board of Regents. This office will also continue to monitor/refine the process, systems, and tools used based on feedback from stakeholders.

 

 

Institution Contact

David Holger, Associate Provost for Academic Programs and Dean of the Graduate College

Karen Zunkel, Director of Undergraduate Education and Academic Quality

Copyright © 2017 - Higher Learning Commission

NOTE: The papers included in this collection offer the viewpoints of their authors. HLC highly recommends them for study and for the advice they contain, but none represent official HLC directions, rules or policies.


Higher Learning Commission • 230 South LaSalle Street, Suite 7-500 • Chicago, IL 60604 • info@hlcommission.org • 800.621.7440

Home | About HLC | Contact Us | Privacy Policy

YouTube graybkgrdLinkedIn graybkgdTwitter graybkgd