Higher Learning Commission

Facilitating Continuous Improvement: The Implementation of UALR’s Decision Support System

University of Arkansas at Little Rock

Overview of the Quality Initiative

1. Provide a brief description of the Quality Initiative. Explain whether the initiative will begin and be completed during the Quality Initiative period or if it is part of work already in progress or will achieve a key milestone in the work of a longer initiative.

 The University of Arkansas at Little Rock (UALR) launched a Decision Support System (DSS) implementation project in fall 2013 to provide support and tools that facilitate a data-informed culture of continuous improvement and demonstrate the university’s academic, economic, and societal impact to the city, state, and region. This effort serves as the Quality Initiative Project (QIP) for UALR. The project has several information system components at various stages of completion that will progress over the QIP period.

UALR planned the QIP over three phases: 1) Metric and System Research, 2) Interim DSS Implementation and 3) Ideal DSS Implementation. QIP will progress in an iterative fashion, ensuring the university continuously responds to the changing landscape of higher education throughout the development of the DSS. The project is currently in phase two.

The Ideal DSS consists of the following primary components identified in Figure 1. These components will be described in section three of the proposal.

Figure 1

Ideal Decision Support System Structure - The ideal DSS will consist of: 1) Business Intelligence Tool, 2) Predictive Analytics 3) Degree Audit 4) Assessment Tools 5) Early Alert System, and 6) Data Warehouse.

UALR Fig1

 
Once implemented, the DSS will provide the needed framework and tools for a comprehensive integrated planning process based on data that is current, complete, and correct. The launch of the Ideal DSS is scheduled for academic year 2019-20.

Sufficiency of the Initiative's Scope and Significance

2. Explain why the proposed initiative is relevant and significant for the institution.

UALR is resolved to be a more effective institution that uses data to identify problems, propose solutions, and make plans. To accomplish this objective, UALR must first provide a consistent, reliable source for reporting and querying. The ultimate goal of UALR’s QIP is to provide decision- makers at all levels of the university with data that they can trust to aid in problem-solving, enrollment planning, student learning, retention, degree completion, interventions, resource allocation, budgeting, and research impact and contributions.

UALR’s use of data has not evolved intentionally. The current student information system was implemented approximately 20 years ago and has limited capacity for the analytics and insights needed by the campus community.

Campus decision makers routinely indicated lack of reliable information as major challenges in their efforts to increase graduation, retention, and enrollment at the program, department, and university levels. Reporting was fragmented, resources were devoted to meeting minimum compliance standards, and data use was decentralized and siloed.

In 2013, while in the midst of an administrative and academic restructuring process, UALR established Vision 2020—to be one of the top metropolitan, community-engaged, research universities among the 16 member states of the Southern Regional Educational Board (SREB). As a unique, urban institution serving multiple populations with varying needs and challenges, the university reaffirmed its focus on student success, its internal network, and relationships with external stakeholders and the surrounding community. To achieve this vision, UALR decided to adopt a data-oriented culture.

A Quality Initiative (QI) Task Force was assembled consisting of internal stakeholders from faculty, staff, and administration. The task force consisted of three teams, Metrics, Analytics, and Systems, charged with transforming UALR into a more efficient institution that uses data to identify problems, propose solutions, and support continuous improvement. Specifically, the teams identified data points (i.e. metrics) to be included in a Data Warehouse (DW) for reporting and planning purposes.

The QI Task Force developed a data governance structure, surveyed faculty, and staff about data and reporting requirements, explored dashboards for analytics, reviewed additional software for student outcomes assessment, and began the selection process for a DW vendor.

In addition to its technology focus, the QI Task Force surveyed the campus community to determine areas of greatest needs. A data requirements survey was launched in the 2013-14 academic year to identify campus data uses and needs. All employees received the survey; 568 responses were collected. Respondents expressed the desire to gain access to a broad number of subject areas. The result was an extensive list of metrics that could be used to assess institutional performance, faculty contributions and impact, and student success.

During the 2015-16 academic year, the QI Task Force further refined the list of metrics and developed a survey to gather input from the campus community on which metrics would provide the “most relevant information to aid decision makers.” The survey was made available to the campus in fall 2015.

The results of the survey fell into multiple categories: student success; research activities and economic impact; graduates’ employment indicators; diversity, equality and global impact; and support for human resources and campus infrastructure.

Retention and persistence, graduation rates and the number of declared majors were the top three most important metrics concerning student success. Respondents indicated the need to use data and insights to increase student success both inside and outside of the classroom.

Faculty and staff also indicated concern about how graduates respond to employers demands for particular skill-sets and economic impact to the community. Other decision makers showed the need for data related to underserved populations identifying race, gender, socioeconomic status, and geographic diversity of students.

Information gleaned from quality data sources can also strengthen and support the human resources infrastructure necessary to fulfill UALR’s mission and vision. Respondents recognized the need to use analytics to calculate return on investment, class sizes, staffing levels, and instructional load.

After careful review of metrics collected in the survey and feedback from the campus community, the QI Task Force started research and preparatory work for the DSS that would address quality data and analytics needs at UALR.

In the midst of UALR’s QIP, state officials launched initiatives to improve higher educational outcomes in Arkansas. Both the Arkansas Department of Higher Education (ADHE) and the University of Arkansas System (UA System) instituted large-scale actions directly affecting the operation of UALR and reinforcing the significance of the QIP. In 2014, the UA System adopted a new strategic plan emphasizing its commitment to being “the most productive and resource efficient system in the nation focusing on student achievement and success.” The plan detailed its strategy to use evaluation metrics to determine success for each institution within the UA System. One year later, ADHE adopted the “Closing the Gap 2020” master plan to increase the post- secondary attainment rate in Arkansas from 43.4% to 60% by 2020.

In July 2016, the ADHE followed this action with the announcement that the ADHE Coordinating Board supported its plan to change the funding formula for state institutions of higher education to a productivity funding model similar to ones in Tennessee, Indiana, Ohio, and Oregon. The assessment of student learning outcomes and the use of data in planning and decision making is more critical than ever in facilitating, maintaining, and documenting educational quality at UALR.

3. Explain the intended impact of the initiative on the institution and its academic quality.

The DSS will provide the needed framework and tools for an integrated planning process based on quality data; facilitate a data-informed culture of continuous improvement; allow for predictive analytics on student success; and demonstrates the university’s impact on the city, state, and region. The DSS will consist of the following six components:

Business Intelligence (BI) Platform
The current BI platform provides basic functionality and insight for decision-making, primarily at the upper administrative level of the institution. The initial implementation of the DSS will use UALR’s current BI platform; however, a new BI platform will be added to the final DW. An upgraded BI platform will provide capacity for informed decision making throughout the campus, not just the upper administrative level.

Predictive Analytics
UALR will successfully integrate data from its student information system into its predictive analytics tool. This functionality provides a mechanism by which administrators, faculty, and staff may utilize predictive analytics in a FERPA-compliant platform to better understand student data.

Degree Audit
UALR is implementing a stand-alone degree audit and transfer equivalency software system that helps students and advisors monitor progress toward degree completion. The DA system will provide students, faculty, and professional advisors with clear visual indicators that showcases course requirement progress, status bars that demonstrate student progression, and worksheets that can be customized to include core course, financial aid, and athletic eligibility audits. Specifically, this tool will provide needed information to reduce time to degree and credits at degree completion in line with the listed goals of the ADHE.

Assessment System
The component of the DSS that will have the most direct effect on the academic quality of the institution will be the adoption of a comprehensive assessment tool. This implementation will provide university personnel the ability to assess learning outcomes throughout the curriculum and create outcomes based assessment plans that include defined measures and targets. This assessment approach will also allow for curriculum mapping so that faculty can illustrate where topics are identified and reinforced throughout their programs.

The assessment system will provide a uniform, secure, and centralized location for all assessment data that can be more efficiently and effectively used for continuous improvement and equip faculty and administration with analytic capabilities that are not currently available.

The first step in the implementation of a comprehensive assessment tool is ensuring that the institution has a systematic approach to assessing the curriculum. This process is led by the faculty senate, core council, undergraduate council, graduate council, and a special group of faculty who serve on the program review pilot task force. Their efforts have included the development of assessment processes for general education, skills in the major, and programs that are not discipline accredited. A pilot assessment including two components of the general education learning outcomes was conducted in 2015-2016, with a full assessment to follow in fall 2016. All assessment of the core curriculum should be completed within three semesters.

The outcomes for the skills in the major are still in development but should be completed by spring 2017. This process includes the creation of university-level student learning outcomes as a component of the skills in the major. The new assessment process for programs that are not discipline accredited was piloted in the 2015-2016 academic year with full deployment planned for 2016-2017. Data from these newly developed processes will be coalesced through the assessment tool as a part of the DSS to contribute to continuous improvement of the learning process for the university.

Early Alert System
The university made significant strides over the last ten years, with fall-to-fall retention improving from 59% to 72% and the six-year graduation rate improving from 19% to 27%. Despite these improvements, there is still much to do to help students succeed. The institution serves a largely non-traditional student population. The majority of students are first-generation, non-traditional, and transfers who have attended one or more institutions before UALR. Approximately 600 to 700 first-time, full-time freshmen attend the university each year. The university administration is committed to reducing “time-to-degree” by decreasing the average number of credits per student at graduation, and improving institutional graduation rates. The university intends to implement an early alert system that will allow institutional personnel to intervene with students identified as at- risk for retention and completion to help accomplish this goal.

Data Warehouse (DW)
The DW will serve as the backbone of the integrated DSS and provide the infrastructure to ensure institutional data is complete, correct, and current. Data currently exist in various systems across campus related to student information, finance, and human resources, learning management, and customer relationship management, among other siloed systems. The data is often of various meaning and quality. The implementation of the DW will enable a standardized view of data across the campus enterprise. As part of the interim solution—and the precursor to the data warehouse—UALR developed a preliminary operational data store (ODS) that contains updated student, instructor, course, and registration data. During the QIP period, the ODS will be expanded, and the DW will be implemented to ensure it contains quality data as listed and defined in the newly created data dictionary.

Clarity of the Initiative's Purpose

4. Describe the purposes and goals for the initiative.

Successful completion of the QIP will equip administrators, faculty, and staff with a DSS intended to create a culture of data-driven decision making and facilitate continuous improvement across the university. The work of the QI Task Force began in 2013 and produced many deliverables in preparation for the QIP. Key milestones include:

  • Adoption of a high-level enterprise system architecture;
  • Completion of a skill-set and gap analysis with the Office of Institutional Research (OIR) and Information Technology Services (ITS);
  • Development of a preliminary data classification framework, data quality findings, and data quality reports;
  • Automation of common administrative and regulatory reports;
  • Development of an ODS to support daily headcount and student semester credit hour (SSCH) analysis; development of a preliminary data dictionary and data governance framework;
  • Initialization of a campus-wide implementation of a DA system to support data-informed decision making for students; and
  • The launch of a predictive analytics tool to support decision making for administration, faculty, and staff.

The work accomplished during the remainder of QIP will build upon these milestones. The following goals represent the significant remaining milestones of the DSS for the duration of the QIP. The goals fall under one of two categories: analytics and systems.

Analytics Goals

  1. Gather input and feedback from the UALR community on updated metrics for data dashboards at the program, department, college, and university levels.
  2. Develop dashboards that include no more than three metrics for each strategic goal that will be the most formative to measure the progress toward the UALR 2020 Vision—to be a top metropolitan, community-engaged research university in the SREB.
  3. Assess the analytical components of software solutions offered by the BI community. A Request for Proposal (RFP) that is used to construct the DW component of the DSS should include the analytical assessment from the QI Task Force.
  4. Determine how to utilize the predictive analytics tool to assess current student success programs.
  5. Review and identify requirements of the DA system.
  6. The QI Task Force will partner with the Core Council to assess the new core curriculum.
  7. Develop a data governance policy framework and solicit feedback from the campus community on a data governance policy framework. Additionally, identify data stewards from various campus units and submit recommendations to the data governance committee.
  8. In collaboration with OIR and the data governance committee, complete development of the data dictionary and business glossary.

Systems Goals

  1. Develop a DW and BI RFP in collaboration with the QI Task Force with input from the campus community.
  2. Assess the information technology infrastructure and the required infrastructure needed to implement the DW and BI platforms.
  3. Review and determine any additional IT infrastructure required to implement the updated DA system.
  4. Recommend knowledge and skill sets required of the various constituents of a data governance committee.
  5. Coordinate with the data governance committee and the data stewards to finalize the data governance framework.
  6. In collaboration with OIR, assess the quality of UALR data and conduct a root cause analysis that will yield actionable information to change processes that will lead to the production of better data. This assessment should include identification of operational processes that impede the flow of quality data to decision makers.
  7. Continue the campus-wide implementation of the predictive analytics tool.

5. Describe how the institution will evaluate progress, make adjustments, and determine what has been accomplished.

UALR established the QIP website (ualr.edu/academics/qi) to share information about the project with the campus community. The task force submits comprehensive status reports annually or bi- annually to the university leadership team. These reports are then made accessible to the campus community via the QIP website.

During the iterative phases of the QIP, status reports are used to document work, share findings, and present recommendations from the QI Task Force to internal and external stakeholders.

The scope of the QIP requires a multi-faceted feedback method to ensure relevant stakeholders are abreast of the project’s status and have the ability to make recommendations. Evidence of success includes the ability of program coordinators, department chairs, and deans to have access to a list of data reports prioritized by the QI Task Force. Training sessions will be offered to internal stakeholders. User surveys will evaluate progress and feedback will be collected to make necessary adjustments to the components of the DSS. The QI Task Force and its subcommittees will establish a central repository for feedback, enhancement requests, and recommendations.

Evidence of Commitment to and Capacity for Accomplishing the Initiative

6. Describe the level of support for the initiative by internal or external stakeholders.

The university recognizes that there are several important initiatives at the state level that have a direct impact on the DSS. In particular, goals outlined by the ADHE’s “Closing the Gap 2020” and the UA System’s “Transformation 2025,” require educational outcomes be supported and informed by data.

In 2014, the UA System released a strategic plan, “Transformation 2025,” that is of particular importance to UALR and this initiative. It highlights the importance of data to ascertain these goals. “‘Transformation 2025’ is based upon seven overarching system-wide goals with evaluation metrics to assess the progress toward meeting them. Annual assessment of progress is reported to the Board of Trustees.”

Similarly, the goals in the ADHE master plan for institutions of higher learning in the state underscore the significance of the objectives in UALR’s QIP: 1) raise completion and graduation rates of colleges and universities by 10%. 2) by fall 2018, increase the enrollment of adult students, age 25 to 54, by 75%, 3) raise the attainment rates of underserved student groups in the state by 10%, and 4) improve college affordability through effective resource allocation.

In addition to the goals outlined in the “Closing the Gap” report, ADHE Coordinating Committee approved a productivity funding model, backed by the governor on July 29, 2016, that places a higher priority on program completion for the state’s higher education institutions. The governor's announcement stated “any new funding model must be built around a set of shared principles embraced by institutions, employing appropriate outcomes metrics, and aligned with goals and objectives for post-secondary attainment in our state and encouraging accountability to stakeholders.”

This mandate affirms the goal of QIP to provide the support and tools needed to facilitate a data- informed culture of continuous improvement and demonstrate UALR’s value to the city, state, and region.

7. Identify the groups and individuals that will lead or be directly involved in implementing the initiative.

The QI Task Force focused on identifying data points to be included in the warehouse and how those data will be defined, outcome measures, dashboarding and reporting needed for integrated operational and strategic planning, and the data, information technology, and the personnel components of the DSS.

The QI Task Force originally consisted of three teams: Analytics, Metrics, and Systems. The teams later consolidated into Analytics and Systems teams to create a high degree of synergy and to ensure the technology components of the DSS support the data needs of faculty, staff, and administration. After successful completion of their charges, the remaining two QI teams consolidated into one task force that now consists of 14 members. The members of the task force represent the following academic and administrative units:

  • College of Education & Health Professions
  • Department of Accounting
  • Department of Business Information Systems
  • Department of Chemistry
  • Department of Criminal Justice
  • Department of Earth Sciences
  • Department of Information Science ¥ Department of Psychology
  • Division of Enrollment Management
  • Division of Student Affairs
  • Office of Information Technology Services
  • Office of Institutional Research
  • School of Law

Dr. Robert Corwyn, Professor of Psychology and Dr. Cody Decker, Director of Institutional Research, co-chair the QI Task Force.

8. List the human, financial, technological and other resources that the institution has committed to this initiative.

A campus cross-section of 14 administrators, faculty, and staff serve on the QI Task Force. ITS and OIR provide primary information technology and data-related assistance to the initiative.

The chief information officer (CIO) is a current member and past chair of the QI Task Force and meets regularly with the chairs of the QI Task Force on the status of the project. A project manager from ITS also serves the QI Task Force.

The university has budgeted $220,000 for the implementation of the degree audit system; $180,000 for implementation of the predictive analytics tool; and an estimated $400,000 to $600,000 for implementation of the DW and BI components.

Appropriateness of the Timeline for the Initiative

9. Describe the primary activities of the initiative and timeline for implementing them.

The QIP is planned over three phases:
Phase 1: Metric and System Research
Phase 2: Interim Decision Support Implementation
Phase 3: Ideal Decision Support System Implementation

The QIP will progress in an iterative fashion, continuously ensuring the university responds to the changing landscape of higher education throughout development of the decision support system. The QIP is currently in phase two.

UALR’s QIP will result in a DSS that will inform and facilitate decisions by faculty, staff and administrators across the university. The following tasks are a snapshot of the actions taken under the direction of QI Task Force from 2013 to the present as well as the remaining actions to be completed for the remainder of the QIP.

Phase I: Metric and System Research (Completed)

  • Campus Metrics Survey to gather input and feedback from the UALR community on the metrics included in the report to determine which of these metrics (or a combination of metrics) provide the most relevant information for decision makers. (2014)

  • Preliminary Data Dictionary, development of data classification framework, data quality findings and data quality reports. (2015-16)

  • Automation of common administrative and regulatory data reports. (2015-16)

  • Operational Data Store (ODS), preliminary system maintains daily enrollment data along with certified snapshots of historic regulatory data. (2016)

  • Implementation and “go-live” of the predictive analytics tool, a unified platform used to evaluate student success through fixed configurations and proprietary algorithms, the system makes student-level predictions through configurations that are tailored to the institution. including two online trainings and two on-site trainings. (March 2016)

  • Interim DSS Diagram, adoption of high-level enterprise system architecture diagram that is compliant with the phase one, data warehouse implementation. (March 2016)

  • Research and findings from two on-site vendor data warehouse and business intelligence visits that may be used, in part, to develop a RFP in compliance with Arkansas Procurement Act 557 of 2015. (2015-16)

  • Staff skill-set assessment for ITS and OIR. (2015-16)

Phase II: Interim Decision Support Implementation (In progress)

  • Upon approval by university leadership, collaborate with the UALR Office of Procurement to develop a RFP for a DW and BI implementation to support the UALR Ideal DSS. Once approved, the timeline for the release of the RFP can take anywhere between six to eight months to be awarded to the chosen bidder. Estimates of when the DW implementation will commence is projected for fall 2016. Table 1 outlines the proposed technical implementation timeline for the DW and BI Tool.

Table 1

Proposed Timeline for Data Warehouse and Business Intelligence Tool of the DSS

UALR Table1

  • Degree audit, a web-based system that includes features such as a degree option analyses, a GPA calculator, and an academic planner to help students plan and set goals for on-time graduation. The DA system will integrate with UALR’s current student information system. It is estimated that the DA system will be in place at UALR in spring of 2017 and will launch fall of 2017.

  • Core Curriculum Assessment, partner with the Core Curriculum Council to assess the new core curriculum taking into consideration mandates by the Arkansas Department of Higher Education’s “Closing the Gap 2020” and University of Arkansas System’s “Transformation 2025.”

  • As an extension of the interim DSS, continue development of the ODS that maintains daily enrollment data along with certified snapshots of historic regulatory data. This will mitigate risks associated with the DW implementation and ensure periodic achievements can be demonstrated to the campus community.

  • Use the predictive analytics tool to perform rapid prototyping of dashboard options and disseminate data among stakeholders for determination of the final model.

Phase III: Ideal Decision Support System (estimated completion 2019-2020)

Phase three of the QIP concludes technical implementations for the Ideal DSS, consisting of: 1) Business Intelligence Tool, 2) Predictive Analytics, 3) Degree Audit, 4) Assessment Tools, 5) Early Alert System, and 6) Data Warehouse. Phase three also emphasizes professional development for administrators, faculty, and staff necessary to ensure effective and efficient use of the DSS.

  • Expand the data dictionary to include a business glossary and data elements beyond those required for state and federal reporting. This effort should solicit input from the campus community to ensure consensus on key definitions.

  • Data Governance Framework, a data governance subcommittee, consisting of members from the QI Task Force, will enhance the data governance framework through development of data policies.

  • Implement and provide training for the predictive analytics tool, focusing on adoption at the program, department, and college levels.

  • Data portal for OIR, develop a website to enable sharing of institutional data and support assessment initiatives.

  • Upgrade the interim BI tool to support data-informed decision making throughout the campus community.

  • Assess commonalities among the three primary external stakeholders’ metrics requirements and the top UALR survey metrics and frame a metric comparison model that includes the “Closing the Gap 2020” and the “Transformation 2025” metrics as well as UALR Vision 2020 priority ranked metrics.

  • Incorporate final higher education funding metrics into the metric comparison model when the final document is available for review.

  • Develop a systematic and sustainable feedback loop to obtain ongoing stakeholder input into the campus dashboards.

The DSS multi-year implementation project will provide support and tools that help facilitate a data-informed culture of continuous improvement and integrated planning, and allow for demonstration of the university’s impact to the city, state, and region. Successful implementation of the DSS will set the foundation for a data-informed decision-making tradition enhancing institutional performance and advancing the educational experience of our students.

 

Institution Contact

W. Cody Decker, Director, Institutional Research

Browse Related Proposals

(by Keyword)

Advising

Student Success

2016 Quality Initiative Proposals

Copyright © 2017 - Higher Learning Commission

NOTE: The papers included in this collection offer the viewpoints of their authors. HLC highly recommends them for study and for the advice they contain, but none represent official HLC directions, rules or policies.


Higher Learning Commission • 230 South LaSalle Street, Suite 7-500 • Chicago, IL 60604 • info@hlcommission.org • 800.621.7440

Home | About HLC | Contact Us | Privacy Policy

YouTube graybkgrdLinkedIn graybkgdTwitter graybkgd