Summary
Integrated planning and advising services (IPAS) comprises student degree planning tools, services that inform students about learning resources and opportunities, and early notifications systems that identify students at risk of failing a course and/or dropping out of the university. To improve student and university outcomes, IPAS aggregates and analyzes student data, including information about an individual’s academic performance, participation in course-related activities, website analytics and/or campus resource use. These tools and services keep students, instructors, and advisors informed about student needs. While the positive impact of IPAS on academic outcomes is clear, the tracking of students’ use of online tools and other behavior raises concerns, particularly related to privacy.
The growing interest in IPAS in higher education requires a deeper understanding of student and faculty perceptions of these services at the University of Washington (UW). The 2015 EDUCAUSE annual ECAR survey investigated student and instructor opinions regarding IPAS and the data collection that is required for these services to operate. Overall, both students and instructors expressed interest in IPAS-related alerts and notifications; students responded more favorably than instructors. With a few exceptions, UW student and instructor opinions were similar to those at peer institutions. Whereas most students and faculty responded favorably about early alert and personalized notification systems, they are considerably less enthusiastic about the data collection practices required to support IPAS. In general, opinions differ for instructors and students, with instructors expressing more hesitance.
Introduction
For the last several years, the UW has participated, along with 250 other institutions of higher education, in the EDUCAUSE Center for Analysis and Research (ECAR) survey. This annual survey gives UW-IT a valuable opportunity to learn about the technology experiences of undergraduates and instructors at UW. Data generated from the survey provides insight into UW students’ and instructors’ use of, and perceptions about, educational technologies compared with their peers at other institutions with doctoral programs.
Integrated planning and advising services (IPAS) includes a variety of services for students, instructors and advisors that assist in degree planning, access to learning resources, and identify students in need of support. For example, one IPAS strategy would be to send an alert to a student and their instructor if the student’s course performance appears to be declining. Or a student may receive personalized course recommendations based on information about their course selection history and academic interests. In order to generate these personalized notifications, IPAS relies on data about student behavior and performance in the academic setting, including information about an individual’s participation in course-related activities, campus resource use, website click data, and, in some cases, social media data. These data have the potential to provide insight into the individualized needs of students, generating personalized alerts and notifications that can be used by any member of the learning team, and provide timely support to address needs as they emerge. While the positive impact of IPAS on academic outcomes is worthwhile, the tracking of students’ use of online tools raises concerns, particularly related to privacy.
The 2015 EDUCAUSE ECAR survey investigated student and instructor opinion regarding IPAS and the data collection that is required for these services to operate. There is a growing interest in IPAS at UW, and thus it is imperative that UW leadership understands student and faculty perceptions of these services.
Findings
In this section, we summarize UW student and instructor perceptions of IPAS. A total of 638 UW students and 426 UW instructors took the 2015 ECAR survey. Pearson’s chi-square test was used to explore potential statistical differences between comparison groups.
Access to academic planning, advising and support tools
Findings from the 2015 ECAR survey reveal that students continue to be interested in opportunities facilitated by IPAS. Students and faculty were asked to rank their interest in the various types of personalized alerts or notifications that support IPAS, indicating if they were not at all interested, not very interested, moderately interested, very interested, or extremely interested. Findings indicate that education planning tools, opportunities to receive personalized academic advising, and early alert systems about student performance, are all services that students and faculty feel would positively impact student academic experience and outcomes. ECAR findings related to each of these services are reported below. Since UW students and instructors’ opinions were very similar to those at peer institutions, we note only statistically significant differences.
Education planning tools. UW students and instructors indicated they were interested in education planning tools (e.g., personalized information on degree progress), with students reporting a greater preference for personalized education planning tools. For example, when asked if they were interested in receiving guidance about courses they might consider taking, 84.4% of UW students responded positively, while just over half of faculty reported this was information they were interested in (p<0.001). UW faculty were less interested in this information than their peers at other institutions, with 52.0% compared to 60.3% (p=0.003).
When asked to reflect generally on personalized support and degree progress information, UW students were more interested than their instructors, with 86.2% of students responding favorably, compared to 69.2% of faculty (p<0.001). UW students also reported being relatively interested in personalized dashboards providing real-time feedback about their progress, with 82.9% of students indicating they would like this service. UW faculty on the other hand, while still positive, were less interested than students, with 71.7% of faculty valuing this notification service for their students (p<0.001).
Figure 1: Student and Faculty Interest in IPAS. Click to explore interactive visualizations.
Personalized approaches to academic advising and counseling. The 2015 ECAR survey evaluated perceptions about responsive counseling and advising systems. UW instructors were less interested than their peers at other doctoral institutions in receiving suggestions for how their students can improve their course performance, with 64.9% indicating interest compared to 72.0%, respectively (p=0.013). UW students displayed more interest in this type of feedback than their faculty counterparts, with 82.4% of students noting that they would like to receive tailored suggestions for how to improve their academic performance (p<0.001). In regard to receiving personalized recommendations for new academic resources, like subject-specific workshops or study resources, UW students and faculty mirrored interest, with 78.8% of students and 78.3% of faculty indicating they would like to receive these types of notifications. While UW students and instructors are equally interested in circulating personalized recommendations for academic resources, students are far more invested in having access to personalized quizzes and practice questions than instructors, with 84% compared to 59.8%, respectively.
The 2015 ECAR survey asked students if they were interested in knowing their academic performance compared to their peers. UW students reported being more interested in receiving this kind of comparative feedback about their in-class performance than their instructors were interested in having this information available for their students. In 2015, 84% of UW students expressed a desire to utilize this tool, compared to 59.8% of UW instructors (p<0.001).
Notifications about performance. UW students and instructors are interested in receiving alerts when it appears that a student’s progress in a course is declining, with students displaying slightly more enthusiasm than their instructors. In 2015, 75.7% of UW students were interested in receiving these types of alerts about their personal performance, while 62.3% of UW instructors said they wanted alerts concerning a decline in student progress (p<0.001). Additionally, that same year, just under half of responding students indicated they wished their instructors used early-alert systems, designed to catch potential academic trouble as soon as possible, more often.
Perception of the collection of personal data
The 2015 ECAR survey inquired about student and instructor perceptions about the data collection that may be required in order to make IPAS possible. Students and faculty were asked to rank their feelings about the various types of data collection, indicating they felt a particular practice was a “very bad idea,” a “bad idea,” “neither good nor bad,” a “good idea,” or a “very good idea.” The results indicate that while most students and faculty feel favorably about the early alert and personalized notification systems they were polled about, they are considerably less excited about the data collection practices that are required to support IPAS. ECAR findings related to opinions about data tracking practices are reported below. Since UW students and instructors’ opinions were very similar to those at peer institutions, we note only statistically significant differences.
Data collection practices surrounding student academic performance, past and present. When asked to reflect on how they felt overall about UW collecting student data to use for generating individualized messages about academic progress and guidance, just over half of students indicated they thought this was a good idea, while 29.8% said it was neither a good nor a bad idea. The remaining 16.6% reported that it was a bad idea. UW faculty were more opposed, with the majority saying it was a bad idea (41.8%), 33.3% stating it was neither a good nor bad idea, and 24.8% saying it was a good idea (p<0.001). Overall, UW faculty were more hesitant than faculty from peer institutions, whose responses to the same question fell roughly into thirds (p=0.002).
When students were asked how they felt about collecting data about their academic performance in past courses, 61.2% indicated that this was a good idea, 25.8% said it was neither a good nor a bad idea, and the remaining 12.9% reported that it was a bad idea. Faculty felt similarly about this type of data collection, with 60.6% indicating it was a good idea, 22.2% stating it was neither good nor bad, and finally, 17.2% of UW faculty indicated that this would be a bad practice.
When asked to reflect on data collection about their academic performance in current courses, students again indicated they generally felt positive about this practice, with 68.6% saying it was a good idea and only 11.3% saying it was a bad idea. The difference observed in student opinions about collecting data on their performance in current classes versus past courses was statistically significant, with students appearing to be slightly more comfortable with data collection of their performance in current courses (p=0.023). UW faculty feel significantly more positive about this practice than UW students, with 72.7% of faculty reporting that it was a good idea to collect data on student performance in current courses (p=0.039).
Data collection that monitors degree progress. When asked how they felt about collecting data on their progress toward a degree or certificate goal, most UW students responded favorably, with 76.5% indicating they thought this was a good or very good idea. UW faculty reported similar impressions about this type of data collection, with 77.8% indicating it was a good or very good idea. Similarly, when UW students were asked how they felt about having data collected about their course performance and compared to the academic performance of their peers, 60.6% said this was a good idea and only 14.8% said it was a bad idea. Among faculty probed about this topic, 56.1% of UW faculty said it was a good idea, while 22.4% said it was a bad idea.
Figure 2: Student and Faculty Opinions about UW collecting data to inform IPAS.
Tracking student activity over university-run websites, services and applications. UW students and faculty were asked how they felt about collecting data on student activity over a university website. In general, UW students and faculty seem more leery of this type of data collection than data collection that focuses on tracking student progress in a course. Of the UW students polled, 42% said they thought tracking activity over university-run website was a good idea, but nearly a third of the total respondents said this was neither good nor bad, and 26.1% of students concluded it was a bad idea to collect this type of data. Conversely, only 22.4% of faculty said this was a good idea, 29.6% said it was neither a good nor bad idea, and the remaining 48% of instructors said this type of data collection was a bad idea.
When UW students and faculty were asked how they felt about collecting data on student activity in a specific application or service provided by the institution, just over half of students said this was a good idea, 17.9% said it was a bad idea, and the remaining respondents, 32.3%, said it was neither a good nor bad idea. Instructor responses were roughly split into thirds, with slightly more UW instructors indicating this was a good idea (36.5%). The difference observed between UW instructors and students was statistically significant (p=0.002).
Data collection through student ID cards and smartphones. Students and faculty were asked how they felt about collecting data on students interactions with campus-based activities logged through their student ID card. Only 40% of students said this was a good idea, while 28.1% suggested it was a bad idea. UW faculty were substantially more opposed to this type of data collection, with only 20.2% of respondents saying it was a good idea, and over half (54.5%) said it was a bad idea. When asked how they felt about collecting data on campus-based activities logged through student smartphones, students and faculty were both uncertain. Of UW students surveyed, 36.5% said this was a bad idea, 30.4% indicated it was a good idea. Over two-thirds, 66.7%, of faculty said this was a bad idea, leaving 21.2% of faculty who felt this data collection was neither a good nor bad idea and 12.1% who felt this was a good idea. The difference observed between UW faculty and students concerning collecting data via student ID cards and student smartphones, was statistically significant (in both cases, p<0.001).
Collecting data about student location on campus. The 2015 ECAR survey also polled students and faculty about various options for collecting data about student location on campus. This type of data collection would provide a fine-tuned and tailored approach to notifications, generating alerts that are responsive to where students spend time on campus. When asked generally about collecting data about student location on campus, students were reluctant to support this type of data collection, with nearly half of UW student respondents reporting that it was a bad idea. UW faculty felt even more strongly, with 71.4% of faculty reporting that it was a bad idea (p<0.001). Responses among students were slightly more favorable when asked how they felt about collecting data about student proximity to a university building, office, or resource, with 32.8% of students saying it was a good idea, 31.4% saying it was neither good nor bad, and 35.8% saying it was a bad idea. Instructors were still quite wary about this data collection, with only 8.2% of UW instructors saying it was a good idea, and 65.3% saying it was a bad idea. The difference observed between UW instructors and students is statistically significant (p<0.001).
Collecting data about students over social media. UW students and faculty were asked how they felt about their institution collecting data on student social media activities to help generate individualized messages and alerts for students. For instance, if a student likes a university-sponsored Facebook group, then the student will receive notifications about related clubs and events on campus. The overwhelming opinion among both populations at UW was negative. UW students felt more negatively than their peers at other institutions, with 60.8% of UW students saying it was a bad idea, 22% who said it was neither good nor bad, and 17.1% who said it was a good idea. This was compared to 58.1% of students at other institutions with doctoral programs who said it was a bad idea, 22.9% who said it was neither good nor bad, and 19.1% who said it was a good idea (p=0.047). UW faculty also felt quite strongly that this type of data collection was a bad idea, with nearly 80% of faculty indicating such. The difference observed among UW faculty and students was statistically significant (p=0.006).
When asked how they felt about UW combining the data on students’ school-related activities with social-media and mobile-device data to enhance student academic experiences, assess institutional impact, and tailor offerings to meet the individual needs and expectations of students, UW students and faculty seemed hesitant. Among UW students, 26.2% felt it was a good idea, 33% said it was neither good nor bad, and 40.8% said it was a bad idea. Compared to UW students and faculty at other doctoral institutions, UW faculty were substantially more negative about this use of data (p
Figure 3: Filter responses based on major, class standing and other demographic factors.
Findings from ECAR indicate a high level of interest among students and faculty in a wide range of IPAS services. IPAS, as described here, can be roughly categorized into three areas: academic planning, campus resources discovery, and academic performance. UW Information Technology (UW-IT) has made strides in all of these areas, with particular emphasis on academic planning.
MyPlan is an academic planning tool that helps students find courses, create plans across multiple quarters, and audit their progress toward the completion of a degree. MyPlan’s academic explorer is a set of features in development that will enable students to find relevant degrees based on a course or area of interest, and to view unmet degree requirements and corresponding courses. Additionally, UW-IT has prototyped an application that will provide students and advisors with a clearer picture of GPA standards for different majors, and the average grade for the most commonly taken courses by students who have declared for a particular major.
ECAR findings indicate that personalized recommendations for new university resources for undergrads, such as subject-specific workshops or study resources, are strongly desired. MyUW, the UW student portal, provides personalized notification to students now and relevant event information (when available) from their degree program. Expanding upon personalized notifications of this nature is the desired direction for MyUW in the future. UW-IT is prototyping an application that helps students find a wide range of campus resources (e.g., 3D printers).
With regard to academic performance, UW-IT is currently assessing Civitas, an analytics platform designed to help advisors identify at-risk students so that they may intervene with the goal of increasing retention.
These tools and services reflect a strong investment of UW-IT resources to enhance IPAS at the UW. Such an investment necessitates a thoughtful, informed response to student and instructor opinions about using performance and other data to increase student success and enhance the student experience. As UW-IT continues to invest time and energy in resources welcomed by students and instructors, it is critical that ongoing work take into account the risks as well as the benefits of tracking student data.