Anthology doesn’t recommend using access and activity logs as the only means of making decisions about students’ academic integrity. Analyzing individuals or small data samples for purposes of high-stakes decisions such as determining cheating is technically possible, but these types of analyses are often influenced by common data biases—in particular, confirmation bias and correlation being equated with causality. Here are two examples of where data analysis biases could lead to improper conclusions:
- A student’s IP changing as a test begins could indicate cheating by having someone else take the test; it can also indicate the student needed to restart their router or that they’re using a VPN when accessing from a public network to better secure their computer.
- Several students starting a test at the same time could indicate cheating by coordinating to take it as a group; it can also indicate these students simply have similar work and personal schedules, leading them to do their course work at very similar times.
For these reasons, features such as the Access Logs in Tests or other point-in-time analysis of access and activity logs are not recommended as the only means of determining academic integrity, though they might reinforce other findings in an investigation of misconduct. Access and activity logs are primarily designed for aggregated analysis and troubleshooting system and user issues. Some reports are constructed from system log files rather than transactional database tables; while rare, these report types can be more susceptible to occasional lost or duplicated data, making use for very small sample sizes such as for individuals at a point-in-time potentially inaccurate.
If you have concerns about cheating or academic integrity, we recommend that you start with the institutional office responsible for handling such incidences of misconduct. This might be an office such as academic affairs or academic technology. They generally will have policies and approved procedures for conducting investigations whether the activity in question occurred online or not. Using a single log or a small activity data set alone to identify misconduct is susceptible to several data bias types, so Anthology doesn’t recommend such use of data alone for concluding misconduct occurred.