This information applies only to the Ultra Course View.
You can share grading responsibilities with multiple graders in your courses.
What is parallel grading?
You can have specific users in your courses grade sets of assessment submissions. Graders can't see other graders' grades, feedback, annotations on student files, and rubrics. They grade in parallel and provide provisional grades. The default grading roles include instructors, graders, and teaching assistants.
The instructor role is the default final grader or reconciler. The reconciler reviews the provisional grades and determines the final grades that students see. You can assign the ability to reconcile grades to one or more other users with grading privileges and remove your ability to reconcile.
Some institutions refer to parallel grading as "blind grading."
Reconciler privileges
- Reconcilers can view the graders' grades and feedback, but not their rubrics or annotations in student files.
- Reconcilers determine final grades or reconcile them.
- Reconcilers can override the final grade.
- Reconcilers can post grades. If anonymous grading is enabled, reconcilers can reveal student names when grades are posted.
At this time, you can't enable parallel grading for group work. You also can't add questions to an assessment with parallel grading.
Why use parallel grading?
Grading and feedback from more than one grader promotes reliability, improves consistency, and removes bias. Reconcilers have more than one critique of the same sample of work as they determine each student's final assessment grade.
Fairness and impartiality in grading are often an institutional requirement or a concern. Parallel grading helps control bias among graders as much as possible.
For large classes, you can divide up the grading tasks among teaching assistants and other graders. For a merged course, you can assign each grader a class section.
Parallel grading workflow
Use these steps to enable and grade with parallel grading:
At the time of assessment creation, you can also enable anonymous grading so graders don't see students' identities. You can also enable the SafeAssign option to prevent plagiarism.
Watch a video about parallel grading
The following narrated video provides a visual and auditory representation of some of the information included on this page. For a detailed description of what is portrayed in the video, open the video on YouTube, navigate to More actions, and select Open transcript.
Video: Parallel grading overview provides an overview of the parallel grading workflow.
1. Enable parallel grading
You can enable parallel grading and assign graders when you create an assessment. You can also enable parallel grading even after students make submissions. The system randomly assigns graders you choose so each student has two graders for the assessment. The grading workload is distributed evenly among the graders. Graders can only open the submissions for the students assigned to them.
Students aren't notified that the assessment has more than one grader unless you inform them. Students see only the final grade and feedback the reconciler provides.
At this time, you can’t create groups or add questions when parallel grading is enabled. Graders, teaching assistants, and course builders can't enable or disable parallel grading.
In the assessment Settings panel, select the check box for 2 graders per student in the Evaluation options section.
Select the Assign graders link that appears.
Assign Graders page
On the Assign Graders page, the roles you assign only pertain to the current assessment.
Assign graders
Your teaching assistants and graders appear in the first Unassigned list.
You can select all with the Assign All as Graders link or open each user's menu and select Grader or Reconciler. Assigned graders appear in the Graders list. Open a grader's menu and select Unassign if needed.
Reminder: The grading workload is distributed evenly among the graders.
Assign reconcilers
Users with the instructor role appear in the Reconcilers list. Open a reconciler's menu and select Unassign if needed. For example, you can have your teaching assistant determine all the final grades and unassign yourself. You can have multiple reconcilers.
No user can be a grader and a reconciler at the same time. However, you can assign a user a different role for an assessment, even after students make submissions.
You can select a list's Unassign All link to remove all users from Graders or Reconcilers.
Select Save when you're finished. Parallel grading works best with multiple graders and at least one reconciler. If you don't have at least two graders and one reconciler, the system warns you. You're allowed to save without the minimal recommendation.
What happens if I unassign users or disallow them course access in the roster?
Use caution before you make changes to graders and reconcilers after grading has begun.
- Unassigned reconcilers lose access to assign final grades, but prior work is maintained. If a reconciler assigns some final grades and then is unassigned or disallowed in the roster, the final grades remain. Another reconciler can make changes to those final grades.
- Unassigned graders' previously completed evaluations are retained. Incomplete evaluations are no longer associated with the students. The system evenly distributes the work to the remaining graders.
If a student already has grades from two graders and then one grader is unassigned, the two grades are retained. A third grader is added. Even if you've reconciled and posted some grades, a third grader is added. The third grader only sees attempts are ready to grade if you haven't reconciled grades. You may have three grades to reconcile for some attempts if a grader is removed during the parallel grading process.
When you disallow a user's access in the roster and then allow access again, the user appears in the Unassigned list on the Assign Graders page.
Disable parallel grading
You can disable parallel grading for an assessment even after submissions exist or graders have provided grades and feedback. The graders' provisional grades are removed, but the system retains them. If you enable parallel grading again, the previous provisional grades and feedback are restored.
If you disable parallel grading after you post grades, the final grades remain. No record of the provisional grades remain.
Graders, teaching assistants, and course builders can't disable parallel grading.
2. Graders > Access and grade attempts
When you enable parallel grading, graders can only access the assessment attempts assigned to them. At this time, graders don't receive notifications in the activity stream when attempts in the parallel grading workflow are ready to grade. Reconcilers receive notifications when grades are ready to reconcile.
As the final grader, the reconciler can't assign grades for graders.
In the anonymous grading workflow, graders also can only access the submissions assigned to them. The grading list is blank until students make submissions.
At this time, the annotations a grader adds to students' files aren't available to students or reconcilers.
After graders provide grades and feedback for their sets of submissions, their work is complete. Graders can't post grades. Instructors and reconcilers can't view a grader's rubric or edit a grader's grade and feedback.
After the reconciler posts grades, a grader can only view the reconciler's final grades, rubrics, and feedback. Graders can't edit their evaluations after grades are posted.
Multiple attempts
When you allow multiple attempts on an assessment, graders can view their workload on the Submissions page. The final grader can reconcile grades even if graders don't grade all the attempts.
3. Final grader: Reconcile grades
When grades are ready to reconcile, you'll receive an activity stream notification. You receive the notification after the assigned graders have graded one student's attempt.
Reconcile a single attempt
You can review each grader's grade and feedback before you determine the final grade. You can reconcile grades even if graders don't grade all the attempts.
- Assign grades. Select the grade pill to assign a grade or open the rubric. Reminder: You can't view a grader's rubric or annotations on student files. You also can't edit a grader's grade and feedback.
- Review feedback. Select a feedback icon to open the Feedback panel and review the graders' feedback.
- Add your feedback. Students won't see the graders' feedback, but your feedback appears when you post the final grades.
After you post the final grades, graders can only see the final grades and feedback. Graders can't access their evaluations to edit them after final grades are posted.
Student view
Students only see the grade, feedback, and rubric from the reconciler. Students don't see anything provided by the individual graders.
Reconcile All
You don't have to reconcile grades individually. In one action, you can reconcile all attempts with one or more grades from graders. For example, you can "batch" reconcile if you have confidence in your graders and want to assign the average of their grades to each attempt.
All graders don't have to finish before you reconcile grades. You also don't have to wait for all students to make attempts. You can automatically reconcile the grades for the attempts that exist. This action is irreversible. Reconciliation doesn't override manually reconciled or previously reconciled grades.
On the Submissions page, select the Reconcile All link. Select the grade to use:
- Average of grades
- Highest of grades
- Lowest of grades
You always see that attempts are ready to reconcile if students have made attempts. Graders don't have to assign grades and feedback before the attempts are ready to reconcile.
If you choose to reconcile all grades and you've associated a rubric with the assessment, an Override label appears on the Submissions list page only.
Reconcile multiple attempts
When you allow multiple attempts for an assessment, you see grades given by graders per attempt. Then, you reconcile grades for each attempt. The final grade for a multiple attempt assessment is calculated based on the Grade attempts setting you chose when you created the assessment.
Select an attempt or the grade pill to view the student's work and reconcile the grades. You can also batch reconcile all the grades for existing attempts.
Parallel grading doesn't impact how the final grade is calculated. However, the final grade can't be calculated until attempts are reconciled. The final grade appears in the gradebook when those attempts are reconciled. For example, if the final grade is calculated based on the highest graded attempt, the final grade appears immediately after at least one attempt is reconciled. But, the grade may change as students submit more attempts.
After reconciliation, you can post the final grades for students.
