What is ETL?
ETL is the process in Blackboard Learn which extracts, transforms, and loads reporting data into the BBLEARN_STATS schema. It takes data from Activity_Accumulator and various other Blackboard Learn tables and processes them into ODS_XXX tables within the BBLEARN_STATS schema. The ETL process runs periodically throughout the day to keep these tables up to date. When you run Course/System Reports, the data is being pulled from these ODS_XXX tables. In order to have the latest data for a course report, the ETL process must be functioning as expected.
ETL_AA and ETL_NON_AA
The ETL process has two main sub processes, when an ETL_REFRESHDATA is executed.
- ETL_AA - Processes Activity_Accumulator data
- ETL_NON_AA - Processes non-Activity_Accumulator data (i.e. rubrics, assessments, etc.)
The ETL_NON_AA process runs periodically throughout the day and is controlled via the bb-tasks.xml file. By default, it runs every hour. The following code can be found in bb-tasks.xml. Learn more about bb-tasks.xml on Behind the Blackboard.
<task-entry key="bb.etl.refresh_noaa" version="60">
<task classname="blackboard.platform.reporting.service.impl.ETLRefreshDataWithoutActivityTask">
<property name="delay" value="3600000" />
<property name="period" value="3600000" />
</task>
</task-entry>
ETL_AA runs only once a day around midnight by default, as part of the Report Scheduler. Within Blackboard Learn, you can go to System Admin > System Reporting > Manage Report Schedule and change the Start Time.
This is not specific to ETL_AA, but also controls when scheduled reports are run within the system as well.
Run ETL manually
You can manually trigger ETL_REFRESHDATA within Blackboard Learn. Go to System Admin > System Reporting > Refresh Report Data. Select Refresh Activity Report Data to run the ETL_AA process, or Refresh Non Activity Report Data to run the ETL_NON_AA process.
Troubleshoot ETL issues
If you notice Course Reports don't contain up-to-date data, it could be that ETL_REFRESHDATA is not functioning properly. If you're a self-hosted a client, you may also begin to notice a performance impact from ETL/ODS related queries in the database.
When these types of issues occur, create a Support Ticket via Behind the Blackboard and together we can begin an investigation into the problem.