What is ETL?

ETL is the process in Blackboard Learn which extracts, transforms, and loads reporting data into the BBLEARN_STATS schema. It takes data from Activity_Accumulator and various other Blackboard Learn tables and processes them into ODS_XXX tables within the BBLEARN_STATS schema. The ETL process runs periodically throughout the day to keep these tables up to date. When you run Course/System Reports, the data is being pulled from these ODS_XXX tables. In order to have the latest data for a course report, the ETL process must be functioning as expected.

ETL_AA and ETL_NON_AA

The ETL process has two main sub processes, when an ETL_REFRESHDATA is executed.

  • ETL_AA - Processes Activity_Accumulator data
  • ETL_NON_AA - Processes non-Activity_Accumulator data (i.e. rubrics, assessments, etc.)

The ETL_NON_AA process runs periodically throughout the day and is controlled via the bb-tasks.xml file. By default, it runs every hour. The following code can be found in bb-tasks.xml. Learn more about bb-tasks.xml on Behind the Blackboard.

<task-entry key="bb.etl.refresh_noaa" version="60">
  <task classname="blackboard.platform.reporting.service.impl.ETLRefreshDataWithoutActivityTask">
    <property name="delay" value="3600000" />
    <property name="period" value="3600000" />
  </task>
</task-entry>

ETL_AA runs only once a day around midnight by default, as part of the Report Scheduler. Within Blackboard Learn, you can go to System Admin > System Reporting > Manage Report Schedule and change the Start Time.

This is not specific to ETL_AA, but also controls when scheduled reports are run within the system as well.

Run ETL manually

You can manually trigger ETL_REFRESHDATA within Blackboard Learn. Go to System Admin > System Reporting > Refresh Report Data. Select Refresh Activity Report Data to run the ETL_AA process, or Refresh Non Activity Report Data to run the ETL_NON_AA process.

Troubleshoot ETL issues

If you notice Course Reports don't contain up-to-date data, it could be that ETL_REFRESHDATA is not functioning properly. If you're a self-hosted a client, you may also begin to notice a performance impact from ETL/ODS related queries in the database.

When these types of issues occur, create a Support Ticket via Behind the Blackboard and together we can begin an investigation into the problem.