The following examples demonstrate the composition of COURSE data feeds while meeting a variety of use cases. These examples utilize the simplest possible data feed required to meet the use case. There are many more COURSE feed headers for use in creating course records - analysis of your institution's information system and registrar requirements and planning will help determine the depth of data necessary to properly populate Learn meeting your data and course lifecycle goals.
The examples are based on default Learn settings which visible in the integration configuration UI. Changing these configuration elements will result in change to the example results. Explanations of these settings are available in SIS Framework Overview. Also, it is assumed unless otherwise noted that the integration is configured to use the same data source for all incoming data.
Courses
Course data is the primary information set which describes a course in Learn. In the context of SIS data COURSE objects are often referred to as "SECTIONS" and this is reflected in existing standards. Predating many of these standards the Learn Snapshot Flat File uses "COURSE" to refer to course related records based on the context.
Operations
Data may be provided to Learn and then subsequently updated, removed, or amended - thus you may start with the simplest data set and augment as your institution's data requirements change.
The one immutable COURSE field is COURSE_ID - this field may not be changed once a record is created and the record must be disabled and purged if the COURSE_ID needs to change. This has ramifications impacting records of activity - it is therefore strongly suggested that you use a data element that is not likely to change during the lifecycle of the course.
Snapshot Flat File data management
The SIS Framework supports Snapshot Flat File data feed uploads via a UI Feed Upload and via a set of URLs provided by the Learn system.
Access HTTP Information and Upload Feed File via the integration menu in the System Admin Data Integration Student Information Systems Integration UI.
In both cases, the behavior of the data operation is driven by the configuration of the integration and the operation type selected. The data operation type selected controls how the data in the feed is 'interpreted' and each URL will provide different results to meet the desired goals of your integration.
| Operation | Description |
|---|---|
| Store | When using this operation type data that is contained in the feed file is stored or updated (per configuration settings) across all data sources that are owned by the integration. (see SIS Framework Overview for data 'ownership' and Data Source ) |
| Complete Refresh | This operation either stores or updates data that is contained in the data feed while disabling data that is not contained in the data feed which associated with the integration across all data sources. |
| Delete | This operation disables, per integration settings, the records contained in the data feed associated with the integration across all data sources. |
| Complete Refresh By Data Source | Introduced in SP 12 this operation performs a complete refresh of data associated ONLY with the integration configured data source. This operation emulates more closely the command-line snapshot process for refreshing data. |
The objects associated with Course operations are:
| Object | Operations |
|---|---|
| Course | Store, Complete Refresh, Delete, Complete Refresh By Data Source |
| Course Association | Store, Complete Refresh, Delete, Complete Refresh By Data Source |
| Course Standard Association | Store, Complete Refresh, Delete, Complete Refresh By Data Source |
Course Association Examples may be found in the Hierarchy example section.
The provided examples are demonstrated using the Snapshot Framework UI Upload Feed File capability. To learn more about automating or using other command line/programmatic operations, see Snapshot Flat File Automation.
A reminder on data source keys
All data objects support the ability to alter the data source key for the grouping of that data set and may be used to alter the associated data source - Note: this is not a required field in Framework based data feeds and unless noted the following examples assume the integration is configured to use a single data source.
Introduced in SP 12 is the ability to specify the Data Source in the data feed separately from specifying a new data source.
See Data Source Key Management and the section on changing a Course record's data source.
A note on field mapping
Field mapping provides the ability to alter incoming data before it is stored in Learn. This allows you to have complete control over the data that is stored and enables you to meet Learn specific rules when the SIS data you are provided is insufficient eg: the creation of a suitable course name entry. When applied to a COURSE object field the associated script is run per course record, altering or providing the data before it is stored in Learn. To learn more about field mapping, see Snapshot Flat File Custom Field Mapping.
Course operation examples
At a high level one can identify three SIS integration data feed patterns which may be applied to all Course data operations and the selection of the pattern depends on the data you are able to provide.
- Using a single feed file you may Store and Update records (Store) utilizing a separate process to disable (Delete) records
- Using a single feed file you may Store, Update, and Disable (Complete Refresh) records
- Using a combination of files you may Store with one, Disable with a Second.
Finally, and this is not an SIS feed pattern, but worth mentioning, you may also disable and purge based on DSK alone utilizing the Data Source Management tool available in the UI. You should exercise great discretion when managing your SIS provided data in this manner. This is extremely useful when purging data which never was or is no longer provided via the SIS or the result of testing operations.
Just the basics: Courses
All course accounts require a basic set of information to establish an account. To learn more, see Snapshot Flat File Data Format and Snapshot Flat File Header Descriptions.
If you are currently using the UI Batch Tools switching to using SIS Framework and using the minimal course data and the UI upload capabilities of the SIS Framework provides you with better logging and reporting of your data uploads w/o altering your data collection processes.
Data in brief
The minimal data set or headers required for creating a course account in Learn consists of:
EXTERNAL_COURSE_KEY - a unique identifier for this course record.
DATA_SOURCE_KEY - a unique identifier for the data set this record. Note: this is provided either in the feed or via the integration configuration)
COURSE_ID - The course's id - this is used as a unique display identifier for the course. COURSE_NAME The course title.
The SIS Framework per an integration configuration provides default values for, or ignores, non-required fields. Two useful fields which are not required for a COURSE feed are AVAILABLE_IND and ROW_STATUS. These will be covered in a following use case.
Each of these headers is described completely in Snapshot Flat File Header Descriptions.
Adding course information
There are two cases for adding COURSE information. The first is to additively STORE COURSE information resulting in the addition or updating of records as presented in the data feed. The second is to REFRESH COURSE information already present in Learn resulting in the addition of new or updating of existing records as presented in the data file while disabling existing Learn records which are not present in the COURSE data file.
Store operation examples
Example #1: Create courses
You wish to add courses to LEARN without impacting existing accounts. You have your integration configured to use the same data source for all incoming data.
Prerequisite
None.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
Solution
Create a COURSES.txt data file containing the required headers and associated data per COURSE you wish to add to the system. For example:
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME
testCourse1|Course.1.1.SP2013|Test Course 1
testCourse2|Course.2.1.SP2013|Test Course 2
testCourse3|Course.3.1.FA2013|Test Course 3
Use the UI to upload this file via the COURSE Data Type using the STORE operation. The course accounts will be created and you can discover them via the System Administrator Course tools.
Postcondition
COURSE records for Course.1.1.SP2013, Course.2.1.SP2013, and Course.3.1.FA2014 are created.
Example #2: Update courses
You created course accounts and need to change them. For example.: the above example did not contain the term in the course name.
Prerequisite
None - existing courses will be updated, any new course data in the feed will create new courses.
Solution
Create a COURSES.txt data file containing the required headers and associated data per COURSE you wish to update to the system. For example:
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME
testCourse1|Course.1.1.SP2013|Test Course 1 (SP2013)
testCourse2|Course.2.1.SP2013|Test Course 2 (SP2013)
Because STORE only operates on the data contained in the file the Course.3.1.FA2013 record previously submitted is not affected.
Use the UI to upload this file via the COURSE Data Type using the STORE operation. The course records in the file will be updated.
Postcondition
The COURSE record for Course.1.1.SP2013 and Course.2.1.SP2013 is updated to include the term in the COURSE_NAME
The COURSE record for Course.3.1.FA2014 is not affected.
Course: Complete refresh course operation
COMPLETE REFRESH operates differently than STORE. COMPLETE REFRESH performs two operations that amount to a comparison of the data in the feed file and the records in LEARN owned by the integration - storing new records, updating existing records, or disabling records in LEARN that are not in the data file.
Example: Complete refresh
The data your SIS provides contains a full snapshot of the COURSEs which should exist in Learn. This data contains COURSE records to add, COURSE records to update, and records which have been removed since previous COMPLETE REFRESH operations which should be handled appropriately per configuration (disable or purge).
Prerequisite
None.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
Solution
Starting with the data from our first COURSE store operation and adding Course.1.2.SP2013 to the data feed and removing Course.3.1.FA2013:
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME
testCourse1|Course.1.1.SP2013|Test Course 1.1 (SP2013)
testCourse1.2|Course.1.2.SP2013|Test Course 1.2 (SP2013)
testCourse2|Course.2.1.SP2013|Test Course 2 (SP2013)
Note that if other COURSE records are managed by this integration they will be disabled or purged due to their absence in the above data feed.
Postcondition
The COURSE record for Course.1.1.SP2013 is retained and updated to include the section in the COURSE_NAME
The COURSE record for Course.1.2.SP2013 is added to Learn
The COURSE record for Course.2.1.SP2013 is retained and not affected.
The COURSE record for Course.3.1.FA2013, because it is absent from the data feed, is marked as disabled or ready for purging per integration configuration.
Course: Complete refresh by data source
COMPLETE REFRESH BY DATA SOURCE performs a COMPLETE REFRESH operation but restricts the data affected to that which is only associated with the integration's data source.
Example: Complete refresh by data source
The data your SIS provides contains a full snapshot of the COURSEs which should exist in Learn. This data contains COURSE records to add, COURSE records to update, and records which have been removed since previous COMPLETE REFRESH operations which should be handled appropriately per configuration (disable or purge). Additionally, all data in this refresh is targeted using the same data source as defined in the integration and ONLY data related to this data source key is to be affected.
Prerequisite
None.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
Solution
Using the data from our last store operation and removing Course.2.1.SP2013 from the data feed:
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME
testCourse1|Course.1.1.SP2013|Test Course 1.1 (SP2013)
testCourse1.2|Course.1.2.SP2013|Test Course 1.2 (SP2013)
Postcondition
The COURSE records for Course.1.1.SP2013 and Course.1.2.SP2013 are retained and not affected.
The COURSE record for Course.2.1.SP2013 is marked as disabled or ready for purging per integration configuration.
If other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration.
Course availability
The COURSE availability setting allows for an account in LEARN to be visible to Students (available) or not visible (unavailable). Note that this is not the same as disabling a course, which not only makes the course unavailable to students and instructors, but also means it is not available to additional operations such as membership management. The addition of this data feed header does not impact the above-demonstrated usage of STORE, COMPLETE REFRESH, COMPLETE REFRESH BY DATA SOURCE for creating COURSE records.
Note the default integration settings when an AVAILABILITY setting is not provided is for the object to be made available on creation/update operations.
Example: Course account availability
Your SIS controls LEARN access availability and the data feed indicates the availability setting on users to control when they have access to Learn and you wish to control this access setting on PERSON create/update.
Prerequisite
None.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
AVAILABILITY_IND
Solution
Add the AVAILABLE_IND header to your data feed and provide the single character of Y for available and N for unavailable.
Postcondition
STORE
Only the COURSE records for Course.1.1.SP2013 and Course.1.2.SP2013 are updated (they were previously created) and Course.5.1.FA2013 is created.
COMPLETE REFRESH
The COURSE records for Course.1.1.SP2013 and Course.1.2.SP2013 are updated (they were previously created) and Course.5.1.FA2013 is created. All other records will be disabled or marked for purging due to their absence in the above data feed.
COMPLETE REFRESH BY DATA SOURCE
The COURSE records for Course.1.1.SP2013 and Course.1.2.SP2013 are updated (they were previously created) and Course.5.1.FA2013 is created with availability explicitly set to Y making the courses available.
If other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration. COMPLETE REFRESH BY DATA SOURCE only operates on records of the integration data source.
Disabling course records
Disabling a COURSE record in Learn makes it inaccessible to all users (disabled status overrides the availability setting) and also makes the record inaccessible for UI operations - eg: you cannot manage a disabled COURSE via the UI. Additionally, to purge a record from Learn that record must first be disabled.
Disabling a record and subsequent purging removes all references to that record from Learn - it is recommended that purging of disabled records take place only after a period of time as determined by your business and or legal practices which may otherwise require a record of activity.
Disabling of records may follow two models: Disabling through feed data exclusion on REFRESH operations, and Disabling through use of the feed header ROW_STATUS.
The above COURSE operations using REFRESH operations demonstrate Disabling thru exclusion, the following case and example demonstrate through the use of ROW_STATUS.
Example: Disabling course records
Policy dictates that you entirely remove courses from Learn after a period of 5 years (unlike making the record unavailable which only restricts visibility). If you are using STORE operations, to purge a course you must explicitly disable the course by using the ROW_STATUS header. This is also useful in manual operations outside the scope of SIS feeds.
Prerequisite
Course exists or course will be created and row_status set as indicated in data feed.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
ROW_STATUS
Solution
Add the ROW_STATUS header to your data feed and provide ENABLED for enabled and DISABLED for disabled.
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME|ROW_STATUS
testCourse1|Course.1.1.SP2013|Test Course 1.1 (SP2013)|enabled
testCourse1.2|Course.1.2.SP2013|Test Course 1.2 (SP2013)|enabled
testCourse5.1|Course.5.1.FA2013|Test Course 5.2 (FA2013)|disabled
Postcondition
STORE
Only the COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated with the ROW_STATUS explicitly updated.
COMPLETE REFRESH
The COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated with the ROW_STATUS explicitly updated. All other records are will be disabled or marked for purging due to their absence in the above data feed.
COMPLETE REFRESH BY DATA SOURCE
The COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated, with the ROW_STATUS explicitly updated.
If other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration. COMPLETE REFRESH BY DATA SOURCE only operates on records of the integration data source.
Course merge
Course Merge provides the ability to present multiple sections of a course as a single course in Learn. This is accomplished by merging the courses in parent-child relationships. The parent is presented to users as the course in which they are participants and the child sections/courses take the memberships etc. Instructors then only need to provide content to a single course and the Learn system maintains a pairing with the SIS in regard to course identifiers facilitating communication of course-related information between Learn and the SIS.
Example: Course merge
The English Department has four entry level courses that require multiple sections each to accommodate their desired student to instructor ratio. They wish to provide a single course for the instructors to manage their course.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
MASTER_COURSE_KEY
Solution
Add the ROW_STATUS header to your data feed and provide ENABLED for enabled and DISABLED for disabled.
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME|MASTER_COURSE_KEY
testCourse1|Course.1.1.SP2013|Test Course 1.1 (SP2013)|
testCourse1.2|Course.1.2.SP2013|Test Course 1.2 (SP2013)|testCourse1
testCourse5.1|Course.5.1.FA2013|Test Course 5.2 (FA2013)|
Leaving MASTER_COURSE_KEY empty means there is no 'parent' for the course or to put another way the course is not merged with a parent course.
Postcondition
STORE
Only the COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated, establishing a parent-child relationship between Course.1.1.SP2013 (parent) and Course.1.2.SP2013.
COMPLETE REFRESH
The COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated, establishing a parent-child relationship between Course.1.1.SP2013 (parent) and Course.1.2.SP2013. All other records are will be disabled or marked for purging due to their absence in the above data feed.
COMPLETE REFRESH BY DATA SOURCE
The COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated, establishing a parent-child relationship between Course.1.1.SP2013 (parent) and Course.1.2.SP2013.
If other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration. COMPLETE REFRESH BY DATA SOURCE only operates on records of the integration data source.
Course template
Course templates are used at the time a course is created to provide consistency in look and feel, gradebooks, content structure, etc. A course created using a template is a mirror of that template with the exception of the specific course settings as presented in the COURSE data feed. You may use any course for this purpose.
Templates are only used when a course is created, thus you cannot create a course and then provide a template via an update feed.
Example: Course template
The Math Department requires that all math courses are of a similar look and feel with similar content layout and populated with some core content.
Prerequisite
A course shell has been created which contains the required content, layout, and settings.
Minimum data feed requirements
EXTERNAL_COURSE_KEY
COURSE_ID
COURSE_NAME
TEMPLATE_COURSE_KEY
Solution
Add the TEMPLATE_COURSE_KEY header to your data feed and provide the external_course_key of the course representing the template.
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME|TEMPLATE_COURSE_KEY
testCourse1|Course.6.1.SP2013|Test Course 1.1 (SP2013)|testCourseTemplate.SP2013
testCourse5.1|Course.5.1.FA2013|Test Course 5.2 (FA2013)|
Leaving TEMPLATE_COURSE_KEY empty means that course does not require a template.
Postcondition
STORE
Only the COURSE record for Course.6.1.SP2013 and Course.5.1.FA2013 are created or updated, with Course.6.1.SP2013 being created using the testCourseTemplate.SP2013 template.
COMPLETE REFRESH
Only the COURSE record for Course.6.1.SP2013 and Course.5.1.FA2013 are created or updated, with Course.6.1.SP2013 being created using the testCourseTemplate.SP2013 template. All other records are will be disabled or marked for purging due to their absence in the above data feed.
COMPLETE REFRESH BY DATA SOURCE
Only the COURSE record for Course.6.1.SP2013 and Course.5.1.FA2013 is created or updated, with Course.6.1.SP2013 being created using the testCourseTemplate.SP2013 template.
If other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration. COMPLETE REFRESH BY DATA SOURCE only operates on records of the integration data source.
Complete example
Combining the above headers into a single file we can address most of the use cases at once.
Prerequisite
For the template copy operation to be successful, testCourse8 must not have been previously created.
To maintain server speed and prevent unnecessary database job queues, only two course copy jobs can run concurrently in the cluster. You can find this setting in config/service-config.properties if you wish to change it: blackboard.service.queuedtaskmanager.param.num-concurrent-copies.background=2
EXTERNAL_COURSE_KEY|COURSE_ID|COURSE_NAME|AVAILABLE_IND|ROW_STATUS|MASTER_COURSE_KEY|TEMPLATE_COURSE_KEY
testCourse8|Course.8.SP2013|Test Course 8 (SP2013)|Y|ENABLED||testCourseTemplate.SP2013
testCourse8.1|Course.8.1.SP2013|Test Course 8.1 (SP2013)|Y|ENABLED|testCourse8|
testCourse8.2|Course.8.2.SP2013|Test Course 8.2 (SP2013)|Y|ENABLED|testCourse8|
testCourse9.1|Course.9.1.FA2013|Test Course 5.2 (FA2013)|Y|DISABLED||
Postcondition
STORE
Only the COURSE records for Course.1.1.SP2013, Course.1.2.SP2013, and Course.5.1.FA2013 are created or updated with the following results:
All courses have availability set to Y, indicating the courses are available.
Course.1.1.SP2013 and Course.1.2.SP2013 are set to ENABLED, while Course.5.1.FA2013 is DISABLED.
Establish a parent-child relationship between Course.1.1.SP2013 (parent) and Course.8.1.SP2013 and Course.8.2.SP2013.
On creating Course.8.SP2013, the contents and settings from the template testCourseTemplate.SP2013 are copied into Course.8.SP2013.
COMPLETE REFRESH
The same results as the store operation with the addition that all other records will be disabled or marked for purging due to their absence in the data feed.
COMPLETE REFRESH BY DATA SOURCE
The same results as the store operation with the addition that if other COURSE records are managed by this integration they will NOT be disabled or purged due to their absence in the above data feed unless they have the same data source as specified by the integration. COMPLETE REFRESH BY DATA SOURCE only operates on records of the integration data source.
Learn more
Snapshot Flat File Header Descriptions
