Uploaded image for project: 'Teacher Preparation Data Model'
  1. Teacher Preparation Data Model
  2. TPDMX-202

Deep dive in multi year ODS for TPDM

    XMLWordPrintable

Details

    • Story
    • Resolution: Won't Do
    • Major
    • None
    • None
    • None

    Description

      In CSU (TPDM v0.8) the decision was made to use a multi year ODS instead of SQL Server temporal tables for the implementation.

      So far the following issues were found with this approach:

      • There's a need to track historical data for tpdm.TeacherCandidateIndicator . This issue was resolved by customizing the TPDM model (v0.8) and making TeacherCandidateIndicator.BeginDate a required field. There's a related ticket for it: https://tracker.ed-fi.org/browse/TPDMX-189
      • There's a need to track historical data for edfi.EducationOrganizationIndicator. CSU decided longitudinal information for EdOrg Indicators was low priority so a fix was not pursued. 
      • Data is being loaded onto the ODS exclusively thru the API. The issue is it overwrites any existing data in the ODS by either POSTING or PUTTING the payload. This means yearly data sources can't be ingested and full historical files must be used instead which leads to issues such as ETLs taking too long to run, additional storage costs, etc. This was not an issue when using SQL Server Temporal tables since the data is moved automatically to the historical tables when the API overwrites record of the table.

       

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              lizbrunet Elizabeth Brunet
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Salesforce