Summary
As part of our broader initiative to move away from our legacy system to a more robust platform, we have been migrating components of our analytics setup in phases for the past few months (starting March 2025) to ensure stability. On May 24, 2025, we completed the final phase of this migration, transitioning the remaining components to a more modern and reliable infrastructure.
Following the migration, we observed a delay in the reflection of user progress data for five specific learning module types — Quick Update, ILT, Checklist, Quest, and Course — in analytics reports.
Impact
User progress data was not reflected for the affected modules in analytics reports. The impacted reports include;
- Out-of-the-box (OOTB) reports
- Self-Serve Reports (SSR)
- OData feeds
- Custom dashboards
The issue was limited strictly to the user’s progress data in analytics.
What Was Not Impacted
- The Rule Engine
- The progress status shown on the user dashboard
- Real-time tracking during user activity (e.g., viewing or completing modules)
Learner experience and actual learning progress within the platform were unaffected.
Root Cause
- As part of the migration effort, events from the legacy system were being transitioned into the new platform via a migration pipeline.
- However, due to historical lag issues in the legacy system, a subset of user progress events—specifically for five identified module types—had not yet been generated or processed within the legacy system. As a result, these events never reached the migration pipeline.
- Despite the pipeline appearing clear for an extended period, the contentious events remained unprocessed within the legacy system and had yet to enter the migration flow, revealing a hidden backlog that challenged initial expectations.
Fix Implemented
- After a thorough and rigorous analysis pinpointed the issue to the legacy system, a backfill process was promptly initiated to accurately identify and carefully ingest the missing user progress events into the new system.
- This process was completed on 29 May 2025, 11:10 am PT / 11:40 pm IST. All the user progress data should now be visible correctly on the platform.
Safeguards Going Forward
The new system features a reliable and observable data pipeline, built with enhanced fault tolerance and end-to-end monitoring. The pipeline is equipped with automated alerting to ensure data integrity, consistency, and anomaly detection, including scenarios such as missed events, duplicate records, and data lag.
We apologize for the inconvenience and remain committed to improving system reliability.