We are conducting the Instructure Data Questionnaire to inform our 2025 roadmap and understand the cloud adoption trends among our users. Your input will help us tailor features, optimize data services, and plan future enhancements. The questionnaire is anonymous, but there is an option to share your email for follow-up. Whether you’re using a cloud provider, data warehouse, or BI tool, your feedback is invaluable. Complete the survey to shape the future of Instructure’s data capabilities. Link to Questionnaire
Customer Reported an Edge Case for DAP Query API
When using the DAP Query API, understanding edge cases is crucial for ensuring smooth operations, especially when working with rarely updated tables like canvadocs_annotation_contexts. This blog post explores a unique edge case around query expiration and how to handle it.
The Edge Case Explained
Imagine a customer querying a rarely updated table. The query result is cached, and the expires_at field is set 24 hours from the original query creation time. However, if the same query is issued just before expiration and reused after it expires, the downloadable links may fail despite the data still existing in our system for debugging purposes. Here’s a UML diagram to illustrate the situation:
In this scenario, if a customer tries to retrieve downloadable links after the query expires (e.g., September 2, 00:01), the system will return a 404 error, even though the data still exists in our system for debugging purposes. The downloadable links are no longer valid because the query has technically expired. To handle this edge case gracefully, your application needs to be aware of the expires_at field and take action accordingly.
Importance of Using Snapshot Only Once
One of the core principles of using the DAP Query API effectively is utilizing snapshots only once and then relying on incremental updates to keep your data up-to-date. Snapshots are resource-intensive, both on the API and client side, and should only be used to initialize your data set. Once the snapshot is established, incremental queries should be used to fetch only the latest data changes. This approach minimizes resource consumption and ensures your data stays fresh.
Upcoming DAP CLI 1.3.0 Release
In the upcoming DAP CLI 1.3.0 release, we’re adding edge case handling for query expiration, as described above, directly into our CLI and library.
Along with this, the release will also introduce support for Microsoft SQL Server.
Upcoming Canvas Data 2 Online Meetup
The 4th meetup is already scheduled for Fri, Nov 21, 2024 05:30 PM CET / 11:30am EST / 8:30am PST, don't miss it, subscribe here!
The previous meetup slides and recorded video can be checked once the survey is filled.
Question for the comment section:
What would be your feedback if we would launch New Quizzes data through DAP Query API in an additional, separate namespace?
