Canvas Data 2 Schema Checking API?

[patdura]

Does Instructure have any plans to add a schema checking endpoint for Canvas Data 2?  Currently, it is a manual process to validate that all of the expected columns for each table have been successfully ingested into our data warehouse (by comparing it to published documentation). This is a highly time consuming process across approximately 90 tables. If there was an endpoint with a table name as an input and a list of columns/column types as output, that would allow us to automate validation to ensure that all columns are landing properly in our data warehouse.

If not, do you have any advice on how to best validate that all tables and columns have landed properly?

[LeventeHunyadi]

First, we have the DAP API schema endpoint

/query/{namespace}/table/{table}/schema

which returns the JSON schema for each table that you can directly validate the output against.

Second, we also publish a recommended table structure for database engines PostgreSQL, MySQL and Microsoft SQL Server.

In our own internal integration tests, we create the database structure as per the referenced SQL scripts, and then ingest data and verify that all output rows for all database tables are successfully inserted or updated. Should there be a schema violation, the respective INSERT INTO ... or UPDATE ... SQL statement would fail.