CD2 - Try to use dap initdb to get all tables but table argement is required

[SunnyYeung]

I am trying to populate a database with ALL the tables in the Canvas namespace.

I am following the instructions on this documentation to initialize my database using the initdb command.

Below is my CLI:

dap initdb --connection-string my_connection_string --namespace canvas

but I get an error saying the table argument is required.

dap initdb: error: the following arguments are required: --table

Is there a way to get all the tables instead of running the initdb command manually for the 89 tables on the Canvas namespace?

Thanks!

[robotcars]

The commands are a little confusing at first, because they use 'db', but they are single table only.

dap --loglevel info --logfile courses.log initdb --namespace canvas --table courses;

[SunnyYeung]

Hi Robotcars,

Thanks for replying.

I tried your code but I got an error:
2024-09-11 17:33:57,268 - ERROR - 404, message='Not Found', url='https://api-gateway.instructure.com/dap/query/canvas/table/courses;/schema'

Is the "--loglevel info --logfile courses.log" part supposed to enable a complete initialization of all the tables or is that your custom command to log the activity of this line?

Is there a way to download all the tables or looping though them is the only way?

Thanks,

Sunny

[SunnyYeung]

Below is my current workaround to initiate all the tables - using loop.

Note: Switch the keyword from initdb to syncdb to update the data. 

FOR %i IN (access_tokens,account_users,accounts,assessment_question_banks,assessment_questions,assignment_groups,assignment_override_students,assignment_overrides,assignments,attachment_associations,attachments,calendar_events,canvadocs_annotation_contexts,comment_bank_items,communication_channels,content_migrations,content_participation_counts,content_participations,content_shares,content_tags,context_external_tools,context_module_progressions,context_modules,conversation_message_participants,conversation_messages,conversation_participants,conversations,course_account_associations,course_sections,courses,custom_gradebook_column_data,custom_gradebook_columns,developer_key_account_bindings,developer_keys,discussion_entries,discussion_entry_participants,discussion_topic_participants,discussion_topics,enrollment_dates_overrides,enrollment_states,enrollment_terms,enrollments,favorites,folders,grading_period_groups,grading_periods,grading_standards,group_categories,group_memberships,groups,late_policies,learning_outcome_groups,learning_outcome_question_results,learning_outcome_results,learning_outcomes,lti_line_items,lti_resource_links,lti_results,master_courses_child_content_tags,master_courses_child_subscriptions,master_courses_master_content_tags,master_courses_master_migrations,master_courses_master_templates,master_courses_migration_results,originality_reports,outcome_proficiencies,outcome_proficiency_ratings,post_policies,pseudonyms,quiz_groups,quiz_questions,quiz_submissions,quizzes,role_overrides,roles,rubric_assessments,rubric_associations,rubrics,score_statistics,scores,submission_comments,submission_versions,submissions,user_account_associations,users,web_conference_participants,web_conferences,wiki_pages,wikis) DO dap --base-url https://api-gateway.instructure.com --client-id=client_id_here --client-secret=client_secret_here initdb --connection-string connection_string_here --namespace canvas --table %i 

 This list of tables can be created using the following code, then add the commas using Excel.

dap --base-url https://api-gateway.instructure.com --client-id=client_id_here --client-secret=client_secret_here list --namespace canvas 

[reynlds]

I solved this a while ago. We maintain a "management" VM for our Canvas instances, so we have an effective proxy for things like this:

https://github.com/reynlds-illinois/enterprise_learning_systems/blob/main/canvas_data_2/cd2_syncdb.s...

[sgergely · Accepted solution]

Hello everyone, I would like to let you know that we have released DAP CLI 1.4.0 where the main feature is that you can define all or multiple tables for one DAP CLI run:

dap initdb --namespace canvas --table all
OR
dap initdb --namespace canvas --table content_tags,courses,submissions

 I hope this helps reducing the complexity in your codebase and makes DAP CLI use easier.

[reynlds]

@sgergely Awesome update! Question: If I use the "all" option for tables, does it still process/log them one at a time?

[sgergely]

It sequentially processes the tables and logs them as well as it goes through all of them. So one at a time.

[ColinMurtaugh]

Hi @sgergely --

Thanks -- this is great! I noticed that the release notes have not been updated since version 1.2.0: 

https://data-access-platform-api.s3.amazonaws.com/client/RELEASE_NOTES.html

Could these be updated with the versions released since then?

Thanks!

--Colin

[sgergely]

Yes, because we have moved all of our documentation to here: https://developerdocs.instructure.com/services/dap

Please remove that old one from your bookmarks, I will remove those old documentation pages soon.

The release notes are also reconciled to here, so both CD2 API and CLI is posted here: https://community.canvaslms.com/t5/Canvas-Change-Log/2025-API-and-CLI-Change-Log/ta-p/626858 and in blogposts by me in the Product Blog. 

[ColinMurtaugh]

Ah - good to know. By the way, the changelog link from PyPi still points to the old version. 

--Colin