Multi-Table Support for Canvas Data will Streamline your Efforts with DAP CLI 1.4.0

[sgergely]

We’re excited to announce the release of DAP CLI 1.4.0, which significantly boosts efficiency and ease of use by introducing multi-table operations. This release addresses a major customer pain point by allowing operations on all or selected tables within a namespace in a single command, saving valuable time and reducing repetitive tasks and additional coding tasks.

What’s New?

Namespace-Level Operations

• Execute commands across all tables in a namespace with a single command:

dap initdb --namespace canvas --table all

• Optionally, specify a subset of tables for more targeted operations.

dap initdb --namespace canvas --table accounts,courses,collaborations

Supported Commands

• Snapshot: Generate snapshots for all or selected tables in a namespace.
• Incremental: Perform incremental updates for all or selected tables.
• Schema: Retrieve schemas for all or selected tables.
• Database Management: Easily manage databases with these commands:
  • initdb: Create schemas for all or selected tables.
  • syncdb: Synchronize data for all or selected tables.
  • dropdb: Drop schemas for all or selected tables.

Improved Error Handling

• Receive detailed logs indicating which tables were processed and which encountered issues.
• If there is an error with one or multiple tables, the application continues with the rest of the tables.

Use Cases

1. New User Convenience

Simplify the onboarding experience by allowing users to initialize, sync, or query all tables in a namespace at once.

Example: 

dap initdb --namespace canvas --table all

2. Bulk Table Management

Streamline operations for data engineers managing multiple tables.

Example: 

dap syncdb --namespace canvas_logs --table accounts,courses,collaborations

3. Enhanced Automation

Enable seamless integration into automated workflows that require data from multiple tables.

Example: Set up nightly incremental updates for all tables with 

dap incremental --namespace canvas --table all

Why This Matters

• Saves Time: Eliminate repetitive tasks and focus on deriving actionable insights from your data without hitting the API limits.
• Improves Workflow: Reduce complexity and simplify automation for data engineers and analysts.
• Addresses Feedback: This feature directly responds to customer requests (1, 2 and numerous solutions on github) for easier multi-table management.

Try it now! To update to DAP CLI 1.4.0, run:

pip install -U instructure-dap-client

For more details, check the documentation. As always, we look forward to hearing your feedback!