CD2 DAP Can't load scores table

[KarlSteinbreche]

Hello, 

I'm hoping someone has seen this and knows how to fix it.

I'm trying to load the scores table in DAP and this is repeatable for me:

Step 1: Drop the scores table
Step 2 Initialize the scores table:

...@ubuntu-vm-karl:~/bin$ dap initdb --namespace canvas --table scores
INFO:pysqlsync.postgres:connecting to postgres@<myip>:5432/postgres
INFO:pysqlsync.postgres:PostgreSQL version 16.0.3 final
INFO:pysqlsync.postgres:connecting to postgres@<myip>:5432/postgres
INFO:pysqlsync.postgres:PostgreSQL version 16.0.3 final
INFO:pysqlsync:synchronize schema with SQL:
CREATE TABLE "canvas"."scores" (
"id" bigint NOT NULL,
"created_at" timestamp, ...

... columns and comments

2024-07-19 03:33:04,129 - INFO - Query started with job ID: f4045d45-1771-48ba-91ad-03cf7506c3c7
2024-07-19 03:33:04,449 - INFO - Data has been successfully retrieved:
{"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7", "status": "complete", "expires_at": "2024-07-20T03:27:31Z", "objects": [{"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7/part-00000-1b875aee-3f51-4040-9183-51d8fa23adb2-c000.tsv.gz"}, {"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7/part-00001-1b875aee-3f51-4040-9183-51d8fa23adb2-c000.tsv.gz"}, {"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7/part-00002-1b875aee-3f51-4040-9183-51d8fa23adb2-c000.tsv.gz"}, {"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7/part-00003-1b875aee-3f51-4040-9183-51d8fa23adb2-c000.tsv.gz"}, {"id": "f4045d45-1771-48ba-91ad-03cf7506c3c7/part-00004-1b875aee-3f51-4040-9183-51d8fa23adb2-c000.tsv.gz"}], "schema_version": 1, "at": "2024-07-19T02:53:17Z"}
Killed

Whatever Killed means here, it creates the table with its schema, but there are no records

Step 3: try to synch it:

...@ubuntu-vm-karl:~/bin$ dap syncdb --namespace canvas --table scores
INFO:pysqlsync.postgres:connecting to postgres@<myip>:5432/postgres
INFO:pysqlsync.postgres:PostgreSQL version 16.0.3 final
INFO:pysqlsync.postgres:connecting to postgres@<myip>:5432/postgres
INFO:pysqlsync.postgres:PostgreSQL version 16.0.3 final
2024-07-19 03:33:54,900 - ERROR - table not initialized, use `initdb`


I'm using dap 1.1.0

Any ideas would be appreciated!!

[ColinMurtaugh · Accepted solution]

"Killed" means that your operating system killed the `dap initdb` process before it could complete. You'll need to do some digging to understand why, but my first guess is that your system doesn't have enough memory. 

https://linuxhandbook.com/oom-killer/

(You can't sync a table until it has been successfully initialized, so step 3 won't work until step 2 is fixed.)

--Colin

[KarlSteinbreche]

yes, I reprovisioned the instance w/ 32gb ram and it went through on the first try