Integrating ClickHouse with Circleback
The Circleback registry item copies a small API reader and raw ClickHouse tables into a chkit project.
Install
Section titled “Install”bunx chkit add circlebackbunx chkit checkbunx chkit generate --name add_circlebackbunx chkit migrate --applybunx chkit ingest run --tag provider:circlebackSet CIRCLEBACK_API_KEY in the runtime environment before the ingestion run. Edit src/integrations/circleback/index.ts to select meetings or skip transcript requests.
| Resource | Default ClickHouse table | Records synced | API reference |
|---|---|---|---|
Meetings (meetings) | circleback_meetings_raw | Meeting metadata and notes, with a transcript where access permits it. | GET/meetings, GET/meeting/{meetingId}/transcript |
Sync behavior
Section titled “Sync behavior”Each new scan cycle reads all meetings with ownership=All, follows provider Link continuations, and rechecks every observed transcript. The meeting listing has no documented modification-time filter or durable cursor lifetime; interrupted cycles restart pagination from the first page and skip only parent enrichment already acknowledged by the destination in that cycle. Unfinished parents are replayed. Completing the cycle clears the acknowledged ID set, so the next run rechecks every accessible meeting. A meeting timestamp is not assumed to track delayed transcript availability.
Sink-acknowledged checkpoints include explicit source/filter scope, bounded active-cycle parent IDs, unavailable transcript diagnostics with distinct forbidden and not_found statuses, and the last fully completed cycle’s completedAt. Neither transcript 403 nor 404 is assumed to mean temporary processing. A new cycle retries listed meetings even when their metadata is unchanged. IDs absent from the listing remain unavailable diagnostics, not executable queued work; they are retried only when listed again or resolved by an explicit reconciliation policy. Recovery can defer updates to acknowledged parents until the next cycle. Listing permission failures fail the stream.
The editable maxRetainedMeetings setting bounds both recovery IDs and diagnostics at 10_000; exceeding the bound fails visibly while preserving acknowledged progress. Source/filter scope changes require a new stream identity.
bunx chkit ingest status --tag provider:circleback --jsonRemoved meetings remain stored; a full scan is not an atomic source snapshot. The raw tables preserve provider fields and require ClickHouse 25.3 or later. See the installed README and registry installation.
Test the reader
Section titled “Test the reader”bunx chkit add circleback --with-testsbun test src/integrations/circleback/tests/basic.test.ts