Custom Fivetran SDK connectors: a pragmatic guide

TL;DR / Key Takeaways

• A custom connector is production software, not a one-off script.
• Pagination, state, schema drift, and retries need design before coding.
• Tests should cover realistic payloads and failure modes.
• A connector is not done until it has logging, docs, and ownership.

Custom Fivetran SDK connectors are useful when the source system matters and the standard connector does not exist.

They are also real production software. Treating them like a quick script is how data teams inherit brittle pipelines.

The connector is only one part

A working connector needs to extract data, define schema, handle pagination, preserve state, and survive API failures. That is the baseline.

Production work also includes:

• Clear naming for tables and columns
• Tests against realistic API payloads
• Retry behavior for temporary failures
• Logging that explains what happened
• Handling for deleted, changed, or missing fields
• Documentation for the next person who owns it

Schema drift is normal

APIs change. Optional fields appear. Nested objects become inconsistent. A connector should expect that reality instead of crashing on the first surprise.

This does not mean swallowing every error. It means deciding which fields are required, which fields are optional, and what should happen when the source changes.

Operational support matters

The best connector is boring after launch. It syncs, alerts when something is wrong, and leaves enough context for a human to fix the issue.

That means tests, runbooks, version control, and a simple deployment workflow are part of the deliverable. Without those pieces, the connector is not finished.

Practical checklist

• Confirm there is no native connector that already solves the problem.
• Define required tables, fields, and sync state.
• Plan pagination, rate limits, and retry behavior.
• Add fixture-based tests.
• Document how failures should be handled.

Related Reading

• Fivetran custom connectors when native connectors are not enough
• Why AI projects need boring plumbing before they need agents