Fivetran custom connectors when native connectors are not enough

TL;DR / Key Takeaways

• Native connectors are best when they exist and cover the needed data.
• Custom connectors are useful when APIs, pagination, auth, or business rules are specific.
• Schema design, state management, and retry behavior matter more than the first API call.
• Production connectors need logging, tests, and clear ownership.

Fivetran is useful because it removes a lot of common data movement work. When a native connector exists and covers the data you need, use it.

When it does not, a custom SDK connector can be the right answer.

When native connectors are enough

Native connectors are usually the best choice when:

• The source system is supported.
• The required objects and fields are available.
• Sync behavior matches the reporting need.
• The cost and schedule are acceptable.
• The team does not need special business logic during extraction.

Do not write a custom connector just to feel more in control.

When custom SDK connectors make sense

Custom connectors make sense when the source is niche, internal, or unusually shaped.

Common reasons include:

• A vendor API is not supported.
• Authentication requires custom handling.
• Pagination is unusual.
• The API response needs business-specific interpretation.
• The data source is internal and important.

What to define before coding

Before writing code, define the tables, fields, sync state, and failure behavior.

This is the part that saves time later. The first API request is rarely the hard part. The hard part is keeping the sync correct after the API changes, a request fails, or a field disappears.

State, pagination, and schema drift

Every production connector needs a plan for state. The connector should know what it has already synced and how to resume safely.

Pagination needs the same treatment. Some APIs use cursors. Some use timestamps. Some use page numbers. Some return partial data without making the failure obvious.

Schema drift is normal. Decide what should fail fast and what should be treated as optional.

Testing and operational handoff

A useful connector should include:

• Fixture-based tests
• Logging
• Retry handling
• Clear deployment steps
• Runbook notes
• Ownership boundaries

If nobody knows how to debug it, the connector is not done.

Practical checklist

• Confirm native connector coverage first.
• Document the source API limits and auth method.
• Define destination tables and required fields.
• Design sync state before coding extraction.
• Test pagination and failure paths.
• Write a short handoff runbook.

Related Reading

• Custom Fivetran SDK connectors: a pragmatic guide
• Why AI projects need boring plumbing before they need agents