Custom Fivetran SDK connectors
Bespoke Fivetran connectors for sources with no native connector, built with the Connector SDK in Python and delivered with tests, retries, and a runbook.
A custom Fivetran connector moves data from a source Fivetran does not natively support into your warehouse, on the same schedule, monitoring, and destination as the rest of your pipelines. We build them with the Fivetran Connector SDK in Python, with tests, retries, and a runbook. This is for teams whose most important data sits behind an API that is not on the native connector list.
The problem
When there is no native connector, the data usually gets moved by hand or by a script somebody wrote once and nobody owns. Both fail quietly, and the failure surfaces as a number that looks wrong in a report three weeks later.
- Someone exports a CSV every Monday, and the process stops when that person is on holiday.
- A cron script works until the API adds a field, changes a nested object, or starts paginating differently.
- Rate limits get discovered in production, usually as a throttle or a block rather than a clean error.
- There is no incremental state, so every run is a full reload that gets slower every month.
- Nothing alerts. The pipeline is judged healthy because nobody has complained yet.
What we build
Schema and table design
Required tables, column names that mean something, primary keys, and composite keys where a parent-child API forces them. Decided before any code is written.
Pagination and checkpointing
Cursor or offset handling that survives a mid-sync failure. State is written so a restart resumes rather than starting the extract over.
Rate limits and retries
Backoff that respects the documented rate limits, retry behavior for transient failures, and a clear distinction between "try again" and "stop and alert".
Schema drift handling
Explicit decisions about which fields are required, which are optional, and what happens when the source adds, renames, or drops one. The connector should not crash on the first surprise.
Fixture-based tests
Tests against realistic recorded payloads, including the malformed and empty responses the API actually returns, not just the happy path from the docs.
Logging and alerting
Logs that explain what happened without dumping sensitive payloads, plus failure alerts wired into wherever your team already looks.
What you get at handoff
- Connector source in your repository, not ours.
- The test suite and the recorded fixtures it runs against.
- A runbook: how to deploy, how to re-sync, what each failure mode means.
- Written notes on the specific quirks of the API, which is usually the expensive knowledge.
- A walkthrough with whoever will own it.
Stack
Connector
Fivetran Connector SDK, Python
Source protocols
REST, SOAP, GraphQL, Webhooks, Flat file drops
Destinations
Snowflake, BigQuery, Postgres, Redshift
Downstream
dbt, Airflow, Dashboards
Operations
GitHub Actions, Structured logging, Failure alerting
Related services
FAQ
How do I know if I need a custom connector?
Check the native connector list first, and check whether a native connector covers the specific objects you need rather than just the platform. If the source is on the list and covers your objects, use it. A custom connector is the right answer when the data matters, the API exists, and nothing off the shelf reaches it.
How long does one take to build?
It depends almost entirely on the API, not on the volume of data. A clean REST API with sane pagination and good docs is fast. A nested API that requires a call per project, or a SOAP interface with large XML payloads and undocumented rate limits, takes considerably longer. We scope it after reading the API documentation and making real calls against it.
What happens when the source API changes?
The connector is built expecting that. Optional fields appearing or disappearing is handled without a crash; changes to required fields raise a clear error rather than silently writing nulls. The runbook covers what to do in each case, and the test fixtures make it fast to reproduce.
Can you work with SOAP APIs?
Yes. SOAP is verbose and unpleasant but still common in enterprise systems, and it is often where the payroll, worker, or finance data lives. The design concerns are the same as REST, with more care needed around payload size, namespaces, and logging XML safely.
Who owns the connector afterwards?
You do. The code lives in your repository, deploys through your workflow, and runs in your Fivetran account. The handoff includes the tests and the runbook so your team or another contractor can maintain it.
Do you support it after launch?
We can, on a retainer, and some clients prefer that for the first few months while the seasonal behavior of the API becomes clear. It is not a requirement. The deliverable is built to be maintained by someone else.
Ready to scope Fivetran SDK Connectors?
Send the workflow, tool stack, or reporting problem. We will tell you what should be automated, what should stay manual, and what is worth building first.