CxAlloy API Extraction Guide: Request Signing, Nested Pagination, Rate Limits and Reconciliation

TL;DR / Key Takeaways

• CxAlloy's TQ API signs every request with HMAC-SHA256 over a Unix timestamp, plus the exact JSON body on a POST. A timestamp older than 1 hour is rejected, so sign each attempt, retries included.
• A request returns at most 500 records. CxAlloy documents no total for GET listings and no sort order, so end a listing at the first page shorter than the page size you asked for, and compare against perpage when you set it, not against 500.
• The budget is 10,000 requests per hour per API key, and checklist lines are listed one checklist at a time. Count requests before you count records: in the runnable example below, 162 of 173 requests are line listings.
• There is no modified-since filter on these endpoints. Take a full snapshot with a checkpoint per page, upsert on a composite key that carries the parent IDs, and mark rows deleted only after a complete run.

A reliable CxAlloy extraction lists projects once, pages each project's records with a checkpoint after every committed page, signs every attempt again, stops cleanly when its request budget runs out, and reconciles the run before anything is marked deleted. This guide builds that against CxAlloy's public API reference for TQ API v1, checked October 12, 2026, and tests it against a synthetic service that fails on purpose. Every CxAlloy behavior it relies on links to CxAlloy's documentation; everything CxAlloy does not document is labeled as this example's own policy.

If you would rather have this built and supported against your CxAlloy account, that is our custom Fivetran SDK connector service.

What is documented and what is not

Nothing in this guide was run against a real CxAlloy account. The paths, parameters, headers and field names come from CxAlloy's public API reference and its OpenAPI file; the data and behavior of the service in the example are synthetic.

Run the example

The example is standard-library Python 3.10 or newer. It has no credentials, makes no network calls, and contains no customer data: every project, record and ID is generated, and the signing key is a labeled placeholder.

mkdir cxalloy-api-extraction && cd cxalloy-api-extraction
base=https://itsmorethansoftware.com/examples/cxalloy-api-extraction
for f in README.md cxalloy.py extract.py destination.py reconcile.py \
    synthetic_cxalloy.py demo.py test_extraction.py test_negative_controls.py; do
  curl -fsSO "$base/$f"
done
python3 -m unittest -v
python3 demo.py

$ python3 -m unittest
......................
----------------------------------------------------------------------
Ran 22 tests in 7.896s

OK

How the API is shaped

CxAlloy TQ is organized around projects. Almost every read takes a project_id, and some records only exist below another record. The example extracts four resources that show each shape:

GET  /project                          one request, not paged
  └─ per project
       ├─ GET  /equipment?project_id=&page=              paged, 500 per page
       ├─ POST /checklist {"project_id", "page", "perpage"}  paged, perpage set
       │    └─ per checklist delivered
       │         └─ GET /checklistline?project_id=&checklist_id=&page=
       └─ each page: rows + load log + checkpoint, one transaction
run end: run check ──► soft-delete rows not seen ──► mark complete

Three things in the OpenAPI file make this harder than one list endpoint:

• Children are listed per parent. /checklistline requires checklist_id, so the request count grows with the number of checklists, not the number of lines.
• Children do not name their parents. The documented Checklistline record has checklistline_id, line_number, answer and similar fields, but no project_id or checklist_id. The extractor has to add them from the request it made.
• The API may grow under you. CxAlloy's versioning notes say new fields and endpoints can appear and field order can change without a new version. Store each record as it arrived and do not depend on field order.

With the stop rule in the next sections, a listing of n records at page size s costs ⌊n/s⌋ + 1 requests: the last page is short, or empty when n is an exact multiple of s. The example's synthetic account shows how quickly the per-checklist term dominates:

Those counts are asserted by the test suite. They are request counts, not timings.

Authentication and request signing

The API reference requires an account-level or project-scoped API key, created by an account admin in TQ. The key has an identifier and a secret, and the secret is shown once (API Credentials). Every request carries five headers: Content-Type: application/json, cxalloy-identifier, cxalloy-signature, cxalloy-timestamp and user-agent.

The signature is HMAC-SHA256 keyed with the secret. For a GET, the message is the Unix timestamp. For a POST with options in the body, it is the JSON body with the timestamp appended. The timestamp must match the header, and one more than 1 hour old is rejected.

This is the request demo.py --sample builds for page 1 of one project's checklists. perpage is 2 only to keep the sample short. The identifier and the key behind the signature are synthetic placeholders.

POST /api/v1/checklist HTTP/1.1
Host: tq.cxalloy.com
Content-Type: application/json
cxalloy-identifier: SYNTHETIC-IDENTIFIER
cxalloy-signature: 6e2c1372826e8146bb9403bcea62ef79869f325151bab0e9377e2fb0b3d2bd82
cxalloy-timestamp: 1791813600
user-agent: cxalloy-extraction-example/1.0

{"project_id":101,"page":1,"perpage":2}

Two details decide whether the signature matches:

• Sign the bytes you send. CxAlloy's PHP sample signs json_encode($body). Python's json.dumps adds spaces after , and : by default, so a body re-serialized after signing will not match. The example serializes once, compactly, and sends exactly those bytes.
• Sign every attempt. A retry that reuses the first attempt's headers carries an old timestamp. With backoff, a paused run, or a slow page, that can cross the 1-hour limit.

def encode_body(options):
    """Serialize once, compactly. The bytes signed must be the bytes sent."""
    return json.dumps(options, separators=(",", ":")).encode()


def sign(signing_key, timestamp, body=None):
    message = (body or b"") + str(timestamp).encode()
    return hmac.new(signing_key.encode(), message, hashlib.sha256).hexdigest()

One inconsistency is worth knowing about. The reference and the example export script use a Unix timestamp in seconds (PHP time()), while the pre-request script for the Postman examples uses JavaScript Date.now(), which is milliseconds. The example follows the reference and uses seconds.

For extraction, create a dedicated key with Read access, scoped to the projects you need. CxAlloy documents that Read keys can only call read endpoints; /checklist and /history are reads sent as POST, so confirm a Read key can call them before you rely on it. Keep the secret in a secret store, load it at send time, and never log headers.

The response

This response is synthetic. It was produced by synthetic_cxalloy.py for the request above, not by CxAlloy. It follows the documented checklistFilterResponse wrapper and uses documented field names, but keeps only a few fields; a real checklist carries more.

{
  "records": [
    {
      "checklist_id": 101001,
      "project_id": 101,
      "name": "PFC-0001",
      "status": "Not Started",
      "type_name": "Pre-Functional",
      "date_created": "2026-03-01T09:00:00Z"
    },
    {
      "checklist_id": 101002,
      "project_id": 101,
      "name": "PFC-0002",
      "status": "Not Started",
      "type_name": "Pre-Functional",
      "date_created": "2026-03-01T09:00:00Z"
    }
  ],
  "total_count": 120,
  "page": 1,
  "description": "120 Checklists"
}

CxAlloy describes total_count as "the total records returned", which does not say whether it counts this page or the whole listing, and the documented example pairs a total_count of 20 with a description of 393 checklists. The example does not use it. CxAlloy's PHP sample treats GET listings such as /equipment as a bare array and counts it; no count field is documented for them.

Pagination and nested listings

The API reference documents the page size and the parameter, not the end of a listing. The example uses the rule CxAlloy's own example export script uses: keep requesting while a page comes back full, and stop at the first page that is shorter.

def is_last_page(records, unit):
    """A page shorter than the page size requested ends the listing. An empty page counts."""
    return len(records) < unit.size

What that rule needs from you:

• Compare against the size you requested. /checklist, /test and /issue accept perpage. If you send "perpage": 100 and keep the vendor sample's === 500 check, the first page of 100 looks short and the listing stops after one page. The example's first negative control does exactly that and loses 20 checklists and their 200 lines.
• Expect one extra request on an exact multiple. A listing of exactly 500 records costs two requests; the second returns nothing. The example has one of each so the suite covers both. CxAlloy does not document what a page past the end returns; the synthetic service returns an empty list.
• Do not page the endpoints that do not page. The reference lists 22 endpoints that do not handle pagination, including /project, /checklisttemplate and the status lookups.
• Refuse a page that repeats. If page 2 returns the same keys as page 1, the page parameter is not being honored. The extractor fails the run instead of looping, and caps each listing at 1,000 pages.
• Do not depend on order. The reference calls page an offset and documents no sort order. Inserts or deletes during a listing can move a record across a page boundary, which the reconciliation section below shows.

Nested listings are discovered from data the same run loaded. After a project's checklists are committed, the extractor reads their IDs back from its own load log and lists each checklist's lines, so a restart rebuilds exactly the same list of listings. The key carries the parents the API leaves out:

    def key(self, record):
        if self.resource == "checklistline":
            # The API returns lines without their parent IDs; the key adds them.
            return (self.project, self.checklist, record["checklistline_id"])
        return (self.project, record[f"{self.resource}_id"])

CxAlloy does not say whether checklistline_id is unique across checklists. The composite key is correct either way, and it keeps lineage in the destination for joins.

The include shortcut. POST /checklist accepts "include": ["lines"], and CxAlloy's Postman pre-request script uses it. That would cut line requests from one per checklist to one per checklist page. The reference does not say whether included lines are complete for long checklists, and it documents two constraints: un-upgraded projects cannot include lines without sections, and upgraded projects have no sections to include. The is_upgraded field on /project tells you which applies. Before switching, compare a sample of included lines against /checklistline for the same checklists in your account. The example uses the documented per-checklist listing.

Incremental extraction and the history endpoint

CxAlloy does not document a modified-since filter on /equipment, /checklist or /checklistline. Since February 6, 2026, list endpoints return date_created, which finds new records but not changed ones. That leaves two options.

Full snapshot, every run. This is what the example does. Each run lists everything, upserts on the composite key, and after the run passes its check, soft-deletes rows it did not see. It is simple and complete, and its cost is the request count above, every run.

Change detection with /history. The OpenAPI file documents POST /history per project, with filters for table, type (INSERT, UPDATE, DELETE), after_date and, since April 28, 2026, after_id. Each record has a history_id, the table and key of the changed row, and before, after and changed as JSON strings. Storing the highest history_id seen per project gives you a cursor: the next run asks for after_id and learns which projects, checklists and equipment changed.

Treat that as an optimization on top of snapshots, not a replacement, because the documentation leaves gaps:

• The before and after objects use database column names such as fk_type and dt_modified, not the API's field names, so you cannot apply them as rows. Use them to decide what to re-fetch.
• The history table names do not map one to one to endpoints; the history enum lists checklistsectionline, for example.
• The response type enum includes SOFT DELETE, which the request filter does not offer.
• The documented response schema is a single History object, with no wrapper or count, and no order is documented.
• Nothing says every change is recorded, or how long history is kept.

A practical pattern is to use /history to skip unchanged projects on most runs and take a full snapshot on a fixed schedule as the backstop. Verify the history behavior in your account first.

Rate limits, retries and backoff

The API reference states the limit plainly: "Maximum requests per hour for any api key is 10000." It does not say whether the hour is fixed or rolling, which status code a rejected request gets, or whether a Retry-After header is sent. It does not publish a concurrency limit.

Because the limit is hourly, a backoff measured in seconds does not help once it is spent. The example therefore separates two kinds of failure:

If CxAlloy reports the limit with a status other than 429, the run retries or fails instead of pausing. The checkpoint is the same either way, so the next run still resumes at the right page; only the log line differs. Log the status and the first part of the body so you can tighten the rule once you have seen a real rejection.

The retry loop signs each attempt, counts it against the budget, and uses exponential backoff with full jitter so parallel jobs do not retry in lockstep (AWS Architecture Blog):

    def call(self, method, endpoint, params, options, *, label):
        for attempt in range(1, self.max_attempts + 1):
            if self.used >= self.max_requests:
                raise BudgetSpent(f"request budget of {self.max_requests} spent before {label}")
            request = build_request(
                method,
                endpoint,
                identifier=self.identifier,
                signing_key=self.signing_key,
                timestamp=int(self.now().timestamp()),
                params=params,
                options=options,
            )
            self.used += 1
            try:
                try:
                    status, headers, body = self.service.send(request, timeout=self.timeout)
                except (TimeoutError, ConnectionError) as exc:
                    raise TransientError(f"{type(exc).__name__}: {exc}") from exc
                if status != 200:
                    classify_failure(status, headers, body)
                return parse_records(body)
            except TransientError as exc:
                if attempt == self.max_attempts:
                    raise ExtractionFailed(f"{label}: {exc}: still failing after {self.max_attempts} attempts") from exc
                if exc.retry_after is not None:
                    delay = exc.retry_after
                else:
                    delay = backoff_delay(attempt, base=self.base_delay, cap=self.max_delay, rng=self.rng)
                self.log(f"{label}: {exc}: attempt {attempt}/{self.max_attempts}, retrying in {delay:.1f}s")
                self.sleep(delay)
            except PermanentError as exc:
                raise ExtractionFailed(f"{label}: {exc}: not retryable") from exc

Request budgeting and throughput

Throughput against this API is a budget problem before it is a speed problem. Some rules that follow from the documented limit:

• Count before you build. Use ⌊n/s⌋ + 1 per listing, summed over projects and checklists, with your real record counts. An account whose per-checklist line listings alone need more than 10,000 requests cannot finish a snapshot within one hour of the key's budget, however fast each request is.
• Retries spend budget. The example counts every attempt. A run that retries 4 times on one page has used 5 requests for it.
• Set a per-run budget below the key's limit. The example defaults to 2,000 requests per run, a fifth of the hourly limit, and pauses when it is spent. A key is shared by everything that uses it, and the extractor cannot see the others' usage, so give the extraction its own key.
• Do not add parallelism to escape the limit. CxAlloy does not publish a concurrency limit, and parallel workers spend the same hourly budget faster. Run one extraction per key, sequentially, unless you have measured otherwise with CxAlloy's agreement.
• Cut requests, not delays. Fewer requests come from larger pages (perpage up to 500), skipping unchanged projects with /history, and dropping listings nobody uses downstream.

The suite proves the pause behaves: with a budget of 100 the first run stops at 100 requests, the second finishes, and the two together use exactly the 173 of a clean run. With the synthetic service's hourly limit set to 120, the run pauses on the 429 and the next hour resumes.

Checkpointing and restart safety

Retries cover a blip inside one run. When they run out, the run fails, and the next run has to continue without losing or repeating work. The checkpoint, the load log and the rows for each page are committed in one SQLite transaction (SQLite UPSERT), so a crash can never leave the checkpoint ahead of the data.

This is the checkpoint after run 1 of the demo fails:

{
  "cxalloy_snapshot_v1": {
    "last_complete": null,
    "run": {
      "cursor": {
        "done": false,
        "page": 1,
        "unit": "checklistline/102/102040"
      },
      "projects": [
        101,
        102,
        103
      ],
      "run_id": "2026-10-12T14:00:00Z"
    }
  }
}

cursor names the last committed page of the listing in progress. projects is the list from /project at the start of the run, so a project added mid-run waits for the next snapshot instead of shifting the run under the cursor. last_complete moves only after a whole run passes its check. The state key is versioned, and the extractor rejects any shape it does not recognize before sending a request.

python3 demo.py runs the scenario. Page 2 of the last checklist's lines returns HTTP 503 five times:

run 1: snapshot; checklistline/102/102040 page 2 returns HTTP 503 five times
  start snapshot 2026-10-12T14:00:00Z: 3 projects
  checklistline/102/102040 page 2: HTTP 503: attempt 1/5, retrying in 0.6s
  checklistline/102/102040 page 2: HTTP 503: attempt 2/5, retrying in 0.6s
  checklistline/102/102040 page 2: HTTP 503: attempt 3/5, retrying in 5.2s
  checklistline/102/102040 page 2: HTTP 503: attempt 4/5, retrying in 1.2s
  FAILED: checklistline/102/102040 page 2: HTTP 503: still failing after 5 attempts
  requests this run: 175
  reconciliation against the synthetic source:
    table          expected  present  result
    project               3        3  OK
    equipment          1540     1540  OK
    checklist           160      160  OK
    checklistline      2732     2502  FAIL
      missing: 102/102040/102040501, 102/102040/102040502, 102/102040/102040503 (+227 more)
    result: FAIL

A failing reconciliation is the correct result at that point: one page was never committed. Ten minutes later, run 2 resumes at that page:

run 2: ten minutes later
  resume snapshot 2026-10-12T14:00:00Z at checklistline/102/102040 page 2
  complete: snapshot 2026-10-12T14:00:00Z, 167 listings, 3 requests this run
  requests this run: 3
  reconciliation against the synthetic source:
    table          expected  present  result
    project               3        3  OK
    equipment          1540     1540  OK
    checklist           160      160  OK
    checklistline      2732     2732  OK
    result: OK

The demo then counts what the checkpoint saved. These are synthetic request counts, not timings:

synthetic request counts
  clean snapshot, no faults:      173
    project 1, equipment 6, checklist 4, checklistline 162
  failed run, resumed at page:    175 + 3 = 178
  failed run, restarted from 1:   175 + 173 = 348

The resumed total is the clean run plus the 5 failed attempts. Restarting from page 1 would have spent another 170 requests repeating work that was already committed.

If your warehouse cannot commit rows and state together, write the rows first and the checkpoint second. A crash between the two replays work, and the upsert absorbs it; the suite moves a checkpoint back to an earlier page and reruns to prove it. Never write the checkpoint first: the example's second negative control shows that loses the page in flight.

Duplicate detection and reconciliation

There are two checks, and they catch different failures.

The run check reads only the load log the extractor wrote, so it needs no extra requests. Every listing the run should contain must be there, its pages must be 1 to n, every page but the last must be full, and the last must be short:

    problems = []
    for unit in expected_units(projects, deliveries):
        pages = deliveries.get(unit)
        if not pages:
            problems.append(f"{unit}: never listed")
            continue
        numbers = sorted(pages)
        if numbers != list(range(1, len(numbers) + 1)):
            problems.append(f"{unit}: pages {numbers} delivered, expected 1-{numbers[-1]}")
            continue
        if pages[numbers[0]][0] is None:
            if numbers != [1]:
                problems.append(f"{unit}: not paged, but pages {numbers} delivered")
            continue
        for number in numbers[:-1]:
            size, keys = pages[number]
            if len(keys) != size:
                problems.append(f"{unit}: page {number} has {len(keys)} of {size} records but was not the last page")
        size, keys = pages[numbers[-1]]
        if len(keys) >= size:
            problems.append(f"{unit}: stopped after page {numbers[-1]}, which was full")
    for unit in set(deliveries) - set(expected_units(projects, deliveries)):
        problems.append(f"{unit}: listed but not part of this run")
    return problems

The load log records the page size each request actually asked for, so a loop that compares against the wrong size is caught here. If one key arrives on two pages of the same listing, the extractor logs that the listing moved while it was paged. Upsert absorbs the repeat.

The destination check compares the live rows with a source of truth, table by table, on the composite key, and reports missing, unexpected, duplicate and stale rows. In the example the synthetic service is the source of truth. Against a real account, use a separate listing of keys, or the next full snapshot.

Offset paging has one failure that no count can see. If a record is deleted from page 1 after page 1 was read, the first record of page 2 slides onto page 1 and is never delivered. The run delivers the same number of records the listing now holds, so counts match, but one live record is missing and one deleted record is still present. The suite reproduces it: only the key-by-key check finds it, and the next complete snapshot repairs it. Because the soft-delete sweep runs only after a complete run, and a row that reappears is undeleted by the upsert, the damage lasts one run.

A check that has never failed proves little, so test_negative_controls.py implements four plausible bugs and asserts the checks catch each one:

I also removed five safeguards from a copy of extract.py, one at a time: re-signing per attempt, Retry-After, the page-repeat check, the run check, and the guard against starting two snapshots in the same second. The suite failed every time. That last guard exists because the deletion test found the bug: two runs with the same ID made the soft-delete sweep blind.

Monitoring and troubleshooting

The API reference documents no response headers for rate-limit usage, so your own logs are the record of what the extractor spent. Log per page, never per record, and never headers or bodies:

• Run ID, listing, page, page size requested, records committed
• Requests used this run against the run's budget
• Each retry: status, attempt number, delay
• Each pause or failure: status and the first 200 characters of the body

Alert on these:

Recovery is the same in every case: the checkpoint is the only thing the next run trusts. Fix the cause, decide whether to keep the run in progress or clear it, and rerun. Clearing run and leaving last_complete alone starts a new snapshot; the upsert keeps that safe.

If a production CxAlloy pipeline already shows these symptoms and the cause is not obvious, that is what Pipeline Rescue is for: a fixed-scope diagnosis of the pipeline you already have.

Known limitations

• Synthetic service. The example models CxAlloy's documented paths, signing, page size and hourly limit, not its real ordering, latency, error bodies or rate-limit status. Nothing was run against a CxAlloy account.
• Four resources. Issues, tests, files and assets other than equipment follow the same pattern but are not modeled. /test and /issue take perpage like /checklist.
• Full snapshots only. The example does not implement /history; the incremental extraction section lists what to verify first.
• Undocumented total_count. Its meaning is unclear and the example ignores it.
• Deletes during a listing can hide a live record until the next complete snapshot, as shown above.
• Project scope changes. If a key loses access to a project, the next complete run soft-deletes that project's rows. That is correct for the key, and may surprise the people reading the table.
• No performance numbers. The clock is fake. The backoff values, the 120-second timeout and the 2,000-request budget are illustrative starting points to tune against your account.

Running this as a Fivetran connector

The extraction logic does not depend on any connector framework. To run it under the Fivetran Connector SDK, the checkpoint becomes Fivetran state and each page commit becomes upserts followed by a checkpoint, with the composite keys declared as primary keys. The Fivetran Connector SDK Production Guide: State, Retries and Restart Safety covers that side, with its own runnable restart test and the Fivetran behaviors it depends on. For an enterprise SOAP API where the paging cache and fault bodies drive the design, see the Workday SOAP API Extraction Guide: Pagination, Throttling, Retries and Restart Safety.

Related reading

• Fivetran Connector SDK Production Guide: State, Retries and Restart Safety
• Workday SOAP API Extraction Guide: Pagination, Throttling, Retries and Restart Safety
• Fivetran custom connectors when native connectors are not enough
• Why AI projects need boring plumbing before they need agents

Sources

CxAlloy documentation, checked October 12, 2026:

• CxAlloy TQ API reference
• CxAlloy TQ API OpenAPI file
• CxAlloy Support: API Credentials
• CxAlloy Support: Example Export
• CxAlloy Support: Example Postman Requests

Specifications and other references:

• RFC 9110, HTTP Semantics: Retry-After
• AWS Architecture Blog: Exponential backoff and jitter
• SQLite: UPSERT