Workday SOAP API Extraction Guide: Pagination, Throttling, Retries and Restart Safety

TL;DR / Key Takeaways

• Page Workday Web Services with Page and Count in Response_Filter, and pin As_Of_Entry_DateTime on every page. Workday's own reference says to, so the result set stays the same between requests.
• A throttled WWS SOAP call comes back as HTTP 500, and so does every SOAP fault. Read the fault body before deciding to retry: validation and authentication faults fail fast, a bare 500 or a Server fault gets exponential backoff with jitter.
• Restart safety is yours to build. Checkpoint after each committed page, resume only while Workday's 2-hour paging cache is likely still there, otherwise replay the window, and upsert by WID so a replay cannot duplicate a row.
• Before you mark a window complete, check that the pages delivered exactly Total_Results distinct workers with no WID on two pages. The runnable example shows four broken extractions that the check catches.

A restart-safe Workday SOAP extraction pins one entry moment for the whole result set, commits a checkpoint with each page, retries only the failures that can succeed later, and reconciles every window by WID before moving on. This guide builds that for Get_Workers on Workday Web Services (WWS) v47.0, the current version as of October 11, 2026, and tests it against a synthetic service that fails on purpose. Every Workday behavior it relies on links to the Workday page it came from; everything Workday does not document is labeled as this example's own policy.

If you would rather have this built and supported against your tenant, that is our custom Fivetran SDK connector service.

What is documented and what is not

Nothing in this guide was run against a real Workday tenant. The request and response element names come from Workday's public v47.0 schema; the behavior of the service in the example is synthetic.

Run the example

The example is standard-library Python 3.10 or newer. It has no credentials, makes no network calls, and contains no real people: every worker, WID and timestamp is generated.

mkdir workday-soap-extraction && cd workday-soap-extraction
base=https://itsmorethansoftware.com/examples/workday-soap-extraction
for f in README.md soap.py extract.py destination.py reconcile.py \
    synthetic_workday.py demo.py test_extraction.py test_negative_controls.py; do
  curl -fsSO "$base/$f"
done
python3 -m unittest -v
python3 demo.py

$ python3 -m unittest
.....................
----------------------------------------------------------------------
Ran 21 tests in 0.642s

OK

Extraction architecture

The extraction runs one window per scheduled run. The first run takes a full snapshot; every later run reads the transaction log from where the last one finished.

scheduler ──► extract.py ──► soap.py ──► Get_Workers (WWS v47.0)
                 │               │
                 │               └── classify faults: retry, or fail fast
                 │
                 ├── per page: rows + load log + checkpoint, one transaction
                 │                        │
                 │                        ▼
                 │              destination (table keyed by WID)
                 │
                 └── per window: pages and Total_Results check, then advance

Each window is defined by three timestamps that never change once the window starts:

The lag keeps Updated_Through safely behind Workday's clock; Workday rejects a request where Updated Through is later than the current moment. Setting the entry moment equal to Updated_Through makes each window a consistent picture as of its end, and the next window starts exactly there. Workday describes the bounds as the earliest and latest moment data was entered, so a change at the boundary instant can be returned by both windows; upsert by WID absorbs that.

The request

This is the request soap.py builds for page 1 of the October 10 window in the example. Count is 2 only to keep the sample short; the extractor defaults to Workday's default of 100.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:wd="urn:com.workday/bsvc">
  <soapenv:Header>
    <wsse:Security soapenv:mustUnderstand="1"
        xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <wsse:UsernameToken>
        <wsse:Username>ISU_USERNAME@TENANT_NAME</wsse:Username>
        <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">REDACTED</wsse:Password>
      </wsse:UsernameToken>
    </wsse:Security>
  </soapenv:Header>
  <soapenv:Body>
    <wd:Get_Workers_Request wd:version="v47.0">
      <wd:Request_Criteria>
        <wd:Transaction_Log_Criteria_Data>
          <wd:Transaction_Date_Range_Data>
            <wd:Updated_From>2026-10-09T12:00:00Z</wd:Updated_From>
            <wd:Updated_Through>2026-10-11T00:00:00Z</wd:Updated_Through>
          </wd:Transaction_Date_Range_Data>
        </wd:Transaction_Log_Criteria_Data>
      </wd:Request_Criteria>
      <wd:Response_Filter>
        <wd:As_Of_Entry_DateTime>2026-10-11T00:00:00Z</wd:As_Of_Entry_DateTime>
        <wd:Page>1</wd:Page>
        <wd:Count>2</wd:Count>
      </wd:Response_Filter>
      <wd:Response_Group>
        <wd:Include_Reference>true</wd:Include_Reference>
        <wd:Include_Personal_Information>false</wd:Include_Personal_Information>
        <wd:Include_Employment_Information>true</wd:Include_Employment_Information>
        <wd:Include_Compensation>false</wd:Include_Compensation>
        <wd:Include_Organizations>false</wd:Include_Organizations>
        <wd:Include_Roles>false</wd:Include_Roles>
      </wd:Response_Group>
    </wd:Get_Workers_Request>
  </soapenv:Body>
</soapenv:Envelope>

What each part comes from:

• Version. wd:version="v47.0" on the request element, and the same version at the end of the endpoint path. v47.0 shipped with Workday 2026R2 on September 18, 2026 (version directory). The directory lists every supported version back to v9, so pin one and upgrade on purpose.
• Endpoint. The WSDL does not carry your tenant's address. Microsoft's Workday provisioning guide documents the format as https://<host>.workday.com/ccx/service/<tenant>/Human_Resources/v<version> (Microsoft Learn). Confirm the host for your tenant with your Workday administrator.
• Authentication. A WS-Security UsernameToken for an integration system user, with the user name formatted as name@tenant (same Microsoft guide; namespace and PasswordText type from the OASIS UsernameToken Profile 1.0). The values here are placeholders. Load the real ones from a secret store at send time and never log the envelope.
• Element order. The binding is document/literal, so order matters: Request_References, Request_Criteria, Response_Filter, Response_Group (Human_Resources v47.0 WSDL).
• Response group. Without one, Get_Workers returns reference, personal, employment, compensation, organization and role data by default. Ask only for what you load. It cuts payload size and keeps personal data you do not need out of your pipeline.
• Full snapshot. The first run sends no Request_Criteria at all, so it returns every worker as of the pinned entry moment.

The response

This response is synthetic. It was produced by synthetic_workday.py for the request above, not by Workday. It uses v47.0 element names and keeps only the elements the example reads; a real response carries much more.

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
  <env:Body>
    <wd:Get_Workers_Response xmlns:wd="urn:com.workday/bsvc" wd:version="v47.0">
      <wd:Response_Filter>
        <wd:As_Of_Effective_Date>2026-10-10</wd:As_Of_Effective_Date>
        <wd:As_Of_Entry_DateTime>2026-10-10T17:00:00-07:00</wd:As_Of_Entry_DateTime>
        <wd:Page>1</wd:Page>
        <wd:Count>2</wd:Count>
      </wd:Response_Filter>
      <wd:Response_Results>
        <wd:Total_Results>12</wd:Total_Results>
        <wd:Total_Pages>6</wd:Total_Pages>
        <wd:Page_Results>2</wd:Page_Results>
        <wd:Page>1</wd:Page>
      </wd:Response_Results>
      <wd:Response_Data>
        <wd:Worker>
          <wd:Worker_Reference>
            <wd:ID wd:type="WID">00000000000000000000000000000007</wd:ID>
            <wd:ID wd:type="Employee_ID">SYN-0007</wd:ID>
          </wd:Worker_Reference>
          <wd:Worker_Descriptor>Synthetic Worker 0007</wd:Worker_Descriptor>
          <wd:Worker_Data>
            <wd:Worker_ID>SYN-0007</wd:Worker_ID>
            <wd:Employment_Data>
              <wd:Worker_Status_Data>
                <wd:Active>false</wd:Active>
              </wd:Worker_Status_Data>
            </wd:Employment_Data>
          </wd:Worker_Data>
        </wd:Worker>
        <wd:Worker>
          <wd:Worker_Reference>
            <wd:ID wd:type="WID">00000000000000000000000000000042</wd:ID>
            <wd:ID wd:type="Employee_ID">SYN-0042</wd:ID>
          </wd:Worker_Reference>
          <wd:Worker_Descriptor>Synthetic Worker 0042 (renamed)</wd:Worker_Descriptor>
          <wd:Worker_Data>
            <wd:Worker_ID>SYN-0042</wd:Worker_ID>
            <wd:Employment_Data>
              <wd:Worker_Status_Data>
                <wd:Active>true</wd:Active>
              </wd:Worker_Status_Data>
            </wd:Employment_Data>
          </wd:Worker_Data>
        </wd:Worker>
      </wd:Response_Data>
    </wd:Get_Workers_Response>
  </env:Body>
</env:Envelope>

Two details are worth copying into a real parser. The entry moment comes back in Pacific time: Workday documents that WWS returns datetimes as UTC-07:00 or UTC-08:00 depending on the API version, and that integration system users retrieve data on Pacific time (limits page). Compare instants, not strings. And key rows by the WID ID, which identifies the worker instance, rather than by display values.

Pagination and the response filter

The Get_Workers reference defines paging entirely in Response_Filter and Response_Results:

Three things in that table drive the design.

Pin the entry moment. Without As_Of_Entry_DateTime, each page defaults to "now", so a hire entered between page 1 and page 2 can shift every later page. The example's first negative control does exactly that with 10 workers per page: one worker is delivered twice, the last worker is never delivered, and Total_Pages from page 1 is now one short.

Read the totals from the response. The Total_Pages description says "A page of data in a WWS has a 100 entry maximum", while Count accepts up to 999. The two statements disagree. The example never computes the page count itself; it uses Total_Pages from page 1 and checks every later page against it.

Respect the paging cache. Workday's limits page says page 1 often takes longer because Workday gathers all results and caches them, so avoid a strict timeout on page 1. The cache is kept 2 hours between paged requests and 30 minutes after you request the last page. The example gives page 1 a 600-second timeout and later pages 120 seconds. Those are illustrative values, not measurements.

The page loop in extract.py fetches, checks, then commits the page and its checkpoint together:

    while window["total_pages"] is None or number <= window["total_pages"]:
        page = call_with_retries(
            lambda: fetch_page(service, window, number, count=count, timeouts=timeouts),
            label=f"page {number}",
            max_attempts=max_attempts,
            base_delay=base_delay,
            max_delay=max_delay,
            rng=rng,
            sleep=sleep,
            log=log,
        )
        check_page(window, number, page)
        window = {
            **window,
            "pages_done": number,
            "total_results": page.total_results,
            "total_pages": page.total_pages,
            "last_page_at": iso(now()),
        }
        destination.commit_page(
            rows=page.workers,
            window_id=window["as_of_entry"],
            page=number,
            state={STATE_KEY: {"complete_through": cursor["complete_through"], "window": window}},
        )

check_page fails the run if a page reports a different Page, a different entry moment, a Page_Results that does not match the workers it contains, or totals that differ from page 1.

Throttling and retries

Workday's limits page is specific about SOAP. Under high load, Workday rejects WWS SOAP requests with HTTP 500 Internal Service Error, not 429. It says the 500 "can sometimes be caused by too many requests", and recommends an exponential back-off retry mechanism and caching. HTTP 429 is what REST and RaaS callers get, and what any integration gets when it generates "an excessive volume of invalid requests". That last point is a good reason never to retry a request Workday has already called invalid.

SOAP itself complicates the 500. SOAP 1.1 requires a server to return HTTP 500 for every SOAP fault (section 6.2), so a malformed request and an overloaded tenant can share a status code. The fault body tells them apart. The v47.0 WSDL declares three faults for Get_Workers: Validation_Fault, Processing_Fault and Authentication_Fault. SOAP 1.1 says a Client-class fault "should not be resent without change", while a Server-class fault "may succeed at a later point in time".

Workday does not document which Processing_Fault codes are transient. The example treats one as retryable only when its faultcode is Server-class, and caps it at the same 5 attempts as everything else. Log the Code and Detail_Message from the fault so you can tighten the rule for your tenant.

def classify_failure(status, headers, body):
    """Raise TransientError or PermanentError for a non-200 response."""
    if status == 429:
        raise TransientError("HTTP 429", _retry_after(headers))
    if status in (502, 503, 504):
        raise TransientError(f"HTTP {status}", _retry_after(headers))
    if status == 500:
        fault = _fault(body)
        if fault is None:
            # Workday: "The 500 error code can sometimes be caused by too many requests."
            raise TransientError("HTTP 500 without a SOAP fault")
        code, details, message = fault
        fault_class = code.split(":")[-1].split(".")[0]
        if "Validation_Fault" in details or "Authentication_Fault" in details or fault_class == "Client":
            raise PermanentError(f"SOAP fault {code} {details}: {message}")
        raise TransientError(f"SOAP fault {code} {details}: {message}")
    raise PermanentError(f"HTTP {status}")

The retry loop uses exponential backoff with full jitter: each delay is random between zero and a capped exponential bound, so parallel jobs do not retry in lockstep (AWS Architecture Blog). A Retry-After value replaces the computed delay.

def backoff_delay(attempt, *, base, cap, rng):
    """Full jitter: a random delay between 0 and the capped exponential bound."""
    return rng.uniform(0, min(cap, base * 2 ** (attempt - 1)))


def call_with_retries(request, *, label, max_attempts, base_delay, max_delay, rng, sleep, log):
    for attempt in range(1, max_attempts + 1):
        try:
            return request()
        except TransientError as exc:
            if attempt == max_attempts:
                raise ExtractionFailed(f"{label}: {exc}: still failing after {max_attempts} attempts") from exc
            if exc.retry_after is not None:
                delay = exc.retry_after
            else:
                delay = backoff_delay(attempt, base=base_delay, cap=max_delay, rng=rng)
            log(f"{label}: {exc}: attempt {attempt}/{max_attempts}, retrying in {delay:.1f}s")
            sleep(delay)
        except PermanentError as exc:
            raise ExtractionFailed(f"{label}: {exc}: not retryable") from exc

What Workday does not publish matters as much. The limits page gives request quotas for RaaS extracts by tenant size, but no requests-per-second or concurrency figure for WWS SOAP. Do not design around a number you found in a blog post. Run one extraction at a time per tenant unless you have measured otherwise, and let backoff absorb the rest. For its own scheduled EIB integrations, the same page recommends a random start time such as 9:10 instead of the top of the hour to avoid peak queuing; the reasoning carries over.

Restart-safe state

Retries cover a blip inside one run. When they are exhausted, the run fails and the next run has to pick up cleanly. Separate what Workday documents in the Get_Workers reference and the limits page from what you add.

Workday does not document what happens if you request page 4 after the paging cache has expired, or whether the order of results survives a rebuilt cache. The 90-minute threshold leaves margin inside the documented 2 hours, and the replay costs a few pages instead of a silent gap.

This is the checkpoint after run 1 of the demo fails on page 3:

{
  "workday_workers_v1": {
    "complete_through": null,
    "window": {
      "as_of_entry": "2026-10-09T12:00:00Z",
      "last_page_at": "2026-10-09T12:05:00Z",
      "pages_done": 2,
      "total_pages": 5,
      "total_results": 230,
      "updated_from": null,
      "updated_through": null
    }
  }
}

complete_through only moves after a whole window passes its check. The in-progress window holds everything needed to repeat the exact same query: the window bounds, the pinned entry moment, the page reached, and the totals page 1 reported. The state key is versioned so a later change to its shape can be migrated instead of misread, and read_state rejects any shape it does not recognize before a single request goes out.

If your warehouse cannot commit rows and state in one transaction, write the rows first and the checkpoint second. A crash between the two replays one page, and upsert by WID absorbs it. Never write the checkpoint first; the example's third negative control shows that loses the page in flight.

Reconciliation for missing and duplicate records

There are two checks, and they catch different failures.

The window check runs at the end of every window, using only the load log the extractor wrote. It needs no second call to Workday:

    problems = []
    # An empty result can report Total_Pages 0, but page 1 was still requested.
    expected_pages = list(range(1, max(total_pages, 1) + 1))
    if sorted(deliveries) != expected_pages:
        problems.append(f"pages {sorted(deliveries)} delivered, expected 1-{max(total_pages, 1)}")
    pages_by_key = {}
    for page, keys in deliveries.items():
        for key in keys:
            pages_by_key.setdefault(key, []).append(page)
    repeated = sorted(key for key, pages in pages_by_key.items() if len(pages) > 1)
    if repeated:
        problems.append(f"{len(repeated)} WIDs delivered on more than one page")
    if len(pages_by_key) != total_results:
        problems.append(f"{len(pages_by_key)} distinct WIDs delivered, Total_Results was {total_results}")
    return problems

The destination check compares the table with a source of truth by WID and reports missing, unexpected, duplicate and stale rows. In the example the synthetic service is the source of truth. Against a real tenant, use a periodic full snapshot pulled with its own pinned entry moment, and compare WID sets and a few key fields.

python3 demo.py runs the whole scenario. Page 3 of the snapshot returns HTTP 500 with no SOAP body five times:

run 1: full snapshot; page 3 returns HTTP 500 five times
  start snapshot as of 2026-10-09T12:00:00Z
  page 1/5: committed 50 workers
  page 2/5: committed 50 workers
  page 3: HTTP 500 without a SOAP fault: attempt 1/5, retrying in 0.6s
  page 3: HTTP 500 without a SOAP fault: attempt 2/5, retrying in 0.6s
  page 3: HTTP 500 without a SOAP fault: attempt 3/5, retrying in 5.2s
  page 3: HTTP 500 without a SOAP fault: attempt 4/5, retrying in 1.2s
  FAILED: page 3: HTTP 500 without a SOAP fault: still failing after 5 attempts
  reconciliation against the synthetic source as of 2026-10-09T12:00:00Z:
    expected workers:  230
    rows present:      100
    missing WIDs:      101, 102, 103, 104, 105 (+125 more)
    unexpected WIDs:   none
    duplicate WIDs:    none
    stale WIDs:        none
    result:            FAIL

A failing reconciliation is the correct result at that point: two pages were committed and three were not. The next run, ten minutes later, resumes the same pinned query at page 3:

run 2: ten minutes later
  resume as of 2026-10-09T12:00:00Z at page 3
  page 3/5: committed 50 workers
  page 4/5: committed 50 workers
  page 5/5: committed 30 workers
  complete through 2026-10-09T12:00:00Z: 230 workers in 5 pages
  reconciliation against the synthetic source as of 2026-10-09T12:00:00Z:
    expected workers:  230
    rows present:      230
    missing WIDs:      none
    unexpected WIDs:   none
    duplicate WIDs:    none
    stale WIDs:        none
    result:            OK

The third run reads the transaction log for October 10: one termination, one rename and ten new hires. It lands 12 workers and reconciles 240 of 240 with current values.

A check that has never failed proves little, so test_negative_controls.py implements four plausible bugs and asserts the checks catch each one:

The positive suite also covers a page served in a different order with matching totals, which only the window check can see, and a run whose window would end in the future, which the synthetic service rejects with Workday's documented validation message and the extractor does not retry. I also removed each safeguard from extract.py by hand, one at a time: the pin, the page check, the window check, the stale-checkpoint replay, the fault classification and Retry-After. The suite failed every time.

Monitoring and recovery

Workday's limits page notes that Workday does not log GET requests or unsuccessful requests, and suggests external integration HTTP headers so your requests can be traced in its server logs. Treat your own logs as the record of what the extractor asked for.

Log per page, never per worker, and never the envelope:

• Window bounds, pinned entry moment, page number, Total_Results, Total_Pages, and rows committed
• Each retry: classification, attempt number, delay
• Each permanent failure: the fault type, Code and Detail_Message, with no request body

Alert on these, in roughly this order:

Recovery is the same in every case. The checkpoint is the only thing the next run trusts, so fix the cause, decide whether to keep the in-progress window or clear it, and rerun. Clearing window and leaving complete_through alone replays that window from scratch with a new entry moment; upsert by WID keeps it safe.

If a production Workday pipeline already shows these symptoms and the cause is not obvious, that is what Pipeline Rescue is for: a fixed-scope diagnosis of the pipeline you already have.

Limitations

• Synthetic service. The example models Workday's documented paging and validation, not its real ordering, latency or throttling. Nothing was run against a tenant.
• Effective-dated changes. Get_Workers returns data "as of the dates in the response filter" (reference). A change entered today with an effective date next month comes back in today's window with today's values, and may not appear in a later transaction log window when it takes effect. A periodic full snapshot reconciliation is the backstop; verify the behavior in your tenant.
• Organization changes. Workday notes that changes to organization data such as name and code are not captured in the transaction log, so a worker-level window will not see them (Transaction_Log_Criteria).
• Rescinds and corrections are not modeled.
• Response size. Workday caps a web service response at 1 million instances and outbound responses at 2 GB (limits page). Pages of 100 stay far below that, but parse large pages incrementally, and follow the Python XML security guidance for the parser you use.
• No performance numbers. The clock is fake. The timeouts and backoff values are illustrative starting points to tune against your tenant.

Running this as a Fivetran connector

The extraction logic does not depend on any connector framework. To run it under the Fivetran Connector SDK, the checkpoint becomes Fivetran state and commit_page becomes upserts followed by a checkpoint. The Fivetran Connector SDK Production Guide: State, Retries and Restart Safety covers that side, with its own runnable restart test and the Fivetran behaviors it depends on. For slow APIs where checkpoint cadence dominates, see the CX Alloy connector notes.

Related reading

• Fivetran Connector SDK Production Guide: State, Retries and Restart Safety
• CxAlloy API Extraction Guide: Request Signing, Nested Pagination, Rate Limits and Reconciliation
• Fivetran custom connectors when native connectors are not enough
• Why AI projects need boring plumbing before they need agents

Sources

Workday documentation, checked October 11, 2026:

• Get_Workers operation, Human_Resources v47.0
• Human_Resources v47.0 WSDL
• Workday Web Services version directory
• Reference: Integrations, EIB, and Web Service Limits

Specifications and other references:

• SOAP 1.1, W3C Note
• OASIS Web Services Security UsernameToken Profile 1.0
• RFC 9110, HTTP Semantics: Retry-After
• Microsoft Learn: Configure Workday for automatic user provisioning
• AWS Architecture Blog: Exponential backoff and jitter
• Python XML security