Back to case studies

PredictAndProfit.io

A self-hosted trading suite for Kalshi weather and inflation markets, and the public website that documents it. Steve Farmer designed and built both, and sells the suite. This case study covers the engineering: release qualification, exchange reconciliation, a forecast-model audit, and the website's content discovery.

Measured evidence

Each figure states what was measured, the sample, the window, the source, and what it does not prove. The categories are kept separate: website engineering figures say nothing about forecast quality, and neither says anything about trading profit.

Website engineering
4 → 36

of 36 blog posts listed in the sitemap, before and after the generated sitemap replaced the hardcoded one

Metric
Blog post URLs in the sitemap, divided by Markdown posts in the content directory with valid frontmatter.
Sample
Before: 4 hardcoded post URLs against 36 posts in the repository. After: the sitemap is generated from the content directory at build time and listed all 36.
Window
The commit of 1 May 2026 that replaced the hardcoded sitemap.
Source
Predict & Profit website repository history (private): the sitemap source before and after that commit, and the content directory at that commit.
Does not prove
Measures whether posts were offered to crawlers. It does not show indexing, rankings, clicks, or traffic.
Website engineering
113

blog post URLs in the live sitemap on 11 October 2026, each with a lastmod date

Metric
Count of /blog/ URLs in the public sitemap that carry a lastmod value, and count of sitemap URLs ending in a trailing slash.
Sample
127 sitemap URLs in total: 14 site pages and 113 blog posts. All 113 posts have lastmod. 0 URLs end in a trailing slash. The RSS feed lists the same 113 posts.
Window
Single snapshot, fetched 11 October 2026.
Source
The public sitemap and RSS feed. predictandprofit.io/sitemap.xml
Does not prove
The count changes as posts are published. A sitemap entry is a crawl hint, not proof of indexing. lastmod is the post’s frontmatter publication date, not its last edit.
Forecast model quality
0.2858 vs 0.2439

Brier score of the retired weather model against a base-rate baseline (lower is better)

Metric
Brier score: mean squared error between forecast probability and the settled 0 or 1 outcome. The baseline always predicts the historical base rate. Scoring above the baseline means the model added no forecasting skill.
Sample
112 settled trades.
Window
About four months of the bot running on Kalshi, ending with the last trade settled on 7 August 2026. Published 12 August 2026.
Source
Steve Farmer’s published audit on DEV. I Audited My Trading Bot Using a Brier Score
Does not prove
Applies only to the retired model. It says nothing about the NBM-based model that replaced it, and it is not a measure of trading profit. Trade-level records are private, so the figure is self-reported.

The product

Predict & Profit is a source-code package for technically comfortable Kalshi users. Customers run it on their own infrastructure with their own account and API key. There is no subscription, licence server, or paid market-data requirement. The hard part is not placing orders. It is keeping two strategies, one account, and the exchange in agreement about what was actually filled, paid, and settled.

What Steve built

  • A Weather bot for Kalshi temperature markets and an Inflation bot for CPI, Core CPI, and PCE markets
  • One shared account-level risk engine, so neither strategy can spend the other’s budget
  • Order, fill, fee, and settlement reconciliation taken from the exchange’s own order records
  • Fail-closed halts on unresolved exposure, incomplete account snapshots, identity conflicts, and state mismatches
  • PostgreSQL for trading, risk, and dashboard state, so state survives restarts and upgrades
  • Immutable versioned installs with upgrade and rollback that leave trading history in place
  • A guided installer that finishes in dry mode, with live activation as a separate, explicit step
  • Migration tooling that validates an earlier installation read-only and reconciles with the exchange before activation
  • A monitoring dashboard that cannot place trades or bypass the risk controls

Source: the public project overview and the 3.0.0 release notes. The full source ships to customers and is not public.

Release qualification

A release does not ship until it passes qualification gates aimed at the failures that matter most for an automated trading system: accounting drift between two strategies, orders the system cannot prove it owns, and upgrades that corrupt history.

  • Fresh installation driven end to end against a disposable PostgreSQL cluster and a real Chromium browser.
  • Migration from supported earlier layouts, rehearsed against realistic historical data.
  • Acceptance run against the built release archive itself, not the working tree.
  • Weather and Inflation exercised together against one account under one risk engine.
  • Order identity bound to authenticated order evidence, with unavailable evidence left unresolved rather than guessed.
  • Fills, fill cost, and fees reconciled from the exchange record, failing closed when that evidence is missing.
  • Immutable release packaging that declares version, supported Python versions, upgrade sources, and rollback compatibility.

The validation page states that 3.0.0 is functionally identical to the release candidate that was qualified and then deployed to production. It also states the limit plainly: passing a test suite is not the same as proving a trading edge.

Auditing the forecast model

An earlier weather model was scored with a Brier score after its live trades settled. It scored worse than always predicting the base rate, so it was retired rather than tuned. Its probabilities were too narrow, and confidence drives position size, so the model traded too big when it was wrong.

The replacement prices temperature contracts from NOAA's National Blend of Models, which publishes calibrated, station-level guidance for the stations Kalshi settles against. Other weather models are used only as disagreement checks. The same audit found that the market scanner had been matching the wrong category name and silently scanning only part of the available series. That defect, settlement stations that did not match Kalshi's, and several ledger bugs were fixed in the rebuild. See the Brier score evidence for the figures and their limits.

Website and content discovery

The public site is a Next.js application. Its original sitemap was a hand-written list that included only a few of the blog posts, so most of the content was not offered to crawlers. It now builds the sitemap and RSS feed from the same post list. See the sitemap evidence for the before-and-after counts.

  • The sitemap is generated from the Markdown content directory at build time, with lastmod taken from each post’s frontmatter date.
  • Static site pages carry no lastmod, because they have no truthful per-page modification date (changed June 2026).
  • Canonical URLs have no trailing slash, and the www host redirects to the apex domain to avoid duplicate indexing (August 2026).
  • An RSS feed is generated from the same post list as the sitemap.
  • Blog posts are statically generated at build time.
  • The home page carries SoftwareApplication, SoftwareSourceCode, and Organization JSON-LD.
  • Most posts include TL;DR and Key Takeaways sections, and some use Markdown tables for comparisons.

What this case study does not claim

  • Trading performance. The public validation page publishes no performance claim for either strategy, because a responsible conclusion needs a meaningful, version-specific sample with reconciled fees and settlements.
  • Search or AI-search results. No dated Search Console export with impressions, clicks, and a measurement window was retained, so this page reports no click-through rates, traffic, rankings, audience geography, or AI-citation results.
  • Revenue, sales volume, or customer counts.
  • Execution quality. The validation page states it has not been measured well enough to compare passive bids with crossing the spread.

Trade-level records, balances, positions, and experimental runs stay in private monitoring and reconciliation systems, as the validation page explains.

Why this matters for clients

Predict & Profit shows the ItsMoreThanSoftware operating model in public: build the system, qualify the artifact that actually ships, reconcile against the system of record, audit the model against a baseline, and publish the limits next to the claims. The same habits apply to client data pipelines, automations, and websites.

  • Release gates run against the artifact that ships
  • Reconciliation against the system of record
  • Fail-closed defaults on ambiguous state
  • Upgrade and rollback without data rewrites
  • Model audits against a baseline
  • Generated sitemaps and feeds
  • Clean canonical URLs
  • Structured data that matches the page
  • Public limits next to public claims

Want the same discipline in your systems?

If you need a pipeline, automation, or website built so its results can be checked rather than taken on trust, start with a short project note.