PredictAndProfit.io
A self-hosted trading suite for Kalshi weather and inflation markets, and the public website that documents it. Steve Farmer designed and built both, and sells the suite. This case study covers the engineering: release qualification, exchange reconciliation, a forecast-model audit, and the website's content discovery.
Measured evidence
Each figure states what was measured, the sample, the window, the source, and what it does not prove. The categories are kept separate: website engineering figures say nothing about forecast quality, and neither says anything about trading profit.
of 36 blog posts listed in the sitemap, before and after the generated sitemap replaced the hardcoded one
- Metric
- Blog post URLs in the sitemap, divided by Markdown posts in the content directory with valid frontmatter.
- Sample
- Before: 4 hardcoded post URLs against 36 posts in the repository. After: the sitemap is generated from the content directory at build time and listed all 36.
- Window
- The commit of 1 May 2026 that replaced the hardcoded sitemap.
- Source
- Predict & Profit website repository history (private): the sitemap source before and after that commit, and the content directory at that commit.
- Does not prove
- Measures whether posts were offered to crawlers. It does not show indexing, rankings, clicks, or traffic.
blog post URLs in the live sitemap on 11 October 2026, each with a lastmod date
- Metric
- Count of /blog/ URLs in the public sitemap that carry a lastmod value, and count of sitemap URLs ending in a trailing slash.
- Sample
- 127 sitemap URLs in total: 14 site pages and 113 blog posts. All 113 posts have lastmod. 0 URLs end in a trailing slash. The RSS feed lists the same 113 posts.
- Window
- Single snapshot, fetched 11 October 2026.
- Source
- The public sitemap and RSS feed. predictandprofit.io/sitemap.xml
- Does not prove
- The count changes as posts are published. A sitemap entry is a crawl hint, not proof of indexing. lastmod is the post’s frontmatter publication date, not its last edit.
Brier score of the retired weather model against a base-rate baseline (lower is better)
- Metric
- Brier score: mean squared error between forecast probability and the settled 0 or 1 outcome. The baseline always predicts the historical base rate. Scoring above the baseline means the model added no forecasting skill.
- Sample
- 112 settled trades.
- Window
- About four months of the bot running on Kalshi, ending with the last trade settled on 7 August 2026. Published 12 August 2026.
- Source
- Steve Farmer’s published audit on DEV. I Audited My Trading Bot Using a Brier Score
- Does not prove
- Applies only to the retired model. It says nothing about the NBM-based model that replaced it, and it is not a measure of trading profit. Trade-level records are private, so the figure is self-reported.
The product
Predict & Profit is a source-code package for technically comfortable Kalshi users. Customers run it on their own infrastructure with their own account and API key. There is no subscription, licence server, or paid market-data requirement. The hard part is not placing orders. It is keeping two strategies, one account, and the exchange in agreement about what was actually filled, paid, and settled.
What Steve built
- A Weather bot for Kalshi temperature markets and an Inflation bot for CPI, Core CPI, and PCE markets
- One shared account-level risk engine, so neither strategy can spend the other’s budget
- Order, fill, fee, and settlement reconciliation taken from the exchange’s own order records
- Fail-closed halts on unresolved exposure, incomplete account snapshots, identity conflicts, and state mismatches
- PostgreSQL for trading, risk, and dashboard state, so state survives restarts and upgrades
- Immutable versioned installs with upgrade and rollback that leave trading history in place
- A guided installer that finishes in dry mode, with live activation as a separate, explicit step
- Migration tooling that validates an earlier installation read-only and reconciles with the exchange before activation
- A monitoring dashboard that cannot place trades or bypass the risk controls
Source: the public project overview and the 3.0.0 release notes. The full source ships to customers and is not public.
Release qualification
A release does not ship until it passes qualification gates aimed at the failures that matter most for an automated trading system: accounting drift between two strategies, orders the system cannot prove it owns, and upgrades that corrupt history.
- Fresh installation driven end to end against a disposable PostgreSQL cluster and a real Chromium browser.
- Migration from supported earlier layouts, rehearsed against realistic historical data.
- Acceptance run against the built release archive itself, not the working tree.
- Weather and Inflation exercised together against one account under one risk engine.
- Order identity bound to authenticated order evidence, with unavailable evidence left unresolved rather than guessed.
- Fills, fill cost, and fees reconciled from the exchange record, failing closed when that evidence is missing.
- Immutable release packaging that declares version, supported Python versions, upgrade sources, and rollback compatibility.
The validation page states that 3.0.0 is functionally identical to the release candidate that was qualified and then deployed to production. It also states the limit plainly: passing a test suite is not the same as proving a trading edge.
Auditing the forecast model
An earlier weather model was scored with a Brier score after its live trades settled. It scored worse than always predicting the base rate, so it was retired rather than tuned. Its probabilities were too narrow, and confidence drives position size, so the model traded too big when it was wrong.
The replacement prices temperature contracts from NOAA's National Blend of Models, which publishes calibrated, station-level guidance for the stations Kalshi settles against. Other weather models are used only as disagreement checks. The same audit found that the market scanner had been matching the wrong category name and silently scanning only part of the available series. That defect, settlement stations that did not match Kalshi's, and several ledger bugs were fixed in the rebuild. See the Brier score evidence for the figures and their limits.
Website and content discovery
The public site is a Next.js application. Its original sitemap was a hand-written list that included only a few of the blog posts, so most of the content was not offered to crawlers. It now builds the sitemap and RSS feed from the same post list. See the sitemap evidence for the before-and-after counts.
- The sitemap is generated from the Markdown content directory at build time, with lastmod taken from each post’s frontmatter date.
- Static site pages carry no lastmod, because they have no truthful per-page modification date (changed June 2026).
- Canonical URLs have no trailing slash, and the www host redirects to the apex domain to avoid duplicate indexing (August 2026).
- An RSS feed is generated from the same post list as the sitemap.
- Blog posts are statically generated at build time.
- The home page carries SoftwareApplication, SoftwareSourceCode, and Organization JSON-LD.
- Most posts include TL;DR and Key Takeaways sections, and some use Markdown tables for comparisons.
What this case study does not claim
- Trading performance. The public validation page publishes no performance claim for either strategy, because a responsible conclusion needs a meaningful, version-specific sample with reconciled fees and settlements.
- Search or AI-search results. No dated Search Console export with impressions, clicks, and a measurement window was retained, so this page reports no click-through rates, traffic, rankings, audience geography, or AI-citation results.
- Revenue, sales volume, or customer counts.
- Execution quality. The validation page states it has not been measured well enough to compare passive bids with crossing the spread.
Trade-level records, balances, positions, and experimental runs stay in private monitoring and reconciliation systems, as the validation page explains.
Why this matters for clients
Predict & Profit shows the ItsMoreThanSoftware operating model in public: build the system, qualify the artifact that actually ships, reconcile against the system of record, audit the model against a baseline, and publish the limits next to the claims. The same habits apply to client data pipelines, automations, and websites.
- Release gates run against the artifact that ships
- Reconciliation against the system of record
- Fail-closed defaults on ambiguous state
- Upgrade and rollback without data rewrites
- Model audits against a baseline
- Generated sitemaps and feeds
- Clean canonical URLs
- Structured data that matches the page
- Public limits next to public claims
Want the same discipline in your systems?
If you need a pipeline, automation, or website built so its results can be checked rather than taken on trust, start with a short project note.