Datasets & Feeds

23,371 problems and 11,033 opportunities, available in bulk. Take the whole thing, or filter it down before you pull.

Bulk export

The response streams, so it starts arriving immediately regardless of size. Anonymous callers get 1,000 rows per request; a free API key raises that to 50,000, which is the entire corpus in one call. The cap is about server cost, not access — the same rows are reachable unauthenticated through /api/v1/signals a page at a time.

Everything, as CSV

curl -L 'https://painsignal.net/api/v1/export?type=problems&format=csv' \
  -H 'X-API-Key: ps_live_…' -o painsignal-problems.csv

One industry, high severity only, as JSONL

curl -L 'https://painsignal.net/api/v1/export?type=problems&format=jsonl&industry=Plumbing&min_severity=4' \
  | jq -r '.pain_statement'

Opportunities, straight into pandas

import pandas as pd

df = pd.read_csv(
    "https://painsignal.net/api/v1/export"
    "?type=opportunities&format=csv&limit=1000"
)
df.nlargest(10, "opportunity_score")[
    ["app_brief_title", "signal_count", "build_complexity", "url"]
]

Parameter	Values
`type`	`problems` (default) or `opportunities`
`format`	`csv` (default) or `jsonl`
`industry`, `category`	Exact names from /industries and /categories
`min_severity`	1–5
`limit`	Up to 1,000 anonymous / 50,000 keyed

A note on the CSV: cells beginning =, +, - or @ are prefixed with an apostrophe, because the source text is arbitrary user writing and spreadsheets execute those as formulas. If you are parsing programmatically rather than opening in Excel, use jsonl and skip the question entirely.

Feeds

RSS 2.0 and JSON Feed, 50 most recent items, refreshed every 30 minutes. Swap .xml for .json on any of them.

Any industry or category works, not just the ones listed: /feeds/industry-{slug}.xml and /feeds/category-{slug}.xml, using the same slug that appears in the URL of the corresponding industry page.

Field dictionary

What each column actually means, including where it is a machine judgement rather than an observation. Read this before you build anything that treats a score as ground truth.

Field	Type	Meaning
`pain_statement`	text	A plain-language restatement of the problem, written by the extraction model from the original post. Not a verbatim quote.
`industry`	text	Assigned per problem by an LLM, then consolidated nightly. Not drawn from a fixed list — new industries appear as the data warrants.
`job_role`	text	Who has the problem, e.g. Owner, Office Manager, Field Technician.
`problem_category`	text	What kind of problem it is, e.g. Scheduling, Invoicing, Compliance. Same dynamic taxonomy treatment as industry.
`severity_score`	integer 1–5	How much pain the person expressed. 1 is a mild annoyance; 5 means the problem is costing them money, customers or staff.
`opportunity_score`	integer 0–100	Composite ranking signal combining severity, willingness to pay, how many independent people reported it, and confidence. Use it for sorting, not as a forecast.
`willingness_to_pay`	none \| implied \| explicit	explicit means someone said outright they would pay for a fix. implied means they described paying for the status quo, or the cost of not fixing it. This is the closest thing in the corpus to a demand signal, and explicit is rare — about 4% of rows.
`existing_solutions_mentioned`	string[]	Tools the person named as what they currently use. Raw and unnormalised here — QuickBooks Online and QBO appear separately. The Tool Gap Index folds them; this column does not.
`implied_software_gaps`	string[]	What a tool would have to do that nothing they have does. The most directly actionable field for a builder.
`workarounds_described`	string[]	The manual process they use instead. A long workaround list is a strong signal that the problem is real and unsolved.
`gap_description`	text	Prose summary of what is missing from the current solutions.
`solution_*`	text / string[]	The AI-generated app brief: title, description, features, target users and revenue model. This is a suggestion, not a validated business plan.
`confidence_score`	float 0–1	How confident the extractor was that it read the post correctly. Filter above 0.7 if precision matters more than recall.
`signal_count`	integer	Opportunities only. How many independent people reported the underlying problem. The single best proxy for demand in the dataset.
`build_complexity`	integer 1–5	Opportunities only, and only on analysed clusters. LLM judgement of engineering effort. 1 is a weekend; 5 needs a team.
`revenue_potential`	integer 1–5	Opportunities only, on analysed clusters. LLM judgement of how much money the problem is worth solving for.
`competition_level`	low \| medium \| high \| unknown	Opportunities only. unknown means the cluster has not been analysed yet, not that competition is absent.
`cluster_id`	uuid	Which opportunity this problem belongs to. Null when the problem has not been clustered.

What this data is not

Not a representative sample. It is what people chose to complain about in public, on the platforms we listen to. Loud industries are over-represented; industries whose workers do not post online barely appear.
Not verbatim. Every pain_statement is an LLM restatement of a source post, and source attribution is deliberately not published.
Scores are judgements. severity_score, build_complexity and revenue_potential come from a language model reading text. They are useful for ranking and filtering. They are not measurements.
Coverage is uneven. All 23,371 problems carry an app brief, but only 1,313 of 11,033 opportunities have been through the analytics pass, so build_complexity and friends are null on the rest.
Time is reported-at, not collected-at. Analytics use the source post’s own date. The created_at column in the export is classification time and will make every trend look like it started in April 2026.

Licence

Free to use, including commercially. A link back to painsignal.net is appreciated. Please do not republish the corpus wholesale as a competing index. Full terms are on the terms page.