What we do
Most organizations have plenty of feedback. Few can say how far it can be trusted.
Complaints, survey comments and reviews arrive every day. They are useful, but they are not a fair sample. Some customers write often. Some never write at all. A theme that appears a lot is not always the thing that matters most.
We read that feedback as measurement. Before any analysis, we agree the decision and what exactly is being estimated. You receive estimates with their limits stated, labels that have been independently checked, and a plain account of what the evidence supports, what it does not, and what would change the answer.
Who the feedback speaks for
Illustration, not data. Feedback comes from some customers and not others. An audit states which group each figure describes.
Who it is for
Organizations that collect feedback and need to act on it.
Public services
Teams responsible for customer service in municipalities, agencies and other public bodies.
- Feedback you may hold
- Complaint and contact-centre records, service requests, open-ended survey comments.
- Decisions it can inform
- Whether a complaint theme is a real pattern. Whether a service change shows up in what people report. Whether the evidence is strong enough to take to a committee.
Tourism and visitor experience
Destinations, attractions, parks and associations who look after the visitor experience.
- Feedback you may hold
- Online reviews, visitor surveys, comment cards, enquiry and complaint emails.
- Decisions it can inform
- Whether a visitor concern is widespread or voiced by a few. Whether reviews and surveys tell the same story. Whether there is enough evidence to support a funding or planning case.
An audit is a good fit when
- One person owns the decision and is able to act on it.
- The decision has a date.
- The feedback already exists, and you have permission to share it.
It is probably not a fit when
- There is no decision in view yet.
- You need a live dashboard or ongoing monitoring.
- The feedback has not been collected, or cannot be shared.
How the audit works
One decision and one body of feedback, in four steps.
The scope is agreed in writing at the start: one decision, one source of feedback and one language, unless we agree otherwise. The tags show who does the work at each step.
Agree the decision
We write down the decision, who owns it, when it is due and the options being considered. We also agree exactly what is being estimated. This happens before any analysis.
- You bring
- The decision, its date and the options on the table.
- You get
- A short written scope that both sides sign.
YouResearcherCheck the data
We review whether your feedback can answer the question. We look at what is missing, what is duplicated and who the records can speak for. Sometimes the data cannot answer the question. If so, we explain why in a short memo and stop there.
- You bring
- One export of feedback records under a signed data agreement. Volume figures too, where you have them.
- You get
- A clear answer: go ahead, or a memo explaining why not.
ResearcherCode and measure
We remove personal details and fix a codebook. Researchers code reference samples by hand. AI-assisted coding is tested against those samples before it is used on the full set. Then we estimate, with the assumptions written down.
- You bring
- The categories that matter most to your decision.
- You get
- Every record coded to the same fixed scheme, with each code linked to the words it rests on.
AI-assistedResearcherReport what holds
We report what the evidence supports, what it does not, and what would change the answer. We go through the findings with you in a meeting and check back after the decision is made.
- You bring
- Your questions, and one round of comments on the draft.
- You get
- A decision report, the coded data, the codebook and the Evidence and Decision Statement.
ResearcherYou
What an audit does not include
We do not build dashboards or collect new data. We do not claim cause and effect from a single export, rank what to fix first, or report findings about named individuals.
Why the results can be trusted
Every claim is checked, and every check is written down.
Measurement validity
We check that each category measures what it is meant to measure, and we state which customers or records each figure describes.
Reliability checks
Two researchers code the reference samples independently. Disagreements are resolved and agreement is reported for each category.
Human review of AI-assisted work
AI helps with coding at scale. A senior researcher frames the question, accepts each category, interprets the results and signs the report.
A logged record
Each AI-assisted step is logged and reviewed. Every figure in the report traces back to that record, and headline figures are recomputed a second way.
Evidence and Decision Statement
Delivered with every audit · Outline- Consistency
- How consistently the feedback was coded.
- Validity
- Whether the codes measure what they were meant to measure.
- Coverage
- Which customers or records each estimate describes.
- Decision support
- Which decisions the evidence can support, and which it cannot.
- Still open
- The uncertainty that remains, and what information would change the answer.
The audit record
Example layout- Decision agreed in writing✓ signed
- Personal details removed✓ checked
- Codebook fixed✓ reviewed
- Reference samples hand-coded✓ two coders
- AI-assisted coding tested✓ reviewed
- Report figures recomputed✓ matched
“The evidence is not sufficient” is an allowed finding. We would rather say so than overstate.