Healthcare Provider Intelligence

The work required a reliable view of healthcare organisations across more than 40 US metro areas and several UK regions. The useful facts existed, but they were scattered across maps, company databases, clinic websites and an existing CRM.

A researcher could assemble one record manually. Repeating that process across hundreds or thousands of organisations made coverage inconsistent, expensive and difficult to audit.

Organisation information was spread across maps, company databases, public clinic websites and an existing CRM. Records were duplicated, incomplete or out of date. Researching each organisation manually did not scale across more than 40 US metro areas and several UK regions.

I defined the research workflow, connected the source APIs, built the deduplication and rejection rules, wrote the web crawler, designed the structured extraction schema and integrated the approved output with Attio.

I also built the operational safeguards: confidence scoring, source retention, checkpoints, rate limits, retries and field-level failure handling. This public case study intentionally excludes individual-level contact handling and confidential implementation details.

1. Search for clinics through Google Places and Apollo across defined regions. Use field masks and pagination so large searches return only the data needed.
2. Normalise website domains and merge duplicates from different sources. Compare them with existing Attio records before creating anything new.
3. Reject parked domains, directory listings, veterinary practices and other false matches. Check website health concurrently across the remaining list.
4. Enrich organisations with employee count, estimated revenue and technology data from Apollo.
5. Map each clinic website with Firecrawl and a custom breadth-first crawler. The crawler handles standard links, JavaScript onclick targets and meta-refresh redirects.
6. Use roughly 60 rules to rank pages likely to contain services, ownership details and organisation information before scraping them.
7. Extract structured organisation facts from the selected pages. Each field includes its source and a confidence score. Retained analysis is anonymised and this case study omits any person-level detail.
8. Score record completeness with code so reviewers can focus on records with missing or conflicting evidence.
9. Write approved records to Attio through its API. If a field is invalid, remove that field, log the reason and retry the rest instead of losing the whole record.
10. Use checkpoints, token-bucket rate limits and retry rules so large regional batches can resume after an API or network failure.

The pipeline does not treat every search result as a healthcare provider. It rejects parked domains, directories, veterinary practices and other false matches before spending time on enrichment or extraction.

Each retained fact carries its source and a confidence score. The system prepares records for review rather than silently presenting uncertain data as fact. Invalid CRM fields are isolated and retried without discarding the rest of an otherwise useful record.

Each proposed organisation record includes normalised identity data, enrichment fields, relevant website findings, source links, confidence flags and a completeness score. Missing or conflicting evidence is made visible so review time is spent on genuine exceptions.

Manual research and CRM entry takes 15 to 25 minutes per clinic. The pipeline reduces this to two to four minutes of final review, saving about 80% of the time.

Across 1,000 clinics, that equates to roughly 250 hours saved. Records arrive with organisation details, source links and confidence flags ready for review.

Collecting more data did not automatically create a better record. Domain normalisation, duplicate detection, rejection rules and traceable sources created more value than adding another enrichment provider.

Large research jobs also need to be resumable. Checkpoints and isolated failures turned a fragile script into an operating system that can survive rate limits, malformed fields and temporary network problems without restarting an entire region.

This project demonstrates how I approach a high-friction operational workflow: define what a trustworthy output looks like, automate the repeatable stages, preserve the evidence and keep a human focused on the decisions that remain uncertain.