A buyer’s checklist for data due diligence

A polished sample proves that a supplier can send a polished sample. It does not, by itself, establish the quality, coverage, or rights of the full product.

Start with a written use case. Without one, diligence becomes an unfocused list of questions and the cheapest product can look like the best deal.

Define the buying decision

Write what the data must enable, who will use it, which systems will receive it, and how success will be measured. Separate mandatory conditions from preferences.

A forecasting team may care about historical availability and revisions. An evaluation team may care about exposure and scoring. A customer-facing product may need redistribution rights. The supplier cannot price or support requirements you discover only after signing.

Verify the supplier and its authority

Confirm the contracting entity and the person authorized to negotiate. Ask how the dataset was collected and what permits its sale for your use. Request evidence appropriate to the asset and risk, with confidential material handled through a suitable process.

If the supplier combines sources, ask how it tracks the rights of each component. A single sentence saying “we own the data” does not explain an upstream license restriction or a customer confidentiality clause.

Keep legal and privacy review separate from statistical quality. A product can be accurate and still be unusable for the intended purpose.

Test sample representativeness

Ask how the sample was selected and compare its distributions with the full product’s disclosed coverage. Look at missingness, time ranges, source mix, rare cases, and known exclusions.

If possible, agree on a selection method before seeing the sample. For a table, you might request stratification by time and source. For labeled tasks, you might request separate results by class or difficulty. The method should suit the product, not a generic checklist.

Synthetic examples are useful for integration tests, but they do not establish real accuracy or coverage. Keep the two evaluation purposes separate.

Reproduce the important claims

Claim	Evidence to request	A test to consider
Broad coverage	Defined population, time period, and exclusions	Compare expected and observed segments
Low duplication	Definition and measured result	Check exact and task-relevant near duplicates
Accurate labels	Annotation method and review evidence	Independently review a suitable sample
Frequent updates	Delivery history and service description	Observe a trial refresh and late-data handling
Historical integrity	Version and revision policy	Reconstruct what was available at an earlier date

Report your tests with denominators, versions, and limitations. If only one batch was tested, do not extrapolate to every future refresh.

Review privacy and confidentiality in context

Inspect both direct identifiers and contextual information. Free text, rare combinations of attributes, and linked records can reveal more than the field names suggest. The ICO’s anonymisation guidance explains why identifiability is broader than removing names; that guidance is currently under review.

Have the appropriate specialists assess the proposed processing and transfer. If EU personal data is in scope, use the GDPR and current applicable guidance to scope the obligations. This guide is not a legal determination.

Test the unglamorous delivery details

Load the data in the environment that will use it. Check encodings, time zones, schema drift, update logic, replacement records, and access expiration. Time the integration work and include it in the economics.

Ask who fixes a broken feed and who communicates a correction. A supplier that can answer promptly during a pilot still needs a defined support commitment for production.

Make the decision auditable

Keep the approved use, tested version, rights findings, quality results, unresolved issues, and acceptance decision together. Name the owner of each unresolved issue. Set a review trigger for source changes, new fields, changed uses, or significant defects.

The aim is not to produce a thick document. It is to ensure that a later team can understand why the purchase was approved and whether the reasons still hold.