Data warehouse · BigQuery data quality for AIThe query runs. The answer might not be true.
BigQuery can feed both analytics and model training. A clean-looking dataset can hide duplicate rows from streaming inserts, schema changes nobody agreed and scheduled queries untouched for months. We compare extracts from BigQuery with exports from the systems it copies, so you know whether the tables a model reads still match reality.
Starts with a 30-minute call. Commits you to nothing. Then exports only: no login, no connected app, nothing installed.
Source of truth
What BigQuery gets right.
Every system is built to be right about something. Knowing what BigQuery owns, and what it doesn’t, is where a cross-system check starts, and where the Truth Score™ (a score out of 100 for how far your data can be trusted) begins to take shape.
- 01
Event and behavioural data
Web, app and product events at volume.
- 02
Consolidated history
Data from many systems, joined in one place.
- 03
Training sets
The tables Vertex AI or other models read.
The record of a sale, a customer or a payment. Those originate in the CRM, store or ledger; BigQuery holds a transformed copy.
Inside the system
Common BigQuery data problems for AI.
What goes wrong inside BigQuery itself, with the objects and fields where it shows. Each one is checkable from an export.
- 01
Duplicates from streaming and backfill
Streaming inserts and re-loaded partitions represent the same record more than once.
insertIdpartition_PARTITIONTIME - 02
Schema evolution without contracts
Added, relaxed or retyped fields, including nested and REPEATED changes, reach datasets unannounced.
INFORMATION_SCHEMA.COLUMNSmode NULLABLE or REPEATED - 03
Unpartitioned and unowned datasets
Missing partitioning and clustering, and datasets with no owner, are costly and hard to trust.
partitioningclusteringdataset labels - 04
Keys and fan-out
Surrogate and natural keys differ across tables; UNNEST and joins silently multiply rows.
join keysUNNESTtable grain - 05
Personal data without controls
Sensitive columns without policy tags or column-level security, copied into ad hoc tables.
policy tagscolumn-level security
Between systems
BigQuery looks clean on its own. The gaps are next door.
The problems that sink AI sit between BigQuery and the systems beside it: the same customer, person or transaction recorded differently in each. These are the gaps to test for each pairing.
Data warehouse
BigQuery
Source of truth forEvent and behavioural dataConsolidated historyTraining sets
BigQuery ↔ Store and marketing
Klaviyoe-commerce platform
- Order counts that differ from the store
- Customers in BigQuery the store doesn’t have
What to export
What we need from you, in plain English.
Your team runs the queries we specify and exports results to CSV or to a file you share. The warehouse is a copy, so we also take source-system exports and compare. We never need a service account or a project role.
- Exports only, no access to your systems
- NDA and data processing agreement first
- The full list of sub-processors comes with our data processing agreement, before any data moves
- Raw exports deleted after the engagement; we keep quality metrics, not your records
AI that depends on it
AI use cases that lean on BigQuery.
If one of these is in a proposal or your plan, BigQuery is one of the systems we compare. Each page sets out what the use case needs and where it breaks.
- Operations Natural-language analytics Plain-English questions run against these tables and their definitions. Assessed with your team, not from exports alone.
- Revenue Personalisation Recommendation and audience models can train on event tables here.
- Operations Demand forecasting Order history at volume, once deduplicated and checked against the source.
- Finance Fraud detection Event-level transaction data, where duplicates look like anomalies.
- Revenue Customer churn prediction Usage events joined to billing and support are the richest churn features, if the joins hold.
How a Truth Score Read tests BigQuery
From your exports to a signed verdict.
Within 5 working days of complete exports. A fixed fee, agreed before we start, that never moves with the verdict.
- 01
Export BigQuery, and one system beside it
The BigQuery exports listed above, plus at least one neighbour, such as store and marketing. Under NDA, using the field list we send.
- 02
Match across systems
Warehouse extracts matched against the source systems’ own exports, key by key, so you can see where the copy and the original have drifted apart.
- 03
Check by hand, then score
Every red flag is checked by hand against sample records before sign-off. Then the Truth Score: each pillar shows what was measured and what wasn’t, and pillars with too little signal are marked, never guessed.
- 04
Get the verdict in writing
A signed memo: the red flags between BigQuery and the other systems, a Leverability read on the AI you’re considering (which AI ideas you can use now, after fixes, or not at all), a verdict (sign, fix first or walk away) and a short fix list your own team can work from.
What you get back
See what comes back: a real sample memo.
One fictional B2B software company, Example Co: 72,837 records in 7 exports from its CRM, finance and HR systems, run through the same engine and the same checks as a paid Truth Score Read. Each system passes its own checks. Matched across systems, only 13% of deals tie to an invoice.
Questions
BigQuery and AI, answered plainly.
Anything else, ask on the 30-minute call. It commits you to nothing.
Book the callDo you need access to our Google Cloud project?
No. Your team runs the queries we specify and shares the results under NDA. No service account, no IAM role.
Our tables pass every query. What could be wrong?
A query can run perfectly on duplicated or incomplete data. We check keys, grain and freshness, and compare totals with the source systems’ own exports.
We plan to train on BigQuery with Vertex AI. What should we test first?
Duplicate rows, join fan-out and whether the training tables still match the systems they copy. A Truth Score Read tests that within 5 working days of complete exports.
Will you redesign our datasets?
No. We never fix what we find; the fix list goes to your team or any partner you choose.
Find out what your BigQuery data can carry. Before the AI does.
Your BigQuery export and one other system in. The gaps between them in writing, within 5 working days of complete exports. Starts with a 30-minute call. Commits you to nothing.
Starts with a 30-minute call. Commits you to nothing.