The AI hype checklist: five claims a local business owner should ignore

Five specific claims appear in every AI sales pitch and keynote recap — from job-replacement fear to adoption-rate pressure to benchmark scores on professional exams. Each one distorts the real decision. Here is what each claim actually means and why none of them should drive what you do this week.

The gap between how AI vendors describe their products and what those products actually deliver in a real business is wide enough to cost you real money — either by rushing you into a purchase you do not need, or by making the whole category feel too complicated to bother with. Five specific claims circulate across every sales deck, keynote recap, and LinkedIn post about AI. A business owner in Northwestern Ontario should treat each one as a reason to ask a harder question, not as a reason to act.

Claim 1: "AI is going to replace your team"

The job-replacement claim gets the most press coverage and generates the most anxiety. The reference point is usually a Goldman Sachs economic analysis from March 2023, which examined AI's potential effects on work across hundreds of occupations. That analysis was turned into headlines about AI replacing hundreds of millions of jobs. What the Goldman Sachs report actually emphasized is that most affected occupations are more likely to be complemented than substituted — AI handles specific tasks within a job, not the job itself. The same analysis framed its finding as a productivity and economic growth story, not a mass-unemployment scenario. For a business in Northwestern Ontario with five to fifteen staff, the near-term reality looks nothing like the headline: one tool saves a few hours per week on a defined task, which usually means the person does more of the work that requires judgment, not that their position disappears. Do not make staffing decisions based on a vendor claim that AI is coming for your payroll.

Claim 2: "Our AI scored in the top 10 percent on the bar exam"

The professional-exam benchmark is one of the most repeated AI marketing claims and one of the most misleading. OpenAI's GPT-4 technical report, published in March 2023 (arXiv 2303.08774), states that GPT-4 passed "a simulated bar exam with a score around the top 10% of test takers." That figure was then repeated across press coverage, keynotes, and sales materials as evidence that AI performs at an expert level. The problem is in the comparison pool. A 2024 peer-reviewed paper by Eric Martínez, published through MIT and available via SSRN (id 4441311), re-evaluated the claim using July bar exam administration data — the administration that draws first-time sitters and a more representative sample. That analysis found GPT-4's actual percentile is below the 69th overall, and approximately the 48th percentile on the essay component. Against licensed attorneys — the group the marketing implicitly invites comparison to — the essay percentile falls further. This is not a minor adjustment: it changes the story from "AI writes at an expert level" to "AI writes at a middling one." More concretely, Canadian courts have sanctioned lawyers twice for submitting AI-generated citations to cases that do not exist. In Zhang v. Chen (2024 BCSC 285), a BC lawyer who used ChatGPT for legal research was ordered to pay costs personally and audit every active file for AI-generated content. In Ko v. Li (2025 ONSC 2965), an Ontario court issued a formal public rebuke after fabricated case citations appeared in filed materials. Benchmark claims do not transfer to real-world task performance. Evaluate any AI tool on the specific task you need it to do, not on a test designed to measure something else.

Claim 3: "Seventy-five percent of businesses are already using AI — you're falling behind"

Adoption-rate claims are almost always vendor-funded survey results, not independent measurement. A vendor surveying its own customers, or a consulting firm surveying large enterprises about "any AI use in any business function," will produce a number in the 70-to-88-percent range. Statistics Canada measured something different: actual use of AI to produce goods or deliver services, across all registered Canadian firms. The 2025 figure, published in 2026, is 12.2 percent — a number that doubled year-over-year, which is meaningful growth, and 14.5 percent of firms planned to adopt within 12 months. A Business Development Bank of Canada study from 2024 probed the definitional gap directly: 39 percent of Canadian entrepreneurs initially said they use AI. When shown a list of common examples — Siri, spam filters, Google Maps, autocomplete — that figure jumped to 66 percent. The BDC's headline finding was that 27 percent of Canadian entrepreneurs use AI without knowing it, because they use tools with embedded AI features they never identified as AI. When nearly a third of the "adopters" in a survey are people who use navigation apps, the adoption rate number stops being a useful competitive signal. The 12.2 percent figure from Statistics Canada is the one that reflects deliberate, operational AI deployment in Canadian businesses. It is not a crisis — it reflects where a genuinely new category of tools is in its adoption curve.

Claim 4: "You need to train your own AI model to get real results"

Training a large language model from scratch is one of the most expensive engineering undertakings in existence — the kind of infrastructure project that requires massive dedicated computing clusters, months of continuous processing, vast curated training datasets, and a skilled engineering team to design, run, and evaluate the whole process. This is what OpenAI, Google, Anthropic, and Meta do. It is not what a small business in Thunder Bay does. The claim that a business needs to train its own model gets aimed at owners who have no engineering department and no machine-learning background. It conflates three very different activities: training a model from scratch (for the largest technology companies in the world), fine-tuning a pre-existing model on specialized data (for organizations with proprietary datasets and technical staff), and using an existing model through an API or a commercial product (what virtually every small and medium-sized business actually needs). For most businesses, the right approach is the third one: use a commercial AI tool, or configure a general-purpose model with clear instructions and your business context. That configuration — called prompt engineering, or retrieval-augmented generation for structured knowledge bases — does not require building anything from scratch. A Thunder Bay trades business or retail shop does not need to train a language model. It needs a good prompt and the right tool.

Claim 5: "This new AI is ten times more powerful than the last one"

AI capability claims use benchmark scores as their primary evidence. The problem with benchmarks is saturation: once every major model scores above a high threshold on a given test, the score differences between competing models become too small to drive any real-world decision. Vendors then move to newer, harder benchmarks and report their scores there — and the cycle repeats. The "ten times more powerful" framing typically compares a new model's score on the benchmark where it performs best to an older model's score on a different benchmark entirely. The Stanford AI Index 2025 documents this pattern across successive model generations in its research and development chapter. There is also a consistent and documented gap between how AI models perform in controlled lab evaluation and how they perform in actual production deployments — the constraints of real business workflows, inconsistent inputs, and edge cases that structured benchmarks do not capture. For a local business evaluating an AI tool, the right test is not a benchmark score. It is whether the tool reliably does the specific task you need, in the time you have, at a cost that makes sense. Run it on your own content and your own questions for two weeks before buying.

Sources: Goldman Sachs Economic Research — "The Potentially Large Effects of Artificial Intelligence on Economic Growth" (March 2023), widely summarized at goldmansachs.com/insights/articles/generative-ai-could-raise-global-gdp-by-7-percent | OpenAI GPT-4 Technical Report — "passing a simulated bar exam with a score around the top 10% of test takers" (March 2023): arxiv.org/abs/2303.08774 | Eric Martínez — "Re-evaluating GPT-4's Bar Exam Performance," MIT / SSRN id 4441311 (2024, published in Artificial Intelligence and Law) — below 69th percentile overall and approximately 48th percentile on essays using July administration data: papers.ssrn.com/sol3/papers.cfm?abstract_id=4441311 | Zhang v. Chen (2024 BCSC 285) — BC Supreme Court sanctions for AI-hallucinated legal citations; lawyer ordered to pay personal costs and audit all active files: mccarthy.ca/en/insights/blogs/techlex/landmark-decision-about-hallucinated-legal-authorities-bc-signals-caution-leaves-questions-about-requirement-disclose-use-ai-tools | Ko v. Li (2025 ONSC 2965) — Ontario court formal public rebuke for AI-generated fabricated case citations in filed materials: cassels.com/insights/fantasies-in-the-footnotes-ontario-court-rebukes-counsel-for-fake-and-misleading-ai-generated-case-citations | Statistics Canada — 12.2% of Canadian firms used AI to produce goods or deliver services in 2025, doubling year-over-year; 14.5% planning adoption within 12 months: www150.statcan.gc.ca/n1/pub/36-28-0001/2026004/article/00002-eng.htm | Business Development Bank of Canada — "The AI Imperative for Canada's Entrepreneurs" (2024): 39% of Canadian entrepreneurs initially said they use AI; 66% recognized use when shown examples including Siri, Google Maps, spam filters, and autocomplete; 27% use AI without knowing it; survey of 1,247 business owners, April 2024: bdc.ca/en/about/analysis-research/the-ai-imperative-for-canada-entrepreneurs (news release: newswire.ca/news-releases/new-bdc-study-reveals-27-of-canadian-entrepreneurs-don-t-know-they-re-using-artificial-intelligence-ai--803724479.html) | Stanford AI Index Report 2025 — documents benchmark saturation and capability-claim patterns across model generations, R&D chapter: hai.stanford.edu/ai-index/2025-ai-index-report/research-and-development