On-device and local AI: why "runs on your own computer" is about to matter for small business

AI models that run entirely on your own hardware — no cloud connection, no data leaving the building — are moving from developer experiments to standard laptop features. For businesses in Northwestern Ontario, the practical advantages are data privacy, offline operation, and no ongoing API cost.

On-device AI — software that runs AI models entirely on your own computer, with no data sent to an external server — is moving from developer experiment to mainstream hardware feature. Apple Intelligence runs a core language model on-device on any Mac with an M1 chip or later. Microsoft's Copilot+ PC certification covers a growing category of Windows laptops built to handle AI inference without a cloud connection. A free tool called Ollama lets any business owner download and run capable open-source models on a modern PC or Mac. For a Northwestern Ontario small business, three things follow from this shift: data that stays on your hardware (relevant to any operation handling client records under Canadian privacy law), AI tools that keep working when internet service is unreliable, and no monthly API subscription fee to run basic language tasks.

What "local AI" means and how it differs from ChatGPT or Copilot

Cloud AI tools — ChatGPT, Claude, Gemini, and Microsoft's cloud-based Copilot — work by sending your input over the internet to a remote server, which runs the model and returns an answer. Your prompt, and any information in it, travels to a third party's infrastructure. Local AI inverts that: the model is downloaded and stored on your own machine, and all the computation happens on your CPU, GPU, or the dedicated neural processing chip now built into many new laptops. The model's output never leaves your device. That distinction matters whenever the information you are working with is client-sensitive — a customer list, a financial record, a contract, a health document. It also matters whenever internet access is not guaranteed.

Three platforms making local AI accessible now

Apple Intelligence is Apple's integrated AI system available on iPhone 16 and later, iPhone 15 Pro and Pro Max, and any Mac with an M1 chip or newer. Apple's own research describes the on-device component as a language model of approximately three billion parameters that handles writing assistance, summarization, notification management, and photo-editing tasks locally on the device. More complex requests are routed to Apple's Private Cloud Compute, which Apple says uses servers that never retain user data. The device compatibility list and the on-device model description are published by Apple directly. On-device features require iOS 18.2 or macOS Sequoia or later and are included free with compatible hardware.

Copilot+ PCs are a Microsoft hardware certification for Windows 11 laptops and desktops equipped with a neural processing unit (NPU) capable of at least 40 trillion operations per second. Microsoft announced the category in May 2024, with qualifying chipsets from Qualcomm, AMD, and Intel and certified devices from Dell, HP, Lenovo, Samsung, ASUS, and Microsoft's own Surface line. The on-device language model built for this platform is called Phi Silica, which runs summarization, rewriting, and Q&A tasks on the NPU without a cloud connection. Microsoft's announcement described these devices as providing "faster responses, added privacy, and less reliance on the cloud." The starting price at launch was $999 USD; confirm current Canadian retail pricing with local retailers or directly at microsoft.com/en-ca.

Ollama is a free, open-source application that runs on macOS, Windows, and Linux. It downloads and runs open-source language models — including Meta's Llama family, Mistral, and Google's Gemma — on your own hardware, with nothing sent to a third-party server. The model library at ollama.com lists dozens of options ranging from very small models suitable for basic tasks on modest hardware to larger models that require more memory. For a small-business desktop or modern laptop, the practical range is 7-8 billion parameter models. LM Studio, whose documentation recommends a minimum of 16 GB of RAM, is a free graphical-interface alternative to Ollama that supports the same model ecosystem and is more accessible for non-technical users.

The three practical advantages for a Northwestern Ontario business

• Privacy: data that never leaves the building. When you enter client information into a cloud AI tool, that data is transmitted to a third-party server. Under PIPEDA — Canada's federal private-sector privacy law — your organization retains accountability for personal information shared with third-party providers, including the safeguards those providers apply. On-device AI removes the third-party transmission from the equation. This is not a ruling that local AI constitutes PIPEDA compliance; other obligations around consent, purpose, and security still apply. Confirm your specific situation with a privacy adviser.
• Offline operation: AI that works without internet. Cloud AI tools require a stable connection. Local models work at a remote job site, in a camp, at a lodge with satellite-only internet, or during an outage. For businesses operating in parts of NWO where connectivity is unreliable, this is a functional difference, not a theoretical one.
• No ongoing API cost: tools like Ollama are free. Cloud AI APIs charge per token or per seat. Apple Intelligence is included with compatible hardware at no subscription fee. Ollama has no cost beyond the hardware you already own. For a business running moderate AI workloads, eliminating the per-use API fee can offset hardware costs over time.

What local AI can and cannot do today

The open-source models that run on consumer hardware — typically 7-8 billion parameter models like Llama 3.1 8B or Mistral 7B, accessible through Ollama or LM Studio — are capable for summarizing documents, drafting text, answering questions from a fixed knowledge base, classifying inputs, and generating code. They are meaningfully less capable than frontier cloud models for complex reasoning, nuanced writing, and tasks requiring broad real-world knowledge. Apple's on-device component (approximately 3 billion parameters) is explicitly designed as a complement to cloud processing for complex tasks, not a replacement. Meta designed its Llama 3.2 1 billion and 3 billion parameter models specifically for on-device deployment on mobile and edge hardware; the trade-off is narrower capability in exchange for compact size. Higher-capability open-source models in the 70-billion parameter range require hardware — particularly GPU memory — that exceeds what most small-business desktops carry.

When this is worth your attention

Three situations make local AI worth evaluating now rather than later. First: your work involves personal or confidential client information — financial records, legal documents, health data, employee records — and you want to avoid transmitting that data off-device. Second: your business operates at a location where internet service is unreliable or intermittent, and cloud AI tools are not dependable. Third: you have identified a specific, repetitive AI task and want to run it without a per-use fee. If none of those apply, cloud AI remains the more capable option for most tasks today, and the decision can wait. The trend worth tracking is that on-device capability improves with each hardware generation — a business buying a new Mac or a qualifying Windows laptop now is likely buying hardware that can already run local AI, whether or not that capability is used today.

Apple Intelligence device compatibility — M1 Mac or later, iPhone 15 Pro/Pro Max or later, iPhone 16 and later, iOS 18.2 / macOS Sequoia required: support.apple.com/en-ca/105112 | Apple on-device foundation language model research — approximately 3 billion parameters, on-device processing for writing, summarization, and photo tasks: machinelearning.apple.com/research/apple-intelligence-foundation-language-models | Apple privacy features — on-device processing and Private Cloud Compute description ("never stored"): apple.com/ca/privacy/features/ | Microsoft Copilot+ PCs announcement, May 2024 — 40+ TOPS NPU requirement, "faster responses, added privacy, and less reliance on the cloud": blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/ | Copilot+ PC certified devices (Dell, HP, Lenovo, Samsung, ASUS, Surface; Qualcomm, AMD, Intel chipsets): learn.microsoft.com/en-us/windows/ai/npu-devices/ | Phi Silica on-device language model hardware requirements — Copilot+ PC NPU or NVIDIA/AMD GPU with 6+ GB VRAM: learn.microsoft.com/en-us/windows/ai/apis/phi-silica | Ollama — free open-source local model runner for macOS, Windows, Linux; model library: ollama.com | Meta Llama 3.2 on-device models — 1B and 3B parameters, designed for Qualcomm and MediaTek hardware, edge and mobile deployment: ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/ | LM Studio system requirements — 16 GB RAM recommended: lmstudio.ai/docs/system-requirements | PIPEDA accountability principle — organizations responsible for personal information transferred to third-party providers: priv.gc.ca/en/privacy-topics/privacy-laws-in-canada/the-personal-information-protection-and-electronic-documents-act-pipeda/pipeda_brief/ | Office of the Privacy Commissioner of Canada on generative AI as a strategic privacy priority: priv.gc.ca/en/privacy-topics/technology/artificial-intelligence/ | This post is commentary, not legal or privacy advice — confirm your PIPEDA obligations with a qualified privacy adviser.