What Happened

On June 30, 2026, a team of seven researchers — Meng Chen, Anya Ji, Tsung-Han Wu, Tobias Maringgele, David M. Chan, Alane Suhr, and Amy Pavel — published a paper on arXiv introducing DigitalCoach, a multimodal dataset specifically designed to evaluate whether AI models can teach humans how to use software.

The dataset is substantial: 72 human expert-novice coaching sessions, comprising 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across five software applications. The researchers used this dataset to run both automated and interactive evaluations of state-of-the-art models acting as coaches.

The findings are confirmed by the paper's evaluation:

• Models provide more direct instructions than human coaches — essentially telling users what to do step-by-step.
• Models provide fewer explanations, error diagnoses, and knowledge-check questions — meaning they don't help users understand why they're doing something or verify comprehension.
• Even when coaching method is controlled (i.e., researchers fixed the approach to match human coaching patterns), model utterances remained poorly grounded in visual context — they don't correctly reference what's on screen.
• Interactive evaluation confirmed that learners coached by models passively follow instructions without deeper engagement.

Why It Matters

The AI industry's focus on agentic computer use — where models operate software autonomously — has overshadowed a parallel use case: teaching humans to use software themselves. Digital adoption platforms, in-app copilots, and AI-powered onboarding tools all depend on this capability.

This research exposes a concrete, measurable gap. Current models default to directive behavior. They tell you to click a button but don't explain what the button does, why it's in that menu, or what happens if something goes wrong. The result is users who can complete a task once but haven't built transferable competence.

For operators, this is not a future risk — it's a present-day product quality issue. If your AI copilot only gives instructions, your users become dependent on the copilot rather than growing in proficiency. That affects retention, satisfaction, and the long-term value proposition of your product.

The paper also provides something actionable: a benchmark. DigitalCoach's dataset and evaluation methodology give builders a way to measure whether their coaching implementations actually teach — or just instruct. This is particularly relevant given the recent surge in agentic AI investment, including General Intuition's $320M raise for training AI agents via gameplay (June 25) and Warp's $60M Series B for AI-native HR and payroll automation (June 25). The industry is pouring capital into agents that do tasks; this paper asks whether they can teach tasks — and the answer is not yet.

Who Is Affected

AI startups building in-app guidance, digital adoption platforms, or copilot-style assistants are most directly affected. Their core value proposition depends on models that can teach, and this research shows current models can't do that well without significant additional engineering.

Enterprise IT and L&D teams evaluating AI-powered training tools should use these findings to pressure-test vendor claims. Ask specifically how the system generates explanations, diagnoses errors, and checks for learner understanding.

Developers building multimodal AI agents that interact with screen content need to internalize that visual grounding remains an unsolved problem. Even when text output looks correct, the model may not be correctly interpreting what's on screen — a critical failure mode for any screen-aware product.

Strategic Implications

For AI Startup Founders

If you're building coaching or guidance products on top of LLMs, don't assume model quality translates to teaching quality. You need explicit scaffolding — explanation generation, error diagnosis logic, and knowledge-check mechanisms — layered on top of the base model. This is a product differentiation opportunity, not just a model limitation. The startups that solve the teaching gap will have a defensible moat that raw model improvements alone won't close.

For Developers/Operators Building with AI APIs

Visual grounding is your weakest link. Even when you fix the coaching method, models produce utterances that don't correctly reference what's on screen. If your product depends on screen-aware guidance, invest in grounding pipelines — screenshot parsing, UI element detection, spatial reasoning — rather than relying on the model to infer visual context from text alone. The DigitalCoach dataset gives you a concrete evaluation framework to measure progress.

For Non-Technical Business Owners Evaluating AI Tools

When evaluating AI-powered training or onboarding tools, ask vendors specifically how their system explains 'why' — not just 'what to click.' Tools that only provide step-by-step instructions will create dependent users, not skilled ones. Request evidence that the system checks for understanding and adapts to errors. If the vendor can't answer these questions, their product is likely just wrapping a base model in a UI shell.

What to Watch Next

Monitor whether model labs (OpenAI, Anthropic, Google) begin addressing visual grounding and pedagogical behavior in future releases — particularly in computer-use agent products. Also watch for startups that explicitly build coaching-layer infrastructure on top of existing models, as this paper defines a clear gap they can fill.

Frequently Asked Questions

Q: What is the DigitalCoach dataset?

A: DigitalCoach is a multimodal dataset of 72 human expert-novice computer use coaching sessions, containing 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across five software applications. It was published on arXiv on June 30, 2026, to evaluate whether AI models can effectively teach humans to use software.

Q: Why do AI models fail at teaching humans to use software?

A: According to the DigitalCoach research, models default to giving direct instructions rather than explanations, error diagnoses, or knowledge-check questions. Even when their coaching method is adjusted to match human patterns, models produce utterances poorly grounded in visual context — meaning they don't correctly reference what's on screen. This causes learners to passively follow steps without building real understanding.