Claude Smart Reports: evaluating AI use across a team

Understand availability and the evidence base

Smart Reports are an Enterprise beta that is disabled by default and must be enabled by an owner. The analysis samples transcripts across periods of up to 28 days. CMEK, HIPAA and Access Transparency configurations are excluded; Enterprise zero data retention also excludes Claude Code. Individual attribution is separately disabled, and revealing names is logged when enabled.

This makes the feature more than a dashboard of sign-ins or token counts. Before generating a report, the company should define which operational question it should answer and who needs the findings. A suitable assignment might ask which obstacles repeatedly affect project reporting. A broad search for the “most productive employees” is not an appropriate task.

Choose one team and one concrete question

A services company might start with project coordination. This team produces status reports, condenses meeting notes and prepares handovers. Select a comparable period and record which assignments actually occurred. A month containing three major acceptance reviews cannot be compared with a quiet holiday period without context.

The question should support a decision: does the team need a shared template, are approved sources missing, or is a process poorly specified? Agree beforehand which finding would trigger which action. This keeps the report focused on improving a workflow. A large collection of colourful charts does not, by itself, produce an actionable decision.

Keep the original question alongside the report. On a later review, the team can then tell whether the same task and organizational context were assessed, or whether new requirements demand a different comparison.

Separate activity, outputs and economic value

Many sessions may indicate intensive use, but they may also reflect repeated corrections. Low spending could mean tasks are completed efficiently, or simply that the feature is rarely used. Add a business review of a sample of genuinely accepted outputs. Ask whether a report was complete, whether its figures were correct and how much rework remained.

Automatically inferred time savings are not measured working time. A budget decision also needs a traceable comparison with the previous process. Separate setup effort, ongoing consumption and human review. Only a usable deliverable with known quality can make a reliable contribution to an economic assessment.

Turn one finding into one process change

Suppose the analysis suggests repeated problems with project briefs. Inspect several relevant examples first. Perhaps the reporting period is regularly missing, or different people use different templates. A shared task definition with fixed inputs, sources and required fields may address the issue. The improvement then comes from a clearer assignment rather than another chart.

Test this change on new cases and compare completeness and review time with the baseline sample. Avoid changing the template, source, model and training simultaneously if you want to understand the effect of one intervention. Also record when a hypothesis is not confirmed. A report can inform a decision without proving every suspicion.

Define purpose and recipients before generating reports

Anthropic explicitly excludes individual performance evaluation and employment decisions as intended uses. For implementation, we recommend clear communication about purpose, data, access and retention. Appropriate internal stakeholders should assess this design before transcripts are used for new analysis. This is an organizational decision that a product toggle does not resolve.

Limit distribution to people responsible for improving the process under review. A downloaded report also needs suitable storage and clear deletion rules. Discuss findings as workflow problems and opportunities: missing sources, unclear templates or unnecessary approval steps. Avoid rankings of individuals and unsupported claims about their performance.

Common questions about Smart Reports

Does every SME need Smart Reports? No. Without Enterprise access or sufficient usage, a manual sample may be more appropriate. Does a report automatically show ROI? No. Business value needs to be checked against accepted results and actual effort.

Should every team be analysed immediately? Start with a clear question and suitable recipients. How can Werkverstand help? The AI System Check can establish whether the priority is improving an initial workflow or evaluating established usage systematically.