---
title: Evaluation plan
summary: How a hotel would know the course works, from completion and records up to what staff do on a shift. Nothing here has been measured yet.
---

# Evaluation plan

**Nothing has been measured.** There is no client and no learner data. This plan says what a hotel that adopts the course would measure, and how. It uses LTEM (Will Thalheimer's Learning-Transfer Evaluation Model), so attendance and smile sheets are not mistaken for results.

| LTEM tier | Measure | Source | When |
|---|---|---|---|
| 1-2 Attendance and activity | Share of guest-contact staff with a completion record by day 180; share with refreshers on time | The roster or the hotel's training records | Monthly |
| 4 Knowledge | Test score per cell, across all learners: which skills are weak | The test results (xAPI-style statements) | Quarterly |
| 5 Decision-making competence | First-try choice in each scene; share of attempts with a dangerous answer | Scene and test statements | Quarterly |
| 6 Task competence | Quality of written reports in lesson 3 (word check, plus a manager's read of a sample) | The written report, and a manager's review | At pilot |
| 7 Transfer | Reports made to the manager on duty: how many, how fast (same shift?), how complete (room, time, facts), and how many were based on looks. Fewer reports is not success on its own; a drop to none is a warning sign | The manager-on-duty log | Six months after rollout |
| 7 Transfer, quick check | A monthly spot check: the manager asks three staff on different shifts, "Who do you tell tonight, and what do you say?" Target: three of three name the right person (including the manager on call at night) and give facts, not guesses | The manager's notes | Monthly, from the first month |
| 8 Effects | Policy, signs and records in order if the Department of Law Enforcement asks; staff confidence in who to tell (short survey) | An internal check; the survey | Yearly |

## Pilot

- **Who:** about ten staff across front desk, housekeeping, bell and valet, and security, including at least two contract workers.
- **What:** timed runs to replace the time model with real minutes; a short interview on wording, especially for staff whose first language is not English; a check that nothing in the course reads as profiling.
- **Before the pilot:**
  - Review by a survivor-informed expert and a Hawaii anti-trafficking organization.
  - Compare with the Attorney General's program once it is out (due July 1, 2027).

## What would change the design

| Finding | Change |
|---|---|
| Reports go up, but many are based on looks | Strengthen lesson 1 and the "not a sign on its own" cards |
| A cell is weak across learners | Rewrite that lesson's teaching slide and add a check |
| Real minutes run well over 40 | Make the cuts listed in the [design document](design-doc.html) |
| Staff don't know who is on shift | An environment fix: names on the pocket card and at the time clock, not more training |