| Table 1. Reproducibility results for TRUST-MA and PCC | ||
| Measure | TRUST-MA (n=51) | PCC (n=24) |
|---|---|---|
| Mean±SD, ChatGPT | 56.21±19.90 | 3.54±2.13 |
| Mean±SD, Claude | 64.28±18.22 | 3.54±2.15 |
| Paired difference, Claude-ChatGPT | 8.07 (95% CI 5.33-10.82) | 0.00 (95% CI -0.51-0.51) |
| Paired comparison ICC, absolute agreement Lin CCC Exact categorical agreement Linear weighted κ |
0.800 (0.698-0.868) 0.798 (0.695-0.867) 68.6% (54.9-80.4) 0.451 (0.265-0.618) |
0.844 (0.541-0.955) 0.838 (0.530-0.954) 66.7% (45.8-83.3) 0.653 (0.353-0.840) |
CI, confidence interval; CCC, concordance correlation coefficient; ICC, intraclass correlation coefficient; PCC, Practice Change Confidence; SD, standard deviation. Confidence intervals in parentheses for ICC, CCC, exact agreement, and κ were obtained by 10,000 bootstrap replicates.