In a field experiment with nearly 1,000 Turkish high-school students, practising with a standard GPT-4 interface raised practice scores by 48% but left students 17% worse than controls on a closed-book exam later in the same session, while a tutor version with safeguards raised practice by 127% and left exam scores statistically indistinguishable from those of controls, who practised with their course notes and textbook.

SocioLens project,

Claim LD04-SL-C11 (v1.1)

Provenance and status

Claim type: Estimate
Publication status: Eligible for publication, with a qualifier
Verification state: Checked against the primary source. Who checked this claim is not recorded for it; claim checks so far were AI-led, and no person has checked claims against their sources.
Does not license: Long-term learning effects; adult or workplace learning
Limitations: The exam measures short-term learning in the same session (the authors say so); one school; five class sessions did not receive their assigned arm (intent-to-treat used); a 2025 correction fixes an affiliation only
Source (as cited): Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025
Source type: Peer-reviewed journal article (recorded as “Peer-reviewed journal article”)
Institutional position: Independent academic
Maturity of the evidence: Emerging evidence
Recorded date: 22 October 2026
Currentness / correction: This is the current version. No correction or concern is recorded on its sources.

Sources this version cites

Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025 (Peer-reviewed journal article)
Read the original

Version

This page shows version v1.1. Its record never changes, even when newer evidence arrives; the site may add a note to the page or withhold a figure not cleared for public use.

This pinned version is also the current version.

Versions: v1.0, v1.1 · current

What changed in this version

This version supersedes version v1.0 (LD04-SL-C11--v1.0), which stays available with its record unchanged.

Changed fields: 2 PUBLIC-FACING CLAIM; 13 COMPARATOR; 14 MAIN RESULT; 25 RECORDED DATE
What changed: The comparator no longer reads “Textbook-only practice”: control students practised with their course notes and the course textbook, without AI. The tutor arm’s exam result now reads “statistically indistinguishable” from controls instead of “indistinguishable”, and the main result gives its point estimate (−0.004 out of 1, against −0.054 for the standard interface), because failing to detect a difference does not show that there is none. The figures, the source and the other fields are unchanged.
Why: An external research-integrity review of the site on 22 October 2026 (an AI reviewer, not a person) found the figures correct but the comparator misdescribed, and asked for “statistically indistinguishable”. SocioLens read the article’s open-access text in PubMed Central (PMC12232635) on 22 October 2026. It says that in the practice phase “students are given access to standard resources (their course notes and the course textbook), as well as additional generative AI resources determined based on a randomly assigned arm”, and that “Student performance in the GPT Tutor arm was statistically indistinguishable from that of the control arm, and the point estimate was smaller by an order of magnitude (−0.004)”, against a GPT Base reduction of 0.054. The check and this revision were done by an AI agent reading the article, not by a person.

Raw record: every recorded field, with its coded values

All 25 canonical Claim fields, verbatim, with the coded values derived from them. The plain-language labels above are read from these.

Claim type (coded): ESTIMATE (as printed: “ESTIMATE”)
Publication status (coded): ELIGIBLE (QUALIFIED) (as printed: “ELIGIBLE WITH QUALIFIER”)
Verification state (coded): CONTENT_VERIFIED_PRIMARY
Source type (coded): PEER_REVIEWED_JOURNAL_ARTICLE
Maturity (coded): EMERGING

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD04-SL-C11	—
2	PUBLIC-FACING CLAIM	In a field experiment with nearly 1,000 Turkish high-school students, practising with a standard GPT-4 interface raised practice scores by 48% but left students 17% worse than controls on a closed-book exam later in the same session, while a tutor version with safeguards raised practice by 127% and left exam scores statistically indistinguishable from those of controls, who practised with their course notes and textbook.	—
3	CLAIM TYPE	ESTIMATE	base_types: ESTIMATE
4	RESEARCH STRAND	B — Skill after the tool is removed	—
5	MATURITY	EMERGING	states: EMERGING
6	SOURCE	Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025	source_ids: SRC-SL-BASTANI-ET-AL-PNAS-2025
7	SOURCE TYPE	Peer-reviewed journal article	classes: PEER_REVIEWED_JOURNAL_ARTICLE
8	INSTITUTIONAL POSITION	Independent academic	—
9	SCOPE	One Turkish high school; about fifty 9th–11th-grade classes; Fall 2023; four 90-minute sessions	—
10	DESIGN	Preregistered field experiment; each session had assisted practice then a closed-book exam on similar problems	—
11	SAMPLE / DATASET	Nearly 1,000 students	—
12	MODEL / TOOL / VERSION	GPT-4 with a standard interface (GPT Base) and a tutor version with safeguards (GPT Tutor); interface details not reported here	—
13	COMPARATOR	Practice with the course notes and the course textbook, without AI	—
14	MAIN RESULT	Practice +48% (GPT Base) and +127% (GPT Tutor); exam: GPT Base 17% of the control mean lower, statistically significant; GPT Tutor statistically indistinguishable from control (point estimate −0.004 out of 1, against −0.054 for GPT Base); both tutors narrowed the spread of practice scores, but that narrowing did not persist when AI was removed; students did not perceive the harm	—
15	INFERENTIAL TARGET	Short-term learning after AI-assisted practice	—
16	DOES NOT LICENSE	Long-term learning effects; adult or workplace learning	—
17	VERIFICATION STATE	CONTENT VERIFIED	state: CONTENT_VERIFIED_PRIMARY · verified_by: UNKNOWN
18	LIMITATIONS	The exam measures short-term learning in the same session (the authors say so); one school; five class sessions did not receive their assigned arm (intent-to-treat used); a 2025 correction fixes an affiliation only	—
19	CONTRADICTORY / SUPPORTING SOURCES	Direct evidence that assisted compression need not survive tool removal; consistent with losses in a programming-library learning experiment	—
20	GEOGRAPHY / LANGUAGE	Türkiye / Not checked	—
21	DISTRIBUTIONAL LEDGER	Limited or no heterogeneity by ability, resources or effort	—
22	R01 / FIELD-MAP LINKS	LD-04, D2	—
23	WHAT WOULD CHANGE THIS CLAIM?	Delayed tests showing the exam loss fading or reversing	—
24	PUBLICATION STATUS	ELIGIBLE WITH QUALIFIER	state: ELIGIBLE · qualifiers: QUALIFIED
25	RECORDED DATE	2026-10-22	—

Field 22 references resolved against Field Map v0.2: LD-04 (LIVING_DEBATE); D2 (DOMAIN)

The sources as recorded, with their record IDs and coded source types:

Pinned Sources (field 6, version-specific)

“Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025” → Bastani et al. (2025), “Generative AI without guardrails…” (SRC-SL-BASTANI-ET-AL-PNAS-2025--v1.0) — Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025; class PEER_REVIEWED_JOURNAL_ARTICLE (explicit public provenance pin)

Cite this page

SocioLens project. (2026, October 22). In a field experiment with nearly 1,000 Turkish high-school students, practising with a standard GPT-4 interface raised practice scores by 48% but left students 17% worse than controls on a closed-book exam later in the same session, while a tutor version with safeguards raised practice by 127% and left exam scores statistically indistinguishable from those of controls, who practised with their course notes and textbook [Claim LD04-SL-C11, version v1.1]. SocioLens. https://sociolens.eu/claims/ld04-sl-c11/v1.1/

@misc{sociolens_ld04_sl_c11_v1_1,
  author = {{SocioLens project}},
  title = {In a field experiment with nearly 1,000 Turkish high-school students, practising with a standard GPT-4 interface raised practice scores by 48\% but left students 17\% worse than controls on a closed-book exam later in the same session, while a tutor version with safeguards raised practice by 127\% and left exam scores statistically indistinguishable from those of controls, who practised with their course notes and textbook.},
  year = {2026},
  date = {2026-10-22},
  howpublished = {\url{https://sociolens.eu/claims/ld04-sl-c11/v1.1/}},
  note = {Claim LD04-SL-C11, version v1.1}
}

The link points to this version, whose record never changes. For a finding, cite the original study as well: SocioLens summarizes research and does not replace it.

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 ab6e42f200e17121ec0296b17662e6f576a120740312b6b512141dfa0c177834.

Repository-authored inputs (not Drive exports)

SocioLens claim revision — LD04-SL-C11 v1.1 (v1.1) — written in the SocioLens repository by Project Lead (Claude, AI) on 2026-10-22; no Drive identity exists. Authority: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and docs/PROJECT.md "Research revisions" (a published claim changes only through a new version that states what changed and why). Block “ENTRY 11 — LD04-SL-C11” SHA-256 8bb0dbf64d1c4601779b4cf325cbe6a4b6a0cd8f576eddfc6ca2c4c294e6def7; fixture fixtures/authored/claim-revisions/LD04-SL-C11--v1.1.

Builder decisions (not Drive data)

VERSION = v1.1 — basis: Claim version taken from the repository-authored revision fixture version (SocioLens claim revision — LD04-SL-C11 v1.1); a new version of LD04-SL-C11 v1.0, which stays routable (AGENTS.md research integrity rule 3)
CURRENT_POINTER = true — basis: repository-authored revision "SocioLens claim revision — LD04-SL-C11 v1.1" (Project Lead (Claude, AI), 2026-10-22); basis: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and docs/PROJECT.md "Research revisions" (a published claim changes only through a new version that states what changed and why)
FIELD_6_SOURCE_REFERENCE = SRC-SL-BASTANI-ET-AL-PNAS-2025--v1.0 — basis: field 6 of ENTRY 11 — LD04-SL-C11 cites "Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025"; the field-6 text of this ClaimVersion's repository-authored ledger block; the same Source record LD04-SL-C11 v1.0 cites.
SOURCE_PROVENANCE_PIN = SRC-SL-BASTANI-ET-AL-PNAS-2025--v1.0 — basis: explicit version-pinned public provenance of this ClaimVersion (Amendment v1.2 §6): the pinned field-6 Source record "Bastani, Bastani, Sungu, Ge, Kabakcı & Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics,” PNAS 122(26), 2025".