Human Test Matrix for Learning Room Challenges

Use this matrix before each cohort to verify that the Learning Room template behaves like a realistic collaborative GitHub project while still working inside private GitHub Classroom repositories.

Required Test Repositories

Create at least one disposable student repository from the current Community-Access/learning-room-template template. For full peer-access testing, create two disposable repositories and use two test student accounts.

Minimum smoke test:

  • learning-room-smoke-a

Full collaboration test:

  • learning-room-smoke-a
  • learning-room-smoke-b

Required Facilitator Setup

The old scripts/classroom/ PowerShell seeding scripts were removed with GitHub Classroom. Seeding now runs through the Student Progression Bot workflow inside the student repository.

After the student repository exists, open the repo's Actions tab, choose Student Progression Bot, select Run workflow, and set:

  • start_challenge: 1
  • assignee: test-student-a

For Day 2-only testing, set start_challenge to 10 instead.

Equivalent CLI:

gh workflow run student-progression.yml -R Community-Access/learning-room-smoke-a -f start_challenge=1 -f assignee=test-student-a

What The Peer Simulation Creates

The progression bot seeds realistic collaboration artifacts inside the student's private repo automatically whenever it creates a challenge issue (and repairs them if any are missing):

  • Peer Simulation: Welcome Link Needs Context issue
  • Peer Simulation: Review Request for Contribution Guidance issue
  • Peer Simulation: Improve contribution guidance pull request
  • docs/samples/peer-review-practice.md on branch peer-simulation/review-pr

Students can use these artifacts whenever a challenge asks them to comment, react, review, compare, or practice collaboration. If facilitators separately provision real buddy access, students may use the real buddy repository instead.

Day 1 Core Test

Complete these steps as a test student.

  1. Challenge 1: Find Your Way Around

    • Verify Challenge 1 issue exists.
    • Complete the scavenger hunt.
    • Comment evidence.
    • Comment or react on the peer-simulation issue.
    • Close Challenge 1.
    • Verify Challenge 2 appears.

  2. Challenge 2: File Your First Issue

    • Find a TODO in docs/welcome.md.
    • Create a new issue with a clear title and description.
    • Comment evidence on Challenge 2.
    • Comment on the peer-simulation issue title/description.
    • Close Challenge 2.
    • Verify Challenge 3 appears.

  3. Challenge 3: Join the Conversation

    • Comment on Peer Simulation: Welcome Link Needs Context.
    • Include @gandalf-bot in the comment.
    • Add a reaction.
    • Verify Gandalf responds if issue-comment workflow permissions allow it.
    • Close Challenge 3.
    • Verify Challenge 4 appears.

  4. Challenge 4: Branch Out

    • Create learn/test-student-a or equivalent feature branch.
    • Compare with the peer-simulation PR branch name.
    • Comment evidence.
    • Close Challenge 4.
    • Verify Challenge 5 appears.

  5. Challenge 5: Make Your Mark

    • Edit the first TODO in docs/welcome.md on the feature branch.
    • Commit with a descriptive message.
    • Compare with the peer-simulation PR title or commit message.
    • Comment evidence.
    • Close Challenge 5.
    • Verify Challenge 6 appears.

  6. Challenge 6: Open Your First Pull Request

    • Open a PR from the feature branch to main.
    • Include Closes #N for the Challenge 6 issue or the issue being solved.
    • Verify Gandalf posts PR feedback.
    • Comment on the peer-simulation PR.
    • Close Challenge 6 when ready.
    • Verify Challenge 7 appears.

  7. Challenge 7: Survive a Merge Conflict

    • Run the facilitator conflict script after the student's PR exists:
scripts/classroom/Start-MergeConflictChallenge.ps1 -Repository Community-Access/learning-room-smoke-a -StudentBranch learn/test-student-a
  • Verify the student's PR reports a conflict.
  • Resolve the conflict.
  • Verify the Challenge 7 conflict-marker workflow succeeds.
  • Close Challenge 7.
  • Verify Challenge 8 appears.

  1. Challenge 8: The Culture Layer

    • Read governance/community files.
    • Use the peer-simulation issue for label/triage discussion.
    • Comment reflection evidence.
    • Close Challenge 8.
    • Verify Challenge 9 appears.

  2. Challenge 9: Merge Day

    • Verify the PR has been reviewed and can merge.
    • Merge the PR or have the facilitator merge it.
    • Leave wrap-up feedback on the peer-simulation issue or PR.
    • Close Challenge 9.

Day 2 Core Test

Seed Challenge 10 if continuing from Day 1 did not naturally reach it.

  1. Challenge 10: Go Local

    • Clone the repo locally.
    • Create a branch.
    • Edit, commit, and push.
    • Verify the local commit workflow succeeds.
    • Close Challenge 10.
    • Verify Challenge 11 appears.

  2. Challenge 11: Open a Day 2 PR

    • Open a PR from the locally pushed branch.
    • Verify Gandalf feedback appears.
    • Review the peer-simulation PR title and description.
    • Close Challenge 11.
    • Verify Challenge 12 appears.

  3. Challenge 12: Review Like a Pro

    • Review the peer-simulation PR.
    • Leave at least two specific comments if GitHub allows inline comments.
    • Submit a review verdict if available.
    • Comment evidence.
    • Close Challenge 12.
    • Verify Challenge 13 appears.

  4. Challenge 13: AI as Your Copilot

    • Use Copilot to improve docs/samples/copilot-improvement-before.md or another document.
    • Record what Copilot suggested.
    • Record what the student accepted, rejected, or changed.
    • Compare the result against the peer-simulation PR or real buddy work.
    • Close Challenge 13.
    • Verify Challenge 14 appears.

  5. Challenge 14: Template Remix

    • Create a new non-challenge issue template in .github/ISSUE_TEMPLATE/.
    • Verify it has name: and description:.
    • Open a PR.
    • Verify the issue-template workflow succeeds.
    • Close Challenge 14.
    • Verify Challenge 15 appears.

  6. Challenge 15: Meet the Agents

    • Browse Community-Access/accessibility-agents.
    • Identify at least three agents.
    • Run or inspect one agent.
    • Compare discoveries with the peer-simulation issue or a real buddy.
    • Close Challenge 15.
    • Verify Challenge 16 appears.
    • Verify Bonus A, Bonus B, Bonus C, Bonus D, and Bonus E also appear.

  7. Challenge 16: Capstone Project

    • Choose Accessibility Agents, GLOW, or another meaningful repository.
    • Create or draft an agentic contribution with a clear mission, responsibilities, and guardrails.
    • Open a PR, prepare a branch, or write a contribution issue/plan.
    • Review a peer PR if available; otherwise review the peer-simulation PR and explain what would matter in an agent review.
    • Verify capstone workflow feedback in the Learning Room if the agent file is also represented there.
    • Close Challenge 16.

Bonus Test

The five bonus challenges are optional, facilitator-reviewed, and should unlock immediately after Challenge 15. Students may complete them in any order, before or after Challenge 16.

  1. Bonus A: Improve an Existing Agent

    • Choose an existing agent.
    • Propose and submit a meaningful improvement.

  2. Bonus B: Document Your Journey

    • Write a reflection document.
    • Verify it is clear and accessible.

  3. Bonus C: Create a Group Challenge

    • Design a collaborative challenge for a future cohort.
    • Confirm it can work with peer simulation or real buddy access.

  4. Bonus D: Notification Mastery

    • Configure notification settings.
    • Document the student's notification strategy.

  5. Bonus E: Explore Git History Visually

    • Use GitHub Desktop or GitHub.com history views.
    • Explain what changed over time.

Pass Criteria

The template is ready only when all of the following are true:

  • Challenge 1 can be seeded.
  • Peer simulation artifacts can be seeded.
  • Closing Challenge 1 creates Challenge 2.
  • Sequential challenge creation works through at least Challenge 5 in smoke testing.
  • Challenge 7 can create a real conflict after the student's branch edits the same TODO line.
  • Gandalf PR feedback appears without failing the workflow if GitHub comment APIs are temporarily unavailable.
  • Challenges 10, 14, and 16 autograders run and post useful feedback.
  • Students can complete all peer tasks using seeded simulation artifacts even without cross-repo buddy access.

Authoritative Sources

Use these official references when you need the current source of truth for facts in this chapter.

Section-Level Source Map

Use this map to verify facts for each major section in this file.