ClassLens

ClassLens is a self-reported tool that claims to diagnose student misconceptions from quiz answers using AI, then generate recheck quizzes tailored to those misunderstandings. The author states it was built in three days as part of an OpenAI hackathon submission and uses GPT-5.6, SymPy, and local retrieval for its core functionality.

The description states ClassLens diagnoses 41.1% @1 accuracy on held-out real student answers when mapping wrong answers to Eedi taxonomy misconceptions — a claim that is self-reported and unverified. It also claims to build executable buggy-math rules to generate distractors, with 27.5% of generated items correctly reproducing student errors.

There is no evidence of revenue, customers, or adoption beyond the author's own account. The tool outputs static HTML reports and Markdown reteach plans, and runs on a local GPT-5.6 model with caching to reduce cost per use.

The single most important open question: Is the 41.1% @1 accuracy claim valid in real-world usage, or is it an artifact of the benchmark setup?

The description states ClassLens:

• Takes a CSV of quiz responses from a class
• Maps wrong answers to named misconceptions from the Eedi taxonomy (2,587 labels)
• Aggregates a class profile ranked by how many students each misconception affects
• Writes a reteach plan
• Builds a recheck quiz using GPT-5.6 for stem generation, SymPy for correct answer computation, and executable buggy-math rules to generate distractors
• Outputs static HTML class report, Markdown reteach plan, and verified recheck quiz

The tool is described as having been built with Codex, GPT-5.6, SymPy, local retrieval (BGE-large hybrid + TF-IDF), and Python-based components including pandas, numpy, scikit-learn, sentence-transformers, pytest.

The description states ClassLens was inspired by the need for teachers to quickly understand why students missed questions on exit tickets. The author's claim evolved from building "the thing that answers why" to providing a number telling how often it is right.

The author claims:

• 41.1% @1 accuracy in diagnosing misconceptions from real student answers
• 55.4% @3 accuracy
• 27.5% of generated distractors correctly reproduce student errors
• Rejection mechanism logs reasons for failed quiz items and shows them in the report

The positioning is described as a tool that helps teachers quickly identify and address student misconceptions, with an emphasis on automation and verification.

The description states ClassLens targets teachers who:

• Give exit tickets or quizzes
• Want to understand why students missed questions
• Need to quickly diagnose class-wide misconceptions
• Want to build recheck quizzes tailored to those misunderstandings

The author describes the tool as designed for teachers running this on "every exit ticket for every class" and emphasizes that it only works if one class costs "cents."

Not evidenced. The description does not state any pricing model, revenue streams, or commercialization strategy.

The description states:

• Built with Codex, GPT-5.6, SymPy, local retrieval stack (BGE-large hybrid + TF-IDF), and Python libraries
• Uses a hybrid retrieval approach combining BGE-large and TF-IDF with reciprocal-rank fusion
• Employs executable buggy-math rules to generate distractors
• GPT-5.6 runs at inference time for hypothesis generation, reranking, verification, stem extraction, and item generation
• All components run on gpt-5.6-luna (a cheaper model) to keep costs low
• Caching is used for GPT responses by model, prompt, and reasoning effort
• The system can be regenerated without an API key due to committed cache

Not evidenced. The description does not mention any customers, revenue, usage metrics, or adoption beyond the author's own account.

Not evidenced. The description does not provide information about competitors or market positioning beyond the author's own claims.

• The 41.1% @1 accuracy claim was initially inflated due to a benchmarking error (GPT-5.6 was included in the "local retrieval only" baseline). The author states they had to manually verify this by running without GPT client instantiated.
• The tool is described as a hackathon submission with no evidence of commercial viability or scalability beyond one developer
• No evidence of real-world testing, customer feedback, or product-market fit beyond the author's own claims
• The system relies heavily on GPT-5.6 and SymPy, which may not be scalable or cost-effective for widespread adoption

1. What was the actual methodology used to validate the 41.1% @1 accuracy claim? How did you ensure that the benchmark wasn't artificially inflated?
2. Can you provide more details about how the executable buggy-math rules are generated and maintained?
3. What is the current state of the Eedi taxonomy integration and how many misconceptions are currently supported?
4. How does the tool handle edge cases or unusual student responses that don't map cleanly to existing misconceptions?
5. What are the specific use cases where ClassLens has been tested in real classrooms?
6. How do you plan to scale this beyond a single developer's prototype?