VibeCheck

What the company appears to be: VibeCheck is a self-reported AI-native tool for grading computer science (CS) courses. It allows students to code with an AI tutor in a web IDE, while capturing every interaction as an event log. Professors receive line-by-line attribution heatmaps, session replays, and automated oral defenses generated by GPT-5.6. The system is built around deterministic attribution rules rather than ML detection.

What changed: The author states that the tool was developed to address a gap in CS education where traditional code evaluation no longer reflects understanding due to AI-assisted coding. It shifts focus from output correctness to comprehension and judgment, using AI not as a detector but as an enabler of formative assessment.

Single most important open question: Does VibeCheck have any real-world adoption or traction beyond the author's own development experience? The description contains no evidence of customers, revenue, usage metrics, or institutional partnerships.

The description states that VibeCheck is a web-based IDE for CS students with an AI tutor in the sidebar. Students code using Codex + GPT-5.6. Every edit, prompt, response, insertion and paste is logged into an append-only event log.

From this log, professors receive:

• Line-by-line attribution heatmap showing human/AI-generated code
• Session replay timeline of student process
• Automated oral defense generated by GPT-5.6
• Course dashboard with AI dependency and comprehension scores

The system uses deterministic rules for attribution (not ML inference) and is built using Next.js, TypeScript, Monaco Editor, Supabase, and OpenAI APIs.

Evidence: The author describes the architecture and functionality in detail, including how GPT-5.6 roles are used for different tasks (tutor, examiner, process analyst, etc.). It also mentions technical stack like React, Node.js, TailwindCSS, and Zod structured outputs.

The author positions VibeCheck as a solution to the mismatch between how software is now written (with AI) and how it's still evaluated in universities. They argue that:

• Banning AI is unenforceable
• AI detectors produce false positives
• Oral exams are effective but scale poorly
• VibeCheck scales oral exam-like assessment by grounding it in student actions

The product is framed as not a cheat detector, but a tool for measuring comprehension and judgment. It assumes AI use is legitimate and focuses on what matters now: understanding, verification, and decision-making.

Inference: The positioning reflects an evolving view of CS education where the value lies in process rather than output.

The description states that VibeCheck targets CS professors teaching large intro courses (e.g., 300-person classes). These are institutions looking to assess student understanding in a world where AI-generated code is common and traditional evaluation methods fail.

It also implies use by students in CS programs, though the primary user is the professor who reviews the generated reports and defenses.

Evidence: The author mentions “universities across disciplines” reviving oral exams and that “programming courses are where that approach matters most.”

Not evidenced. There is no mention of pricing models, monetization strategies, or commercial plans in the description.

The system uses:

• Codex + GPT-5.6
• Next.js + TypeScript + Monaco Editor
• Supabase Postgres (server-side only)
• Zod structured outputs for multiple roles
• Parallel agents working on git worktrees with frozen contracts
• Rate limiting, token caps, retry logic

The author emphasizes that attribution is deterministic, not ML-based, and that the system was built using a schema-first approach ("AGENTS.md").

Evidence: The write-up includes details about architecture, agent design, and implementation choices like RLS deny-all DB rules.

Not evidenced. There is no mention of:

• Customers or users
• Revenue or funding
• Product adoption or usage metrics
• Institutional partnerships
• Any form of market validation beyond the author’s own development experience

The project was submitted to a hackathon, but this does not constitute traction.

Not evidenced. The description does not mention competitors, existing tools in CS education, or how VibeCheck compares technically or functionally to anything else.

• No traction: No evidence of real-world usage or adoption.
• Unverified claims: The author makes strong claims about accuracy and effectiveness without supporting data.
• Single-person team: Only one member listed (Carlos Sanchez), which raises questions about scalability and execution capability.
• Limited scope: The tool is described as a hackathon submission, not a production-ready product.
• Assumption of AI integration: Relies heavily on GPT-5.6 being available and usable, which may not be true in all environments.

Inference: Without external validation or user feedback, the risk of misalignment between stated goals and actual utility is high.

1. What specific CS courses or institutions are currently using VibeCheck?
2. How does the deterministic attribution model handle edge cases like copy-paste from AI responses?
3. Has there been any external testing or feedback from professors or students?
4. Are there plans for monetization, and if so, what is the business model?
5. What are the technical limitations of GPT-5.6 that might affect scalability or accuracy?
6. How does VibeCheck ensure data privacy and compliance with educational regulations?