RAG Params Finder

The description states that RAG Params Finder is a tool for evaluating RAG application parameters—specifically vector databases, embeddings, chunking, and retrieval methods—before building a full RAG app. It claims to sweep over 13 embedding models, 5 chunking methods, and 3 retrieval methods, scoring combinations against persona-organized questions. The author describes it as a "free pre-eval tool" that supports real-world setups and includes features like a dashboard, live phase indicators, and results export.

The project is self-reported as being built by one person (Mani Sarkar), using technologies including Docker, FastAPI, JavaScript, MongoDB, Node.js, Python, Shell, SIE, TypeScript, Vite, VoyageAI. It was submitted to the OpenAI 2026 hackathon on Devpost.

Key commercial due-diligence read: The tool appears to be a pre-evaluation utility for RAG system configuration, but there is no evidence of revenue, customers, or adoption beyond its own self-description. The core feature—sweeping every combination—is emphasized as central to the product identity. However, the description does not indicate whether this tool has been used in production or by any external users.

Most important open question: Is there any evidence that this tool is being used or tested by developers or organizations outside of its creator?

The description states:

• RAG Params Finder is a practical RAG sweep tool.
• It evaluates vector databases, embeddings, chunking, and retrieval methods in real-world setups.
• It helps users find the best parameters for RAG applications by sweeping over:
  • 13 embedding models (Voyage AI hosted, plus local sentence-transformers / SIE open-source)
  • 5 chunking methods (Fixed, Recursive, Token, Sentence, Semantic)
  • 3 retrieval methods (Dense, Sparse, Hybrid)
• It scores combinations against persona-organized questions.
• It includes a dashboard, live phase indicators, elapsed/ETA timers, a Search Explorer, and results export.

Inferred from the description:

• The tool is designed to pre-evaluate RAG configurations before full implementation.
• It supports multiple vector databases and embedding models, suggesting it's intended for experimentation and optimization.
• It uses a grid search approach across all combinations (as stated in the "Challenges" section), which is described as a core feature.

Not evidenced:

• Whether the tool has been used by others or integrated into larger workflows.
• Whether any of the listed models or methods are actually supported in practice.
• Any pricing, usage metrics, or user feedback.

The description states:

• The tool is positioned as a free pre-eval tool before running expensive evals on RAG apps.
• It aims to help users find optimal parameters for RAG systems.
• It builds upon previous work: a lighter version that ran in the browser (https://github.com/neomatrix369/pre-rag-explorer-dashboard).

Inferred from the description:

• The positioning is developer-focused, targeting those building or optimizing RAG applications.
• It positions itself as a preliminary evaluation tool to reduce risk and cost of experimentation.

Not evidenced:

• How it differentiates from other tools in the market.
• Whether there are any competitors or prior art referenced.
• Any claims about performance improvements, accuracy, or adoption beyond its own description.

The description states:

• The tool is aimed at developers working with RAG systems.
• It helps users find optimal parameters for their RAG apps.
• It supports real-world setups, implying it targets practical use cases rather than academic or experimental ones.

Inferred from the description:

• The primary user is likely a technical practitioner (developer, data scientist) working in NLP or AI.
• The ICP may include those who are experimenting with RAG systems and need to optimize performance before full deployment.

Not evidenced:

• Specific personas or customer segments beyond general developers.
• Any evidence of actual users or target market size.
• Whether the tool is used by teams or individuals.

The description states:

• The tool is described as free.
• It is presented as a pre-eval tool, suggesting it’s not a paid product but rather an open-source or utility-based offering.

Inferred from the description:

• There is no explicit mention of monetization or pricing.
• It may be intended for open-source or community use.

Not evidenced:

• Any revenue model, subscription plans, or paid features.
• Whether there are any commercial partnerships or licensing terms.
• No evidence of a business model beyond its self-description as a free tool.

The description states:

• Built with: Docker, FastAPI, JavaScript, MongoDB, Node.js, Python, Shell, SIE, TypeScript, Vite, VoyageAI.
• The project was built in stages, starting from a prototype (https://github.com/neomatrix369/pre-rag-explorer-dashboard).
• It uses simple solutions, including support for one vector database, local embedding model, and a few chunking/retrieval methods initially.
• Coding agents like Codex were used during development.

Inferred from the description:

• The tool is built using modern stack components typical of backend + frontend web apps.
• It was developed iteratively with early prototypes.
• It supports grid search as its core feature, which implies a structured evaluation process.

Not evidenced:

• Whether it has been scaled or optimized for production use.
• Any performance benchmarks or scalability claims.
• No evidence of API access, integrations, or extensibility beyond what is described.

The description states:

• The project was submitted to the OpenAI 2026 hackathon.
• It has a single team member, Mani Sarkar.
• The author mentions accomplishments like adding parallelization and Bayesian search methods.
• There is a backlog of features planned for future development.

Inferred from the description:

• The tool is in an early stage, likely a prototype or MVP.
• It has been validated through a hackathon submission.
• It shows signs of iterative improvement but lacks evidence of traction or adoption.

Not evidenced:

• Any user base, customer data, or real-world usage.
• No metrics on how many times it’s been used or by whom.
• No evidence of product maturity beyond its own description.

The description states:

• The tool is inspired by two prior projects (one being a browser-based version).
• It sweeps over multiple embedding models, chunking methods, and retrieval methods.
• It supports real-world setups.

Inferred from the description:

• There may be similar tools in the RAG space for evaluating configurations.
• It positions itself as a comprehensive sweep tool, which could differentiate it from simpler explorers or dashboards.

Not evidenced:

• Any direct competitors or market analysis.
• No mention of pricing, features, or adoption levels of other tools.
• No indication of how this tool compares to existing solutions in the market.

The description states:

• The tool was built by a single person (Mani Sarkar).
• It uses coding agents like Codex, which failed or were slow at times.
• It includes parallelization and Bayesian search as additions, suggesting complexity in implementation.

Inferred from the description:

• Risk of limited scalability or maintenance due to single-person ownership.
• Risk of technical debt or instability if coding agents are relied upon heavily.
• Risk that Bayesian search is an add-on, not core functionality, which may dilute the value proposition.

Not evidenced:

• Any evidence of technical failures in production.
• No mention of security, data privacy, or compliance issues.
• No indication of long-term roadmap or sustainability beyond current development plans.

1. What is the actual utility of sweeping every combination versus using Bayesian search or other optimization techniques?
2. How does the tool handle large datasets or complex RAG setups in practice?
3. Has anyone else besides the creator used this tool? If so, what were their experiences?
4. Are there any plans to monetize or commercialize the tool beyond its current open-source nature?
5. What are the limitations of the current version, and how do you plan to address them?
6. How does it compare to existing tools in the market for RAG parameter tuning?
7. Is there a way to integrate this into existing development workflows or CI/CD pipelines?