EdgeSplit

The description states that EdgeSplit runs LLM prefill on a laptop and decode on a phone, using two handoff mechanisms:

• V1: saves state to a file and uploads it over HTTP.
• V2: patches llama.cpp to export raw sequence state directly via a custom TCP protocol.

The system uses:

• CUDA for laptop-side prefill
• Termux on Android for phone-side decode
• FastAPI router on the laptop
• Custom binary framing protocol over TCP

It is built with technologies including: android, arm, c++, clang, cmake, codex, cuda, fastapi, gguf, gpt-5.6, llama, llama.cpp, neon, nvidia-smi, pydantic, python, qwen3, sockets, sqlite, tcp, termux, ubuntu, uvicorn, wsl2.

The author claims that EdgeSplit:

• Turns everyday hardware into a distributed inference system.
• Splits LLM inference phases (prefill vs decode) across devices, similar to large-scale production systems like NVIDIA’s Dynamo architecture.

This positioning implies:

• A novel approach to edge computing for LLMs
• Use of existing consumer-grade hardware instead of dedicated infrastructure

Inference The project is positioned as a technical innovation in distributed inference, not a commercial offering. The author emphasizes reproducibility and correctness over scalability or market readiness.

Not evidenced.

The description does not identify:

• Specific customer segments
• Use cases beyond the hackathon context
• Target industries or personas

Inference If this were to evolve into a product, it would likely target developers or researchers who want to experiment with LLM inference on low-cost hardware. However, no such intent is stated.

Not evidenced.

There is no mention of:

• Revenue model
• Pricing structure
• Monetization strategy
• Paid features or tiers

Inference The project appears to be a proof-of-concept with no commercial business model described. Any future monetization would need to be inferred from how the technology might scale or be packaged.

The author reports:

• Implementation in WSL2 Ubuntu (laptop) and Termux (phone)
• Patching of llama.cpp for V2
• Use of FastAPI, TCP sockets, and custom binary framing protocol
• Benchmarking methodology: warmed repetition with statistical tracking
• Challenges including device-specific crashes, profiling issues, and unreliable power sensors

Inference The technical execution shows a high level of engineering effort and attention to detail. However, there is no evidence of production-grade delivery or deployment.

Not evidenced.

There are no signs of:

• Customers
• Revenue
• Product usage
• Adoption metrics
• Market traction

Inference This remains a hackathon project with no demonstrated traction or maturity beyond prototype development.

Not evidenced.

The description does not reference:

• Competitors
• Existing solutions in the space
• Market positioning relative to others

Inference While the idea of distributed inference is not new, there is no evidence that EdgeSplit is part of a competitive landscape or has differentiated itself from other tools or platforms.

• Proof-of-concept only: No commercial product or adoption.
• Device-specific limitations: One phone failed due to hardware incompatibility; performance may not generalize.
• Unreliable instrumentation: Power sensor issues suggest lack of robustness in measurement tools.
• Single founder: Team size is listed as 1, which raises questions about scalability and execution capacity.
• No verification of results: Multiple false positives were caught only after deeper inspection.

1. What are the actual performance gains in real-world usage vs. controlled benchmarks?
2. Has this been tested across multiple phone models or platforms?
3. Are there plans to move beyond a hackathon prototype into a product or service?
4. How would you handle edge cases like network interruptions during handoff?
5. Is there any interest from developers or enterprises in using this system?
6. What are the limitations of the current implementation that prevent broader adoption?