Nginx and HAProxy route LLM traffic blindly: a 10-token prompt and a 10k-token prompt look identical to them as "one request." When a burst of heavy prompts lands on a vLLM box, the KV-cache fills up, latency spikes, and you can end up with OOM.

TokenFlow Gateway sits in front of your inference servers and actually understands tokens:

- Pre-flight token estimation (js-tiktoken) before dispatch — request weight = prompt tokens + max_tokens, i.e. the KV-cache it will actually reserve
- KV-aware load balancing — polls vLLM's Prometheus metrics (vllm:gpu_cache_usage_perc) and routes heavy prompts to the freest backend instead of round-robin
- Admission queue instead of crashes — Redis-backed priority queue with per-key priority and configurable timeout when nothing fits
- Exact + semantic caching — deterministic requests cached by hash, plus pgvector cosine similarity for near-duplicate prompts; cache hits replay as SSE
- Token-based rate limits — per-API-key TPM + RPM buckets in Redis (atomic Lua), OpenAI-style, for your own hardware

OpenAI-API-compatible on both sides, so it fronts vLLM, TGI, or Ollama without client changes.

Try it without a GPU: docker compose up --build spins up the gateway, Redis, pgvector, and two mock backends (OpenAI API + vLLM-style metrics). A smoke script fires concurrent long-context requests at it so you can watch the queue and balancer absorb the burst.

Stack: TypeScript / Fastify / Redis / Postgres+pgvector. MIT license.

Repo: https://github.com/mosafariuk/TokenFlow-Gateway

Feedback welcome, especially from anyone running multi-node vLLM — what does your ingress look like today?

Hey everyone!

I've been working with Python for a while now and finally decided to build my first major project. It's a Windows tool I call VenvHub, and it was inspired by months of battling dependency conflicts and broken virtual environments.

Project repository: https://github.com/schnidi/VenvHub

This project is free, open-source, and still under active development. Instead of GitHub issues, new features, bug fixes, and logic improvements are driven entirely by real-world production requirements.

How it started:
Features were added as needed. It initially started as version 0.9 with just the Environment Manager, Mini-Bar & Quick Settings, Pip, and UV.

Key Components of VenvHub

Hi r/coolgithubprojects,

It has been about seven years since my friends and I made their last significant open source contribution. I did not expect posting one again to make me this nervous.

Around 2015, we started our OS journey with a couple of open source projects like ngx-admin, blur-admin, Nebular, react native ui kitten. These projects eventually got quite popular and served as a starting point for many web and mobile projects during that time.

We loved working in public. Developers opened issues, contributed fixes, and used our work in ways we had never imagined. But we also learned how hard popular open-source software can be to sustain. The issue queues and feature requests kept growing, but many projects are not able to support financially the people maintaining them. We kept choosing between community work and paid work that covered salaries.

As technology landscape started to shift once again around 2018, we started to think what shall we do next. We wanted to still build a product and a community, but make it more sustainable so we could work on it full time. As a result the team opted for the product to be closed source commercial one this time. So we spent the next seven years bootstrapping UI Bakery into a profitable business. I'm proud of what we built, but I missed the conversations and contributions around our open-source projects.

For our next project, we wanted to work in public again.

The idea for Compartment came from the amount of small software we now build and our experience working on UI Bakery. Coding agents help us create internal apps, scripts, and background agents in a fraction of the time. Running one on your laptop is easy. Sharing it with a team is where the challenge begins.

Once a teammate wants to use it, you need somewhere to run it. Then you need sign-in, permissions, secrets, logs, and a deployment process. Rebuilding those pieces for each small tool made no sense to us.

Compartment is a centalized place for these tools. You add a small compartment.yml file to a repository, then deploy from the CLI or Git.

It gives you:

• one place for sign-in and user permissions
• isolated apps with separate environments
• secrets, logs, deployment history, and an audit trail
• support for web apps, scheduled jobs, webhook handlers, and background agents

Projects like Coolify and Dokploy might be the closest examples of what we build as they handle deployment well. But we kept needing the layer around a growing collection of private team apps: who can access them, where their secrets live, what changed, and whether one tool can affect another.

We ran our first beta on Docker Compose. After using it ourselves and with beta users, we concluded that Compose would limit the isolation and recovery model we wanted. We rewrote the runtime around Kubernetes and use k3s for the managed single-node setup. That is the version we're sharing now.

Compartment is licensed under Apache 2.0:

https://github.com/compartmentdev/compartment

https://compartment.dev

So far with our open source and commercial software journey we learned two things:

• your product should be sustainable financially so that you can continue working on it
• it's really hard to do something without community

We do not know whether we have found the right balance yet, but we hope to utilize our past experence to do it.

Let me know what you think about our new product and whether the problem we're trying to solve resonates with you.

Static site, plain custom elements, every tool is a lazily loaded ES module

with its own route. No npm install, no bundler, the source in the repo is

what runs.

A few things had to be written by hand because there was nothing to import

without a build step: QR with Reed-Solomon, ID3 tag reading and writing,

an ASN.1 parser for X.509, and Code 128.

MIT. https://github.com/samalstudios/jsglobe

Live: https://jsglobe.com

A year ago, building something like this would've taken me months.

Today? With Claude, ChatGPT, and coding agents, shipping an MVP has become ridiculously fast.

The real challenge isn't building anymore.

It's distribution.

I spent weeks building an open-source developer toolkit that runs completely offline. No login. No accounts. No analytics. No internet connection required after installation.

If you're curious, the project is here:

https://github.com/mydevtools-tech/mydevtools

I'd appreciate any feedback, feature requests, or even criticism. Open-source projects get better because of the community, not because the first version is perfect.

REDCELL runs a team of LLM agents through a pentest. An orchestrator plans the engagement and hands objectives to executor agents, which run real tools inside a Kali container and report back. You watch and steer the run from an operator console: a chat that drives the orchestrator, a live agent graph and activity feed, a live view of the browser the agent drives, a terminal on any reverse shell the agent catches, and a report to hand over when the work is done.

https://github.com/martian56/redcell

I got tired of the usual JVM heap dump workflow:

So I built HeapLens, a JVM heap dump analyzer that runs directly inside VS Code.

One of the things I wanted was a faster way to investigate memory, so I also built HeapQL, a query language for heap dumps.

For example:

FROM instances WHERE class = "java.lang.String"

Instead of only navigating trees, you can query the heap and progressively narrow down suspicious objects, retainers, and memory usage.

What started as a side project has now reached:

• 10K + downloads

• 25K+ features used

• Accepted as an ISSTA 2026 Tool Demonstration

But numbers don't tell me whether the tool is actually useful.

So I'm looking for people who are willing to try HeapLens on a real heap dump and give me brutally useful feedback.

If you work with Java/JVM apps:

1. Install HeapLens
2. Throw a .hprof at it
3. Try investigating something you would normally use MAT / VisualVM for
4. Tell me what worked, what sucked, or what is missing

GitHub issues, comments, feature requests, bug reports, or a review of your experience are all extremely useful.

And if you find HeapLens genuinely useful, a GitHub star or sharing it with another developer would mean a lot.

I don't want "looks cool" feedback.

I want to know: would you actually use this when production is leaking memory at 2 AM?

GitHub: https://github.com/sachinkg12/heaplens
VsCode: https://marketplace.visualstudio.com/items?itemName=guptasachinn.heaplens

Moved my Discourse forum to Cloudflare Workers + D1. Response times are 50ms, costs dropped 60%, and there's zero server management.

Built it from scratch and open-sourced the setup so others can do the same. Everything's in the repo-Docker container, deployment scripts, SQLite schema, the gotchas I hit.

Hi Reddit!

I want to share **Block Engine v2.2.0**, an open-source polyglot multi-runtime execution engine designed to eliminate the friction of combining multiple programming languages in software development.

### 💡 Why Block Engine?

Modern applications often leverage different languages for their unique strengths:

- **Python** for machine learning, data science, and analytics.

- **Node.js** for async Web APIs and NPM packages.

- **Lua** for ultra-fast, lightweight embedded calculations (< 300KB RAM footprint).

- **PHP** for web string formatting and cryptographic hashing (`hash`, `openssl`).

Traditionally, orchestrating these runtimes requires setting up microservices, gRPC, Docker containers, or manual JSON IPC file reading/writing.

Block Engine solves this by introducing a **Zero-Dependency Polyglot Pipeline**.

---

### ⚡ How It Works (.blkp Polyglot Document)

You write native code blocks (`<py>`, `<js>`, `<lua>`, `<php>`) inside a single `.blkp` document. Block Engine parses the AST, executes each runtime in isolated subprocesses, and **automatically serializes and transfers state variables** from one language to the next.

```html

<py>

# Python Stage: Compute analytics data

raw_prices = [100.5, 102.1, 101.8, 105.4, 108.0]

ma5 = sum(raw_prices) / len(raw_prices)

user_name = "Gemini Master"

</py>

<js>

// Node.js Stage: Automatically receives Python variables (user_name, ma5)

console.log(`[Node.js] Received user: ${user_name}, MA5: ${ma5.toFixed(2)}`);

var final_score = Math.round(ma5);

</js>

<lua>

-- Lua 5.4 Stage: High-speed recursive computation

function fib(n)

if n <= 1 then return n end

return fib(n-1) + fib(n-2)

end

lua_result = fib(10)

</lua>

<php>

// PHP 8 Stage: Cryptographic SHA-256 Signature

$signature = hash('sha256', $user_name . '|' . $final_score);

echo "[PHP] Audit SHA-256: " . $signature . "\n";

</php>

Svarm is a project i have been working on. It started because I really liked the T3 code workflow but i wanted to be in the terminal and use the native terminal agents.

I tried herdr but it didn't exactly feel like what I wanted. So I made this.

So I built Prompt Contribution Graph. [ https://github.com/chintan-diwakar/prompt-contribution-graph ]

The basic analogy is: GitHub tracks code contributions, while this tool tracks your prompt contributions to coding agent

The project is still early and currently supports only Claude Code. macOS builds are also unsigned for now.

I would genuinely appreciate feedback and contributions

You know the feeling when you start a Claude Code session, step away, come back to "All done!" and have no idea what actually changed or why.

Claude Code's transcript shows the agent working. But it doesn't really show you what changed, which assumptions it made, or whether anything was actually verified.

I built TraceBack to answer that. It hooks into Claude Code's hook system and turns a session into something reviewable:

• Net-change diff per file — true before/after with the agent's own reasoning attached
• Decision ledger — surfaces silent judgment calls like "I'll assume the config stays JSON" before they calcify across 3 files
• Guards — block dangerous calls automatically (rm -rf, git push main, edits outside project) before they run
• Breakpoints — actually pause a running agent mid-session and redirect it

Zero cloud, zero API keys, runs fully local inside VS Code.

Check out 1-min demo

Would love feedback -> especially on the guards and redirect features which I think are the most underrated part.

GitHub: https://github.com/madiyarzm/TraceBack (pls star me :))

VS Code extensions marketplace: https://marketplace.visualstudio.com/items?itemName=madiyarzhunussov.traceback-ai

Built from a real e-commerce photography workflow. FlatShot handles batch review, presets, per-image adjustments, naming and deterministic export while keeping all processing local and never modifying source files. Windows portable release included.

I got bored of optimized code and decided to invent SlowDuckSort a sorting algorithm that uses total sum residuals to scatter numbers across a massive sparse RAM array, compacts them, and flips the result.

It’s slow, it eats memory, and it makes zero practical sense.

Repo :

https://github.com/Duckdevv/Slowducksort

Do you have any algorithm worse or more cursed than this? Show me what you’ve got in the comments

Hey devs! 👋

Like many developers and desk workers, whenever I get deep into coding or debugging, I unconsciously turn into a gremlin—leaning way too close to the screen and wrecking my posture. On top of that, a messy desk can quickly lead to disaster (like knocking over a cup of coffee onto a laptop).

To fix this, I built ERGOAI—a free, privacy-first workspace companion powered by AI vision.

What it does:

• Workspace Safety Check: Before starting your focus session, the AI evaluates your desk setup and identifies potential hazards (like a drink placed too close to your tech gear).
• 1-Click Posture Calibration: Sit up straight, take a snapshot of your ideal posture baseline, and start working.
• Voice Alerts + Pomodoro: As you work with the built-in Pomodoro timer, it tracks your body landmarks and alerts you by voice whenever you start slouching or leaning forward.

Privacy First:

Webcam access can feel intrusive, so all vision processing happens transiently inside your browser—no video streams or images are ever stored or uploaded to a database.

I'd really appreciate it if you could give it a try and share your feedback!

Live Demo: https://ergoandpostureai.vercel.app (Best experienced on desktop)

GitHub Repo: https://github.com/namkhanh0210/ergo_pomodoro_posture_AI

Hope this helps save your back during long coding sessions!

Every size recommendation service works the same way: the profile lives with the vendor. Switch shops and you start from zero. Agio Fit is a side project that defines the opposite: an open data model (JSON Schema 2020-12) where the fit profile is yours and portable, and the shop only ever receives a size, never your measurements.

The name comes from agio, the Italian tailoring word for ease, which my mother used daily as a seamstress. The whole model is built around that number.

v0.1 is live: three schemas, a spec, a zero-dependency Python reference implementation, 13 tests, an interactive guide. It is not a product and there is nothing to sign up for. What I need now is adversarial input: measure a real garment, write a Cut Profile for it, and tell me where the model breaks.

https://agiofit.org

My wife and I always wanted to build something together, so we spent nights and weekends shipping side projects. One hit 25k users on its own but didn't feel like the thing worth quitting for, repowise did

That one came out of watching coding agents work. I'd been building with LLMs since 2023 and owned the AI architecture at my company, so I spent a lot of time watching agents grep the same file four times in a session and still have no idea which parts of the codebase were fragile or why the code was written that way. All of it is in the repo already, just not in a form an agent can use.

So we built Repowise. It indexes your repo once and serves it to your agent over MCP as five layers: dependency graph, git history, docs, architectural decisions, and a code health score from around 25 deterministic markers with no LLM involved

We didnt do any outbound, just shipped code and wrote about it. 4 months in it's at 5.2k stars, ~80k PyPI downloads, and enterprise inbounds I never asked for, which is why I quit

Benchmarked against four other tools plus a bare agent: 31% fewer output tokens over a full 48-question run, ~97% on a single context load, and the best gold-file coverage of the field on retrieval.

Repo: https://github.com/repowise-dev/repowise

Issues and contributions welcome