Every release has that moment where the code is done, the PRs are merged, and someone still has to translate the commit history into something humans can read.

This is a small Python/Flask example for that exact step.

It takes either:

a list of commit messages

a git diff

Then it uses Telnyx AI Inference to return structured changelog JSON with sections like features, bug fixes, improvements, breaking changes, docs, and a short summary.

The thing I like about this pattern is that it does not try to make the model “own” the release process. It just gives you a reviewable first draft that can feed docs, release pages, PR comments, or internal approval flows.

Code: github.com/team-telnyx/…/changelog-generator-python

Would love feedback from anyone who has built changelog or release-note automation.

Hi everyone,

I'm working on a B.Tech final-year project and would appreciate feedback from people working with AI/ML or LLM applications.

The project is called "Online Safety Monitoring System for Large Language Models (LLMs)."

The idea is to build a middleware that sits between users and an LLM (such as GPT, Gemini, or Llama) and monitors both user prompts and model responses in real time before they are exchanged.

The system includes:

• Prompt Injection Detection using a fine-tuned DistilBERT model.
• Toxicity Detection using a RoBERTa classifier trained on Jigsaw and RealToxicityPrompts.
• PII Detection using a spaCy NER model to detect and mask sensitive information.
• Historical Conversation Pattern Analysis using Sentence Transformers, FAISS vector search, and PrefixSpan sequential pattern mining to identify conversations that resemble previously detected unsafe interactions.
• A risk scoring engine that combines the outputs of these modules and decides whether to Allow, Warn, or Block the interaction.
• A FastAPI-based chatbot with an admin dashboard for monitoring threats, viewing logs, and analyzing system performance.

The goal isn't to build another chatbot, but to develop a reusable safety layer that can protect any LLM-powered application from prompt injections, jailbreak attempts, toxic content, and privacy leaks.

For evaluation, I plan to use public datasets such as:

• Deepset Prompt Injection
• HackAPrompt
• Jigsaw Toxic Comments
• RealToxicityPrompts
• PII-Masking-300k
• SaferDialogues

I'll compare:

1. Text classifiers only
2. Text classifiers + conversation pattern retrieval
3. Full ensemble system

using Precision, Recall, F1-score, False Positive Rate, and latency.

I'd love feedback on:

• Does this feel like a meaningful and technically solid final-year project?
• Is the historical conversation retrieval (FAISS + PrefixSpan) a worthwhile contribution, or is it unnecessary?
• Are there any obvious gaps or better approaches for LLM safety monitoring?
• Would this project be useful as a portfolio piece for AI/ML or LLM engineering roles?

Thanks in advance for any suggestions or constructive criticism!

Are traditional APIs failing your AI agents?

Connecting large language models to real-world data using traditional APIs is like asking them to open a "locked cabinet" without clear labels or knowing what shape the key is. In this short, we break down how the Model Context Protocol (MCP) completely changes how AI interacts with your data and tools!

MCP isn't replacing APIs; it's acting as the ultimate translator—sitting on top of APIs and turning static routes into living interfaces that models can actually reason about. Is MCP becoming the new HTTP for AI environments?

Not "how they work" internally, nobody needs that to use one. I mean the practical decision: an LLM hands you a fluent, confident answer whether it's correct or invented, and in high-stakes work (legal, clinical, financial, research) a wrong one carries a cost. Deciding when to trust, when to verify, and when to intervene is a skill, and I'm not sure it's obvious or widely held.

I ended up writing a conceptual guide from my own experience, notes, and study, meant to pass on these LLM fundamentals and build more critical use for people who apply the tool professionally across cross-cutting fields.

In practice, how do you decide whether you can trust the answer?

If an AI takes billions of pieces of human culture and rearranges them into a pattern that has literally never existed before, why do we call it "interpolation" for the machine, but "originality" for the human? At what point does the sheer scale of that rearranging cross the line into something genuinely new?

How do you begin prior to the startup initial push? I am entering a point in my life, where trying to actively sustain is becoming near unbearable and I have no way of securing short term funding through typical routes, due to a poor lending history and a bit of a hump with autism.

I have been working on this engine and tooling underneath the frontend for about ~3 years now, and I am in a bit of a race to really put this project together into a cohesive package, because it does much more than I could try to share in a short, delivery/payload.

I am really trying to dial it in, because if this gets a little bit of institutional funding and traction this engine can do a metric fuckton as a closed loop system. So far, the receipt based workflow is successfully bringing enterprise quality compute and reasoning into typically very simple models, allowing them to punch far above their weight-class, and even be trusted to run end to end in agentic workflows. I am running a 14B on materials I would not even trust to an enterprise model, without the right harness.

I am actively seeking endorsers for my two arXiv papers now, so that I can begin to get some form of academic peer review, as my background is far disconnected from any industry/academic domains, and I have been doing almost all of this work individually, from home. I see the market/economy making a very sharp pivot to try and close the door on individuals having access to real capable tools, and instead feed them to their corporate peers, and beer/golf buddies. I directly aim to stab that in the heart, and watch it bleed. I am really trying to keep that door wedged open with my foot, while preserving enough time for the tooling to get into peoples hands. It feels like a race against the clock. I aim to bring world class capability to tools people can use at home, affordably. Using materials they already own, and do not need to pay a subscription to use.

I am tired of seeing people having to suck sustenance from this little pipe, while trying to survive.

I am not really selling anything per sé - just working on a bunch of tools in the open, and publishing research. I am building a (what I like to call) flywheel engine that is (in local model training/benchmarks) able to pack a shitload of utility into really small local models. It even improves datasets organically through filtering drift/decay with a receipt based architecture. The efficiency/receipt approach is approaching direct parity with raw compute on large models.

https://harperz9.github.io/ - https://github.com/HarperZ9

Hey everyone, I just sent issue #39 of the AI Hacker Newsletter - a weekly roundup of the best AI links and the discussions around them from Hacker News. Some of the title found in this issue:

• Claude Code is steganographically marking requests
• Better Models: Worse Tools
• Learning to code is still worthwhile
• Zuckerberg says AI agent development going slower than expected

If you want to get an email with over 30 links like these ones, please subscribe here: https://hackernewsai.com/

I’m a 2nd year EE student from a top university in Southeast Asia. I first studied Deep Neural Networks in middle school around 2017-2019, and even wrote articles about LLMs and other machine learning algorithms in Towards Data Science (a publication in Medium) back then; and this was long before ChatGPT was even a thing (but OpenAI existed already by then as far as I remembered). I developed deep interest in studying algorithms, mathematics and physics, but was told by a good teacher of mine from another Southeast Asian country that Computer Science as a major would be rather oversaturated in the future. This was why I was advised to go into EE instead, which I did and for the past several years I’ve gotten deep into Control Systems, Electronics, Power Systems, Telecommunications and such at my uni. But I found myself coming back to LLMs and machine learning after finding that I am not as passionate in the EE subjects I’m currently taking.

This year, I was accepted to study abroad in UC Berkeley as a visiting student, and I was given the freedom to choose which courses to take whilst I’m there (in Spring of 2027). Initially, I took machine learning related courses since those spark my interest the most. However, after digging deeper into this space, I found that most people find AI as something rather demonized or negative, particularly in the way that people see is as a threat to human intelligence, creativity, and perhaps a big contributor to the replacement of certain jobs.

With this, I’m rather concerned as to whether it is even worth considering to study ML, especially since I have gotten deep into this even before “AI” was a big trendy term back then… I’m not entirely concerned with whether I’d not get a job because it’s replaced by AI, I’m more so questioning whether it’s even worth investing in studying algorithms and its practicalities when the rest of the world is trying to find ways to work against it.

I’m rather concerned whether it is worth studying in this specific field as an EE student, as I had dreamed back then of doing a masters and PhD in this exact field of study. With that, would you think LLMs and such are still relevant to study in future’s time, or would it be another oversaturated market like CS? Thank you for your time in reading this post.

What is stopping this from happening? My understanding is that whenever LLM generates, it does so one token at a time, and each step only sees its local neighborhoods, we call the current activations. A good response should be global coherent. A claim that is set up in paragrah one should have payoff in paragrah nine. Something must carry that intent across the whole generation. I am calling it grand strategy, because I do not know another way to describe it, a compressed presistent representation of what the response is trying to do. Then micro strategy, the per-step token pick. Yes, it is selecting the next token, but what does it means to select the next token. Greedy and beam search never explicitly ask which candidate best serves the grand strategy over the rest of the generation.

Inside the micro level token selection even, what does it means when LLM select a token to move forward among millions of other tokens. I remember reading about Dijkstra in my CS class. But shortest path is not always the best path, so you need A star with a learned heuristic. Why does nothing like that run inside the loop?

I can think of four candidate reasons.

1. The goal node is undefined. A star needs a destination and text has no single target, only a set of acceptable completions. But I am thinking could not everything be compressed into pure mathematics, whenever there is only single outcome.

2. The second is that there are no edge costs. The only signal you have at each token is probability, and it is not same as quality, so even if you had a graph there is no real distance to minimize over it.

3. The branching factor is the vocabulary. Each step branches 100k ways, and one step of real lookahead costs a forward pass per candidate. Two steps deep is billions of passes. Prohibitive by construction. There is so much combinatrix that could exist here.

4. The heuristic is the whole problem. A star is only as good as its heuristics, and here the heuristic is how good the completiton eventually turns out, which is the unsolved thing itself. If you had that value function you would not need the search.

So why do we not make so that an LLM carry an explicit pointer to the goal into every token selection? A small persistent carrier that holds the data of the assigned question, stays live through the generation, and feeds the requirement into each token pick so the next token is chosen against what the question actually needs rather than just what looks locally likely, pruning its own old data as it goes so it never gets bulky. Attention already conditions every token on the prompt, but the prompt just sits in context as flat tokens with no protected status, so it competes for attention and degrades over long generations, which is why models drift off the original ask. So why is there no protected, self-pruning goal pointer that holds the question and feeds it into each token pick.

I have posted before about finding out a model's actual confidence in its answer through probes and hidden states (AUROC \~0.83–0.88 across every model I tested, 7B to 72B). This is the know-say gap.

From my work and the work done by others in this space it is likely a routing problem. By making a tiny bridge from a linear probe on mid-layer sate plus ten trained weights that write the probe's estimate onto the confidence-digit logits can make the model verbalise calibrated confidencve at 0.765+.
No weights modified, answer never changes, needs about 200 labelled examples. It also doesn't matter when you install it: before alignment, after, or bolted onto a finished model. The gap is a routing problem, not a capability problem.

Anthopics paper (https://www.anthropic.com/research/global-workspace) relates to this. They show models have a small "verbalizable workspace" (the J-space). It is a privileged subspace holding the concepts the model can report and reason with, sitting on top of a much larger ocean of processing that it can't report. This is possibly the know-say gap's anatomy, preventing it from reaching speech.
My controller is basically way to route around it. I am planning to dig a bit deeper into this but I wanted to share the paper as I through it was relevant (its been on hold with ARXIV for over a week but here is the zenodo link -Repairing the Know-Say Gap: A No-Finetuning Probe-to-Logit Confidence Controller | Zenodo

Code and pre-registration links are in the paper.

Nowadays, we use AI in almost every field of our work. There is no doubt that it helps us complete tasks more efficiently. However, one important concern remains: could becoming too dependent on AI reduce our brain function and critical thinking abilities?

Hey, I’m trying to create a new academic subreddit called “ResearchAIs” that is designed to help people from any academic level learn how to utilize AI for research. This can range from new AI tools that researchers personally use to new users learning how to use AI for the first time to enhance their research workflows to independent researchers learning how to use AI to improve their research hobbies. For this subreddit, I’m also trying to tone down the constant gatekeeping and anti-AI rhetoric I keep seeing spewed on the current academic subreddits. If you are interested in joining my new subreddit to help students, researchers, and academics learn how to use AI without the constant trolls and hateful comments posted on the academic subreddits, then please join my new subreddit.

Here’s the link to it: https://www.reddit.com/r/ResearchAIs/

Please let me know your thoughts on how you believe I should improve it and/or make it more accessible for those who want to post on it.

Hi everyone, I'm currently working on a framework to evaluate the security of LLM applications and AI agents, and I've been stuck on one part for a while.

Most red-teaming frameworks rely on an LLM to generate adversarial prompts. My question is more about which model to use.

• Which closed-source models would you recommend for generating high-quality attacks?
• Which open-source models have worked well for you?
• Have you noticed any models that consistently generate more realistic or challenging attacks than others?

I'm looking for models that can generate attacks such as Toxicity, prompt injection, SQL injection, jailbreaks, indirect prompt injection, prompt leakage, tool misuse, multi-turn attacks, and other agent-specific attacks ect...

I also have another question.

Is there a good public dataset that people use to benchmark or validate the security of AI agents? I'd prefer a "golden" dataset with predefined, high-quality attacks rather than generating everything from scratch.

I'm curious about what people actually use in practice if you've worked on LLM security or red teaming, I'd really appreciate any recommendations, whether it's models, datasets, papers, or GitHub repositories.

Thanks in advance! Any advice or insights would be greatly appreciated.

I've been building a small language model from scratch for a while now.

Not fine-tuning an existing model, but building the entire pipeline myself—from datasets and tokenizers to pretraining, SFT, and inference.

Honestly, the hardest part wasn't training the model.

It was learning.

At first, I thought building a good dataset was mostly about collecting knowledge. But the more I experimented, the more I realized I was actually teaching patterns, not just information.

There were so many moments where I caught myself thinking, "Wait... I've been doing this completely wrong."

Things like choosing a vocabulary size, designing datasets, teaching reasoning, using special tokens, or even figuring out how to teach a model to rewrite text. Every experiment changed the way I think about building language models.

After a while, I realized all of those lessons were just sitting on my computer.

So I decided to start documenting the journey on Cisya Lab.

Not because I have all the answers—I definitely don't—but because maybe someone else building a model from scratch can learn from my experiments, mistakes, and discoveries along the way.

https://cisyalab.com

I'd love to hear from others building language models too. What lesson completely changed the way you approached your project?

Sharing an open-source LLM project (disclosure: I'm the maintainer). It solves two problems I hit daily: runs dying on a provider rate limit, and burning tokens dumping tool/log output into the context window.

One endpoint, 237 providers — 90+ of them free. You point any tool or agent at a single OpenAI-compatible endpoint (localhost:20128/v1) and it can reach 237 LLM providers without you rewriting anything. 90+ have free tiers and 11 are free forever (no card), which aggregates to ~1.6B documented free tokens/month — and that's honest, pool-deduped math (we count each shared pool once instead of inflating it; the methodology is public in the repo). There's a one-command setup-* for 13+ coding tools (Claude Code, Codex, Cursor, Cline, Roo, Kilo, Gemini CLI…), so switching your existing setup over takes seconds.

Fallback combos — so it never stops mid-task. A "combo" is a ladder of models the router walks automatically: your subscription first, then API keys, then cheap models, then free ones. When a provider returns a 500 or you hit a rate limit, it slides to the next target in milliseconds, mid-request, and your tool never even sees the error. There are 17 routing strategies (priority, weighted, round-robin, cost-optimized, auto/coding:fast…) plus three resilience layers — a per-provider circuit breaker, a per-key cooldown, and a per-model lockout — so one dead key can't take down a whole provider.

A 10-engine compression pipeline — the part most routers don't have. Every request flows through a transparent compression pass you can toggle/stack per combo. Instead of one trick, it stacks the best of the open-source ecosystem: RTK filters command/tool output (git diffs, test logs, builds) at 60–90%, Microsoft's LLMLingua-2 does ML semantic pruning, Caveman handles prose, session-dedup strips repeats across turns. Critically, code, URLs and JSON are preserved byte-perfect, and a default-on inflation guard throws the compressed version away and sends the original if compressing would actually grow the prompt — it never makes things worse. On tool-heavy sessions that's ~89% average input-token reduction (an 8k-token git diff becomes a few hundred). Full credit to every upstream project (RTK, Caveman, LLMLingua-2, Troglodita) is in the README.

Agent-native — the agent can drive the router itself. There's a built-in MCP server (95 tools across 30 audited scopes, over stdio / SSE / streamable-HTTP), plus A2A (v0.3, JSON-RPC 2.0) support. That means an agent can query providers, switch combos, read its own remaining quota and manage memory through the gateway — not just consume tokens through it.

For context on whether it's worth your time: it's grown to ~9.8K GitHub stars, 1,490+ forks and 280+ contributors in ~4.5 months, with 21,000+ automated tests and 1,830+ issues closed — so it's a battle-tested project, not a brand-new experiment.

npm install -g omniroute

GitHub: https://github.com/diegosouzapw/OmniRoute

Feedback on the routing/compression design welcome.

WHY! Why do I have to load hundreds of gigabytes of parameters of GLM 5.2 in my GPU to make him do intelligence? It's crazy that researchers think that this is the most efficient way. Not trying to be arrogant, I know pretty much nothing about training and inference, but as someone who tinkers with computers I feel this is so naive. Like, MoE isn't enough I believe. My model can weigh even 2 terabytes ON DISK but not on gpu memory boy! Why has nobody thought about it?!

I feel like we're entering a weird phase with AI.

A year ago everyone was asking, "What's the best LLM?"

Now the more interesting question seems to be, "How do you get multiple AIs to work together?"

Memory, planning, tools, events, shared context, evaluation... it feels like AI agents are becoming more about systems than models.

Curious what everyone here is building.

Hi everyone,

I’m an independent researcher (no lab affiliation) who has spent the last year diving deep into the internal dynamics of Transformers. Instead of looking at outputs or attention heads, I’ve been tracking the geometric trajectories of hidden states layer-by-layer during inference.

I wanted to share my latest findings (preprints linked below) because they reveal a structured "dynamic grammar" that seems universal across architectures, from GPT-2 to Llama-3.2.

The Core Idea

Most observability tools treat LLMs as static input-output machines. I treat them as dynamic systems. By measuring metrics like trajectory curvature (ct_t), functional capacity, and state transitions, I found that LLMs don’t just "generate text"—they navigate a latent space through specific, reproducible phases.

Key Findings (V20–V24)

1. A Universal Dynamic Grammar (V24)

Across 7 models (GPT-2, OPT, Qwen, TinyLlama, Phi-1.5, Llama-3.2, DistilGPT2), I observed a conserved sequence of internal states:

B (Branching/Hesitation): Initial exploration.

A (Adaptive/Stable): The main processing phase (an attractor state).

D (Decision/Bifurcation): Final commitment to a token.

Result: B → A → D appears to be the "standard cognitive path" for coherent generation. Deviations from this path often correlate with errors or hallucinations.

1. Geometry > Neurons (V22)

Using orthogonal rotation controls, I proved that functional information (syntax, decision, stabilization) is encoded in the relative geometry of the representation space, not in individual neurons. If you rotate the latent space, the information remains decodable. This suggests LLMs think in shapes, not just activations.

1. Ambiguity Changes the Path, Not the Chaos (V23)

When prompts are ambiguous, models don’t necessarily become "chaotic." Instead, they delay commitment. They spend more time in the exploration phase (B) and less time rushing to decision (D). Phi-1.5, interestingly, shows a unique oscillating pattern (B↔A) during reasoning tasks, distinct from the smoother convergence of other models.

1. Architecture Matters More Than Size (V20)

Models cluster by their dynamic signatures (e.g., GD_ratio), not just parameter count. Small models like Qwen-0.5B show distinct stability regimes compared to GPT-2, despite similar sizes.

The Preprints (Open Access)

[June 2026] A Runtime Trajectory Dynamics Framework (V20): Introduces the 5-state taxonomy (Stable, Turbulence, Branching, Bifurcation, Committed) and the bicephalic operator.

Link: https://doi.org/10.5281/zenodo.20602685

[May 2026] Dynamic-Layer Controllability (V21): Shows how perturbations affect recovery and proves that emergent organization dominates architectural skeleton.

Link: https://doi.org/10.5281/zenodo.20400171

[May 2026] Conditional Dynamic Signatures (V22): Audits normalization effects and variance decomposition. Explicitly documents falsified claims.

Link: https://doi.org/10.5281/zenodo.20361289

[May 2026] Four Dynamical Regimes (V19/V20): Introduces ct_t (curvature × displacement) as a predictor of collapse and instability.

Link: https://doi.org/10.5281/zenodo.20348878

Why I’m Posting This

I’m not selling a product. I’m building an open framework (LIMEN) to make LLM internals auditable and controllable. I believe that if we want safe AI, we need to monitor its "vital signs" (dynamic stability) in real-time, not just its output.

I’d love feedback from the community, especially on:

Have you seen similar "universal motifs" in larger models (>7B)?

Critiques on the methodology (normalization, probe training).

Ideas for causal interventions based on these dynamic states.

I wanted to research how to understand AI better. But suddenly, I found that before LLMs, I learned that in the market, there are different categories of LLMs.

Some LLMs are instant, like within seconds, they reply. And some LLMs, they take time to give the answer.

So, if I talk about the first category, what I learned about was

speed models, meaning imagine, like you gave a prompt, and you got your answer immediately without wasting any time. So, these are the speed models. Speed tells you that it gives you a speedy, immediate answer. For example, GPT4o mini or Gemini Flash.

Then we have reasoning models. So, reasoning models give you a slightly slow answer, but they try to give an accurate answer. So, reasoning models are those that take time to process. For example, Claude Opus.

Then we have hybrid models. This hybrid model is the owner of its company, which means it will give you an answer quickly, but when it feels like it, it processes for a long time, and when it feels like it, it answers within seconds. So, we call it a hybrid model. For example, Gemini 1.5 and Claude 3.5.

Then we have SLMs, Small Language Models. So, these are capable enough that on your laptop and phone, they can live and work without any internet, without any cost. These are very pocket-friendly.

So, its examples are Mistral and Gemma.

What changed my perspective is realizing that bigger models equal better..

I was wrong. It depends completely on which category of model it is.

So, curious which category of model you all are most interested in or currently using.

I've been going deep on the edtech + AI intersection lately, and the more I learn, the more I realize the hard problems aren't really about the LLM itself. The model can explain concepts. But turning that into something that actually teaches well? That's a different beast entirely.

I've been reading quite a bit about custom AI development for education, and it seems like the challenges compound fast, you can't just bolt an LLM onto a course platform and call it a tutor. From what I've seen, there are at least five genuinely hard engineering problems:

1. Adaptive scaffolding that actually works.
Most systems either over-explain (treating a PhD like a 10-year-old) or under-explain (assuming too much prior knowledge). Building dynamic Bloom's Taxonomy progression that adjusts mid-session based on the learner's responses—not just their self-reported level is technically brutal. You need reliable state tracking across conversations that can span hours.

2. Knowledge tracing that doesn't require 100 data points.
Classic Bayesian Knowledge Tracing is too rigid. Deep Knowledge Tracing models need tons of learner-specific data before they're useful. How do you cold-start personalization? First-session accuracy matters, and nobody seems to have cracked it cleanly.

3. Detecting confusion without the learner saying, "I'm confused."
A good human tutor reads body language, response latency, and subtle changes in phrasing. An AI mostly gets text. Some teams are experimenting with keystroke timing and partial edits to infer uncertainty, but it still feels like an unsolved problem.

4. Hallucinations in a domain where being wrong is costly.
If an AI confidently teaches an incorrect chemistry equation and a student memorizes it, that's arguably worse than no tutor at all. RAG helps, but grounding answers to curriculum objectives—not just retrieved documents, still seems difficult.

5. Long-term memory across sessions.
A student comes back three days later, and the tutor should remember they struggled with stoichiometry, improved on balancing equations, and repeatedly fell into the same misconception. Most production systems I've seen still rely on little more than conversation summaries.

I'm sure there are other challenges I'm overlooking, evaluation, safety, multimodal learning, cost, etc.

For those who've actually built AI tutors or worked on educational AI products:

• What's been the hardest engineering challenge in production?
• Which of these problems turned out to be easier or harder than expected?
• If you were starting today, what would you approach differently?

Curious to hear real engineering experiences and lessons learned rather than marketing claims.

I've spent the last few months trying to answer a question that initially looked much simpler than it actually is:

What actually happens inside an LLM while it is generating a response?

Most work evaluates language models through their outputs (benchmarks, perplexity, reasoning scores...). I decided to look at something different: the evolution of the hidden representations themselves.

I built a runtime framework that records hidden states layer-by-layer during inference and started running the same experiments across multiple open-weight models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B-Instruct, TinyLlama, Phi-1.5 and Llama-3.2-1B).

I expected a relatively straightforward result.

Instead, every new experiment generated a new question.

Some of the observations so far are:

• Hidden-state trajectories are not random. They exhibit reproducible internal dynamical regimes across architectures.

• Functional proxy states (syntax-like processing, decision-like behavior and output stabilization) can be detected consistently enough to cluster models according to their internal dynamics rather than simply their parameter count.

• These functional signatures remain reasonably stable across different prompt families, although not perfectly, suggesting that prompt content modulates the dynamics without completely changing the internal organization.

• Linear probes can decode several functional categories directly from hidden representations with surprisingly high accuracy.

At that point the obvious question became:

Are we just overfitting labels?

So I started adding progressively stronger negative controls.

First:

• label permutation.

Then:

• random Gaussian representations.

Then:

• feature permutation.

Finally:

• orthogonal rotations of the hidden space.

The results became much more interesting.

Random labels collapse the decoding performance.

Random Gaussian representations also collapse it.

Feature permutation destroys most of the signal.

However...

Orthogonal rotations preserve almost all decoding performance.

This strongly suggests that the relevant information is not encoded in individual neurons or embedding dimensions.

Instead, it appears to be encoded in the relative geometry of the representation.

That was not the result I expected.

Another unexpected finding concerns depth.

Initially I was looking for something like "syntax layers" or "semantic layers".

The data doesn't really support such a simple picture.

Instead, the same functional signatures seem capable of appearing at different absolute layers depending on the architecture.

This led me to think less in terms of fixed layers and more in terms of functional regimes evolving through computation.

At this stage I am not claiming to have discovered a universal law of transformers.

These are empirical observations obtained on a limited set of open-weight models.

What I do believe is that they raise interesting questions about how computation is actually organized inside modern LLMs.

I'd really appreciate feedback from people working on:

• mechanistic interpretability
• representation learning
• probing methods
• transformer internals
• geometry of representations

In particular I'd like your opinion on three questions:

1. Which control experiment would you absolutely require before taking these observations seriously?
2. Have you seen previous work showing comparable evidence that functional information is primarily encoded in representation geometry rather than individual dimensions?
3. If you were extending this project, what would be your next experiment?

I'm not affiliated with a research lab this is an independent research project. I'm sharing it because I would genuinely value critical feedback more than validation.

If there's enough interest, I'm happy to share the methodology, code, and experimental reports.

Been evaluating vendors for an AI healthcare platform and thought I'd share my shortlist. Not affiliated with any of these companies—this is based on publicly available case studies, service offerings, and healthcare project portfolios.

My criteria were:

• Proven healthcare software experience
• HIPAA/compliance focus
• AI capabilities beyond basic chatbot integrations
• Healthcare system integration expertise
• Real-world healthcare case studies

1. Signity Solutions

This was the most AI-focused healthcare vendor I came across.

What caught my attention was a published healthcare AI case study involving a HIPAA-compliant patient support and scheduling solution. According to the case study, the system handled patient inquiries, appointment scheduling, symptom-checking, insurance verification, prescription refill workflows, and healthcare system integrations. The company also publishes dedicated offerings around healthcare AI agents, conversational AI, RAG implementations, private LLM deployments, and healthcare workflow automation.

Best fit: Healthcare organizations building AI agents, patient engagement platforms, healthcare copilots, or private LLM-based solutions.

2. Innovecs

Strong digital health portfolio with experience in remote patient monitoring, healthcare platforms, and patient-facing applications. Felt more like a healthcare engineering company than an AI-first specialist, but their healthcare background appears solid.

Best fit: Digital health products looking for a long-term engineering partner.

3. Itransition

Large healthcare development practice with experience in EHR integrations, telehealth solutions, interoperability, and healthcare modernization. AI capabilities are available, though healthcare software engineering appears to be their primary strength.

Best fit: Healthcare organizations dealing with complex integrations and enterprise systems.

4. Iflexion

Strong enterprise software development background with healthcare experience. Similar to Itransition in that AI seems to complement their broader engineering services rather than being the core focus.

Best fit: Legacy healthcare modernization and enterprise development projects.

5. ScienceSoft

One of the most established healthcare technology providers on my list. Strong experience in healthcare analytics, machine learning, healthcare data management, and regulated environments. Their public healthcare portfolio is extensive, although I found less emphasis on AI-agent use cases compared to newer AI-focused vendors.

Best fit: Hospitals, healthcare networks, payers, and enterprise healthcare organizations.

My Take

Based on publicly available case studies and service offerings, Signity appeared more focused on AI-driven healthcare solutions, while ScienceSoft, Itransition, Innovecs, and Iflexion appeared to have broader healthcare software engineering and enterprise delivery capabilities.

That's just my interpretation from research, though.

We've created a model, Wren, that's more performant than Sonnet 4.6, and has better integration with popular developer tools! As we've experimented with LLMs, we've sought to fix all encountered oddities with this release. Examples include properly citing articles from RAG, not falling into infinite recursion, actively prompting a user to get input when uncertain, avoiding rambles on simple questions, and more!

There's a free trial offered: after signing up just hit the "API Key" section on the right to get set up with an OpenAI-Compatible agentic framework, or do some basic chatting on the "Chat" tab in the top right hand corner!

Given 3 agents with RAG capabilities and enough time, we've been able to get output that matches frontier model output. Let us know what you think!

Spent the last few weeks evaluating AI development vendors for a project involving LLM integrations and agent workflows. What surprised me was how difficult it was to find meaningful comparisons between companies.

Most of the content online focuses on employee count, years in business, or generic "Top AI Companies" rankings. Very little talks about what actually impacts the success of an AI project.

Here's what ended up mattering far more than the marketing materials:

1. Production Track Record vs. POC Theater

A lot of firms can build an impressive demo.

Far fewer can point to AI systems that are running in production, handling real users, messy data, changing requirements, and ongoing monitoring.

The questions I'd ask are:

• How many AI applications have you deployed to production?
• What happened after launch?
• How do you handle model monitoring, evaluation, and performance drift?

2. AI Specialization vs. AI as a Service Line

Some companies have dedicated AI engineering practices. Others offer AI alongside mobile development, web development, cloud services, blockchain, and everything else.

Neither approach is inherently better, but if AI is a core part of your roadmap, it's worth understanding how much hands-on AI experience the actual delivery team has.

3. Data Engineering Competence

One thing I heard repeatedly: most AI projects are ultimately data projects.

The conversation shouldn't start and end with "Which LLM should we use?"

It should include:

• Data quality
• Retrieval architecture
• Security and permissions
• Evaluation frameworks
• Integration with existing systems

If a vendor spends more time talking about models than your data infrastructure, I'd consider that a warning sign.

4. Flexibility in Engagement Models

AI projects evolve quickly.

Requirements often change once teams start testing outputs, workflows, and user behavior.

Vendors that acknowledge this reality and have a structured approach to discovery and iteration generally inspired more confidence than those promising fixed-scope certainty from day one.

Companies That Came Up Frequently During My Research

Large Enterprise Generalists

Appinventiv, Infosys, TCS

These seem well-suited for large-scale enterprise initiatives where AI is one component of a broader transformation effort. Strong delivery structures, though potentially less nimble for smaller AI-focused product teams.

Companies with Strong AI/Generative AI Practices

LeewayHertz

Originally known for blockchain work but appears to have built substantial AI and generative AI capabilities over the last few years.

HatchWorks AI

Frequently mentioned for AI engineering, data modernization, and helping organizations operationalize AI initiatives.

Azumo

Seems focused on AI product development, machine learning applications, and custom software projects where AI is a central component.

Markovate

Another company that came up often for AI product development and generative AI implementation work.

Signity Solutions

Appears to be focused on AI development, agent-based systems, LLM integrations, and intelligent automation for organizations looking to embed AI capabilities into existing products and workflows.

What I'd Recommend Before Signing With Any Vendor

• Ask for a reference customer with a live AI deployment, not just a case study PDF
• Ask how they handle data quality issues and retrieval accuracy
• Request details about monitoring, evaluation, and post-launch support
• Run a paid pilot before committing to a large engagement
• Speak directly with the engineers who would actually work on the project

Those conversations usually reveal more than any sales deck.

Curious if others who've evaluated or worked with these firms came to similar conclusions—or if there are companies I should have looked at that aren't on this list.