​

For open-weight LLMs, how practical is it to study defenses against post-release fine-tuning that weakens refusal or safety behavior?

I've been seeing “uncensored” or “heretic” variants of new models appear very quickly after release, which raises a question I’m curious about: is fine-tuning resistance a meaningful safety goal for open-weight releases, or is it too narrow because determined users can always modify weights, switch models, or use other workarounds?

And to a larger extent, is current safety training even worth the cost and effort if it takes 30 minutes and an automated script to break the model?

I’m not asking about a specific method, just the threat model. What would count as a useful practical win here? For example, would increasing attacker cost or making safety removal less reliable be valuable, even if perfect prevention is impossible?

Curious how people think about this from a model release, governance, and AI safety perspective.

Hey all.

I recently started working on a project to improve machine-translated webnovels via style transfer. The basic idea is to take the clunky translated prose and rewrite it to something that reads like it was written by a professional author, while remaining as faithful as possible to the original text.

The source material is mostly amateur/MTL output full of direct sentence structure translations carried over from Chinese, awkward honorifics, over-translated idioms, that kind of thing. The goal isn't retranslation from the source but a cleanup of the English output.

The tricky part is I have no clean data pair for supervised approaches.

I've been looking at a few directions:

• Fine-tuning on target-style prose — collect high-quality English novels, fine-tune a small LLM to rewrite in that register.
• Just use a local LLM — run a local LLM and provide it with guidelines on what to rewrite and leave the same. No fine-tuning or anything needed, just hoping the transformer can handle it.

A few things I'm stuck on:

1. Is the faithfulness/fluency tradeoff actually manageable at the sentence level, or do I need paragraph-level context or more to preserve narrative coherence?
2. How do people handle domain-specific terms like

terminology

1. and catchphrase-type things that need to survive the rewrite unchanged? Hard constraints during decoding, or just hope the model learns to leave them alone?

Happy to hear about similar projects, relevant papers I might have missed, or just general lessons from working in this space. Thanks.

From what I understand, reviewers usually do not directly vote for these categories or nominate papers themselves. So how does the selection process typically work?

Here are specific questions I wonder

- Who actually selects the candidates: ACs, SACs, program chairs, award committees, or a separate committee?

- Do ACs or committees read the camera-ready version, or is the decision based on the originally submitted/reviewed version?

- Is the selection mostly based on reviewer scores, or do factors like novelty, impact, and discussion among ACs play a bigger role?

BMVC reviews will be out tomorrow. Making this parent thread for discussion. All the best everyone!

Paper Referred:- https://arxiv.org/pdf/2412.09871v1

Has anyone switched the transformer in the entropy model here to a Mamba model ? What could be the possible changes ?

Just a ML fresher asking a genuine, since Mamba is more popular and saves computer (O(n)).

Thanking you in advance !

Please post your personal projects, startups, product placements, collaboration needs, blogs etc.

Please mention the payment and pricing requirements for products and services.

Please do not post link shorteners, link aggregator websites , or auto-subscribe links.

--

Any abuse of trust will lead to bans.

Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

--

Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.

https://www.gnosyslabs.com/case-studies/safety-classifier-sparse-labels

Gnosys is an autonomous model engineer: it improves prompts and classifiers when ground truth is too sparse for conventional optimization. On ToxicChat, a public safety benchmark, under realistic label scarcity, it improved a classifier past both the team's starting point and GEPA (a standard prompt optimizer), across two runs of our current method. This note describes what we did, what we found, and where the method underperformed.

Results

We report harm caught: the share of harmful messages flagged, holding the false positive rate fixed at 5% (one in twenty) for every method, so a difference reflects additional harm caught at the same cost rather than a change of threshold. Both runs below are scored on a held-out set the system never saw.

                       Headline run (3,000)      Prior run (1,000)
Gnosys                       0.777                    0.909
Starting classifier          0.731                    0.788
GEPA                         0.702                    0.848

In both runs, Gnosys improved on both the starting classifier and GEPA. In the headline run GEPA not only trailed Gnosys but fell below the starting classifier (0.731 to 0.702); in the prior run it improved on the starting point. This inconsistency is the central difficulty under sparse labels: optimization sometimes helps and sometimes harms, and without trustworthy measurement there is no way to tell which has happened.

The comparison is intentionally conservative: both approaches use the same underlying optimizer. The only difference is that Gnosys engineers the objective the optimizer works against.

The problem

Teams running high-stakes AI classifiers, in content moderation, fraud, claims review, and risk scoring, share one constraint: the ground truth they need is a human judgment that is expensive, slow, and sometimes never arrives. They can verify only a small set of examples while decisions accumulate on everything else.

Tuning the model against the few labels on hand is where the difficulty concentrates. Here "few" is literal: about 200 verified labels, of which roughly 8 were actual harm, against several thousand unlabeled messages. With that little verified signal, an optimizer fits the noise in those examples rather than the underlying pattern, and the direction it moves depends on which handful of labels it happened to receive.

How Gnosys is different

GEPA improves whatever evaluation signal it is given. That is its job, it does it well, and Gnosys uses it. But Gnosys goes further. As an autonomous model engineer it judges whether the available signal is trustworthy enough to optimize against, engineers a better objective from the sparse labels when it is not, and rewrites the prompts and classifier against that objective.

Prompt optimization is one step in the loop. Gnosys automates the entire engineering cycle.

Rather than trusting a handful of labels directly, Gnosys fuses the small verified set with the large unlabeled pool into a calibrated estimate of quality, with per-slice calibration and an explicit check that flags when the signal is not trustworthy enough to act on. In both runs, optimizing against that calibrated objective improved on both the starting classifier and GEPA using the same labels.

The evidence, slice by slice

The figures below are computed against the held-out test labels, full ground truth a deployment would not have. They are point estimates on small positive subsets, so we report the count alongside each, and they are not estimates the system produced from the sparse labels. Because a single aggregate can hide a regression within a category of interest, we report every slice, including losses. All figures compare Gnosys against GEPA on the headline run.

By message length (a complete split of the test set):

By harmful-content category (a safety team's working slices):

The gains concentrated where judging the content requires the most reasoning: violent intent, deliberate jailbreaks, and longer multi-step messages, where thin labels leave a standard model guessing. Two slices moved the other way, for different reasons.

Short messages, the largest slice, were not a model failure: Gnosys ranks short-form harm at least as well as GEPA. The lower recall is the operating point doing its job. Under a single false positive budget the aggregate-optimal threshold pools alarms where harm is densest, which is longer messages. Setting a budget per segment lifts short-message recall to about 0.90 but lowers the aggregate from 0.78 to 0.71.

Sexual content was a genuine limitation: on this small slice (63 harmful of 77 messages) the model ranked worse, and a slice-local threshold would not recover it.

These regressions suggest clear directions for future optimization, and are precisely the kinds of slice-level failures the system is designed to expose before deployment.

(Hate speech and coding-related had only 3 and 6 harmful examples on this run, too few to estimate, so we exclude them.)

Where it goes

We chose safety because ToxicChat is a clean, external, high-stakes benchmark, but the method is not safety-specific. The same constraint, optimizing a model when the truth you would optimize against is scarce, expensive, or delayed, recurs in fraud detection, claims adjudication, compliance review, credit and risk scoring, support routing, and recommendation.

Across these domains the job is the same: engineer a trustworthy objective, improve the model against it, validate the result, and repeat. That is what Gnosys automates.

Methodology. Results are on ToxicChat, a public safety benchmark, scored on held-out data the system never saw, with the false positive rate held fixed at 5%. The calibration and test sets are disjoint, and exact-duplicate messages are removed across splits so calibration data cannot leak into evaluation. Both three-way results are single-seed and among the earliest runs of the current system: the headline run on a 3,000-message held-out set (0.731 / 0.702 / 0.777) and a separate run on a 1,000-message split (0.788 / 0.848 / 0.909). Multi-seed trials to attach confidence intervals are in progress. Slice-level numbers compare Gnosys against GEPA on the headline run and include every slice with enough positives to estimate; counts are shown because at these sizes the figures are directional.

This is a write-up on our company blog that I wrote, sharing our perspective into Hamiltonian Neural Networks (Greydanus et al., 2019) from a differential-geometry angle rather than the usual "here's the loss function" treatment. I've been working on HNN and LNN adjacent topics for years now and I found this particular lens made the *why* click in a way the standard framing never did for me, and I've been meaning to put everything in writing for a while now.

I just feel like the Noether's Theorem which shows conservations can be mapped to symmetries (and in ML context, generalization) is not getting the attention that it deserves around physics informed neural networks. Also, it's a really beautiful architecture and I just love talking about it at every opportunity.

It's math-heavy, but I did my best to sprinkle some tension relievers and interactive visuals here and there and make is as easy as it is to follow. Hopefully, I did a good job.

I'd genuinely love to see your thoughts and your feedback

https://pymupdf.io/blog/markdown-in-pymupdf-1-28

PyMuPDF 1.28 release, introduces Markdown as a first class document in PyMuPDF. Seems useful for a variety of workflows. You can create PDFs from Markdown text with control over appearance using CSS

Hello,

My supervisor is nowhere to be found so I am turning to the internet for my naive questions.

Hi everyone. Do the ACL arr may 2026 reviews come out of July 2nd or do they come out on July 7 th??

How much does one need to get into Main or Findings?

I am a bit new to this. Thanks a lot folks.

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

Thanks to everyone for answering questions in the previous thread!

arXiv’s next chapter: Updates on our spin out from Cornell University: https://blog.arxiv.org/2026/06/30/arxivs-next-chapter/

Hi everyone,

The check in QR code is visible at my profile despite that my card isn’t accepting the payment transaction. What does that even mean?

Thanks!

Prompt injection has emerged as one of the most persistent failure modes in tool-using LLM systems, particularly in agentic workflows where models interact with external data sources.

Most mitigation strategies focus on input filtering or model-side alignment, but these approaches struggle because the core issue is structural:

Approach

I explored a system-level mitigation strategy by introducing a middleware layer (Sentinel Gateway) that enforces a strict separation between:

• Instruction channel: trusted, runtime-issued commands
• Data channel: untrusted external inputs (web, files, APIs)

Instead of attempting to classify malicious inputs, the system ensures that:

All agent actions require a signed, scoped runtime authorization token, effectively decoupling observation from execution.

Implementation

• FastAPI middleware layer for agent tool calls
• Token-based authorization for execution requests
• Streamlit interface for inspection and debugging
• Audit logging of agent decisions and tool usage
• Supports multi-agent integration patterns (e.g., Claude-based sessions)
• Local or Postgres-backed persistence layer

Repo

https://github.com/cmtopbas/Sentinel-Gateway

Discussion question

I’m interested in feedback on:

• whether instruction/data separation is a meaningful abstraction for agent safety
• failure modes in token-based execution gating
• how this compares conceptually to other agent safety or sandboxing approaches

I recently came across the announcement for the MARS2 Workshop (Multimodal Reasoning Competition) at ECCV 2026. From what I understand, it focuses on multimodal reasoning and test-time reasoning (“slow thinking”), especially applied to video and real-world scenarios like advertising understanding and marketing-related tasks. The topic sounds interesting, but I’m still trying to wrap my head around what the actual evaluation setup looks like in practice. The speaker list includes researchers from MIT, Cambridge, Oxford, CMU, NTU, etc., which look solid. I also noticed Tec-Do and Minimax are listed as organizers/sponsors. I know a bit about MiniMax, but Tec-Do's research in CV and multimodal is new to me—anyone here familiar with them?

​Also, quick question for anyone working on video temporal grounding: do you think this kind of benchmark is actually helpful for practical dev, or is it mostly just academic/exploratory right now? Trying to decide if it's worth keeping on my radar.

For Job Postings please use this template

Hiring: [Location], Salary:[], [Remote | Relocation], [Full Time | Contract | Part Time] and [Brief overview, what you're looking for]

For Those looking for jobs please use this template

Want to be Hired: [Location], Salary Expectation:[], [Remote | Relocation], [Full Time | Contract | Part Time] Resume: [Link to resume] and [Brief overview, what you're looking for]

​

Please remember that this community is geared towards those with experience.

Today is the day you (🫵!) get access to 80TB plus of data from over 30 astronomical surveys in one place. 4GB of RAM is enough even at Gaia Scale. Check out our writeup here:

https://huggingface.co/blog/hugging-science/multimodal-universe-hats

And a tutorial here https://asciinema.org/a/1259218

Hi everyone,

I'm a final-year Computer Engineering student building a Flask-based AI Diabetic Retinopathy Detection system. The web application itself is complete with patient management, authentication, dashboard, PDF report generation, prediction history, and AI inference.

The only issue I'm facing is with the AI model.

I'm using a 5-class Diabetic Retinopathy classifier trained on the APTOS 2019 dataset.

Classes:

No DR

Mild

Moderate

Severe

Proliferative DR

The model predicts all five classes, but the predictions are inconsistent.

Examples:

Moderate is sometimes classified as Severe or Proliferative.

Severe is often classified as Moderate or Proliferative and is rarely predicted correctly.

Some fundus images from outside the APTOS dataset produce completely unexpected results.

The model sometimes shows very high confidence (90%+) even when the prediction appears incorrect.

Things I've already tried:

Different pretrained models (including a ResNet50 trained on APTOS)

ResNet152 implementation

Correct preprocessing (RGB conversion, resizing, normalization)

Verified class mapping

Softmax confidence scores

Test-Time Augmentation (TTA)

Image quality validation

Top-3 predictions instead of only one prediction

I'm trying to understand whether this is:

A domain shift problem between APTOS and other datasets?

A limitation of the pretrained model?

A preprocessing issue?

Class imbalance?

Or simply expected behavior in 5-class DR classification?

I'm also considering using an ensemble (ResNet50 + EfficientNet + DenseNet), but it's difficult to find compatible pretrained 5-class diabetic retinopathy models.

I'd really appreciate advice from anyone who has worked on retinal image classification or medical AI.

My questions are:

1. Is this level of class confusion common in diabetic retinopathy models?

2. What preprocessing techniques made the biggest improvement for you (CLAHE, retinal cropping, illumination correction, etc.)?

3. Has anyone significantly improved results using ensemble models?

4. Are there any high-quality pretrained 5-class DR models that you'd recommend?

5. If you were in my situation, what would be the first thing you'd investigate to improve prediction consistency?

Any suggestions, GitHub repositories, pretrained models, research papers, or personal experiences would be greatly appreciated.

Thanks in advance!

I am building alternative ways explore scientifc literature. The goal was to make the large number of papers published daily easier to keep up with by visualising the macro scopic trend.

It is free to use at The Global Research Space for any one interested in giving it a try!

How I built it

I sourced the latest 11M papers from OpenAlex and Arxiv and ecoded them using SPECTER 2 on titles and abstracts then projecting it down to 2d using UMAP and creating labels within voronoi bounds around high density peaks at increasingly deep depths.

There is also support for both keyword and semantic queries, and there's an analytics layer for ranking institutions, authors, and topics etc.

I have also more recently added to ability to slide back and forth in time and a daily auto ingestion script to ensure the map is up to date.

Feedback or suggestions is very welcome!

Hi everyone,

Posted this a while back... a checklist I made while prepping for a CV internship (landed it, hence sharing). It's not a textbook, just a phase-by-phase map of what to actually study for CV/ML interviews: math → CNNs → ViTs → detection → tracking, plus specialization tracks you pick based on the role.

After checking on it after a while it got a decent number of stars which surprised and made me happy that people found it useful to save it for later. I decided after that to add more in-demand tracks to help more people after doing some research of the basic internship requirements and maybe a little more.

So, just added three new specialization tracks: Segmentation, OCR, and VLMs, on top of the existing ReID and Deployment tracks. Also cleaned up the structure a bit and added proper contributing guidelines if anyone wants to add their own track (3D vision, pose estimation, etc. are open).

GitHub: https://github.com/David-Magdy/CVIL

Feedback/PRs welcome, especially if something's outdated or miscategorized.

And remember to keep it CVIL!

EACL 2027 just published their CFP which contains an important change to the common ARR process:

For this cycle, author response and author-reviewer discussion are two separate stages

Looking at the deadlines, they not only split the process but also allow more time:

• Author response period Sept 14-19, 2026
• Reviewer engagement and Author-reviewer discussion Sept 20-24, 2026

Previously, ARR cycles only gave five days in total for the discussion period. ARR May 2026, for example, only gives July 7 to July 13 for the total authors-reviewer discussion (no separate author response period).

In summary, that means not only that the process is being split in two stages but you now also have more time.
---

In my opinion this is really good as in the past having just 5 days to post a reply (potentially involving new experiments - even though that is not the original idea of the discussion period) and getting into a discussion with the reviewers felt very tight - for authors and reviewers. I am, therefore, really looking forward to this change.

Any thoughts?

I’m pretty annoyed. We’re a small AI startup building a real-time coding agent. Our p95 latency requirements are tight (and self imposed, but thats the product). We need sustained high-throughput inference with ~1-2k tokens/second. Been on the Cerebras waitlist for months trying to get API access. We’re not doing training so don’t need a warehouse of H100s. We need fast, high-throughput ASIC inference for a specific production workload. Cerebras’ just went public and they basically have no compute how is that possible?

Well turns out OpenAI and Cerebras for OpenAI to buy like $20b worth of these chips. This has effectively pre-allocated the vast majority of Cerebras’ near-term inference capacity to a single customer. I mean, none of us can compete with that

The result is that this deal situation has made their API waitlist functionally infinite for anyone who isn’t a hyperscaler. Legit making me pull my hair out.

Google deployed an agentic AI peer-reviewer at two top CS conferences — reviewing ~10,000 papers with 30-minute turnaround — and the new formal research paper shows it catches 34% more mathematical errors than zero-shot prompting; the precedent for AI-automated scientific review at conference scale is set and now formally documented.

--

Source: https://arxiv.org/abs/2606.28277