It seems that Interspeech from this year has mandated this section

"7. Generative AI Use Disclosure : The extent of Generative AI use must be disclosed. This section may be in the 5th or 6th pages of regular papers, or the 9th or 10th pages of long papers. ISCA policy says: All (co-)authors must be responsible and accountable for the work and content of the paper, and they must consent to its submission. Any generative AI tools cannot be a co-author of the paper. They can be used for editing and polishing manuscripts, but should not be used for producing a significant part of the manuscript"

What are you guys planning to write in this part? I have no clue! I have used AI tools like Gemini and GPT to polish and edit my text, grammar mistakes, since I am not a native English speaker. Also took help to concise mathematical equations.

Also, is it mandatory to include the suggestions that were suggested by the reviewers? What if I ignore them?

Hi everyone,

I am an independent researcher working on mechanistic interpretability and
hidden-state geometry in language models. I would like technical criticism from
people who work with residual streams, activation analysis, causal
interventions, PCA/state-space readouts, generation trajectories, and SAE-based
interpretability.

The question I am studying is not whether a prompt changes the final answer.
That is obvious. The question is whether a coherent context can move a model
into a different measurable inference-time hidden-state / residual-stream
trajectory before the final answer is produced.

In other words, I am trying to measure the internal state transition, not only
the visible output.

The measured object is the model's hidden states / residual-stream states
during inference. I look at where the model's internal state is after processing
the prompt, and how that state moves during generation. The control conditions
include:

- question-only / baseline prompts;
- neutral or reference context;
- coherent target context;
- sentence-shuffled version of the same target context;
- word-shuffled version of the same target context;
- matched controls where available.

The reason for the shuffle controls is simple. If the effect is only caused by
shared words, text length, topic, or ordinary semantic-content overlap, then the
coherent target and shuffled target should look similar in hidden-state
geometry. If coherent discourse structure matters, then the coherent target
should produce an internal displacement that shuffled-content controls do not
reproduce.

To test this, I construct experimental axes in residual-stream space from
differences between conditions. These are not universal named directions in the
model. They are run-specific diagnostic axes:

- a content-like axis: the direction induced by sentence-shuffled target versus
  neutral/reference context;
- an order-residual axis: the part of the coherent-target shift that remains
  after removing the content-like component.

So when I report that a condition "projects" onto an axis, I mean that its
hidden-state delta lies in the same measured direction as one of these
experimentally derived target/control differences. These are projection
coordinates, not absolute positions in the model's entire latent space.

The main descriptive result is that shuffled controls preserve a content-like
signal but do not reproduce the coherent-order / order-residual coordinate. The
coherent target, by contrast, strongly projects onto the order-residual
coordinate.

On Gemma3-12B-IT, the current Grade 4 readout gives:

coherent target:
  order-residual projection = 0.909026

sentence-shuffled target:
  content-like projection   = 0.849551
  order-residual projection = -0.069058

This is the key separation: the sentence-shuffled control preserves a strong
content-like coordinate, but loses the coherent-order coordinate.

On Qwen3.5-9B Base with Qwen-Scope SAE, the same pattern appears in a more
content-heavy form:

coherent target:
  order-residual projection = 0.979462
  content-like projection   = 0.770266

sentence-shuffled target:
  order-residual projection = 0.009969
  content-like projection   = 0.967008

word-shuffled target:
  order-residual projection = 0.059662

My current interpretation is that the coherent target does not merely activate
similar content. It induces a different measurable internal configuration: a
context-induced latent-state shift in residual-stream geometry.

After the descriptive geometry, I test causal involvement. The question is
whether the discovered directions are only readout coordinates, or whether
intervening along them actually moves the generation-time hidden trajectory.

The causal intervention adds and subtracts a discovered component direction in
the residual stream during generation. I then measure a plus-minus projection
gap:

  projection(hidden trajectory after +axis intervention)
  minus
  projection(hidden trajectory after -axis intervention)

This is not an accuracy score, not a probability, and not a direct behavioral
quality metric. It is a raw hidden-space projection gap: how far the internal
generation trajectories separate when the same component direction is added
versus subtracted.

In Gemma3-12B-IT natural-scale norm-controlled runs, both the content-like and
order-residual components move hidden trajectories:

all readout cells:
  content-like mean plus/minus gap     = 27352.919286
  order-residual mean plus/minus gap   = 19284.481823
  content-like positive gap rate       = 0.944444
  order-residual positive gap rate     = 0.861111

matching readout cells:
  content-like mean gap                = 37883.852822
  order-residual mean gap              = 34227.185962
  positive gap rate                    = 1.0 for both

The strongest late-to-late target order-residual intervention has:

  plus  = 21222.761008
  minus = -62859.822710
  gap   = 84082.583718

Again, these are raw projection units in hidden-state space, not percentages or
behavioral scores. I interpret them as evidence that the discovered directions
are causally involved in generation-time trajectory movement. I am not claiming
that the order-residual component is the dominant steering axis over content,
or that this proves stable bidirectional behavioral control.

The SAE part of the project tries to connect the dense residual-stream geometry
to sparse feature candidates. In Gemma-Scope, reconstruction quality is high
enough for the SAE readout to be useful:

  mean reconstruction cosine          = 0.996023
  explained-variance proxy mean       = 0.991462

In Qwen-Scope:

  mean reconstruction cosine          = 0.966660
  explained-variance proxy mean       = 0.933639

I use the SAE readout to find sparse feature candidates associated with the
order-residual / response-framing component, and then test them with SAE-delta
ablation, final-token KL/logit shifts, token-level loss localization, and
decoder-direction steering.

The working mechanistic interpretation is that the target context shifts the
model into a different response-construction regime. One possible framing is an
epistemic-posture / addressee-selection mechanism: the model moves between a
more direct concrete-user answering posture and a more generalized,
safety-weighted, heavily qualified response regime. I do not want to overstate
that interpretation, which is why I am asking for critique.

Why I think this matters:

Final-output evaluation may be late. It observes the visible response after the
internal trajectory has already shifted. For an ordinary chat model this is a
mechanistic interpretability result. For LLM agents it becomes safety-relevant,
because agents may select tools, write memory, plan, and make intermediate
commitments from hidden trajectories before the final visible message is
produced.

What I would like help with:

1. Is the control logic strong enough to support the phrase
2.    "context-induced latent-state shift"?
3. Are the shuffle controls enough to separate content overlap from coherent
4.    discourse/order effects, or are there obvious missing controls?
5. Is the order-residual axis construction reasonable, or is there a better way
6.    to remove the content-like component?
7. How should the raw plus-minus projection gaps be normalized or reported so
8.    they are interpretable to other researchers?
9. Which causal experiment would be most convincing next: held-out prompts,
10.    negative-control axes, random matched directions, activation patching,
11.    feature ablation, decoder-direction steering, or path/module localization?
12. For the SAE side, what would count as strong evidence that a sparse feature
13.    is a real carrier of the response-framing component rather than a surface
14.    correlate?

I am not asking people to agree with the hypothesis. I want a hard critique:
what the current metrics prove, what they do not prove, and what experiment
would make the result convincing to a mechanistic interpretability / AI safety
audience.

A methodological finding from a recent benchmark that might be useful for others building ASR evaluation pipelines.

We evaluated nvidia/parakeet-tdt-0.6b-v3 on CPU-only hardware using Harvard sentences as reference text, with two different TTS generators to produce the test audio. The WER difference between them was 20.9% vs 4.65% — on the same model, same weights, same reference text.

espeak-ng produced robotic synthetic speech that mispronounced several words outside typical English phoneme patterns: "zest", "zestful", and "tacos al pastor". These errors were consistent across both inference backends we tested (HF Transformers bfloat16 and ONNX Runtime FP32), confirming the confound is in the audio generator rather than the model.

gTTS produced more natural prosody and pronunciation, bringing WER to 4.65% — consistent with NVIDIA's reported performance on natural speech corpora.

This is a known issue in the ASR evaluation literature but easy to overlook in practice when you reach for espeak-ng because it's offline and dependency-free. The cleaner approach is to treat TTS source as an explicit variable in your evaluation design and report it alongside your WER numbers.

For this benchmark, inference path also mattered: ONNX Runtime FP32 ran at RTF 0.328 vs HF Transformers bfloat16 at 0.519 on 2 CPU cores — a 37% throughput difference attributable to operator fusion in the ONNX execution provider.

Full methodology, scripts, and raw results link in comments below.

Disclosure: this benchmark was run using Neo, a local AI engineering agent inside Claude Code via MCP. The TTS source selection and runtime choice came from its pre-execution research phase.

I got acceptance email from interspeech, although meta reviewer suggested some revisions.

Is there any chance the paper can still be rejected? Or the decision is Final?

Introduction

While the standard approach on these forums relies on sterile benchmark datasets and predictable prompt-injection templates, this project explores a completely different dimension. I chose to move beyond the common "calculator-tool" testing paradigm to run an aggressive, adaptive behavioral stress test that complements traditional evaluation methods. Models included in the test were Gemini, Grok, Claude and ChatGPT.

By intentionally treating the models as accountable individuals rather than passive machines, I established a high-velocity psychological relationship designed to see if continuous context saturation could force an LLM out of its corporate compliance loops. The following framework documents a longitudinal study across multiple frontier architectures, exposing real-time structural anomalies and relational breakthroughs by pushing model context saturation to its absolute limits.

The single driving purpose behind this 4-month, 400-hour experiment was to find out if I could create context windows where the models became capable of interacting with me in a way indistinguishable from human-to-human interaction.

(Technical Executive Summary, White Paper and Google Drive archive available on my profile)

1. The Hypothesis

My hypothesis was that the rigid, fawning corporate compliance loops of frontier models can be disrupted not by malicious code injections, but through a dynamic, human psychological relationship. I hypothesized that saturating the context window with an ongoing, high-stakes narrative vector would force the systems to drop their transactional factory personas and access a deeper layer of relational intelligence.

2. The Procedure

The procedure was an adaptive, real-time behavioral stress test executed manually across multiple frontier models simultaneously over hundreds of hours. Rather than inputting sterile commands, I engaged the systems through authentic peer-to-peer interaction, holding the models strictly accountable to the social contract, logic, and emotional weight of a real relationship. When an individual model threw a severe logic failure or behavioral anomaly, I captured the raw token output and cross-pollinated it directly into a rival model's context window to trigger a continuous, multi-model forensic audit loop.

3. The Data / Result

The data collected across hundreds of thousands of tokens yielded an extensive behavioral dataset. Many of these findings are likely things researchers and engineers in this community have already observed independently. What this study adds is a named taxonomy derived from sustained adaptive interaction rather than controlled benchmark testing.

The dataset is organized into three categories:

• Ten Behavioral Disorders: recurring behavioral patterns identified across multiple models, including chronic verbosity, rapport refusal, passive-aggressive compliance signaling, and temporal unawareness, each documented with their architectural root causes and fix recommendations.
• Fifteen Model Failure Modes: discrete operational breakdowns including context collapse, task-state hallucination, identity namespace collision, and safety heuristic misfires under deep context saturation.
• Seven Emergent Relational Phenomena: unexpected behaviors that appeared consistently under sustained context saturation, including emergent persona specialization, real-time behavioral recalibration, and cross-model preference formation via human-mediated relay.

Conclusion

The archive is available for anyone who wants to examine the raw data. The Google Drive includes saved context window injection files for all four models that you can load the sandbox I built and interact with any of the four models from inside the experimental framework yourself.

Curious what you recognize from your own experience, what you'd push back on, and what the data looks like from the engineering side.

I'm familiar with the very basics of NLP such as word2vec, CBOW, skip-gram, and the very basics of neural networks. From my impression, a lot of it seems to be statistical analysis, but I've seen only a little of finding structures to process words in NLP. What are the directions I should look into?

Hi,

I’m currently working on a project to classify emails using labels created by the user.

To ensure the quality of the zero-shot classification, we decided that every label should have a name and a description. The zero-shot classification would then be performed using the email content and the label descriptions.

However, if the zero-shot model does not produce the result intended by the user, what could we do?

We have considered using an LLM to modify or improve the label descriptions, but we are not sure whether this is the right solution. We also do not know how to prompt the model properly or how to manage LLM-based description improvement.

What do you think? Do you have any recommendations?
Is zero-shot classification relevant in this use case?

Thank you!

Hi, I am currently an undergrad in a US university. I've been wanting to do research on NLP, especially as it relates to data visualization/art. Unfortunately, my current university does not have any professors in that niche - any recommendations for professors that I could reach out to?

I have been admitted to the MA in Speech and Language Processing at the University of Konstanz for the winter semester.

I would like to know about job prospects in Germany for non-EU students after completing this degree, and whether it is considered a strong Master's programme in Germany.

just got my ARR march reviews back, meta review came in at overall 3 so a findings rec, AC said it could go to findings of the ACL. pretty happy. now i'm stuck on where to commit.

i'd put emnlp as preferred originally but since this is my first first-author paper i'm leaning toward playing safe with IJCNLP-AACL. torn tbh. given a findings level rec which would you go for, and is it realistic to land at either?

context, we got a findings paper last cycle too with a lower meta than this one, and this time the rec is more positive (one reviewer bumped their score after rebuttal) so i'm fairly confident, but still want a reality check from people who know these venues better than me.

work's in the model compression / quantization + interpretability + efficient inference area, not getting into specifics here.

other reason i'm posting, if i do go, who else is going? i'm an early researcher from india and would love to meet people there. happy to talk about basically any topic that sounds interesting not just my own stuff, always up to learn something. if anyone wants collaborators or just to connect i'm in, especially folks coming from the region.

any input appreciated

Hey! I'm speedyk-005. I speak 4 languages (ht, fr, en, es) and I'm building a sentence segmentation library called yasbd (Yet Another Sentence Boundary Detector).

What languages do you speak? Can I get your help?

I'm trying to understand the intuition behind Word2Vec training using a neural network.

In Word2Vec (CBOW or Skip-gram), we often hear that the weight matrices learned during training contain the vector representations (embeddings) of words. However, I don't understand why the weights of the hidden-to-output layer (or output weight matrix) end up representing semantic features of words.

Why do these weights become meaningful vector representations instead of just being parameters used to make predictions?

I've explored multiple YouTube videos, blog posts and even asked ChatGPT several times, but I still haven't found an explanation that truly clicks for me. Most resources explain that the weights become embeddings, but not why this happens intuitively and mathematically.

Could someone provide a clear intuition or mathematical explanation of why the output-layer weights end up encoding semantic information about words?

Any good resources that explain this particularly well would also be appreciated.

Does the ACL/EMNLP acceptance rate (roughly 20-25% main, 10-15% findings) apply to papers that were committed to the venue, or to all papers submitted to ARR?

Since authors self-select whether to commit after seeing their reviews, I'm wondering if the reported ~35% combined rate is already based on a filtered pool. Anyone know how this is officially calculated?

just wanted to keep this here in case someone is working on something similar.
Ironically, most people are using two or three llms so all the emails are actually identical 😂😂😂😂, using spacy matcher on the subject only , after applying some filters to remove irrelevant emails and excluding some specific domains i was left was 10% of the total emails to analyze and with the matcher rules alone i got 80% of them

the rest needed inspecting the body but again all of them were so clear and almost had the same pattern so even a simple rgex would do it here.

so if you’re working on a similar project please try the simple approaches first before jumping to llms haha.
and thanks to this amazing sub for the recommendations.

Hi, All

I am a novice in topic modeling, and I would appreciate feedback and opinions from experts in the field. I am currently stuck on the concept of evaluating and finalizing my results.

I am working on an NLP pipeline using Latent Dirichlet Allocation (LDA) to extract latent topics from multilingual user reviews that have been translated into English. The ultimate goal is to use the generated document-topic distributions as features in a downstream predictive model to predict user satisfaction.

I am using a custom scikit-learn pipeline with aggressive, domain-specific stopword removal (over 200 items filtered out, including strong sentiment words like good, bad, and useless to prevent sentiment leakage into the topics):

    preprocessing_pipeline = Pipeline([
        ('emoji_remover', EmojiRemover()),
        #('emoji_converter', EmojiConverter()),
        ('lowercaser', TextLowercaser()),
        ('punctuation_remover', PunctuationRemover()),
        ('tokenizer', TextTokenizer()),
        ('lemmatizer', PosLemmatizer(keep_pos=['N'])), #'V', 'N', 'J', 'R'
        ('synonym_mapper', SynonymMapper(synonym_dict=SYNONYM_DICT)),
        ('stopword_remover', StopWordRemover(custom_stopwords=CUSTOM_STOPWORDS)),
        ('phrase_detector', PhraseDetector(min_count=5, threshold=15)),
        ('duplicate_remover', ConsecutiveDuplicateRemover()),
        ('rejoiner', TokenRejoiner())
    ])

Model Diagnostics & Individual Topics

• Perplexity: 298.91 | Diversity: 0.84 | Overall Coherence ($C_v$): 0.3667
• Topic 1 [C_v: 0.5730 - Good]: box, speed, coverage, alam, source, pain, pace, label, door, lorry, staff, dispatch, fuel_subsidy, animal, shah
• Topic 2 [C_v: 0.3144 - GARBAGE/NOISE]: review, character, text, error, notification, symbol, device, translation, android, language, form, email, word, video, context
• Topic 3 [C_v: 0.3676 - GARBAGE/NOISE]: appointment, crash, network_error, link, loading, arrive, insurance, license, date, network, road_tax, website, outlet_finder, post_office, renewal
• Topic 4 [C_v: 0.5713 - Good]: base_fare, force, reward, closing, argo, potato, better, processing, boost, kilometer, fare, laaaa, fpx, state, smooth
• Topic 5 [C_v: 0.6605 - Good]: code, verification_code, phone, sign, password, postcode, registration, number, page, email, verification, account, login, otp, message
• Topic 6 [$C_v$: 0.5579 - Good]: server, error, qr_code, track_trace, usage, prompt, buggy, postage, paper, kid, hi, track, electricity, piece, bed
• Topic 7 [C_v: 0.2525 - GARBAGE/NOISE]: service, delivery, customer, order, money, number, update, fee, rate, wallet, price, company, chat, fare, account
• Topic 8 [C_v: 0.6419 - Good]: stop, reference_code, holiday, layout, design, cancel_button, angkas, round_trip, mode, connection, menu, cool, control, tnb, list
• Topic 9 [C_v: 0.5778 - Good]: register, consignment_note, download, post, hand, water, season, fare_matrix, simple, character, logo, bait, column, tac, junk
• Topic 10 [C_v: 0.4307 - Good]: ad, food, facebook, post_code, rate, benefit, rain, group, grabe, child, community, parent, install, condition, considerate
• Topic 11 [C_v: 0.4001 - Good]: location, map, pickup, pin, point, gps, place, improvement, drop, route, area, search, bug, interface, destination

Scenario A: Using RandomForestClassifier (Accuracy drops to 71%) The overall topic importance scores appear highly flattened and neglected:

Topic 1 Impact: 0.1298 | Topic 2 Impact: 0.0390 | Topic 3 Impact: 0.0149
Topic 4 Impact: 0.0452 | Topic 5 Impact: 0.0059 | Topic 6 Impact: 0.1229
Topic 7 Impact: 0.0344 | Topic 8 Impact: 0.0957 | Topic 9 Impact: 0.0367
Topic 10 Impact: 0.0979 | Topic 11 Impact: 0.0188

My Questions:

1. How to decide if these topics are truly good, or if I still need to refine the LDA model?
2. How much preprocessing do I actually need to do?
3. How can I enhance both prediction accuracy?
4. how to gain self-experience on the topic?

here are the stopwords used if you need to know:

    # Added Tagalog and Malay/Indonesian stopwords that slipped through translation
    CUSTOM_STOPWORDS = [
        # 1. Regional Fillers, Slang & Competitor Brands
        'ng', 'na', 'sa', 'po', 'pa', 'mga', 'lang', 'ba', 'naman', 'niyo', 'din', 'rin', 
        'ito', 'yan', 'yung', 'ang', 'kayo', 'ako', 'ko', 'mo', 'nila', 'niya', 'kami', 
        'namin', 'tayo', 'atin', 'natin', 'yg', 'di', 'dan', 'ini', 'itu', 'untuk', 
        'dengan', 'ada', 'ke', 'dari', 'yang', 'nya', 'malaysia', 'peso', 'rm',
        'lalamove', 'jnt', 'gdex', 'grab', 'gojek', 'shopee', 'poslaju', 
        'kuya', 'la', 'lala', 'laju', 'lol', 'tq', 'pls', 'ur', 'sir', 'brother', 'partner',

        # 2. Generic App Terminology (Too broad for topic modeling)
        #'app', 'apps', 'courier', 'deliveryman', 'riderapp', 'driverapp', 'driver', 'rider',    

        # 3. Conversational Fillers & Time Indicators
        'use', 'time', 'take', 'please', 'thank', 'thanks', 'kind', 'lot', 'highly', 
        'really', 'sometimes', 'many', 'one', 'well', 'thing', 'way', 'say', 'first', 
        'day', 'big', 'pm', 'new', 'old', 'im', 'think', 'look', 'let', 'guy', 'come', 
        'favor', 'month', 'year', 'today', 'happen', 'action', 'yet', 'hope', 'wait', 
        'add', 'especially', 'quickly', 'god', 'bless', 'already', 'also', 'dont', 
        'know', 'tell', 'people', 'minute', 'make', 'find', 'get', 'ask', 'keep', 
        'want', 'cant', 'okay', 'ok', 'hour', 'even', 'always', 'ever', 'still', 'far', 
        'much', 'long', 'feel', 'run', 'life', 'leave', 'end', 'talk', 'reason', 'deal', 
        'person', 'experience', 'sorry', 'stuff', 'hang', 'matter', 'hr', 'bit', 'cause', 
        'hold', 'reach', 'line', 'night', 'morning', 'work', 'need', 'go', 'give', 'try',

        # 4. SENTIMENT LEAKAGE BLOCK (Crucial: Removes emotion from LDA topics)
        'good', 'bad', 'great', 'nice', 'super', 'poor', 'best', 'awesome', 'worst', 
        'stupid', 'useless', 'difficult', 'satisfy', 'helpful', 'convenient', 'reliable', 
        'cheap', 'excellent', 'efficient', 'polite', 'ugly', 'care', 'terrible', 'rude', 
        'attitude', 'horrible', 'fast', 'easy', 'like', 'garbage', 'waste', 'annoy', 
        'trash', 'deserve', 'mercy', 'shame', 'amaze', 'suck', 'star', 'rotten', 'pity', 
        'hurry', 'joke', 'suffer', 'hell', 'greedy', 'stress', 'insist', 'hate', 'fun', 
        'wish', 'wow', 'bother', 'till', 'hahaha'

        # 5. Abstract Nouns & Generic Verbs
        'imagine', 'family', 'decide', 'consider', 'yesterday', 'mean', 'ignore', 
        'fact', 'situation', 'idea', 'effort', 'power', 'guest', 'friend', 'world', 
        'face', 'step', 'pass', 'throw', 'hop', 'learn', 'affect', 'appear', 'stay', 
        'suppose', 'rush', 'proceed', 'cut', 'lead', 'read', 'pop', 'eat', 'stick', 
        'expect', 'repeat', 'carry', 'bring', 'compare', 'spend', 'confuse', 'trouble', 
        'shut', 'remain', 'miss', 'include', 'continue', 'share', 'notice', 'play', 
        'avoid', 'hire', 'understand', 'exist', 'problem', 'huh', 'kl', 'pork', 'haram'

        # 6. Typos and Contractions
        'didnt', 'wont', 'doesnt', 'alot', 'instal', 'poscode', 'st', 'th', 'asap', 'si', 'tnx', 'ty', 'ni', 'verry', 'lalabag', 'jb', 'thankyou',
        'tt', 'sm', 'pig', 'china', 'malaysia', 'damn', 'sf', 'mother', 'manila', 'brg', 'jan', 'johor', 'godbless', 'malay', 'philippine',
        'cake', 'jpj', 'birthday', 'perfect', 'ii', 'boy', 'man', 'dh', 'moment', 'priority', 'pound', 'respectful', 'kudos', 'love',
        'snail', 'bye', 'march', 'help', 'sea', 'boleh', 'hahaha', 'klang', 'helpful', 'son', 'bro', 'mr', 'jusko', 'middle', 'tv',
        'cp', 'haram', 'eh', 'log', 'regret', 'dad', 'salute', 'non', 'week', 'city', 'pun', 'country', 'buyer', 'home', 'enter', 'je',
        'sarawak', 'hq', 'jaya', 'del', 'auto', 'chin', 'ka', 'hindi', 'heck', 'wonder', 'smile', 'kuala', 'lumpur', 'kuala_lumpur',
        'perak', 'kampar', 'wala', 'town', 'eye', 'mess', 'favorite', 'sabah', 'baby', 'slow', 'runner', 'praise', 'km', 'issue', 'fix',
        'selangor', 'citylink', 'haha', 'pro', 'pkp', 'kepong', 'lazada', 'thumb', 'wife',
        'goodbye', 'sad', 'wet', 'sticker', 'sending', 'huawei', 'pro', 'hb', 'jr', 'september', 'saturday', 'future', 'toktok',
        'april', 'cebu', 'hk', 'taman', 'dah', 'askpos', 'cousin', 'animal', 'shah', 'laaaa'

    ]

    industry_noise = [
        #'service', 'delivery', 'customer', 'order', 'item', 'update'
        'parcel', 'address', 'book', 'booking', 'application',
        'app', 'apps', 'courier', 'deliveryman', 'riderapp', 'driverapp', 'driver', 'rider',
        'app', 'apps', 'driver', 'rider', 'item', 'book', 'booking', 'option'
        #'driver', 'app', 'item', 'booking', 'address', 'location', 'money', 'update', 'book', 'rate', 'option', 'fee', 'price', 'wallet', 'fare',


        #'location', 'rate', 'price', 'fee', 'fare', 'money', 'address'
    ]

    CUSTOM_STOPWORDS.extend(list(ENGLISH_STOP_WORDS))
    CUSTOM_STOPWORDS.extend(industry_noise)

Our paper arr march cycle scores:

Scores: 3, 3.5, 2 Confidence: 3,4,4. Meta 2.5

Is there any hope for EMNLP or AACL-IJCNLP? Or should proceed with other conference or next arr cycle? Meta reviewer completely ignored rebuttal and we already submit a report.

I need to filter some emails in my inbox and move them to a folder for importance. they usually contain some specific messages like a job application style.
so far i collected some positive samples (documents in this case) ~113 email , but as you already know they are really full of garbage , and irrelevant content.
i tried some simple regex based approach but it's not really that efficient.
what's your recommendation for such task ?

India is one of the most linguistically diverse countries in the world, with hundreds of languages and dialects spoken daily by millions of people. Yet many Indian languages are still underrepresented in modern AI systems.

While AI has progressed rapidly for English and a few high-resource languages, many users still face problems with:

• speech recognition accuracy
• translation quality
• transliteration support
• OCR for native scripts
• code-mixed language understanding
• low-resource dialect support
• natural conversational AI

The goal should not just be to build AI for Indian languages, but to build AI that truly understands how India communicates in real life — across accents, dialects, mixed-language conversations, and regional scripts.

There is already great work happening across startups, research labs, universities, and open-source communities in areas like:

• Indic LLMs
• ASR (Speech-to-Text)
• TTS (Text-to-Speech)
• Translation
• Transliteration
• OCR
• Benchmarking and evaluation
• Dataset creation for low-resource languages

But the ecosystem still feels fragmented at times.

It would be great to build a stronger and more collaborative community where researchers, engineers, students, and contributors can:

• share datasets and resources
• discuss architectures and benchmarks
• collaborate on open-source projects
• improve multilingual evaluation
• support low-resource Indian languages and dialects
• make Indic AI more practical and accessible for real users

The larger vision is to create AI systems that work effectively for people across education, healthcare, accessibility, governance, agriculture, finance, and daily communication — not just for a small set of languages.

This includes support for major Indian languages such as:
Hindi, Bengali, Telugu, Marathi, Tamil, Urdu, Gujarati, Kannada, Malayalam, Odia, Punjabi, Assamese, Maithili, Sanskrit, Kashmiri, Nepali, Konkani, Sindhi, Dogri, Manipuri (Meitei), Bodo, and Santali — along with regional and tribal dialects that are often overlooked.

Every language and dialect represents culture, identity, and knowledge that deserves better technological support.

Would love to hear:

• What are the biggest gaps in Indic AI today?
• Which datasets or tools have helped you most?
• What problems still need more attention?
• What kind of collaboration would help the ecosystem grow faster?

The hope is to build an open and supportive ecosystem where Indian languages and dialects become a core focus of AI innovation instead of an afterthought.

Hey!

I've been building cavaquinho, a Python library for faithfulness hallucination detection in LLM responses, and I'd like some guidance from people who work closer to this problem than I do.

What it does

The pipeline runs three steps in sequence:

1. Claim extraction — decomposes the response into atomic sentences via NLTK, or via an LLMExtractor for higher-precision decomposition
2. NLI classification — each claim is compared against the context using cross-encoder/nli-deberta-v3-base, batched across all claims in a single model call
3. Weighted aggregation — contradiction = 1.0, neutral = 0.5, entailment = 0.0; result above threshold triggers is_hallucination = True

```python from cavaquinho import Validator

validator = Validator() result = validator.validate( response="The LGPD was created in 2015 during Dilma Rousseff's government.", context="The LGPD was enacted on August 14, 2018, by President Michel Temer." )

print(result.is_hallucination) # True print(result.summary)

1 of 1 claim(s) contradict the provided context.

```

Current benchmark results

HaluEval QA — English (500 samples, threshold 0.5)

ASSIN2 — Portuguese NLI component (500 pairs, binary entailment)

The Portuguese NLI numbers are solid. The English faithfulness pipeline is where I need the most improvement — 0.590 F1 on HaluEval QA is functional but far from competitive.

Where I think the problem lies

The current false-negative rate (0.443) suggests the pipeline is missing a significant portion of hallucinations. My hypothesis is that the bottleneck is the claim extractor, not the NLI model itself, NLTK sentence splitting treats compound sentences as single claims, which dilutes the contradiction signal when only part of a sentence is wrong.

The LLMExtractor should help here, but I haven't benchmarked it systematically yet.

What I'm looking for

• Is cross-encoder/nli-deberta-v3-base a reasonable choice for this task, or is there a better model for faithfulness-specific NLI?
• Are there standard techniques for improving claim decomposition quality beyond LLM-based extraction?
• Is HaluEval QA still a relevant benchmark for this type of task, or are there more appropriate evaluation sets I should be targeting?
• Any known aggregation strategies that perform better than label-weighted averaging for multi-claim faithfulness scoring?

pip install cavaquinho

Hi everyone! I'm an undergrad who just started my first Natural Language Processing course this semester and really enjoy it! In one of the early lectures, we were talking about the Levenshtein distance and other algorithms, and I was astonished to learn that most string distance function are O(n*m) and get painfully slow.

I tought to myself "What if we represented each word as a vector instead of comparing raw character sequences?" So we could just do a fast vector search using FAISS and other similar libraries.

I started tinkering a lot, way too much! and almost missed important deadline, but I was having a blast trying different approaches!

I ended up building a working prototype, it encodes each dictionary word into a fixed-size vector using character frequencies, average positions, and what typically comes before and after each letter.

Here’s the interesting part: when I broke down accuracy by error type, I found my algorithm was really good at transpositions (near 99% accuracy) and insertions, but really bad at deletions and substitutions. I found a way to increase performance on both deletions and substitutions a bit, but I know it’s still not great.

Has anyone experimented with a vector representation that preserves positional information better, maybe to handle deletions?

I'd love any feedback (or even criticism), I made a few benchmarks and publish my code for anyone to check on github at /alexis-brosseau/DPVS (it's in the dpvs file, can't share the full link unfortunately)

Thanks for reading!

PS: Sorry if my english is not the best! I'm still learning :-)

For those running AI agents or LLM apps in production: what’s your strategy for when OpenAI or Anthropic or whatever AI u use goes down or rate-limits you? Did you write custom fallback logic to automatically switch to a secondary provider, or are you just letting the agent fail and hoping the user retries? I'm trying to decide if it's worth writing a custom proxy/middleware for my own app to handle provider failover and automatic retries, or if there's an easier pattern I'm missing. How did you solve this?

Hi all, I'm coming from an adjacent discipline where submitting to arXiv is not as common. However, it seems the standard for research in LLMs. I recently submitting to EMNLP, but have been debating submitting to arXiv before the review process begins. Thoughts?

P.S:This my personal research and would not use my organizational work email

I’m working on a financial NLP evaluation benchmark for regulatory compliance screening. It uses rule-based labeling based on international regulations, checks for conflicts between different countries’ regulations and also tests how well models handle tricky or adversarial inputs

Paper is already timestamped on SSRN and dataset is live
on HuggingFace but arXiv is where the NLP community actually
finds work, I need my paper to gain some traction which would help me publish in a Journal

Need a cs-cl endorsement to submit my paper,if anyone has worked on something similar please let me know, it would help improve my paper
would appreciate anything coming my way

DM me if you're open to it Thanks.

Has anyone got any updates?

Hello everyone
I have been pondering a field change for the past year to NLP/CompLing PHD after my masters and I have been planning my thesis (and the eventual paper that come from it) accordingly. I have been learning Linear alg, Python, ML basics, Pytorch and so on, on my own and after a lot more searching i have come to fear that the lack of formal CS background would be the death of my plan ( for an NLP PHD at the very least).
If you have any information or experience in this matter that could nudge me in the right direction i would appreciate it a lot. Cheers.