LLM Infrastructure Engineering, Layer-Selective Unlearning, Long-Context Memory, and Verifiable Inference

Today’s selection of 8 noteworthy AI/ML papers from arXiv, covering LLM infrastructure engineering benchmarks, quantization-resilient unlearning, convolutional memory for long-context reasoning, adaptive vision token pruning, low-resource Greek TTS, automated exception handling code generation, LLM-assisted emergency department quality review, and private verifiable LLM inference.

1. $Φ$-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

Authors: Leilei Ding, Shumin Wang, Yuting Huang, Fanqi Wan, Yinmin Zhang et al. | Categories: cs.CL Link: arxiv.org/abs/2609.10226

Existing benchmarks for LLM code generation focus on isolated kernels or pre-specified optimization targets, failing to capture open-ended, long-horizon infrastructure engineering. $Φ$-Bench addresses this gap with tasks derived from frontier research and real-world repositories, spanning localized kernel-level function completion to end-to-end system optimization. Experiments on frontier LLMs reveal both their current capabilities and limitations in engineering complex LLM infrastructure.

Takeaway: This benchmark asks a pointed question—can LLMs optimize the very stack they run on?—and provides a rigorous testbed for tracking progress toward autonomous AI infrastructure.

2. Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Authors: Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou | Categories: cs.LG, cs.AI Link: arxiv.org/abs/2609.10439

FOM-UL is a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score, concentrating updates where they are most effective while leaving most of the model unchanged. Across TOFU, KnowUnDo, and MUSE-style evaluations, it reduces residual memorization compared with GA, NPO, KLD, SURE, ReLearn, and LUNAR baselines while preserving retain-set utility. Under 8-bit and 4-bit post-training quantization, FOM-UL maintains stronger suppression and utility preservation, and shows lower recovery of forgotten content under adversarial prompts.

Takeaway: Quantization resilience is an underappreciated failure mode for unlearning, and this work directly confronts it—an important step for deployment-realistic privacy compliance.

3. ConvMem: Convolutional Memory for Long-Context Reasoning

Authors: Hongming Zhang, Zhaozhen Gu, Fengshuo Bai, Ming Hao, Qingyang Zhang et al. | Categories: cs.AI, cs.CL Link: arxiv.org/abs/2609.10441

ConvMem is a training-free, highly parallelizable framework that reformulates long-context reasoning as hierarchical convolution, treating an LLM prompted with a query as a convolutional kernel that summarizes text segments in a logarithmic tree rather than a linear chain. Configurable strides, skip connections, and multi-kernel convolution ensure robust evidence capture while mitigating error accumulation. On RULER-HotpotQA and RULER-2WikiMultiHopQA, ConvMem outperforms training-free baselines and avoids the overfitting risks of RL-trained models on out-of-distribution tasks.

Takeaway: Reframing sequential memory updates as convolution is an elegant way to trade latency for parallelism—worth reading for anyone wrestling with long-context inference costs.

4. Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMs

Authors: Haiji Liang, Pengfei Zhou, Zhenglin Wan, Wei Wang, Yang You et al. | Categories: cs.CV, cs.AI Link: arxiv.org/abs/2609.10346

The authors show that ranking vision token pruning methods by average benchmark accuracy hides substantial sample-wise complementarity, with alternative strategies outperforming the average-best on many individual samples. VIP-Router is a lightweight router that adaptively selects the pruning strategy best suited to each input, conditioned on low-cost visual and textual features, and can fall back to full-token inference. It achieves a 26.9% relative improvement in average accuracy and 22.0% relative utility gain over the best fixed strategy, with only 0.017% additional trainable parameters.

Takeaway: A plug-and-play router that exploits per-sample strategy diversity—practical, cheap, and a reminder that average-case benchmarks can mask real deployment gains.

5. Deterministic Prompting for Speaker-Stable Low-Resource Greek TTS

Authors: Georgios Syllas, Efthymios Georgiou, Kosmas Kritsis, Alexandros Potamianos | Categories: cs.SD, cs.CL, cs.LG Link: arxiv.org/abs/2609.10022

The authors curate TTS-ready Greek data from audiobooks via WhisperX alignment and filtering, then fine-tune Parler-TTS (880M) with a speaker-specific LoRA stage trained on 3.5 hours of single-speaker data. They find that LLM-generated style prompts cause speaker drift, and replacing them with deterministic prompts resolves the issue. The system reaches WER 10.7% (2.9 above the ASR floor), MOS-I 4.00 vs. 4.36 for human speech, and near-human speaker consistency (MOS-C 4.24 vs. 4.30).

Takeaway: A tidy case study in low-resource TTS that isolates a subtle prompting failure mode—useful for anyone adapting multilingual models to underrepresented languages.

6. Retrofitting Code Using LLMs to Support Exceptional Behavior

Authors: Linghan Zhong, Jiyang Zhang, Jayanth Srinivasa, Junyi Jessy Li, Milos Gligoric | Categories: cs.SE, cs.CL Link: arxiv.org/abs/2609.10397

The paper introduces retrofitting existing code with Exception Related Code (ERC), given code without ERC and Exceptional Behavior Tests. EXCODER performs context engineering that integrates static and dynamic program analysis with LLMs, and is evaluated on a benchmark built from 304 methods across 75 GitHub Java projects. With Qwen 2.5 Coder 32b, EXCODER achieves pass@1 of 85.92% (12.56 p.p. over baseline), pass@5 of 86.18%, and pass@10 of 86.51%.

Takeaway: Matching exceptional behavior to tests is a genuinely useful formulation of test-driven development for error handling, and the gains over baseline are substantial.

7. Emergency Department Revisit Quality Review Screening: Exploring Human Decision-Making and Artificial Intelligence Support

Authors: Jonathan A. Handler, Marlene I. Robles-Granda, Jacob E. Mefford, Jeremy S. McGarvey, Gregory S. Podolej et al. | Categories: cs.CY, cs.AI Link: arxiv.org/abs/2609.10421

In a retrospective study of 99 ED diagnosis pairs with revisits within 1–14 days, GPT-4 poorly correlated with clinician raters, flagging 94% of pairs for follow-up (4.4–13.3× more than clinicians), though prompt engineering was minimal. An algorithm leveraging an LLM-populated knowledge graph (KGA) achieved 83–100% positive predictive value for at least one clinician determining further assessment was warranted.

Takeaway: A candid look at where off-the-shelf LLMs fail in clinical screening—and how structured knowledge graphs can salvage useful signal without inflating reviewer workload.

8. Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation

Authors: Ben Merbaum, Mohammad Amin Raeisi, Wenhao Wang, Charalampos Papamanthou, Katerina Sotiraki et al. | Categories: cs.CR, cs.LG Link: arxiv.org/abs/2609.10264

Maverick builds private and verifiable LLM inference on a delegation protocol for matrix-vector multiplication, providing (to the authors’ knowledge) the first information-theoretically sound verification protocol for this operation with transparent preprocessing, efficient batch verification, and virtually no server overhead. Combined with LPN-based pseudorandom masking for input privacy, it is evaluated on Qwen3-4B, achieving throughput gains over local inference of up to 17× with online masks, 45× with precomputed masks, and 44× for verification only.

Takeaway: Making delegated inference both private and verifiable without punishing server overhead is a hard combination—this is a meaningful advance for trust-minimized LLM serving.

This content was generated with AI assistance. Paper information sourced from arXiv.