Preprint - Under Review
TL;DR: We investigate a strong baseline for LLM Agents Memory: retrieving experience from similar tasks. We show that LLM agents generalize better to never-before-seen tasks when they are trained not only to act, but also to use retrieved experience during training. We analyses key choices behind effective experience retrieval fine-tuning.
EMNLP, 2024 & Sys2Reasoning @ NeurIPS, 2024
TL;DR: Introducing RealInstruct to evaluate LLMs on real multi-constrained instructions, and DeCRIM self-correction that improves instruction following by decomposing requests and refining responses, enabling open LLMs to outperform GPT-4 with strong feedback.
Learning on Graphs Conference (LoG 2024)
TL;DR: Investigate the impact of incorporating user's text into GNNs to enhance fake news detection. Found some content features are more important than network features, and that contextual text representations brings big improvement to the task in two benchmarks.
ICASSP, 2024
TL;DR: Proposes a lightweight adaptation method bridging the gap between small and large speech models on under-represented languages by leveraging language-specific experts and knowledge distillation, outperforming fine-tuning and LoRA with minimal overhead.
Master Dissertation, Télécom Paris, Institut Polytechnique de Paris
TL;DR: We find that Whisper speaker and model-related biases are worsened by light compression (quantization), for low-resource languages and small models, and to address this we propose a distilled modular variant that markedly improves it for under-represented languages while preserving its multilingual, multitask strengths with minimal overhead.
PROPOR, 2022
TL;DR: ZeroBERTo combines topic modeling with language models for zero-shot text classification, overcoming input size limitations and reducing runtime, achieving a 12% better F1 score and 13x faster inference compared to XLM-R on a Portuguese benchmark.