https://hdl.handle.net/20.500.12008/48532 2026-05-13T22:53:57Z 2026-05-13T22:53:57Z Sastre, Ignacio Rosá, Aiala https://hdl.handle.net/20.500.12008/54654 2026-04-28T17:42:47Z 2025-01-01T00:00:00Z

Título: Memory Tokens: Large Language Models can generate reversible sentence embeddings Autor: Sastre, Ignacio; Rosá, Aiala Resumen: In this work, we observe an interesting phenomenon: it is possible to generate reversible sentence embeddings that allow an LLM to reconstruct the original text exactly, without modifying the model’s weights. This is achieved by introducing a special memory token, whose embedding is optimized through training on a fixed sequence. When prompted with this embedding, the model reconstructs the fixed sequence exactly. We evaluate this phenomenon across English and Spanish datasets, sequences of up to approximately 240 tokens, and model scales ranging from 100M to 8B parameters. Notably, Llama 3.1 8B successfully reconstructs all tested sequences. Our findings highlight an interesting capability of LLMs and suggest potential applications in memory-based retrieval, compression, and controlled text generation.

2025-01-01T00:00:00Z Sastre, Ignacio Rosá, Aiala https://hdl.handle.net/20.500.12008/54653 2026-04-28T17:42:07Z 2026-01-01T00:00:00Z

Título: Concept Tokens: Learning behavioral embeddings through concept definitions Autor: Sastre, Ignacio; Rosá, Aiala Resumen: We propose Concept Tokens, a lightweight method that adds a new special token to a pretrained LLM and learns only its embedding from multiple natural language definitions of a target concept, where occurrences of the concept are replaced by the new token. The LLM is kept frozen and the embedding is optimized with the standard language-modeling objective. We evaluate Concept Tokens in three settings. First, we study hallucinations in closed-book question answering on HotpotQA and find a directional effect: negating the hallucination token reduces hallucinated answers mainly by increasing abstentions, whereas asserting it increases hallucinations and lowers precision. Second, we induce recasting, a pedagogical feedback strategy for second language teaching, and observe the same directional effect. Moreover, compared to providing the full definitional corpus in-context, concept tokens better preserve compliance with other instructions (e.g., asking follow-up questions). Finally, we include a qualitative study with the Eiffel Tower and a fictional "Austral Tower" to illustrate what information the learned embeddings capture and where their limitations emerge. Overall, Concept Tokens provide a compact control signal learned from definitions that can steer behavior in frozen LLMs. Descripción: Aceptado para su publicación en : 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026) San Diego, California, July 2 - 7, 2026.

2026-01-01T00:00:00Z Dai, Zuoheng Manitto, Martín Chiruzzo, Luis Rosá, Aiala https://hdl.handle.net/20.500.12008/54243 2026-04-07T17:34:11Z 2025-01-01T00:00:00Z

Título: Error detection and correction for English learners using neural models Autor: Dai, Zuoheng; Manitto, Martín; Chiruzzo, Luis; Rosá, Aiala Resumen: In recent years, the importance of English language learning in Uruguay has been steadily growing. However, the country faces a significant challenge: a shortage of teachers, which makes it difficult for students to achieve an adequate English level. This problem particularly affects children at the initial levels, limiting their progress in language learning. In this context, it is crucial to assess student progress in order to define educational policies. This project builds on the work of previous initiatives that seek to support teachers in correcting exercises written by students. The central purpose of the project is to develop a tool that facilitates the automatic correction of texts written by English learners at the initial stages. The tool must receive the texts and return a corrected version, highlighting the identified errors. Initially, we explored the exclusive use of Large Language Models (LLMs) to address this problem. However,after several experiments, the results were not satisfactory. Given this limitation, we opted for an alternative solution that combines the use of LLMs, a module for detecting differences, and a classifier trained to predict error types, achieving the same correction objective, developing a system composed of three independent modules. We obtained an error corrector based on the Mistral LLM with an F0.5 score of 0.81, and an error classifier with an F1 score of 0.76. Overall, the system achieved an F0.5 performance of 0.68.

2025-01-01T00:00:00Z Dufrechou, Ernesto Favaro, Federico https://hdl.handle.net/20.500.12008/53879 2026-03-13T17:17:02Z 2025-01-01T00:00:00Z

Título: Modelar : Modelado del desempeño de métodos numéricos en plataformas de hardware heterogéneas Autor: Dufrechou, Ernesto; Favaro, Federico Descripción: El video lo realizó el Área de Comunicación de la Facultad de Ingeniería.

2025-01-01T00:00:00Z