UCL Foundational AI CDT PhD Student

UCL Foundational AI CDT PhD Student

Publications

You can also find my articles on my Google Scholar profile.

Journal Articles

Latte: Latent Attention for Linear Time Transformers

Published in NGSM ICML Workshop, 2024

This paper is about a linear latent variable re-parametrisation of attention and combining it with local standard attention

Recommended citation: Dolga, Rares, Marius Cobzarenco, and David Barber. "Latent Attention for Linear Time Transformers." arXiv preprint arXiv:2402.17512 (2024).
Download Paper

RotRNN: Modelling Long Sequences with Rotations

Published in NGSM ICML Workshop, 2024

This paper is about parameterising linear recursive neural networks with rotation matrices.

Recommended citation: Dolga, Rares, et al. "RotRNN: Modelling Long Sequences with Rotations." arXiv preprint arXiv:2407.07239 (2024).
Download Paper