Plenary Talk WIRN 2026
The Transformer After Heuristics: Geometry, Redundancy, and the Incremental Architecture
Abstract
Transformers dominate modern AI, yet their architecture is still chosen mostly by intuition, scaling rules, and trial and error. In this talk, I will argue that this phase is ending. I will present a mathematical theory of the Transformer that reveals why standard architectures contain deep structural redundancy, identifies the geometric source of directional information flow in attention, and explains how the behavior of heads and layers can be understood through a small set of principled operators.
On this basis, I will introduce the Incremental Transformer (INCRT), an architecture that does not fix its own size in advance. Instead, it grows during training only when the task demands new structure, and stops when the learned representation becomes sufficient. The broader implication is a universal law of structural complexity: across architectures (MLPs, CNNS, RNNS and so on), the number of units a task truly requires is not arbitrary, but governed by measurable geometric quantities.
This talk is meant both as a theoretical proposal and as an invitation: to move from architecture design by search to architecture design by derivation.
Biography
Giansalvo Cirrincione is Associate Professor HDR at the University of Picardie Jules Verne, Amiens, France, and an IEEE Senior Member. His research lies at the intersection of mathematics, neural networks, deep learning, and signal processing, with applications in medicine, bioinformatics, energy systems, and intelligent diagnostics. He has authored more than 200 scientific publications, developed several original neural models and theoretical frameworks, and received multiple Best Paper Awards for his research. He has recently completed a monograph on the Transformer, currently in publication, presenting a comprehensive mathematical theory of the architecture together with the Incremental Transformer, a self-determining model derived from task geometry. He is a frequent invited speaker and has delivered keynote and plenary talks at international conferences.