Frank Nielsen

What is information geometry?

Dreaming of a future enriched by computer science, I engage in research on information and geometry. This is because I believe that beyond existing theories lies a new world that can only be seen by transcending them.

We perceive the world using language, and sometimes differences in language can lead to differences in perception. Geometry itself is a language. One of the major characteristics of geometry is that it allows humans to understand the world more intuitively. This is because in geometry we should be invariant to the choice of coordinate systems defining objects. Coordinate systems can be chosen purposely to ease the computations. Due to its purity, geometry can dynamically capture intrinsically the diversity of structures of information spaces.

Historically, when attempting to perceive the world anew through the language of geometry, one notices fresh perspectives and approaches emerging.

For example, Kepler first postulated his three laws of planetary motions from empirical data collected by telescopes. Those laws were later recovered from first principles by Newton's laws of motions. Nowadays, geometric mechanics rely on symplectic geometry and contact geometry and are used to intrinsically analyze dynamical systems.

I am pursuing theories that emerge when viewing the world anew through the lens of geometry and develop new technologies based on these novel insights.

In our daily life, we may perceive the world with many types of geometries beyond the usual familiar Euclidean geometry. Let me give you 4 examples showing how we change our geometric representation of space in our daily lives (Figure 1):

1. When moving to a new house, we load a truck by packing boxes using our familiar　Euclidean geometry.
2. When riding a train, we see that the parallel rail tracks merge in a single point at the horizon. Our brain implicitly switches the geometric model to the projective model to reason about our visual perception.
3. When moving by taxi in Manhattan, New York, we think about the L1-geometry where the distance between two points p1=(x1,y1) and p2=(x2,y2) is not the Euclidean distance but |x1-x2|+|y1-y2|. This reflects the fact that streets define a grid where we move on.
4. When flying transcontinental, we visualize the trajectory of the airplane on a 2D Mercator map not as a straight line, but rather as a curved line called a geodesic. This indicates that the airplane is following a great circle arc surrounding the Earth.

Figure 1. Managing different geometries in our daily life

In information geometry (https://franknielsen.github.io/IG/index.html), our spaces are not physical but statistical in essence. That is, a space denotes a statistical model of probability models, and we seek to infer from data the right model in that space. This geometric viewpoint allows us to build efficient algorithms (Figure 2). Information geometry finds many applications in signal processing (like Independent Component Analysis or radar processing), machine learning (deep learning with natural gradient), sound processing (geometric factorization), medical imaging (diffusion tensor imaging), etc.

Figure 2. Visualization of Geometric and Statistical Means

Breaking through the "walls" of theories

To surpass existing theories, it's crucial to identify and approach the "walls" that tell us the limits of those theories. I believe it's the duty of mathematicians to touch and explore these boundaries. In fields like machine learning and artificial intelligence, some fundamental questions are: How do we learn models from datasets? How do we extract information from datasets? How do we compare the performance and fitting of models? How do we simplify large models? I'm dedicated to pursuing principles that tackle these questions, aiming to overcome some of the existing walls established by current theories. My research spans a broad spectrum of geometric science applied to fields ranging from machine learning to data science, visual computing, and artificial intelligence. Geometry provides the most advanced calculus (Figure 3): I'm dealing with inherently non-Euclidean, high-dimensional, noisy, large-scale, heterogeneous dynamic datasets.

In addition, I'm developing geometric computational methods and toolboxes to construct advanced models and learning machines that capture both regularity and diversity in datasets.

Geometry provides the most advanced calculus: We can define and solve partial differential equations on manifolds or equivalently PDEs on curved surfaces. Geometry by its language and logic inspires us to build new techniques powerful for machine learning and AI. For example, when learning a deep neural network (DNN), we seek parameters of the DNN which minimizes a given loss function. The set of parameters forms a neuromanifold. We then visualize the dynamics of learning as a trajectory on the neuromanifold and design fast optimization methods for finding good parameters with guaranteed performance and invariance properties.

The future of information geometry

Since 2013, I've been instrumental in organizing the Geometric Science of Information (GSI, https://franknielsen.github.io/GSI/), fostering a community dedicated to the geometric science of information (In 2023, we hosted the sixth edition. See photo of participants at GSI'23). With each passing year, the number of participants has grown, and the community has expanded rapidly. This indicates a growing recognition of the relationships between data, model, information and geometry, and I feel that people are increasingly embracing it.

Figure 5. Group photo of GSI'23

When I contemplate what geometry is, I believe there isn't a clear definition. Because I believe that the act of capturing things that cannot be defined precisely is the essence of research unleashing creativity. I thoroughly enjoy this pursuit of research aimed at exploring the possibility of viewing the world through new perspectives.