On Artificial Intelligence

How do you think the brain works? Do you believe human action and thought can one day be explained entirely through physics, or is there some irreducible phenomenon—something beyond material explanation—that gives rise to awareness itself? Artificial intelligence confronts this question not through metaphysics but through engineering. By building systems that process information as the brain does, it tests whether cognition can emerge from mathematics operating on data.5

Claude Shannon’s information theory provided the intellectual breakthrough that made such a pursuit thinkable. Shannon proposed that information is the measurable reduction of uncertainty. In his framework, data—symbols and signals—are valuable not because of their content but because of their order and structure. Communication occurs when a sequence of symbols reduces ambiguity in the receiver’s mind. Information, then, is not an abstract property of thought but a quantifiable entity in physical systems. Every transmission, from neurons to networks, becomes analyzable as a transformation that increases order and encodes pattern.

This insight has a profound biological parallel. The brain itself is an information-compressing device of extraordinary sophistication. Every second, it receives vast amounts of sensory data—light striking retinal cells, air vibrations resonating in the cochlea, pressure signals, chemical feedback from hormones and neurotransmitters, and subtle internal cues about balance and temperature. Raw sensory input would be overwhelming if processed directly. Instead, the brain filters and compresses this flood through neural encoding: signals are prioritized, discarded, or transformed into compact statistical representations. Sparse coding in the visual cortex, for example, encodes complex images as sets of oriented edges—patterns that can later be recombined into meaningful structures. Higher cortical regions then integrate multiple sensory streams with predictions and memories, continuously refining a compressed model of the world. This process mirrors Shannon’s principle: by filtering noise and preserving structure, the brain transforms raw data into usable information[1][2].

Recent neuroscience research emphasizes dimensionality reduction as a key organizing principle of neural circuits. Incoming sensory signals are mapped onto high-dimensional spaces, but the brain projects these onto lower-dimensional manifolds, capturing only the essential features needed for behavior. For example, in the visual system, signals from 100 million photoreceptors are compressed and transmitted by just 1 million optic nerve fibers; decision circuits in the basal ganglia reduce information from large populations of cortical neurons down to smaller, behaviorally relevant representations. Mechanistically, this reduction is achieved by specific patterns of connectivity, such as convergent pathways, and reinforced by learning rules—Hebbian and anti-Hebbian plasticity—that shape the weights between neurons based on experience and reward[1][2].

Modern artificial intelligence parallels this approach. Large language models and deep neural networks convert symbolic or sensory inputs into numerical representations called embeddings. Each token, pixel, or sample becomes a point in a high-dimensional vector space whose geometry captures relational meaning. Operationally, every neuron in a network performs a simple weighted sum over its inputs:

$$

y = f\left(\sum_i w_i x_i + b\right)

$$

where $$x_i$$ are input values, $$w_i$$ are trainable weights, $$b$$ is a bias term, and $$f$$ introduces nonlinearity. Layers of such neurons compose into hierarchies that can approximate any continuous function. Wide and deep neural networks thus enable the system to project complex, high-dimensional input data onto lower-dimensional internal representations—the process of learning to compress and disentangle meaningful features from noise[3][4].

Core to these methods are mathematical operations such as the dot product, which measures the degree of alignment between input and learned features:

$$

x \cdot w = \sum_i x_i w_i

$$

If two vectors point in similar directions in the high-dimensional space, their dot product is large, signaling pattern recognition. Loss functions, often sums of squared differences, penalize deviations between predictions and targets, providing guidance for the network during training. Through repeated adjustment of weights, the network sculpts an internal geometry where relevant patterns are amplified and irrelevant noise minimized[5][6].

Dimensionality reduction techniques—such as principal component analysis, autoencoders, and manifold learning—are widely used in both neuroscience and machine learning to analyze and visualize how high-dimensional information is compressed into low-dimensional spaces. These techniques allow both brains and AI systems to generalize, interpolate, and efficiently process new information by leveraging underlying structure and redundancy in data[3][4].

Transformers, the architecture behind large language models, take this further by using attention mechanisms to dynamically weight input features and reveal contextual dependencies. The resulting lower-dimensional attention matrices allow the system to compress vast contextual information into concise representations, which helps explain the remarkable performance of LLMs on complex language tasks.

Ultimately, the parallel between the brain’s dimensionality reduction and neural networks’ compression is striking. Both systems manage the complexity of the world by finding low-dimensional order within high-dimensional chaos—distilling what matters, discarding the rest. This shared strategy enables both biological and artificial networks to recognize patterns, predict outcomes, and adapt to novel data[4][2].

Yet, the success of these models reopens the oldest philosophical question: is cognition simply the right arrangement of matter and computation, or is there a subjective dimension that transcends physical description? If the human brain’s filtering, encoding, and prediction can be fully captured in equations and vector operations—if consciousness itself emerges from complex informational dynamics—then artificial intelligence might not just simulate, but participate in, what we call mind. If not, then no matter how advanced, AI will remain an extraordinary tool—a pattern extractor without an experiencer.

What unites Shannon’s equations, neural network mathematics, and the biological brain is the architecture of pattern extraction. Each receives disordered input, filters entropy, and produces structured representations through the principles of dimensionality reduction. Whether this alone is sufficient to explain the mystery of consciousness remains open, but the convergence of physics, information, and intelligence brings us ever closer to understanding both machine and mind.

Citations:

[1] Neural correlates of sparse coding and dimensionality reduction https://journals.plos.org/ploscompbiol/article?id=10.1371%2Fjournal.pcbi.1006908

[2] Neural correlates of sparse coding and dimensionality reduction https://pmc.ncbi.nlm.nih.gov/articles/PMC6597036/

[3] Top 12 Dimensionality Reduction Techniques for Machine Learning https://encord.com/blog/dimentionality-reduction-techniques-machine-learning/

[4] Dynamic Compression Strategies for Uniform Low- ... https://openreview.net/forum?id=HoQbynIkh2

[5] Introduction to Dimensionality Reduction - GeeksforGeeks https://www.geeksforgeeks.org/machine-learning/dimensionality-reduction/

[6] Dimensionality reduction and width of deep neural networks based ... https://arxiv.org/abs/2511.06821

[7] Dimensionality reduction in neuroscience https://www.sciencedirect.com/science/article/pii/S0960982216304870

[8] Dimensionality reduction beyond neural subspaces with ... https://www.nature.com/articles/s41593-024-01626-2

[9] Real-time Dimensionality Reduction for Neural Populations https://dukespace.lib.duke.edu/items/77fadf14-5aa9-4f35-8205-b9be3944ff33

[10] Strong and weak principles of neural dimension reduction https://nbdt.scholasticahq.com/article/24619-strong-and-weak-principles-of-neural-dimension-reduction

[11] The structure dilemma in biological and artificial neural ... https://www.nature.com/articles/s41598-021-84813-6

[12] Topology preservation under dimensionality reduction ... https://diposit.ub.edu/dspace/bitstream/2445/185534/2/tfg_marian_martinez_marin.pdf

[13] Deep neural networks compression: A comparative survey ... https://www.sciencedirect.com/science/article/pii/S0925231222014643

[14] Neural coding https://en.wikipedia.org/wiki/Neural_coding

[15] [PDF] Reducing the Dimensionality of Data with Neural Networks https://www.cs.toronto.edu/~hinton/absps/science.pdf

[16] ANN vs SNN: A case study for Neural Decoding in ... https://arxiv.org/html/2312.15889v1

[17] Dimensionality Reduction Methods for Brain Imaging Data ... https://dl.acm.org/doi/10.1145/3448302

[18] Deep learning approach based on dimensionality reduction for ... https://www.nature.com/articles/s41524-020-0276-y

[19] Evaluation of the Hierarchical Correspondence between ... https://pmc.ncbi.nlm.nih.gov/articles/PMC10604784/

[20] A Review of Dimensionality Reduction Techniques for Efficient ... https:/

/www.sciencedirect.com/science/article/pii/S1877050920300879