Part III: How AI Works: A Technical Primer

Section 3.1: The Architectures of Thought: Symbolic vs. Connectionist AI

The history of AI has been shaped by a deep philosophical and technical divide between two competing approaches to creating intelligence: the symbolic and the connectionist schools.

Symbolic AI (GOFAI - Good Old-Fashioned AI)

Symbolic AI, which dominated the field's early decades, is a top-down approach. It is founded on the hypothesis that intelligence can be achieved by manipulating symbols (like words or numbers) according to a set of explicit, formal rules, much like a mathematician solves an equation. The mind, in this view, is a device that operates on bits of information according to these rules.
A classic implementation of this paradigm is the Rule-Based System. Such a system consists of two main components:

1. A Knowledge Base: A database containing facts about a specific domain, representing the system's "knowledge" (e.g., "Fever is a symptom of infection").
2. An Inference Engine: A set of rules, typically in an "IF-THEN" format, that dictates how to manipulate the facts in the knowledge base to reach conclusions (e.g., "IF a patient has a fever AND a sore throat, THEN diagnose strep throat").

In this model, intelligence is explicitly programmed by human experts who codify their knowledge into these facts and rules. While powerful for well-defined, logical problems, symbolic AI proved to be brittle and inflexible when faced with the ambiguity and complexity of the real world.

Connectionist AI

In contrast, connectionism is a bottom-up approach inspired by the structure and function of the human brain. It posits that intelligence is not the result of following pre-programmed rules but is an emergent property arising from the interactions of a vast network of simple, interconnected processing units, or artificial neurons.

In this paradigm, a system is not explicitly coded with knowledge. Instead, it learns by being exposed to vast amounts of data. Through a process of trial and error, the system adjusts the strength (or "weight") of the connections between its neurons, gradually forming patterns and relationships associated with the data. The "intelligence" of a connectionist system resides in the complex web of these learned connection weights, not in a set of human-written rules. This approach is philosophically aligned with empiricism—the idea that knowledge comes from sensory experience—whereas symbolic AI aligns with rationalism, which emphasizes innate knowledge and logical rules. The recent explosion in AI capabilities is almost entirely due to the success of the connectionist approach, powered by massive datasets and computational power.

Section 3.2: A Taxonomy of Reasoning Models

Reasoning is the process of drawing inferences or conclusions from information. AI systems employ several types of reasoning, each suited to different kinds of problems.

• Deductive Reasoning: This is a top-down, logic-based approach where a conclusion is guaranteed to be true if the premises are true. It moves from general principles to specific instances. For example: Premise 1: All software has bugs. Premise 2: ChatGPT is software. Conclusion: ChatGPT has bugs. Deductive reasoning is used in formal logic systems and expert systems where the rules are considered absolute.
• Inductive Reasoning: This is a bottom-up approach that generalizes from specific observations to form broader conclusions. It is probabilistic, not certain. For example: Observation: Every time I have run this program, it has crashed. Conclusion: The program is unstable. This is the foundational logic of most machine learning, where a model learns a general rule (e.g., how to identify a cat) by observing thousands of specific examples (images of cats).
• Abductive Reasoning: This form of reasoning seeks the most plausible explanation for an incomplete set of observations. It is often described as "inference to the best explanation". For example: Observation: The street is wet. Explanation: The most likely reason is that it rained. Abductive reasoning is crucial in medical diagnosis and cybersecurity for identifying potential threats based on limited evidence.
• Case-Based Reasoning (CBR): This method solves new problems by retrieving and adapting solutions from a database of similar past problems, or "cases". Instead of reasoning from first principles, a CBR system finds the most analogous past case and reuses its solution. This is commonly used in customer support helpdesks and legal analysis.
• Model-Based Reasoning: This approach uses an explicit model of how a system or the world works to make inferences. For example, a diagnostic AI for a car would have an internal model of the engine, allowing it to reason about how a fault in one component might affect others.

Section 3.3: The Building Blocks: How Neural Networks Learn

Connectionist AI is implemented using artificial neural networks (ANNs). These networks are built from layers of interconnected nodes, or "neurons," that are mathematical evolutions of the early MCP neuron.

The conceptual ancestor was the MCP Neuron, which could only execute pre-programmed logic. The first major leap forward was the Perceptron, developed by Frank Rosenblatt in 1957. The Perceptron was a single-layer neural network that could learn from data. It could automatically adjust its internal weights and threshold to correctly classify inputs, a critical advance that moved the field from static programming to dynamic learning.

Modern deep neural networks are far more complex, with many layers of neurons, but the fundamental learning process remains conceptually similar:

1. Prediction: The network receives an input (e.g., an image of a cat) and, based on its current weights, makes a prediction (e.g., "it's a dog").
2. Error Calculation: The prediction is compared to the correct label ("cat") to calculate an error signal. The size of the error reflects how wrong the prediction was.
3. Backpropagation: This is the core learning algorithm. The error signal is propagated backward through the network, from the output layer to the input layer. At each connection, the algorithm makes a tiny adjustment to the weight, assigning more "blame" to the connections that contributed most to the error.
4. Iteration: This process of prediction, error calculation, and weight adjustment is repeated millions or even billions of times with a vast dataset. With each iteration, the network's predictions become slightly more accurate, as it gradually "learns" the intricate patterns in the data that define, for instance, what a cat looks like.

Section 3.4: Understanding Modern AI: What is a Model? What is an LLM?

The terminology of modern AI can be confusing. It is helpful to understand the hierarchy of key terms.

• AI Model: This is a broad term for any program or algorithm that uses statistical techniques to find patterns in data and then uses those patterns to make predictions or decisions. A simple linear regression model is a type of AI model, as is a complex deep neural network.
• Generative AI: This is a category of AI models that are capable of creating new, original content, rather than just classifying or predicting from existing data. This content can be text, images, music, or code.
• Large Language Model (LLM): An LLM is a specific, highly advanced type of generative AI model that is specialized for understanding and generating human-like text. LLMs like OpenAI's GPT series or Google's Gemini are the engines behind conversational AI applications like ChatGPT.

The inner workings of an LLM can be simplified as follows:

1. Training Data: They are pre-trained on an unimaginably vast corpus of text and code, often scraped from a large portion of the public internet.
2. Architecture: They are built using a specific deep learning architecture called a Transformer, introduced by Google researchers in 2017. The Transformer's "attention mechanism" allows the model to intelligently weigh the importance of different words in a long passage of text, giving it a sophisticated grasp of context.
3. Core Function: At its most fundamental level, an LLM's task is astonishingly simple: predict the next most statistically probable word (or, more accurately, "token") in a sequence, given the preceding text. When a user provides a prompt, the model generates a response one token at a time, each time calculating the most likely next piece of the sequence.
4. Parameters: The "large" in LLM refers to the number of adjustable parameters (the connection weights) in the model, which can run into the hundreds of billions or even trillions. More parameters allow the model to capture more complex and subtle patterns in language.
5. Fine-Tuning: After pre-training, models are often fine-tuned using Reinforcement Learning from Human Feedback (RLHF). In this stage, humans rate the model's responses, and this feedback is used to "reward" the model for generating answers that are more helpful, harmless, and aligned with human expectations, steering it away from the raw, unfiltered nature of its training data.

A crucial distinction in the modern AI landscape is between a foundation model and a fine-tuned model. A foundation model, like GPT-4, is a massive LLM trained on a wide variety of general internet data. It is incredibly expensive to build but can perform a vast range of tasks. A fine-tuned model is created by taking a pre-existing foundation model and continuing its training on a smaller, specialized dataset (e.g., legal documents or medical research) to make it an expert in that specific domain. This two-step process underpins the business strategy of major AI labs: build one hugely expensive foundation model, then license access to it for countless companies to adapt for their specific needs.

Works cited

[21] Rule-Based System - Engati
[22] Rule-based system - Wikipedia
[23] AI winter - Wikipedia
[24] AI for Beginners - The Difference Between Symbolic & Connectionist AI
[25] What is Reasoning in AI? Types and Applications in 2025 - Aisera
[26] www.ionos.com
[27] What is case-based reasoning? - IONOS
[28] en.wikipedia.org
[29] Model-based reasoning - Wikipedia