∫ A definition of intelligence and its measure

Kernel:

Intelligence is one's ability to want things and capability to achieve them.
Intelligence is measured by the success rate of achieving various goals.
And a measure theory based definition of intelligence.

We are living in the edge time where synthetic intelligence is just around the corner. But not just only after the advent of the computers, that we have thought about intelligence. Before that, intelligence has always been a subject of advancement, investment, and great historical interest. Along with curiosity, we use it to expore the immeasurable extent of the universe. Along with bravery, we equip it and land ourselves on the moon. Along with pride, we decorate our bank notes not with the faces of our strongest but the brightest. Intelligence is perhaps unarguably the most versatile trait of human beings.

But what is intelligence? How do we measure it? In this blog, I would like to present my perspective on the matter; and, if you would indulge me, provide a definition of intelligence and how to measure it.

∂ Definition of intelligence

Let's start with perhaps the most famous definition of intelligenceLegg, S., & Hutter, M. (2007). Universal intelligence: A definition of machine intelligence. Minds and machines, 17, 391-444.:

Intelligence measures an agent's ability to achieve goals in a wide range of environments

Shane Legg and Marcus Hutter

There are several questions that arise from this definition.

What if we consider the receiving reward instead of goals?

This is often the case in reinforcement learningSutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press. where the agent is trained to maximize some reward function. Suppose that there is an agent that always miss its goals but still manage to achieve an overall high rewards. Even though the agent can achieve high rewards, the agent has failed to control its surrounding to get what it wants. Is this agent considered intelligent in this case? Or better, does the agent have full autonomy of its actions at all?

Can something without a goal or purpose be intelligent?

This relates to the previous question. We cannot distinguish between a random agent that unintentionally reaches a certain state and an agent whose interest is implicit but nevertheless reaches the same state. Therefore, there is no way to be sure that those agents have control over itself. Trying to measure intelligence without knowing the subject's goal is like asking "why" something exists. Unless we consider everything inherently intelligent (which would make the definition of intelligence meaningless), it is not useful to measure the intelligence of something whose motives we do not know.

How wide of the range of environments for intelligence measure to be meaningful?

In the definition, the range of environments should be wide. But how wide is wide? Can we say that a person is intelligent if they can only achieve goals in a limited range of environments? Or can we say that a person is intelligent if they can achieve goals in a wide range of environments but only with a low success rate? Which is more intelligent? Is it fair to compare them at all?

From the analysis of these questions, we know that

we should not consider the agent intelligent based on rewards,
intelligence is meaningful only when purpose or the interest to achieve goals is explicit, and
intelligence quotient ties on the environment in which the agent performs.

With respect to these points, I propose the following definition of intelligence:

Intelligence is a context-dependent ability to achieve goals with explicit interest.

This definition encompasses the idea that intelligence involves not only the capability to achieve goals but also clear signs of interest or desire to achieve them. It also emphasizes the context-dependent nature of intelligence, recognizing that it cannot be easily compared between agents with different interests or operating in different environments.

But suppose that we cautiously prepare everything for the sake of measuring intelligence. How can we actually measure it?

∂ Measuring intelligence

To measure intelligence, we can assess the alignment between the importance an agent places on achieving a goal and its actual performance in doing so. A straightforward way to do this is through a controlled trial.

Given a sequence of trials, where each trial consists of a starting state $s$ and a goal state $g$, an agent must attempt to achieve the goal. If the agent's success rate for any given trial $t = (s, g)$ is $\mathcal{V}(t)$, then the agent's intelligence $ \mathcal{I} $ is defined as the expected success rate over all trials:

$$ \begin{align*} \mathcal{I}^{M} \triangleq \mathbb{E}_P \left[\mathcal{V}\right] \end{align*} $$

where $P$ is the probability distribution over the trials, as some trials may be more important for the agent to succeed than others.

Let's redefine this using the language of measure theoryhttps://en.wikipedia.org/wiki/Measure_space. Consider a measure space $ M = (E, T, P) $ for measuring intelligence, where:

$E = S \times G $ is the sample space, representing all possible pairs of start states $S$ and goal states $G$ in an environment.
$T$ is the set of all trial groups (e.g., $\{(s_0, g_0), (s_1, g_1), \ldots \}$).
$P$ is the probability measure, which assigns an importance to each trial group ($P: T \rightarrow [0, 1]$ such that $P(E) = 1$).

Intelligence can then be defined as the Lebesgue integral of the success rate function $\mathcal{V}$ over the environment:

$$ \begin{align*} \mathbb{E}_P\left[\mathcal{V}\right] = \int_{E} \mathcal{V} d P \end{align*} $$

I find it easier to grasp this concept by thinking of the function $ \mathcal{V} $ as a collection of mutually exclusive "neurons" $ \nu_i $, where each neuron activates for a designated group of trialsCybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems, 2(4), 303-314.:

$$ \begin{align*} \mathcal{V}(t) = \nu_0\mathbf{1}_{T_0}(t) + \nu_1\mathbf{1}_{T_1}(t) + \ldots \end{align*} $$

Here, $ \mathbf{1}_{T_i} $ is an indicator function for the trial group $ T_i \subset T $. It equals 1 if the trial $ t $ is in group $ T_i $, and 0 otherwise. Each group $ T_i $ contains trials for which the agent achieves a consistent success rate of $ \nu_i $. This signifies a skill set or domain of expertise that requires the same capability to achieve the goals. We can interpret the intelligence as a weighted sum of the performance over these skill sets:

$$ \begin{align*} \int_{E} \mathcal{V} d P = \nu_0 P(T_0) + \nu_1 P(T_1) + \ldots \end{align*} $$

where $ P(T_i) $ is the importance measure of the skill set $ T_i $.

Therefore, intelligence can be defined using the measure theory as

$$ \begin{align*} \mathcal{I}^{M} \triangleq \int_{E} \mathcal{V} d P. \end{align*} $$

∂ Application to education

This definition of intelligence mirrors real-world actions. When we use tests to assess students' cognitive abilities, we're essentially measuring their success at achieving the test's specific goals. However, since intelligence is also defined by the desire to achieve goals, test scores are only truly meaningful when the test-taker is genuinely interested in the test.

Therefore, educators should focus not just on improving test scores, but on cultivating students' genuine interest in the subject matter. A student's measured intelligence is most reflective when they are aware of the value and importance of their education, and are motivated to achieve their goals.

∂ Final words

My entire life has been dedicated to the pursuit of the meaning of intelligence and how to simulate it. I have read many papers and books, and have spent countless hours thinking about it. This post is the culmination of my thoughts on the matter. I hope that this post can be a stepping stone for those who want to understand intelligence even better, and a final closure for those who have been seeking the answer for a long time.