Under the Hood of AI Timelines: How the AI 2027 Forecast is Made

While forecasting how AI will impact my career in Consumer Lending Credit Risk Management (Article Link), I explored the timeline forecast methodology used in AI 2027.

Prerequisites

Time Horizon

The concept of time horizon, proposed by the Model Evaluation & Threat Research (METR) team, states that the "50%-task-completion time horizon is the time humans typically take to complete tasks that AI models can complete with a 50% success rate." A notable result from their research is that the length of tasks AI can perform is doubling approximately every 7 months.

Monte Carlo Simulation

Monte Carlo simulation was largely developed in the 1940s by mathematicians and physicists working on secret projects. They were dealing with incredibly complex problems, such as simulating neutron diffusion, which were too difficult to solve with traditional analytical mathematics. The fundamental idea was to determine probability distributions by running numerous trials and observing the outcomes. The core ideas involve using probability distributions to model variables (instead of single, deterministic values) and conducting repeated trials to ensure a sufficient number of random scenarios are explored for patterns to emerge. The name "Monte Carlo" is thought to have originated because one of its inventors, Stanislaw Ulam, enjoyed gambling at the Monte Carlo Casino, and the project required a secret codename.

The process of Monte Carlo simulation is as follows:

1. Define the problem: Identify the key variables and their relationships.
2. Create a model: Use probability distributions to represent the variables.
3. Run many trials: Simulate the model multiple times to observe the probability distribution of the outcomes.
4. Analyze the results: Look for patterns and make predictions.

The variables used in the AI 2027 model are:

I was confused about the difference between "doubling time speed up" and "intermediate speed up". Doubling time speed up affects the time (number of work hours based on current capacity) required to reach the target capability level, while intermediate speed up affects the actual time it takes to reach the target capability level if efficiency is improved.

Lognormal Distribution and Normal Distribution

Assuming familiarity with the normal distribution, a lognormal distribution arises when the logarithm of a random variable is normally distributed. First, a variable following a lognormal distribution can only take positive values, as its logarithm is normally distributed (and the exponential of any real number is positive). Second, the lognormal distribution is typically skewed to the right. This implies that while most values may cluster around a central point, there is a possibility of observing significantly larger values, albeit less frequently. An underlying data generation process that often leads to a lognormal distribution involves the multiplicative accumulation of random effects. For example, stock returns (\(r\)), which result from compounding, can often be modeled this way.

Gaussian Copula

To account for correlations between variables, a Gaussian Copula can be employed. The core concept is to define a multivariate joint distribution (for correlated variables) using their individual marginal distributions and a copula function, which captures the correlation structure. The process is as follows:

1. Specify the marginal distributions for each variable.
2. Specify the correlation matrix.
3. Generate random samples from the multivariate normal distribution derived from the correlation matrix.
4. Transform the normal samples to uniform samples.
5. Transform the uniform samples to the desired marginal distributions.

In this model, T_t, cost_speed, and the growth type probabilities (is_superexponential, is_subexponential, is_exponential) are treated as correlated variables with a correlation coefficient of 0.7. Similarly, present_prog_multiplier and SC_prog_multiplier are modeled with a 0.7 correlation.

The Process

The following table summarizes the process of each Monte Carlo simulation run.