Lesson 4 of 12 · Numbers and Algebra
Exponents and Logarithms
Understand exponents, square roots, scientific notation, the number e and logarithms in plain English, with graphs and Python, and see why logs appear everywhere in machine learning.
- 9 min read
- Beginner
- Reviewed
- What exponents are and the rules for working with them
- Square roots and scientific notation
- What the special number e is
- What a logarithm is, and how it undoes an exponent
- Why logarithms appear so often in AI
Exponents: repeated multiplication
In Python, the power operator is **:
print(2 ** 3) # 2 x 2 x 2 print(10 ** 2) # 10 squared print(5 ** 1) # anything to the power 1 is itself print(7 ** 0) # anything (except 0) to the power 0 is 1 print(2 ** -1) # a negative power means "one divided by": 1 / 2
"Squared" means to the power 2, and "cubed" means to the power 3.
The rules of exponents
| Rule | Example | Why |
|---|---|---|
| multiply: add the powers | 2³ × 2² = 2⁵ | 3 twos times 2 twos is 5 twos |
| divide: subtract the powers | 2⁵ ÷ 2² = 2³ | two of the twos cancel |
| power of a power: multiply | (2³)² = 2⁶ | two groups of 3 twos |
| power 0 | 5⁰ = 1 | |
| negative power | 2⁻³ = 1 ÷ 2³ |
print(2 ** 3 * 2 ** 2, "==", 2 ** 5) print(2 ** 5 / 2 ** 2, "==", 2 ** 3) print((2 ** 3) ** 2, "==", 2 ** 6)
Square roots
import math print(math.sqrt(9)) print(9 ** 0.5) # a power of 0.5 is a square root print(math.sqrt(2))
Scientific notation
Very big and very small numbers are common in AI: billions of parameters, and learning rates such as 0.0001.
parameters = 7e9
learning_rate = 1e-4
print(f"{parameters:,.0f}")
print(learning_rate)
print(learning_rate == 0.0001)Exponential growth and the number e
When something grows by multiplying, rather than adding, it is exponential. Doubling every step gives 1, 2, 4, 8, 16, 32... and it quickly becomes enormous.
import math print(math.e) print(math.exp(1)) print(round(math.exp(2), 4))
Logarithms: the opposite of an exponent
An exponent asks: "2 to the power 3 is what?" (answer: 8). A logarithm asks the reverse: "2 to the power what is 8?" (answer: 3).
import math print(math.log2(8)) # 2 to the power what is 8? print(math.log10(1000)) # 10 to the power what is 1000? print(math.log(math.e)) # e to the power what is e? print(math.log(1)) # anything to the power 0 is 1 # Undoing each other: print(math.log(math.exp(5)))
Why logarithms matter in AI
Logarithms turn multiplication into addition: log(a × b) = log(a) + log(b). That is extremely useful:
- Tiny probabilities. Multiplying many small probabilities gives numbers too small for a computer. Adding their logs is safe.
- Loss functions. The most common way to measure a classifier's error, called log loss or cross-entropy, uses logarithms.
- Huge ranges. Charts of things like model size often use a log scale, so 1 million and 1 billion fit on the same picture.
import math
probabilities = [0.01, 0.02, 0.005, 0.01]
product = 1
for p in probabilities:
product *= p
print("Multiplied:", product)
log_total = 0
for p in probabilities:
log_total += math.log(p)
print("Sum of logs:", round(log_total, 3))
print("Back again:", math.exp(log_total))The sum of logs carries the same information as the product, but in a size the computer handles comfortably.
Common mistakes
Use an exponent to grow a dataset, and a logarithm to work backwards.
import math # A dataset doubles in size every month, starting at 1,000 rows. # 1. Print the size after 0, 1, 2, 3 and 4 months (1000 * 2 ** month). # 2. Use math.log2 to work out how many months until it reaches 1,024,000 rows, # and print it as a whole number. # Output must be exactly: # Month 0: 1000 # Month 1: 2000 # Month 2: 4000 # Month 3: 8000 # Month 4: 16000 # Months to reach 1,024,000: 10
What is log₁₀(1000)?
Summary
- An exponent says how many times to multiply the base by itself. In Python, use
**. - Multiply powers by adding exponents. A power of 0 is 1, and a negative power means one divided by.
- A square root is the power ½. Scientific notation writes
7e9and1e-4. - e is about 2.718, and
math.exp(x)is e to the power x. - A logarithm undoes an exponent. It turns multiplication into addition, which is why AI uses it so much.
You have finished the first module. Next: functions, the idea at the heart of every AI model.
Finished reading? Tick it off to track your progress.