Velamuvi AIAcademy

Lesson 4 of 12 · Numbers and Algebra

Exponents and Logarithms

Understand exponents, square roots, scientific notation, the number e and logarithms in plain English, with graphs and Python, and see why logs appear everywhere in machine learning.

  • 9 min read
  • Beginner
  • Reviewed
In this lesson you will learn
  • What exponents are and the rules for working with them
  • Square roots and scientific notation
  • What the special number e is
  • What a logarithm is, and how it undoes an exponent
  • Why logarithms appear so often in AI

Exponents: repeated multiplication

In Python, the power operator is **:

Try it yourselfPython
print(2 ** 3)      # 2 x 2 x 2
print(10 ** 2)     # 10 squared
print(5 ** 1)      # anything to the power 1 is itself
print(7 ** 0)      # anything (except 0) to the power 0 is 1
print(2 ** -1)     # a negative power means "one divided by": 1 / 2

"Squared" means to the power 2, and "cubed" means to the power 3.

The rules of exponents

RuleExampleWhy
multiply: add the powers2³ × 2² = 2⁵3 twos times 2 twos is 5 twos
divide: subtract the powers2⁵ ÷ 2² = 2³two of the twos cancel
power of a power: multiply(2³)² = 2⁶two groups of 3 twos
power 05⁰ = 1
negative power2⁻³ = 1 ÷ 2³
Try it yourselfPython
print(2 ** 3 * 2 ** 2, "==", 2 ** 5)
print(2 ** 5 / 2 ** 2, "==", 2 ** 3)
print((2 ** 3) ** 2, "==", 2 ** 6)

Square roots

Try it yourselfPython
import math

print(math.sqrt(9))
print(9 ** 0.5)          # a power of 0.5 is a square root
print(math.sqrt(2))

Scientific notation

Very big and very small numbers are common in AI: billions of parameters, and learning rates such as 0.0001.

Try it yourselfPython
parameters = 7e9
learning_rate = 1e-4

print(f"{parameters:,.0f}")
print(learning_rate)
print(learning_rate == 0.0001)

Exponential growth and the number e

When something grows by multiplying, rather than adding, it is exponential. Doubling every step gives 1, 2, 4, 8, 16, 32... and it quickly becomes enormous.

-3-2-101234051015
y = 2ˣy = eˣ
Exponential curves start flat and then rise faster and faster.
Try it yourselfPython
import math

print(math.e)
print(math.exp(1))
print(round(math.exp(2), 4))

Logarithms: the opposite of an exponent

An exponent asks: "2 to the power 3 is what?" (answer: 8). A logarithm asks the reverse: "2 to the power what is 8?" (answer: 3).

Try it yourselfPython
import math

print(math.log2(8))         # 2 to the power what is 8?
print(math.log10(1000))     # 10 to the power what is 1000?
print(math.log(math.e))     # e to the power what is e?
print(math.log(1))          # anything to the power 0 is 1

# Undoing each other:
print(math.log(math.exp(5)))
0246810-4-3-2-101234
y = ln(x)y = log₂(x)
Logarithms grow quickly at first, then more and more slowly. They are only defined for numbers above 0.

Why logarithms matter in AI

Logarithms turn multiplication into addition: log(a × b) = log(a) + log(b). That is extremely useful:

  • Tiny probabilities. Multiplying many small probabilities gives numbers too small for a computer. Adding their logs is safe.
  • Loss functions. The most common way to measure a classifier's error, called log loss or cross-entropy, uses logarithms.
  • Huge ranges. Charts of things like model size often use a log scale, so 1 million and 1 billion fit on the same picture.
Try it yourselfPython
import math

probabilities = [0.01, 0.02, 0.005, 0.01]

product = 1
for p in probabilities:
    product *= p
print("Multiplied:", product)

log_total = 0
for p in probabilities:
    log_total += math.log(p)
print("Sum of logs:", round(log_total, 3))
print("Back again:", math.exp(log_total))

The sum of logs carries the same information as the product, but in a size the computer handles comfortably.

Common mistakes

Practice exercise

Use an exponent to grow a dataset, and a logarithm to work backwards.

Your solutionPython
import math

# A dataset doubles in size every month, starting at 1,000 rows.
# 1. Print the size after 0, 1, 2, 3 and 4 months (1000 * 2 ** month).
# 2. Use math.log2 to work out how many months until it reaches 1,024,000 rows,
#    and print it as a whole number.

# Output must be exactly:
# Month 0: 1000
# Month 1: 2000
# Month 2: 4000
# Month 3: 8000
# Month 4: 16000
# Months to reach 1,024,000: 10
Knowledge check

What is log₁₀(1000)?

Summary

  • An exponent says how many times to multiply the base by itself. In Python, use **.
  • Multiply powers by adding exponents. A power of 0 is 1, and a negative power means one divided by.
  • A square root is the power ½. Scientific notation writes 7e9 and 1e-4.
  • e is about 2.718, and math.exp(x) is e to the power x.
  • A logarithm undoes an exponent. It turns multiplication into addition, which is why AI uses it so much.

You have finished the first module. Next: functions, the idea at the heart of every AI model.

Finished reading? Tick it off to track your progress.