Newlines and escape sequences in Python

Have you ever seen \n in a Python string? What does that represent?

Newline characters

This string contains a newline character:

>>> text = "Hello\nworld"
>>> text
'Hello\nworld'

That's what \n represents: a newline character.

If we print this string, we'll see that \n becomes an actual newline:

>>> print(text)
Hello
world

Why does Python represent a newline as \n?

Escape sequences in Python

Every character in a Python string represents that literal character, with the exception of escape sequences. Escape sequences represent something other than the literal character that we type.

All escape sequences start with a backslash (\) in Python.

When I try to run this code in Python, I'll see a warning:

>>> print("C:\Windows")
<unknown>:1: SyntaxWarning: invalid escape sequence '\W'
C:\Windows

The \W is an invalid escape sequence, so Python ends up treating the backslash as a literal backslash.

If we wanted to properly represent a literal backslash character, we could use a double backslash (\\):

>>> print("C:\\Windows")
C:\Windows

Doubling the backslash lets Python know that we want a literal backslash character.

Using escape sequences to avoid syntax errors

Escape sequences can help us avoid invalid syntax while we're creating a string.

For example, if we try to use the Enter key to insert a newline into a string, we would see a SyntaxError:

>>> text = "Hello
  File "<python-input-0>", line 1
    text = "Hello
           ^
SyntaxError: unterminated string literal (detected at line 1)

We could fix this by creating a multi-line string using triple quotes, and then hitting Enter to make a newline:

>>> text = """Hello
... world"""

Or we could make a string that uses \n to represent a newline character:

>>> text = "Hello\nworld"

Escape sequences can also help us avoid syntax errors due to quote characters.

Using a double quote in a string literal that's created with double quotes results in a SyntaxError:

>>> message = "It said "I won't" in large letters."
  File "<python-input-0>", line 1
    message = "It said "I won't" in large letters."
                             ^

SyntaxError: unterminated string literal (detected at line 1)

We can fix this problem by escaping the double quotes in our string:

>>> message = "It said \"I won't\" in large letters."
>>> message
'It said "I won\'t" in large letters.'

This tells Python that those quotes represent literal quote characters, instead of ending our string.

>>> print(message)
It said "I won't" in large letters.

Whitespace characters in Python

Escape sequences are pretty common for whitespace characters as well.

Whitespace characters are characters that don't represent an actual visible character, but instead represent some sort of blank space.

For example, this string looks like it only has two characters in it when we print it:

>>> string = " .\n."
>>> print(string)
 .
.

But if we look at the length of this string, we'll see that there are four characters in it:

>>> len(string)
4

One of those characters is a space, and the other is a newline character. Those are whitespace.

Space and newline are the most common whitespace characters, but they're not the only ones.

For example, \t represents a tab character:

>>> print("Tab:\t.")
Tab:    .

Tab characters can also be made by hitting the Tab key on your keyboard. Though, sometimes your text editor may convert those into space characters instead (particularly when writing in a code editor).

The width of a tab character varies depending on how many characters come before it. So it's more common to see them entered in code using the escape sequence \t, than using the Tab key on your keyboard. Using \t makes it crystal clear that the character that we're seeing is in fact a tab character, and not a series of space characters.

In addition to space, newline, and tab, there are hair spaces, en spaces, em spaces, non-breaking spaces, and many more whitespace characters. Most whitespace characters you'll probably never see.

But there is one more whitespace character that we should probably discuss. You'll almost always see it alongside newline characters.

Carriage returns

This is a carriage return:

>>> carriage_return = "\r"

On Windows machines, and on much of the Internet, line breaks are represented by a carriage return followed by a line feed, sometimes called CRLF for Carriage Return, Line Feed:

>>> text = "Line 1\r\nLine 2\r\nLine 3"
>>> print(text)
Line 1
Line 2
Line 3

Wait, did I just call \n a line feed? Isn't it a newline? 🤔

Well, officially, \n is called a line feed character. But it's also often called a newline character because it's used to represent a new line.

Sort of.

Some programs require a carriage return just before a line feed to treat a line as properly ended.

The history of line endings

In the 1960s, teleprinters used a carriage return (CR) to move the printhead back to the beginning of the line, and they used a line feed (LF) to move to the next line. Except, some teleprinters allowed just a line feed to accomplish the same thing.

Teletype machines came after teleprinters, and early operating systems either required a carriage return followed by a line feed, or just a line feed on its own.

Although, early Macintosh machines actually allowed just a carriage return on its own for their newlines. So things are complicated.

You don't need to know any of this history to use newline characters, but in case you're curious, see this Scott Hanselman video on newlines for more on the history of newline characters.

Newline normalization in Python

These different ways of representing a newline character can cause headaches.

So when Python opens files, it normalizes all line endings to simply be represented as line feed characters (\n).

This file has lines separated by CRLF (carriage return, line feed) line endings:

>>> with open("au-states.csv") as csv_file:
...     contents = csv_file.read()
...

So in the land of escape sequences, that's \r\n (\r is carriage return and \n is line feed).

But Python only shows \n (line feed) because it's normalizing our line endings:

>>> contents[:60]
'New South Wales,Sydney\nQueensland,Brisbane\nSouth Australia,A'

To force Python to preserve the original line endings in our file, when we open the file, we can specify newline="":

>>> with open("au-states.csv", newline="") as csv_file:
...     contents = csv_file.read()

This way Python won't normalize the line endings, so we can see the actual \r\n characters that are in the file:

>>> contents[:60]
'New South Wales,Sydney\r\nQueensland,Brisbane\r\nSouth Australia'

But usually, this line ending normalization that Python does automatically is a good thing. So in most cases, I recommend that you let Python do newline normalization for you.

Writing files with custom line endings

Python also normalizes newlines when writing files. But it does so based on the operating system that your code runs on.

Whenever Python writes a line feed character (\n) to a text file on Windows, that line feed will automatically be converted to a carriage return (\r) followed by a line feed (\n):

So in Windows, this code that writes LF (\n) newlines will end up writing CRLF (\r\n) to the file:

>>> with open("file.txt", mode="wt") as file:
...     file.write("Line 1\nLine 2\n")
...
14

If you don't want this behavior when writing certain text to certain files, you can specify newline="" when opening your file for writing:

>>> with open("file.txt", mode="wt", newline="") as file:
...     file.write("Line 1\nLine 2\n")
...
14

When we write our file this way, the original line endings will be maintained from our string.

It's most common to specify newline="" when you know that the string that you're writing already has carriage return characters within it. For example, Python's csv module writes carriage returns at the end of each line, because that's how CSV files are usually formatted:

>>> import csv
>>> cities = [
...     ["New York City", 8175133, 81, .314],
...     ["Los Angeles", 3971883, 66, .121],
...     ["Chicago", 2695598, 54, .186],
...     ["Houston", 2296224, 78, .081],
...     ["Phoenix", 1563025, 79, .045],
... ]
...
>>> with open("cities.csv", mode="wt", newline="") as csv_file:
...     writer = csv.writer(csv_file)
...     writer.writerows(cities)
...

So when writing a file using Python's csv module, you're expected to always specify newline="" when opening the file you're writing to, to make sure that you don't accidentally double up your carriage returns on Windows.

Python uses \n to represent newline characters

Python sees a line feed (\n) as the canonical way to represent a newline.

Though, sometimes you'll see a carriage return followed by a line feed (\r\n) used to represent a newline, usually when your text is coming from over a network, or when you're reading it from a database.

Technically \n is called a line feed, though we usually just call it a newline character.