File Input/Output

Common tasks in programs are reading data in from a file and writing data out to a file.
To read or write data to a file in python, you can use the built-in python function, open().
It will also often be necessary to write your own functions to either parse data from a file, or write data to a file.
Suppose we have the following text file and it is called "file.txt" :

This is line 1
This is line 2
This is line 3

To open and read in the first line we use the following code :

>>> file = open("file.txt", "r")
>>> firstLine = file.readline()
>>> firstLine
'This is line 1\n'

Notice the open() function takes two arguments.

The first argument takes a string that represents the system path to the file you want to open and the filename. The path can be either an absolute full system path to the file, or it can be a relative path with respect to the directory where the program is running.
The second argument in the open function specifies what you want to do with the file. This is called the mode. In this case, we want to read the file, so we give it the argument "r", specifying read mode.

When we call the open() function, it returns a special file object data type that allows us to manipulate the file.
To use the file, we need to assign the returned value to a variable, in this case the variable named file, but we could have named this anything we wanted.
To read a line from the file, we use the command file.readline() where file is whatever the name of the variable is that we assigned to the file object. The file.readline() command will return a string representing the next line of text in the file. Whenever a file.readline() command is issued, the file variable will set its position in the file to the next line. This means subsequent calls to file.readline() will return the line after the last one that was returned.

>>> file = open("file.txt", "r")
>>> file.readline()
'This is line 1\n'
>>> file.readline()
'This is line 2\n'
>>> file.readline()
'This is line 3\n'

Make note that the string returned includes the '\n' character at the end. The '\n' character represents a new line in the file. This means if we print a line, it will print a new line after the string.

>>> file = open("file.txt", "r")
>>> print file.readline()
This is line 1

>>> print file.readline()
This is line 2

>>> print file.readline()
This is line 3

>>>

If we want to easily remove the new line character we can use the string method strip()

>>> file = open("file.txt", "r")
>>> file.readline().strip()
'This is line 1'
>>> file.readline().strip()
'This is line 2'
>>> file.readline().strip()
'This is line 3'

If we try to read a line when there are no more lines in the file, the file.readline() statement will just return an empty string

>>> file = open("file.txt", "r")
>>> file.readline()
'This is line 1\n'
>>> file.readline()
'This is line 2\n'
>>> file.readline()
'This is line 3\n'
>>> file.readline()
''

If we want to do processing on every single line in a file, python gives us a much easier way to do this using a for loop

>>> file = open("file.txt", "r")
>>> for line in file:
...     line
... 
'This is line 1\n'
'This is line 2\n'
'This is line 3\n'

In this method, each line will be looped over, and at each iteration, the line is stored in the variable line
Like with the file.readline() approach, a new line character is placed at the end of each line, which means if we print the line we'll get the following

>>> file = open("file.txt", "r")
>>> for line in file:
...     print line
... 
This is line 1

This is line 2

This is line 3

>>>

Again, we can use the strip() method to get rid of the ending new line characters

>>> file = open("file.txt", "r")
>>> for line in file:
...     print line.strip()
... 
This is line 1
This is line 2
This is line 3
>>>

Whenever you've finished using a file, you should close the file.
You close a file using the close() method.

>>> file = open("file.txt", "r")
>>> for line in file:
...     print line.strip()
... 
This is line 1
This is line 2
This is line 3
>>> file.close()

Writing to a File

Suppose we want to write to a file to save the results of a program to a text file.
To write to a file we will again use open(). This time however, we will specify the mode, the second argument, to be "w", where "w" indicates that we want to write to the file.
If we use the "w" argument, and a file with the same name specified already exists, it will erase the old file and start a new one, so be careful!
If we want to add data to an already existing file we can use the "a" argument, which signifies append mode. If a file already exists with the same name, then anything we write to the file will be appended to the file, below the previously existing data.
To create or erase a previous file with the name "newfile.txt" we would use the following code :

>>> file = open("newfile.txt", "w")
>>> file.write("New file line 1")
>>> file.write("New file line 2")
>>> file.close()

If we now look at the file "newfile.txt" we created, we'll find the following content :

New file line 1New file line 2

Notice the lines all run into each other!
To fix this, make sure we always add the new line character '\n' at the end of the strings

>>> file = open("newfile.txt", "w")
>>> file.write("New file line 1\n")
>>> file.write("New file line 2\n")
>>> file.write("New file line 3\n")
>>> file.close()

The "newfile.txt" file will now have the following contents

New file line 1
New file line 2
New file line 3

If we want to write a number or some other data type to a file, we must convert it to a string before passing it to the write method otherwise we will get an error:

>>> file = open("newfile.txt", "w")
>>> file.write(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: argument 1 must be string or read-only character buffer, not int

Suppose we want to get as input a student's grades, and then write a file for that student containing their grade record and average.


# Filename:    gradeReport.py
# Authors:     Sue Evans & James MacGlashan
# Date:        8/6/09
# Section:     12
# Email:       bogar@cs.umbc.edu
# Description: This program generates grade 
#              report files for two students.

# generateGradeReport() gets four homework grades 
# from the user, calculates their average, 
# generates a unique filename for each student, 
# and writes a grade report out to a file.
#
# Input:  a student's name, studentName
# Output: none
# Effect: A grade report file is produced.

def generateGradeReport(studentName) :

    # generate personalized prompts for each homework
    prompt1 = "Enter " + studentName + "'s grade for HW1: "
    prompt2 = "Enter " + studentName + "'s grade for HW2: "
    prompt3 = "Enter " + studentName + "'s grade for HW3: "
    prompt4 = "Enter " + studentName + "'s grade for HW4: "

    # get input from user
    grade1 = input(prompt1)
    grade2 = input(prompt2)
    grade3 = input(prompt3)
    grade4 = input(prompt4)

    average = (grade1 + grade2 + grade3 + grade4) / 4.0

    # generate a unique filename for each student
    filename = studentName + ".txt"

    # write the information to the file
    file = open(filename, "w")
    file.write("HW1: " + str(grade1) + "\n")
    file.write("HW2: " + str(grade2) + "\n")
    file.write("HW3: " + str(grade3) + "\n")
    file.write("HW4: " + str(grade4) + "\n")
    file.write("Average: " + str(average) + "\n")
    file.close()

def main() :
 
    generateGradeReport("Alice")
    print
    generateGradeReport("Bob")

main()

The above code will produce the files "Alice.txt" and "Bob.txt"
Let's run the program.

Enter Alice's grade for HW1: 93.0
Enter Alice's grade for HW2: 84.0
Enter Alice's grade for HW3: 87.0
Enter Alice's grade for HW4: 90.0

Enter Bob's grade for HW1: 89.0
Enter Bob's grade for HW2: 91.0
Enter Bob's grade for HW3: 80.0
Enter Bob's grade for HW4: 91.0

The grade report for Alice will have the following contents :

HW1: 93.0
HW2: 84.0
HW3: 87.0
HW4: 90.0
Average: 88.5

The grade report for Bob will have the following contents :

HW1: 89.0
HW2: 91.0
HW3: 80.0
HW4: 91.0
Average: 87.75

Appending a File

You can add new data to the end of a file using append mode, "a".
So, if you have a file named sample.txt with these contents:


Sample - line 1
Sample - line 2
Sample - line 3

We can open it in append mode and write some more lines to the file.


>>> file = open("sample.txt", "a")
>>> file.write("1st line appended\n")
>>> file.write("2nd line appended\n")
>>> file.close()
>>>

Let's look at sample.txt now.


Sample - line 1
Sample - line 2
Sample - line 3
1st line appended
2nd line appended

What happens if you try to append a file that doesn't exist ?
A new file will be made with the filename you used in the call to open()

What is "r+" mode?

"r+" mode allows you to both read from and write to a file.

When using this mode, you can read through a file until you find the place where you want to start writing, and then write to the file.

Here is sample.txt currently:


Sample - line 1
Sample - line 2
Sample - line 3
1st line appended
2nd line appended

Let's try using "r+".


>>> file = open("sample.txt", "r+")
>>> line = file.readline()
>>> line
'Sample - line 1\n'
>>> line = file.readline()
>>> line
'Sample - line 2\n'
>>> file.write("Writing a new line\n")
>>> file.close()
>>>


Sample - line 1
Sample - line 2
Writing a new line
 line appended
2nd line appended

Oh, my! What happened here ?

I'll fix the sample.txt file back to the way it was and we can try our experiment.


Sample - line 1
Sample - line 2
Sample - line 3
1st line appended
2nd line appended


>>> file = open("sample.txt", "r+")
>>> line = file.readline()
>>> line
'Sample - line 1\n'
>>> line = file.readline()
>>> line
'Sample - line 2\n'
>>> line = file.readline()
>>> line
'Sample - line 3\n'
>>> file.write("Exact length line\n")
>>> file.close()
>>>


Sample - line 1
Sample - line 2
Sample - line 3
Exact length line
2nd line appended

We were right. When we write to the file, what is written has to be exactly the same length as the original text that gets replaced, if it is to leave the remainder of the file unchanged.

This behavior severely limits the usefulness of this mode.

When could you use this mode ?

Double Space

Let's write a program that will read in a file that is single-paced and write it out to a file double spacing it.

# File: doubleSpace.py
# Author: Sue Evans
# Date: 10/1/09
# Section: All
# EMail: bogar@cs.umbc.edu
#
# This program reads in lines from a file that
# is single spaced and writes out to a file so
# that it will be double-spaced.

import string

# printGreeting() prints an explanation of the
#                 program for the user
# Inputs:  None
# Outputs: None

def printGreeting():

   print
   print "This program will read in the lines from a file that"
   print "is single-spaced and write those same lines out to"
   print "a file that will be double-spaced"
   print



def main():

   printGreeting()

   # get the filename from the user
   filename = raw_input("Enter the name of the file to double-space : ")
   
   # construct outfile name
   list = string.split(filename, '.')
   outFilename = list[0] + '.ds.' + list[1]
      
   # open infile and outfile
   infile = open(filename, 'r')
   outfile = open(outFilename, 'w')

   # process each line in the file
   for line in infile:
       double = line + '\n'
       outfile.write(double)

   # close the files
   infile.close()
   outfile.close()


main()

Let's look at both files

Here's sample.txt

This is line 1.
This is line 2.
This is line 3.
This is the last line.

And here's sample.ds.txt

This is line 1.

This is line 2.

This is line 3.

This is the last line.

File Exercise

Write a similar program that will read in lines from one file and write out those same lines to another file with the lines numbered. The line numbers should be right-justified in a total field width of 2.

Here's the contents of ioExercise.txt

This is line one.
This is line two.
This is line three.
This is line four
This is line five
This is line six
This is line seven
This is line eight.
This is line nine.
This is line ten.
This is line eleven.
This is line twelve.

Here's the contents of ioExercise.num.txt

 1 This is line one.
 2 This is line two.
 3 This is line three.
 4 This is line four
 5 This is line five
 6 This is line six
 7 This is line seven
 8 This is line eight.
 9 This is line nine.
10 This is line ten.
11 This is line eleven.
12 This is line twelve.