29. File Management

By Bernd Klein. Last modified: 08 Nov 2023.

Files in General

It's hard to find anyone in the 21st century, who doesn't know what a file is. When we say file, we mean of course, a file on a computer. There may be people who don't know anymore the "container", like a cabinet or a folder, for keeping papers archived in a convenient order. A file on a computer is the modern counterpart of this. It is a collection of information, which can be accessed and used by a computer program. Usually, a file resides on a durable storage. Durable means that the data is persistent, i.e. it can be used by other programs after the program which has created or manipulated it, has terminated.

The term file management in the context of computers refers to the manipulation of data in a file or files and documents on a computer. Though everybody has an understanding of the term file, we present a formal definition anyway:

A file or a computer file is a chunk of logically related data or information which can be used by computer programs. Usually a file is kept on a permanent storage media, e.g. a hard drive disk. A unique name and path is used by human users or in programs or scripts to access a file for reading and modification purposes.

The term "file" - as we have described it in the previous paragraph - appeared in the history of computers very early. Usage can be tracked down to the year 1952, when punch cards where used.

A programming language without the capability to store and retrieve previously stored information would be hardly useful.

The most basic tasks involved in file manipulation are reading data from files and writing or appending data to files.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See our Python training courses

See our Machine Learning with Python training courses

Reading and Writing Files in Python

The syntax for reading and writing files in Python is similar to programming languages like C, C++, Java, Perl, and others but a lot easier to handle.

We will start with writing a file. We have a string which contains part of the definition of a general file from Wikipedia:

definition = """
A computer file is a computer resource for recording data discretely in a
computer storage device. Just as words can be written 
to paper, so can information be written to a computer
file. Files can be edited and transferred through the 
internet on that particular computer system."""

We will write this into a file with the name file_definition.txt:

open("file_definition.txt", "w").write(definition)

283

If you check in your file browser, you will see a file with this name. The file will look like this: file_definition.txt

We successfully created and have written to a text file. Now, we want to see how to read this file from Python. We can read the whole text file into one string, as you can see in the following code:

text = open("file_definition.txt").read()

If you call print(text), you will see the text from above again.

Reading in a text file in one string object is okay, as long as the file is not too large. If a file is large, wwe can read in the file line by line. We demonstrate how this can be achieved in the following example with a small file:

with open("ad_lesbiam.txt", "r") as fh:
    for line in fh:
        print(line.strip())

V. ad Lesbiam
VIVAMUS mea Lesbia, atque amemus,
rumoresque senum severiorum
omnes unius aestimemus assis!
soles occidere et redire possunt:
nobis cum semel occidit breuis lux,
nox est perpetua una dormienda.
da mi basia mille, deinde centum,
dein mille altera, dein secunda centum,
deinde usque altera mille, deinde centum.
dein, cum milia multa fecerimus,
conturbabimus illa, ne sciamus,
aut ne quis malus inuidere possit,
cum tantum sciat esse basiorum.
(GAIUS VALERIUS CATULLUS)

Some people don't use the with statement to read or write files. This is not a good idea. The code above without with looks like this:

fh = open("ad_lesbiam.txt")
for line in fh:
        print(line.strip())
fh.close()

A striking difference between both implementation consists in the usage of close. If we use with, we do not have to explicitly close the file. The file will be closed automatically, when the with blocks ends. Without with, we have to explicitly close the file, like in our second example with fh.close(). There is a more important difference between them: If an exception occurs inside of the ẁith block, the file will be closed. If an exception occurs in the variant without with before the close, the file will not be closed. This means, you should alwawys use the with statement.

We saw already how to write into a file with "write". The following code is an example, in which we show how to read in from one file line by line, change the lines and write the changed content into another file. The file can be downloaded: pythonista_and_python.txt:

with open("pythonista_and_python.txt") as infile:
    with open("python_newbie_and_the_guru.txt", "w") as outfile:
        for line in infile:
            line = line.replace("Pythonista", "Python newbie")
            line = line.replace("Python snake", "Python guru")
            print(line.rstrip())
            # write the line into the file:
            outfile.write(line)

A blue Python newbie, green behind the ears, went to Pythonia.
She wanted to visit the famous wise green Python guru.
She wanted to ask her about the white way to avoid the black.
The bright path to program in a yellow, green, or blue style.
The green Python turned red, when she addressed her.
The Python newbie turned yellow in turn.
After a long but not endless loop  the wise Python uttered:
"The rainbow!"

As we have already mentioned: If a file is not to large and if we have to do replacements like we did in the previous example, we wouldn't read in and write out the file line by line. It is much better to use the readmethod, which returns a string containing the complete content of the file, including the carriage returns and line feeds. We can apply the changes to this string and save it into the new file. Working like this, there is no need for a withconstruct, because there will be no reference to the file, i.e. it will be immediately deleted afeter reading and writing:

txt = open("pythonista_and_python.txt").read()
txt = txt.replace("Pythonista", "Python newbie")
txt = txt.replace("Python snake", "Python guru")
open("python_newbie_and_the_guru.txt", "w").write(txt)
;

''

Resetting the Files Current Position

It's possible to set - or reset - a file's position to a certain position, also called the offset. To do this, we use the method seek. The parameter of seek determines the offset which we want to set the current position to. To work with seek, we will often need the method tell which "tells" us the current position. When we have just opened a file, it will be zero. Before we demonstrate the way of working of both seek and tell, we create a simple file on which we will perform our commands:

open("small_text.txt", "w").write("brown is her favorite colour")
;

''

The method tell returns the current stream position, i.e. the position where we will continue, when we use a "read", "readline" or so on:

fh = open("small_text.txt")
fh.tell()

0

Zero tells us that we are positioned at the first character of the file.

We will read now the next five characters of the file:

fh.read(5)

'brown'

Using tellagain, shows that we are located at position 5:

fh.tell()

5

Using read without parameters will read the remainder of the file starting from this position:

fh.read()

' is her favorite colour'

Using tellagain, tells us about the position after the last character of the file. This number corresponds to the number of characters of the file!

fh.tell()

28

With seekwe can move the position to an arbitrary place in the file. The method seek takes two parameters:

fh.seek(offset, startpoint_for_offset)

where fh is the file pointer, we are working with. The parameter offset specifies how many positions the pointer will be moved. The question is from which position should the pointer be moved. This position is specified by the second parameter startpoint_for_offset. It can have the follwoing values:

0: reference point is the beginning of the file
1: reference point is the current file position
2: reference point is the end of the file

if the startpoint_for_offset parameter is not given, it defaults to 0.

WARNING: The values 1 and 2 for the second parameter work only, if the file has been opened for binary reading. We will cover this later!

The following examples, use the default behaviour:

fh.seek(13)
print(fh.tell())   # just to show you, what seek did!
fh.read()          # reading the remainder of the file

13
'favorite colour'

It is also possible to move the position relative to the current position. If we want to move k characters to the right, we can just set the argument of seek to fh.tell() + k

k = 6
fh.seek(5)    # setting the position to 5
fh.seek(fh.tell() + k)   #  moving k positions to the right
print("We are now at position: ", fh.tell())

We are now at position:  11

seek doesn't like negative arguments for the position. On the other hand it doesn't matter, if the value for the position is larger than the length of the file. We define a function in the following, which will set the position to zero, if a negative value is applied. As there is no efficient way to check the length of a file and because it doesn't matter, if the position is greater than the length of the file, we will keep possible values greater than the length of a file.

def relative_seek(fp, k):
    """ rel_seek moves the position of the file pointer k characters to
    the left (k<0) or right (k>0) 
    """
    position = fp.tell()