Opening and processing files in Python

Code junkies hangout here

Moderators: ChrisThornett, LXF moderators

Opening and processing files in Python

Postby PhilC » Wed Aug 17, 2005 3:16 pm

Hi,

I'm trying to make a start in Python programming. I consider myself competant with Perl, and I want to do some of the same things which are easy in Perl, in Python.

In particular, I want to be open a file and process each line at a time.
In Perl I just do:
while (<>) {
doSomethingWith ($_);
}

I've had a browse around python.org, but can't find any good references to opening and processing files.

Can anyone tell me how to do the above Perl code in Python? I really would appreciate it, thanks.
PhilC
 
Posts: 6
Joined: Tue Apr 26, 2005 5:08 pm

RE: Opening and processing files in Python

Postby nelz » Wed Aug 17, 2005 5:35 pm

Code: Select all
lines = open('foo/bar').readlines()
for line in lines:
   dosomethingwith(line)
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
User avatar
nelz
Site admin
 
Posts: 8577
Joined: Mon Apr 04, 2005 11:52 am
Location: Warrington, UK

RE: Opening and processing files in Python

Postby PhilC » Thu Aug 18, 2005 12:34 pm

Thanks for getting me started nelz, I'll be using that a lot in the future.

I did a little bit more digging though since the perl diamond operator can do a little bit more than this code.
i.e. it will read each file given as an argument to the program and process each file in turn. Much more useful than that though is that it can read input from stdin or a pipe.

Just in case anyone else is looking for this feature in Python, I had another look around python.org and found this code: (http://www.python.org/doc/2.2.3/lib/mod ... input.html)

import fileinput
for line in fileinput.input():
dosomethingwith (line)
PhilC
 
Posts: 6
Joined: Tue Apr 26, 2005 5:08 pm

Postby overflow » Wed Aug 24, 2005 11:40 am

It is even easier than the example posted above.
Code: Select all
for line in file('foo.bar'):
    print line

file (which is a synonym for open) support iterators and it yields a class which simply passes records back to you one at a time.

The example posted above would read the whole file into memory in the readlines() function which might be a performance problem for large files.
overflow
User avatar
overflow
LXF regular
 
Posts: 158
Joined: Tue Aug 23, 2005 1:40 pm
Location: London

Postby overflow » Wed Aug 24, 2005 12:00 pm

Python doesn't have the equivalent of the Perl <> operator but it's easy enough to reproduce it. Here's a simple cat program written in Python which supports multiple filenames on the command line and '-' for stdin.
Code: Select all
#! /usr/bin/python
import sys
def fileargs():
   for filename in sys.argv[1:]:
      if filename == '-':
         for line in sys.stdin:
            yield line
      else:
         for line in file(filename):
            yield line
      
for line in fileargs():
   print line

The fileargs function is an iterator (or generator, I can't remember the difference) and passes back records from each input file in turn. It switches to stdin if '-' is specified.

I hope that helps.
overflow
User avatar
overflow
LXF regular
 
Posts: 158
Joined: Tue Aug 23, 2005 1:40 pm
Location: London

Postby PhilC » Tue Oct 04, 2005 12:51 pm

Thanks overflow,
That was a very insightful answer and a great example to demonstrate.
(Still wondering why people prefer to use this to perl though)
PhilC
 
Posts: 6
Joined: Tue Apr 26, 2005 5:08 pm

Postby nelz » Tue Oct 04, 2005 1:01 pm

Because the code doesn't look like line noise :)
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
User avatar
nelz
Site admin
 
Posts: 8577
Joined: Mon Apr 04, 2005 11:52 am
Location: Warrington, UK


Return to Programming

Who is online

Users browsing this forum: No registered users and 2 guests