Bash recursive collection script

The place to post if you need help or advice

Moderators: ChrisThornett, LXF moderators

Bash recursive collection script

Postby Lancer » Sat Jun 25, 2005 12:10 am

Probably this can be done with a simple cp command and options...

I have a school fileserver with each student having their own account in /home as shown here...

/home/judy
/home/larry
/home/greg
/home/karen
(etc)

Students are in three class group (junior, middle and senior known as $GROUP1 $GROUP2 and $GROUP3) although I don't really want to add that complication to my script right now except to say that the wanted scripts COULD be formed along the lines of...

for STUDENT in $GROUP3 do { ..... }

...ignore that if you want (it could just do all students in /home dir), as it's a side issue. Just something you can use if you like. Here's the real question:

Students in the SENIOR group (for example) have made a lot of pov ray modeler files (which end in the *.kpm prefix). I want to make a colection.sh script which will go through the senior /home dirs and copy all files in those directories into a new one at /home/collection, RENAMING each file as tom1.kpm tom2.kpm tom3.kpm tom4.kpm and so forth, depending on their username. What this means is that I will have a "collected works" (/home/collection) directory where I can quickly sort through student files to look at the items of the file type I am wanting.

The script must not of course also reproduce stuff from the /home/collection directory itself
The script should be recursive just in case students decided to organise their work into subdirs

Preferably, the script would be executed along the lines of

./colection.sh GROUP3 *.kpm

...to collect all the student files in the list $GROUP3 which end in .kpm

It's the sort of thing I love to nut out myself (for my own learning) although I'm kind of pressured for time at the moment, so your help is appreciated.
Lancer
LXF regular
 
Posts: 232
Joined: Thu Apr 07, 2005 11:59 am

RE: Bash recursive collection script

Postby Guest » Sat Jun 25, 2005 5:47 am

OK I got you setup but I did it in Perl...

Code: Select all
#!/usr/bin/perl

$returnval = system('find /home | grep -v "collection" | grep -i "\.kpm" > /tmp/xyz');
$infile = '/tmp/xyz';
open(INFILE, $infile);
while (<INFILE>){
        if ($_ =~ m/\/home\/(\w+).*\/(\w+\.[kK][pP][mM])$/)
        {
                $fullpath = $_;
                $username = $1;
                $filename = $2;
                chomp($fullpath);
                chomp($username);
                chomp($filename);
                #print "VALUES: fullpath $fullpath username $username filename $filename\n";
                $args = '/home/collection/' . $username . '_' . $filename;
                #print "COMMAND " . "mv $fullpath $args\n";
                system("mv $fullpath $args");
        }
}


Paul Doherty
http://powerusers.info
Power Tools for Power Users
Guest
 

Postby ped » Mon Jun 27, 2005 8:33 pm

Man, for being "pressured for time" you seem to be taking your time getting back to this thread... :roll:
Paul Doherty
http://powerusers.info
Power Tools for Power Users
ped
 
Posts: 3
Joined: Sat Jun 25, 2005 5:53 am

Postby Rhakios » Mon Jun 27, 2005 9:35 pm

Obviously so pressured for time that he hasn't had sufficient, yet ;)
Bye, Rhakios
User avatar
Rhakios
Moderator
 
Posts: 7634
Joined: Wed Apr 06, 2005 11:18 pm
Location: Midlands, UK

Postby Lancer » Tue Jun 28, 2005 4:58 am

Actually, the reason I didn't get back yet was because the perl solution somewhat confused me, as I do not know perl (It does not look harder than other languages, but the /\/\/\/\/\ syntax throws me a bit, which is why I specified BASH). I thereby was in no position to modify the perl script for a few other things I wanted to do, but did not want critique a solution which, if I only right now had the time to study perl, was in all likelyhood a perfectly viable solution (hence lack of a reply). Instead used the following script to get the job done...

Code: Select all
#!/bin/bash
DIRNAME=sximpress

FILELIST='
./kaitb/slideshow.sxi
./miko/my_robot.sxi
./jessf/slideshow.sxi
./alexd/slideshow.sxi
./marp/TUX.sxi
./maxk/slideshow.sxi
./andyw/slideshow.sxi
./chanp/slidshow.sxi
./student/slideshow_yasmin.sxi
./needp/slideshow.sxi
./hang/trees.sxi
./harryg/slideshow.sxi
./chrisp/spt.sxi
./chrisp/slide show.impress.sxi
./troyh/slide show.sxi
./josg/slideshow.sxi
./olivr/slideshow.sxi
./ryanb/slideshow.sxi
./ryanb/myperson.sxi
./harh/slideshow.sxi
./juliah/slideshow.sxi
./morgb/slideshow.sxi
./genec/slideshow.sxi
./zain/slideshow.sxi
./camb/SLIDESHOW.sxi
./levih/levi and .sxi
./jakabc/Happy little monster.sxi
./alexh/slindshow.sxi
./maxt/slideshow.sxi
'

# ----------------- Create new collection and copy objects.
mkdir $DIRNAME
for i in $FILELIST; do
mkdir -p $DIRNAME/$i
rmdir $DIRNAME/$i
cp -rf /home/$i $DIRNAME/$i
done


...where $DIRNAME was given the name of the target "collection" directory for the wanted file type and the $FILELIST were results of a manual [/b]find[/b] for the file type I was wanting to collect in the /home/dirs.

Not an ideal solution, as I had to manually edit my code each run for every file type based on my manual find query. Also, the inability of my own code to handle file names with spaces meant I had to manually read through screeds of "file not found" errors each run and manually mget the misnamed files via ftp from the file server, one by one.

A damned lengthy process which took hours. I still haven't managed to mark the work I collected. Fortunately, the classes were cancelled due to outside courses etc or I would have certainly missed the deadlines altogether.
Lancer
LXF regular
 
Posts: 232
Joined: Thu Apr 07, 2005 11:59 am

Postby skecs » Tue Jun 28, 2005 10:17 am

Lancer wrote:the /\/\/\/\/\ syntax throws me a bit
- is quite understandable :lol:

The backslashes "" are escape characters in regular expressions. They mean "use the ASCII character that follows". The special characters "/" (forward slashes) and "." (periods) are used to separate or group strings of characters but they also signify the directory structure "/home/username/filename.ext" or special matching characters like \w ("a-zA-Z0-9_" = alphanumeric characters & _) and \n (newline).

There are a number of online resources to help with Perl (when you get time :wink: ) like Learn Perl and The CGI Site. I know LXF is doing some shell scripting but Perl is also very powerful and can also interact with Web Servers very well.
Regards from
Downunder!
.... _
... (0)>
... / /\
.. / / .)
.. V_/_
Linux Powered!
User avatar
skecs
 
Posts: 76
Joined: Fri Apr 22, 2005 9:22 am
Location: Bathurst, NSW Australia


Return to Help!

Who is online

Users browsing this forum: No registered users and 3 guests