Zip files, then delete them.

The place to post if you need help or advice

Moderators: ChrisThornett, LXF moderators

Zip files, then delete them.

Postby thusgaard » Wed May 01, 2013 8:14 pm

Hi

I'm running a script that generates a lot of files. But 4000 files in the same directory is a bit much...

So I have added a few lines to my script and currently I have 4173 files in my data directory.

I want to zip/gzip/tar the files in the directory when there are X files or more in the directory. And the delete them. I have the part with when to run the zip+delete job, I even have a simple zip running. But how do I delete the files in the folder that have been added to the zip file.


Bonus info:
the files are .jpg average filesize is less than 200kB

Bonus question:
What is the difference between Zip/Gzip/tar?


J;-)
thusgaard
LXF regular
 
Posts: 102
Joined: Wed Jun 07, 2006 10:21 am
Location: Skanderborg, Denmark

Postby nelz » Wed May 01, 2013 9:32 pm

If the files have no spaces in their names

Code: Select all
FILES=$(ls -1 *.jpg)
tar cf somearchive.tar $FILES
rm -f $FILES


Zip is a DOS/Windows archive format, universal but not particularly good compression. Tar creates uncompressed archives. gzip, bzip2 and xz compress single files or streams and are using in conjunction with tar to produce compressed archives.

However, JPEG files are also compressed and are likely to expand when further compression is attempted, so plain tar is the sensible choice.
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
User avatar
nelz
Site admin
 
Posts: 8545
Joined: Mon Apr 04, 2005 11:52 am
Location: Warrington, UK

Postby nelz » Wed May 01, 2013 10:41 pm

It might be a good idea to make sure the archive is valid before throwing away the original files

Code: Select all
FILES=$(ls -1 *.jpg)
tar cf somearchive.tar $FILES
tar tf somearchive.tar && rm -f $FILES
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
User avatar
nelz
Site admin
 
Posts: 8545
Joined: Mon Apr 04, 2005 11:52 am
Location: Warrington, UK

Postby Dutch_Master » Wed May 01, 2013 11:48 pm

Are you sure the argument to ls is -1 Nelz? Shouldn't that be -l (letter instead of digit)? (haven't checked the man page, so you could well be right though ;))
Dutch_Master
LXF regular
 
Posts: 2458
Joined: Tue Mar 27, 2007 1:49 am

Postby nelz » Thu May 02, 2013 12:12 am

Yes, -1. -l would break it horribly.
"Insanity: doing the same thing over and over again and expecting different results." (Albert Einstein)
User avatar
nelz
Site admin
 
Posts: 8545
Joined: Mon Apr 04, 2005 11:52 am
Location: Warrington, UK

Postby thusgaard » Thu May 02, 2013 2:11 pm

nelz wrote:It might be a good idea to make sure the archive is valid before throwing away the original files

Code: Select all
FILES=$(ls -1 *.jpg)
tar cf somearchive.tar $FILES
tar tf somearchive.tar && rm -f $FILES


Works like a charm.
Beauitiful.

Thanks.
thusgaard
LXF regular
 
Posts: 102
Joined: Wed Jun 07, 2006 10:21 am
Location: Skanderborg, Denmark


Return to Help!

Who is online

Users browsing this forum: No registered users and 3 guests