Monday, November 15, 2010

Ignore Missing Files with tar

I've been experiencing an annoying problem with tar in which it is not creating archives if at least one file in the tar list does not exist. This behavior is totally acceptable and appreciated when I am tarring on demand. However, in this case, I am running a command across several machines each of which will have a different sample of a set of files and I need this done as a cron. And to make matters more complicated the file set involves specific files in a set of directories.
# The novice command
bash$ tar cfz yesterday-logs.tgz \
log-type1/log-type1-2011-11-15.log \
log-type2/log-type2-2011-11-15.log \
log-type3/log-type3-2011-11-15.log \
log-type4/log-type4-2011-11-15.log
This command will work fine if all the directories exist. However, if at least one of the directories or files does not exist, I will experience the following error and an empty tar archive.

tar: log-type3/log-type3-2011-11-15.log: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.

I solved this issue by using the wonders bash expansion, sub-shell, and std error redirection
bash$ tar cfz yesterday-logs.tgz $( ls log-type{1,2,3,4}/*-2011-11-15.log} 2>/dev/null )
Fun. I suppose there is most likely a flag to tar that I couldn't find in my scan of the manual entry that completely obviate this complexity, but in the meantime...

2 comments:

  1. So it turns out that tar will ignore missing files by default, so simply piping error output to dev null will do the trick of streaming the tar. Just like I figured: I was overcomplicating things!

    ReplyDelete
  2. Original idea with bash expansion is still good if using the above in a script because even though tar works with the missing file it will return a non-zero status which could freak out the calling script.

    ReplyDelete