Sponsored Content
Top Forums Shell Programming and Scripting Problem running zip from cron - bad zipfile offset Post 302937041 by Agreppa on Monday 2nd of March 2015 07:02:56 PM
Old 03-02-2015
Problem running zip from cron - bad zipfile offset

I've created a pretty straightforward shell script to do backups via rsync. Every night it rsyncs a copy of a website (approx 60GB) to the local machine. Once a week, after the rsync step, it zips the local copy of the whole site into 2GB chunks and places them in another folder.

When running 'unzip filename.zip' I get a long string of errors similar to this:

Code:
file #17934:  bad zipfile offset (lseek):  1726201856
file #17935:  bad zipfile offset (lseek):  1726775296
file #17936:  bad zipfile offset (local header sig):  1702568762
file #17937:  bad zipfile offset (EOF):  1545311941

Approx 60% of the archived series of 2GB chunks unzips without any problem, but the rest shows those errors above. Those errors "seem" to be happening on files that were already .zip files on the webserver, and which were originally created with a mix of windows WinRAR and command line windows 'zip'. (I don't see how the original method of creation would impact them being zipped into a larger archive, but I'm mentioning it to be thorough).

ALSO: If I ftp the entire linux-created series of 2GB chunks to a Windows machine and open them in WinRAR, they all extract without any problem.

Here's the crontab I'm using:
Code:
2 * * 2-7 /home/user/Scripts/backup.rsync.sh > /mnt/data/Backups/logs/$(date +\%Y\%m\%d_\%H\%M\%S)_website.rsync.cron.log 2>&1
0 2 * * 1 /home/user/Scripts/backup.full.sh > /mnt/data/Backups/logs/$(date +\%Y\%m\%d_\%H\%M\%S)_website.full.cron.log 2>&1

And here are the 2 scripts referenced in that same cron with a lot of additional commands removed for clarity:

backup.rsync.sh
Code:
# Rsync website.com to local machine
/usr/bin/rsync -Pavzhe "ssh -p 22" --log-file=$dirlog/$date\_website.rsync.log --bwlimit=2500 --skip-compress=jpg/zip --delete user@website.com:'/home/user/' "/mnt/data/Backups/website.com"

backup.full.sh
Code:
# Rsync website.com to local machine
/usr/bin/rsync -Pavzhe "ssh -p 22" --log-file=$dirlog/$date\_website.rsync.log --bwlimit=2500 --skip-compress=jpg/zip --delete user@website.com:'/home/user/' "/mnt/data/Backups/website.com"
# Zip site to 2G chunks
/usr/bin/zip -r -s 2g -y -2 $dirzip/website_$date.zip $dircom/

Something in that final zip command seems to be the cause but I can't figure it out. The fact that I can take those resulting files into windows and extract them with winRAR is even more confusing, since linux 'unzip' throws thousands of errors... Can anyone suggest what's wrong with my batch script?

PS: The 'zip' version is 3.0-6 and 'unzip' is 6.0-8+deb7u2. The local machine is Crunchbang 11.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find files within a zipfile on AIX5.3

In a directory I have several zipfiles, every zip files has some csv files within. I need to find out which zipfiles have the particular csv files. I know we can use a zipgrep on linux. Does AIX have a similar command. Any help appreciated. Thanks & Regards, Ram (1 Reply)
Discussion started by: ramky79
1 Replies

2. Shell Programming and Scripting

Behavior of Bad Script in Cron Job

Hi A Ksh script is deployed in a server and executed through cronjob. If one of the line in the middle of the script fails . Are the remaining lines executed ? (3 Replies)
Discussion started by: Sivaswami
3 Replies

3. UNIX for Dummies Questions & Answers

Problem running a cron job

I have created a cron job for the vtiger workflow to execute the shell file named com_vtiger_workflow.sh to run the workflow. I've created the following line in crotab -e : 00 13 * * * /var/www/html/prashant/cron/modules/com_vtiger_workflow/com_vtiger_tiger_workflow.sh | mail -s 'Check... (2 Replies)
Discussion started by: anaigini45
2 Replies

4. Shell Programming and Scripting

Cron job running problem

Hi Guys, I am trying to run a script through contab. The script can only be executed once user logs in as su - oracle. I have tested the script other then cronjob and it executes successfully, more over the paths used in the script are absolute paths. Crontab entries are as as below.... (3 Replies)
Discussion started by: Asteroid
3 Replies

5. AIX

Partition offset problem after re-mirror vg

We had a mirrored disk failed (not the rootvg), there are 3 lvs (transfer, applogs, arch) from extvg gone open/stale state. After replaced failed disk and run cfgmgr, the new replaced disk is visible: ) I did the following to re-mirror new disk: # extendvg prodvg hdisk3 # lspv (got new pvid on... (2 Replies)
Discussion started by: jalite19
2 Replies

6. Shell Programming and Scripting

bad interpreter when running script

Hi All, I'm not confortable in writing script, can someone can help me, when I run that script below i found this error code : -bash: ./script.sh: /bin/sh.: bad interpreter: Here is the script for i in * x=${i##*.} z=$(perl -e 'print time;') t=$(echo $z-$x|bc)... (12 Replies)
Discussion started by: bzb23
12 Replies

7. Solaris

! bad user (adm).. in cron log

I have noticed this error in /var/cron/log: > CMD: /usr/lib/acct/ckpacct > adm 6739 c Tue Oct 11 10:00:00 2011 < adm 6739 c Tue Oct 11 10:00:00 2011 rc=1 ! bad user (adm) Tue Oct 11 11:00:00 2011but when I try to list crontab of user 'adm': solarni/~# crontab -l adm crontab: you are... (4 Replies)
Discussion started by: orange47
4 Replies

8. UNIX for Advanced & Expert Users

Running multiple php scripts into one php only, cron mail alert problem...

hi, while separated they produce the usual mail alert and i can see the output... if i write into the php script: <?php system('php -f /var/www/vhosts/domain.com/httpdocs/folder/script1.php'); system('php -f /var/www/vhosts/domain.com/httpdocs/folder/script2.php'); system('php -f... (0 Replies)
Discussion started by: 7stars
0 Replies

9. Solaris

Cron job running even after cron is removed

Hi , I have removed a cron for particular user , but cron job seems to be running even after the cron entry is removed. The purpose of the cron was to sendmail to user ( it uses mailx utility ) I have restarted cron and sendmail service still user is getting mail alerts from the cron job. And... (4 Replies)
Discussion started by: chidori
4 Replies

10. UNIX for Advanced & Expert Users

Grep --byte-offset not returning the offset (Grep version 2.5.1)

Hi, I am trying to get the position of a repeated string in a line using grep -b -o "pattern" In my server I am using GNU grep version 2.14 and the code is working fine. However when I am deploying the same code in a different server which is using GNU grep version 2.5.1 the code is not... (3 Replies)
Discussion started by: Subhamoy
3 Replies
SVN::Notify::Mirror::Rsync(3pm) 			User Contributed Perl Documentation			   SVN::Notify::Mirror::Rsync(3pm)

NAME
SVN::Notify::Mirror::Rsync - Mirror a repository path via Rsync SYNOPSIS
Use svnnotify in post-commit: svnnotify --repos-path "$1" --revision "$2" --handler Mirror::Rsync --to "/path/to/local/htdocs" [--svn-binary /full/path/to/svn] --rsync-host remote_server [--rsync-delete=[yes|no]] [--rsync-dest "/path/on/remote/server"] [--rsync-args arg1 [--rsync-args arg2...]] [[--rsync-ssh] [--ssh-user remote_user] [--ssh-identity /home/user/.ssh/id_rsa]] or better yet, use SVN::Notify::Config for a more sophisticated setup: #!/usr/bin/perl -MSVN::Notify::Config=$0 --- #YAML:1.0 '': PATH: "/usr/bin:/usr/local/bin" 'path/in/repository': handler: Mirror to: "/path/to/www/htdocs" 'some/other/path/in/repository': handler: Mirror to: "/path/to/local/www/htdocs" rsync-host: "remote_host" rsync-dest: "/path/on/remote/www/htdocs" ssh-user: "remote_user" ssh-identity: "/home/user/.ssh/id_rsa" DESCRIPTION
Keep a directory in sync with a portion of a Subversion repository. Typically used to keep a development web server in sync with the changes made to the repository. This directory can either be on the same box as the repository itself, or it can be remote (via SSH connection). USAGE
Depending on whether the target is a "Local Mirror" or a Remote Mirror, there are different options available. All options are available either as a commandline option to svnnotify or as a hash key in SVN::Notify::Config (see their respective documentation for more details). Working Copy on Local host Because 'svn export' is not able to be consistently updated, the local rsync'd directory must be a full working copy. The remote server will only contain the ordinary files (no Subversion admin files). The files in the working copy must be writeable (preferrably owned) by the user identity executing the hook script (this is the user identity that is running Apache or svnserve respectively). Local Mirror Please see " SVN::Notify::Mirror " for details. Remote Mirror Used for directories not located on the same machine as the repository itself. Typically, this might be a production web server located in a DMZ, so special consideration must be paid to security concerns. In particular, the remote mirror server may not be able to directly access the repository box. o rsync-host This value is required and must be the hostname or IP address of the remote host (where the mirror directories reside). o rsync-delete The default mode of operation is to delete remote files which are not present in the local working copy. NOTE: this will delete any unversioned files in the remote directory tree. Unless you have all of your files under version control, you should pass the "--no-rsync-delete" or "--rsync-delete no" option. o rsync-dest This optional value specifies the path to update on the remote host. If you do not specify this value, the same path as passed in as the "--to" parameter will be used (this may not be what you meant to do). o rsync-args This optional parameter can be used to pass additional commandline options to the rsync command. You can use this multiple times in order to pass multiple options. The default args are "--archive --compress". See the "rsync-ssh" options for using SSH instead of RSH (rather than pass those commands via "--rsync-args" o rsync-ssh This optional parameter signals that you wish to use SSH instead of whatever the default remote shell program is configured in your copy of rsync. You may need to set one or more of the "ssh-*" parameters as well. o ssh-user If the remote user is different than the local user executing the postcommit script, you can specify it with this parameter. You would often use this in conjunction with the next parameter. o ssh-identity This value may be optional and should be the full path to the local identity file being used to authenticate with the remote host. If you are setting the ssh-user to be something other than the local user name, you will typically also have to set the ssh-identity. AUTHOR
John Peacock <jpeacock@cpan.org> COPYRIGHT
Copyright (c) 2005-2008 John Peacock This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
SVN::Notify, SVN::Notify::Config, SVN::Notify::Mirror perl v5.14.2 2012-07-04 SVN::Notify::Mirror::Rsync(3pm)
All times are GMT -4. The time now is 03:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy