Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extracting data from rsync log
Top Forums Shell Programming and Scripting Extracting data from rsync log Post 302945817 by treds on Wednesday 3rd of June 2015 10:50:59 AM
Old 06-03-2015
Extracting data from rsync log
Hi

I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied.

I would like to use this information to see if i can increase the amount of rsyncs that run in a day.

so the log will reside in the normal place /var/log/rsync.log

I like to extract the information and cat to another file. The idea is i could gather the data daily over the course of a week.

Thanks in advance
Treds
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

2. UNIX for Dummies Questions & Answers

Extracting Data Using SED

Given the following text in a file named extract.txt: listenPort:=25 smtpDestination:=2 enableSSL:= I am trying to extract only the value 2 following smtpDestination:= Someone had suggested I use: sed -e "s/^smtpDestination:=\(.*\)$/\1/" extract.txt but this returns: listenPort:=25 2 ... (2 Replies)
Discussion started by: cleanden
2 Replies

3. UNIX for Advanced & Expert Users

rsync data corruption

Dear Peoples I am using the following command to transfer the files inbetween two servers, i am using this command in a crontab for doing it in every 1 hour on all days. rsync --stats --archive --verbose --compress --force --rsh=ssh --exclude-from=/root/cfg/mkt_scn.exclude... (1 Reply)
Discussion started by: thameema
1 Replies

4. Shell Programming and Scripting

Extracting data from a log file with date formats

Hello, I have a log file for the year, which contains lines starting with the data in the format of YYYY-MM-DD. I need to get all the lines that contain the DD being 04, how would I do this? I tried using grep "*-*04" but it didn't work. Any quick one liners I should know about? Thank you. (2 Replies)
Discussion started by: cpickering
2 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

extracting data

I have a txt file of the following format >ab_ qwerty >rt_ hfjkil >Ty2 hglashglkasghkf; >P2 aklhfklflkkgfgkfl >ui_ vnllkdskkkffkfkkf >we32 vksksjksj;lslsf'sk's's .... ..... I want to split this big file based on the header (>) (5 Replies)
Discussion started by: Lucky Ali
5 Replies

7. Shell Programming and Scripting

Extracting the last 10mins worth of data in a log file

Hi all, Hope someone here will be able to help me. Our system has some scripts that are run from a cron job every ten mins and is used to see how many error there are in that time frame. Problem is that in the scripts grep is used to find the data, but as the day goes on these log file grow to a... (7 Replies)
Discussion started by: Goldengreen
7 Replies

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file... (2 Replies)
Discussion started by: Buddyluv
2 Replies

9. Shell Programming and Scripting

Tail over rsync data

Hi, I have written a shell script (that will be called in every 5 seconds) that used rsync to copy logs of our production server from server A(production) to server B (Dev), because we don't allow team members to hit server A. The shell script is working fine. But we are facing a problem that... (2 Replies)
Discussion started by: rish_max
2 Replies

10. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies

LEARN ABOUT DEBIAN

git-annex-shell

git-annex-shell(1)					      General Commands Manual						git-annex-shell(1)

NAME

       git-annex-shell - Restricted login shell for git-annex only SSH access

SYNOPSIS

       git-annex-shell [-c] command [params ...]

DESCRIPTION

       git-annex-shell is a restricted shell, similar to git-shell, which can be used as a login shell for SSH accounts.

       Since  its  syntax  is  identical to git-shell's, it can be used as a drop-in replacement anywhere git-shell is used. For example it can be
       used as a user's restricted login shell.

COMMANDS

       Any command not listed below is passed through to git-shell.

       Note that the directory parameter should be an absolute path, otherwise it is assumed to be relative to the user's home directory. Also the
       first "/~/" or "/~user/" is expanded to the specified home directory.

       configlist directory
	      This outputs a subset of the git configuration, in the same form as git config --list

       inannex directory [key ...]
	      This checks if all specified keys are present in the annex, and exits zero if so.

       dropkey directory [key ...]
	      This drops the annexed data for the specified keys.

       recvkey directory key
	      This runs rsync in server mode to receive the content of a key, and stores the content in the annex.

       sendkey directory key
	      This runs rsync in server mode to transfer out the content of a key.

       commit directory
	      This commits any staged changes to the git-annex branch.	It also runs the annex-content hook.

OPTIONS

       Most options are the same as in git-annex. The ones specific to git-annex-shell are:

       --uuid=UUID
	      git-annex uses this to specify the UUID of the repository it was expecting git-annex-shell to access, as a sanity check.

HOOK

       After  content is received or dropped from the repository by git-annex-shell, it runs a hook, .git/hooks/annex-content (or hooks/annex-con-
       tent on a bare repository). The hook is not currently passed any information about what changed.

ENVIRONMENT

       GIT_ANNEX_SHELL_READONLY

	      If set, disallows any command that could modify the repository.

       GIT_ANNEX_SHELL_LIMITED
	      If set, disallows running git-shell to handle unknown commands.

SEE ALSO

       git-annex(1)

       git-shell(1)

AUTHOR

       Joey Hess <joey@kitenet.net>

       <http://git-annex.branchable.com/>

																git-annex-shell(1)
All times are GMT -4. The time now is 10:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy