Sponsored Content
Top Forums Shell Programming and Scripting Extracting data from rsync log Post 302945959 by treds on Friday 5th of June 2015 05:46:53 AM
Old 06-05-2015
hi Jim

start of a log

Code:
2015/06/05 21:14:06 [32390] building file list
2015/06/05 21:14:06 [32395] building file list
2015/06/05 21:14:06 [32400] building file list
2015/06/05 21:14:06 [32405] building file list
2015/06/05 21:14:06 [32410] building file list
2015/06/05 21:14:18 [32390] .d..t...... STUDIOS/HM_MASTERS/2012_MASTERS/
2015/02/07 21:14:18 [32390] .d..t...... STUDIOS/HM_MASTERS/2012_MASTERS/1202_SEASON_START/Files from HQ/ME/
2015/02/07 21:14:18 [32390] .d..t...... STUDIOS/HM_MASTERS/2012_MASTERS/1202_SEASON_START/Files from HQ/ME/KSA/

End of log
Code:
LR_CRP_ASSET/
2015/06/05 10:18:27 [27914] .d..t...... TIER2/TAG_REPOSITORY_JOB_ASSETS/97/04/47/97044710/JOB_ASSETS/RGB_ASSET/
2015/06/05 10:18:27 [27914] .d..t...... TIER2/TAG_REPOSITORY_JOB_ASSETS/97/04/47/97044710/JOB_ASSETS/RGB_CRP_ASSET/
2015/06/05 10:18:27 [27914] Number of files: 9172
2015/06/05 10:18:27 [27914] Number of files transferred: 0
2015/06/05 10:18:27 [27914] Total file size: 0 bytes
2015/06/05 10:18:27 [27914] Total transferred file size: 0 bytes
2015/06/05 10:18:27 [27914] Literal data: 0 bytes
2015/06/05 10:18:27 [27914] Matched data: 0 bytes
2015/06/05 10:18:27 [27914] File list size: 242568
2015/06/05 10:18:27 [27914] File list generation time: 4.261 seconds
2015/06/05 10:18:27 [27914] File list transfer time: 0.000 seconds
2015/06/05 10:18:27 [27914] Total bytes sent: 247360
2015/06/05 10:18:27 [27914] Total bytes received: 4797
2015/06/05 10:18:27 [27914] sent 247360 bytes  received 4797 bytes  16268.19 bytes/sec
2015/06/05 10:18:27 [27914] total size is 0  speedup is 0.00

I have tried using sed but to be honest my scripting skills are limited.
thanks
treds

Last edited by vbe; 06-05-2015 at 08:41 AM.. Reason: Code tags please
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

2. UNIX for Dummies Questions & Answers

Extracting Data Using SED

Given the following text in a file named extract.txt: listenPort:=25 smtpDestination:=2 enableSSL:= I am trying to extract only the value 2 following smtpDestination:= Someone had suggested I use: sed -e "s/^smtpDestination:=\(.*\)$/\1/" extract.txt but this returns: listenPort:=25 2 ... (2 Replies)
Discussion started by: cleanden
2 Replies

3. UNIX for Advanced & Expert Users

rsync data corruption

Dear Peoples I am using the following command to transfer the files inbetween two servers, i am using this command in a crontab for doing it in every 1 hour on all days. rsync --stats --archive --verbose --compress --force --rsh=ssh --exclude-from=/root/cfg/mkt_scn.exclude... (1 Reply)
Discussion started by: thameema
1 Replies

4. Shell Programming and Scripting

Extracting data from a log file with date formats

Hello, I have a log file for the year, which contains lines starting with the data in the format of YYYY-MM-DD. I need to get all the lines that contain the DD being 04, how would I do this? I tried using grep "*-*04" but it didn't work. Any quick one liners I should know about? Thank you. (2 Replies)
Discussion started by: cpickering
2 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

extracting data

I have a txt file of the following format >ab_ qwerty >rt_ hfjkil >Ty2 hglashglkasghkf; >P2 aklhfklflkkgfgkfl >ui_ vnllkdskkkffkfkkf >we32 vksksjksj;lslsf'sk's's .... ..... I want to split this big file based on the header (>) (5 Replies)
Discussion started by: Lucky Ali
5 Replies

7. Shell Programming and Scripting

Extracting the last 10mins worth of data in a log file

Hi all, Hope someone here will be able to help me. Our system has some scripts that are run from a cron job every ten mins and is used to see how many error there are in that time frame. Problem is that in the scripts grep is used to find the data, but as the day goes on these log file grow to a... (7 Replies)
Discussion started by: Goldengreen
7 Replies

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file... (2 Replies)
Discussion started by: Buddyluv
2 Replies

9. Shell Programming and Scripting

Tail over rsync data

Hi, I have written a shell script (that will be called in every 5 seconds) that used rsync to copy logs of our production server from server A(production) to server B (Dev), because we don't allow team members to hit server A. The shell script is working fine. But we are facing a problem that... (2 Replies)
Discussion started by: rish_max
2 Replies

10. Shell Programming and Scripting

Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error: awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log Error: date: 0551-402 Invalid character in date/time... (8 Replies)
Discussion started by: oberoi1403
8 Replies
STRSPN(3)						     Linux Programmer's Manual							 STRSPN(3)

NAME
strspn, strcspn - get length of a prefix substring SYNOPSIS
#include <string.h> size_t strspn(const char *s, const char *accept); size_t strcspn(const char *s, const char *reject); DESCRIPTION
The strspn() function calculates the length (in bytes) of the initial segment of s which consists entirely of bytes in accept. The strcspn() function calculates the length of the initial segment of s which consists entirely of bytes not in reject. RETURN VALUE
The strspn() function returns the number of bytes in the initial segment of s which consist only of bytes from accept. The strcspn() function returns the number of bytes in the initial segment of s which are not in the string reject. ATTRIBUTES
For an explanation of the terms used in this section, see attributes(7). +--------------------+---------------+---------+ |Interface | Attribute | Value | +--------------------+---------------+---------+ |strspn(), strcspn() | Thread safety | MT-Safe | +--------------------+---------------+---------+ CONFORMING TO
POSIX.1-2001, POSIX.1-2008, C89, C99, SVr4, 4.3BSD. SEE ALSO
index(3), memchr(3), rindex(3), strchr(3), string(3), strpbrk(3), strsep(3), strstr(3), strtok(3), wcscspn(3), wcsspn(3) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. 2015-08-08 STRSPN(3)
All times are GMT -4. The time now is 04:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy