Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Extracting part of data from files
Top Forums Shell Programming and Scripting Extracting part of data from files Post 302978015 by RudiC on Monday 25th of July 2016 09:10:41 AM
Old 07-25-2016
Adapting this post:
Code:
awk '
BEGIN                   {SRCH1 = "numberoffiles[^,]*"
                         SRCH2 = "numberofrows[^,]*"
                        }
/^Table/                {TBL = $2
                         sub (/\./, ",", TBL)
                         match ($0, SRCH1)
                         NoF = substr ($0, RSTART+14, RLENGTH-14)
                         match ($0, SRCH2)
                         NoR = substr ($0, RSTART+13, RLENGTH-13)
                         FND = 1
                        }
/Time/ && FND           {print TBL, NoF, NoR, $3, "PASS", FILENAME
                         FND = 0
                        }
/^FAILED/               {print "NULL,NULL,NULL,NULL,NULL", $0, FILENAME
                        }
' OFS="," log*.txt
astcms,dfile1,2,2000,10.23,PASS,log1.txt
astcms,dfile2,5,100,15.12,PASS,log1.txt
NULL,NULL,NULL,NULL,NULL,FAILED:Exception [Error 56700],log1.txt
astcms,dfile4,3,60,12.123,PASS,log2.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies

2. Shell Programming and Scripting

extracting uncommon part between two files

Hi, I need to extract the uncommon (better say incremental) part from 2 files say file_1 and file_2. file_2 contains everything that is in file_1. That is file_2 has been created internally somehow : cat file_1 temp_file > file_2 My objective is to extract the temp_file part from... (2 Replies)
Discussion started by: sabyasm
2 Replies

3. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

4. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

5. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

6. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >... (12 Replies)
Discussion started by: p_sun
12 Replies

7. UNIX for Dummies Questions & Answers

Extracting data from PDF files into CSV file

Hi, I have several hundreds of PDFfiles number 01.pdf, 02.pdf, 03.pdf, etc in one folder. These are vey long documentd with a lot of information (text, tables, figures, etc). I need to extract the information asociated with one disease in particular (Varicella). The information I need to... (5 Replies)
Discussion started by: Xterra
5 Replies

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file... (2 Replies)
Discussion started by: Buddyluv
2 Replies

9. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of... (4 Replies)
Discussion started by: nans
4 Replies

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

LEARN ABOUT DEBIAN

mysqmail-dovecot-logger

mysqmail-dovecot-logger(8)				      System Manager's Manual					mysqmail-dovecot-logger(8)

NAME

       mysqmail-dovecot-logger - logs dovecot-imapd and dovecot-pop3d traffic to a mysql database

SYNOPSIS

       mysqmail-dovecot-logger subprogram [ args ...  ]

DESCRIPTION

       mysqmail-dovecot-logger reads the syslog using tail -F, and for each dovecot lines of POP3 and IMAP corresponding to a disconnection, mysq-
       mail-dovecot-logger writes the total used bandwidth corresponding to the username. Configuration is read from /etc/mysqmail.conf.

       mysqmail-dovecot-logger uses a table corresponding to this one:

       CREATE TABLE IF NOT EXISTS pop_access (
	 id varchar(32) NOT NULL default '',
	 uid int(11) NOT NULL default '65534',
	 gid int(11) NOT NULL default '65534',
	 home varchar(255) NOT NULL default '',
	 shell varchar(255) NOT NULL default '',
	 mbox_host varchar(120) NOT NULL default '',
	 crypt varchar(50) NOT NULL default '',
	 passwd varchar(50) NOT NULL default '',
	 active int(11) NOT NULL default '1',
	 start_date date NOT NULL default '0000-00-00',
	 expire_date date NOT NULL default '0000-00-00',
	 quota_size int(11) NOT NULL default '0',
	 type varchar(20) NOT NULL default 'default',
	 memo text,du bigint(20) NOT NULL default '0',
	 another_perso varchar(5) NOT NULL default 'no',
	 redirect1 varchar(255) default NULL,
	 redirect2 varchar(255) default NULL,
	 localdeliver varchar(10) NOT NULL default 'yes',
	 pop3_login_count int(9) NOT NULL default '0',
	 pop3_transfered_bytes int(14) NOT NULL default '0',
	 imap_login_count int(9) NOT NULL default '0',
	 imap_transfered_bytes int(14) NOT NULL default '0',
	 last_login int(14) NOT NULL default '0',
	 PRIMARY KEY  (id,mbox_host) ) TYPE=MyISAM

VERSION

       This documentation describes  mysqmail-dovecot-logger  version  0.4.4  or  superior.   See  http://gplhost.com/softwares-mysqmail.html  for
       updates.

SEE ALSO

       qmail-send(3), syslog(3), logger(8)

															mysqmail-dovecot-logger(8)
All times are GMT -4. The time now is 01:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy