Extracting part of data from files Post: 302978015

Sponsored Content

Top Forums Shell Programming and Scripting Extracting part of data from files Post 302978015 by RudiC on Monday 25th of July 2016 09:10:41 AM

07-25-2016

Registered User

Adapting this post:

Code:

awk '
BEGIN                   {SRCH1 = "numberoffiles[^,]*"
                         SRCH2 = "numberofrows[^,]*"
                        }
/^Table/                {TBL = $2
                         sub (/\./, ",", TBL)
                         match ($0, SRCH1)
                         NoF = substr ($0, RSTART+14, RLENGTH-14)
                         match ($0, SRCH2)
                         NoR = substr ($0, RSTART+13, RLENGTH-13)
                         FND = 1
                        }
/Time/ && FND           {print TBL, NoF, NoR, $3, "PASS", FILENAME
                         FND = 0
                        }
/^FAILED/               {print "NULL,NULL,NULL,NULL,NULL", $0, FILENAME
                        }
' OFS="," log*.txt
astcms,dfile1,2,2000,10.23,PASS,log1.txt
astcms,dfile2,5,100,15.12,PASS,log1.txt
NULL,NULL,NULL,NULL,NULL,FAILED:Exception [Error 56700],log1.txt
astcms,dfile4,3,60,12.123,PASS,log2.txt

RudiC

View Public Profile for RudiC

Find all posts by RudiC

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,...

2. Shell Programming and Scripting

extracting uncommon part between two files

Hi, I need to extract the uncommon (better say incremental) part from 2 files say file_1 and file_2. file_2 contains everything that is in file_1. That is file_2 has been created internally somehow : cat file_1 temp_file > file_2 My objective is to extract the temp_file part from...

3. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec...

4. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,...

5. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt

6. Shell Programming and Scripting

awk - extracting data from a series of files

Hi, I am trying to extract data from multiple output files. I am able to extract the data from a single output file by using the following awk commands: awk '/ test-file*/{print;m=0}' out1.log > out1a.txt awk '/ test-string/{m=1;c=0}m&&++c==3{print $2 " " $3 " " $4 ;m=0}' out1.log >...

7. UNIX for Dummies Questions & Answers

Extracting data from PDF files into CSV file

Hi, I have several hundreds of PDFfiles number 01.pdf, 02.pdf, 03.pdf, etc in one folder. These are vey long documentd with a lot of information (text, tables, figures, etc). I need to extract the information asociated with one disease in particular (Varicella). The information I need to...

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file...

9. Programming

Python script for extracting data using two files

Hello, I have two files. File 1 is a list of interested IDs Ex1 Ex2 Ex3File 2 is the original file with over 8000 columns and 20 millions rows and is a compressed file .gz Ex1 xx xx xx xx .... Ex2 xx xx xx xx .... Ex2 xx xx xx xx ....Now I need to extract the information for all the IDs of...

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late...

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - extracting data from .csv files

Discussion started by: kregh99

2. Shell Programming and Scripting

extracting uncommon part between two files

Discussion started by: sabyasm

3. Shell Programming and Scripting

extracting data from files..

Discussion started by: clx

4. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

Discussion started by: Boltzmann

5. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Discussion started by: Fahmida

6. Shell Programming and Scripting

awk - extracting data from a series of files

Discussion started by: p_sun

7. UNIX for Dummies Questions & Answers

Extracting data from PDF files into CSV file

Discussion started by: Xterra

8. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Discussion started by: Buddyluv

9. Programming

Python script for extracting data using two files

Discussion started by: nans

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

Discussion started by: pawannoel