Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: parsing data for certain conditions
Top Forums Shell Programming and Scripting parsing data for certain conditions Post 302323039 by cfajohnson on Friday 5th of June 2009 07:47:11 AM
Old 06-05-2009
Quote:
Originally Posted by PAW
I have got this working OK but I am sure there is a more efficient/elegant way of doing it, which I hope you can help me with.

It can be done in whatever is most suitable i.e perl/awk..

You are calling awk 6,400,000+ times, and sed 800,000+ times.

With 800000+ rows, you need awk, but you only need one call to awk, not eight (including one that does nothing) and one to sed for every line of the file.

Here's a start to an awk script:

Code:
awk -F'"' '
    {
     test1 = $1
     fields = split($0,a,"|")
     test2 = a[fields - 1]
     test3 = $3

     if ( length(test2) > 0 && length(test3) == 0 ) ...
}
' head_out_orig_phil

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies

2. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

3. Shell Programming and Scripting

Organization data based on two conditions applied problem asking...

Input file: HS04636 type header 836 7001 ID=g1 HS04636 type status 836 1017 Parent=g1.t1 HS04636 type location 966 1017 ID=g1.t1.cds;Parent=g1.t1 HS04636 type location 1818 1934 ID=g1.t1.cds;Parent=g1.t1 HS04636 type status 1818... (8 Replies)
Discussion started by: patrick87
8 Replies

4. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

5. Shell Programming and Scripting

Data parsing

Hi, I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed The file has the format: CATGC Best GO enrichment: Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)): ... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

6. Shell Programming and Scripting

take a section of a data with conditions

I have a data file like below: 2011 0701 2015 21.2 L 37.692 46.202 18.0 Teh 4 0.3 2.1 LTeh 1 GAP=233 E Iranian Seismological Center, Institute of Geophysics, University of Tehran 6 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO SNR AR TRES W DIS CAZ7 TBZ SN EPg 0 2015 31.19 -0.3... (3 Replies)
Discussion started by: saeed.soltani
3 Replies

7. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

8. Shell Programming and Scripting

[Script] Conditions on parsing file

Hello, I open a new POST, i consider that this is resolved https://www.unix.com/shell-programming-scripting/215803-create-file-comment-script.html But i wish improve it. In case 1, I would like to test the input file $1. If $1 exist with no parameters but only comments, then send a message... (2 Replies)
Discussion started by: amazigh42
2 Replies

9. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

10. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file based on certain conditions

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
Discussion started by: Vivekit82
1 Replies

LEARN ABOUT CENTOS

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 06:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy