Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: parsing data file picking out certain fields
Top Forums Shell Programming and Scripting parsing data file picking out certain fields Post 302172504 by summer_cherry on Tuesday 4th of March 2008 01:28:38 AM
Old 03-04-2008
Hi,

A complex one, but should be ok for you.

Code:
grep -n lead e | cut -d":" -f1 > temp
for i in `cat temp`
do
t=`expr $i + 4`
sed -n "$i,${t}p" e >> 000${i}
done
rm temp
for i in `ls -l 000* | awk '{print $9}'`
do
	sort $i | awk '{
				if(NF<2)
				{
					NF=2
				}
				for(j=1;j<=NF;j++)
				{
					if($j=="")
						$j=0
					printf("%s ",$j)
				}
				printf("\n")
			}'> ${i}.temp
	rm $i
	mv ${i}.temp $i
done
ls -l 000* | awk '{print $9}' | sort -n | paste - - > tt
n=0
while read line
do
	n=`expr $n + 1`
	join $line > t.$n
done < tt
join t.* > final
rm 000*
rm t*
awk '{
if ($1=="tim" || $1=="tad" || $1=="lead1")
print $1" "$3" "$4" "$7" "$8" "$17" "$18" "$19
}' final
rm final

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing file and extracting the useful data block

Greetings All!! I have a very peculiar problem where I have to parse a big text file and extract useful data out of it with starting and ending block pattern matching. e.g. I have a input file like this: sample data block1 sample data start useful data end sample data block2 sample... (5 Replies)
Discussion started by: arminder
5 Replies

2. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies

3. Shell Programming and Scripting

urgent<parsing data from a excel file>

Hi all, I wud like to get ur assistance in retrieving lines containing l1.My excel dataset contains around 8000 lines.I converted it into a text tab delimiter file and got the lines containing l1,My output is a list of lines containing l1 saved in a outfile.Some of d lines from my outfile s... (5 Replies)
Discussion started by: sayee
5 Replies

4. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi All I have two files: file 1 >AB_1 MLKKPIIIGVTGGSGGGKTSVSRAILDSFPNARIAMIQHDSYYKDQSHMSFEERVKTNYDHPLAFDTDFM IQQLKELLAGRPVDIPIYDYKKHTRSNTTFRQDPQDVIIVEGILVLEDERLRDLMDIKLFVDTDDDIRII RRIKRDMMERGRSLESIIDQYTSVVKPMYHQFIEPSKRYADIVIPEGVSNVVAIDVINSKIASILGEV >AB_2... (5 Replies)
Discussion started by: Lucky Ali
5 Replies

5. Shell Programming and Scripting

parsing data and incorporating it into another file

Hi, I have a folder that contains many (multiple) files 1.fasta 2.fasta 3.fasta 4.fasta 5.fasta . . 100's of files Each such file have data in the following format for example: vi 1.fasta >AB_1 gi|15835212|ref|NP_296971.1| preprotein translocase subunit SecE... (3 Replies)
Discussion started by: Lucky Ali
3 Replies

6. Shell Programming and Scripting

parsing a portion of Data from a text file

Hi All, I need some help to effectively parse out a subset of results from a big results file. Below is an example of the text file. Each block that I need to parse starts with "Output of GENE for sequence file 100.fasta" (next block starts with another number). I have given the portion of... (8 Replies)
Discussion started by: Lucky Ali
8 Replies

7. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

8. Shell Programming and Scripting

Parsing data using keys from one file

I have 2 text files where I need to parse data from file 2 using the data from file 1. Below are my sample files File 1 (tab delimited) 257 350 670 845 725 1025 767 820 ... .... .... file 2 (tab delimited) 220..450 TA AB650 ABCED 520..850 GA AB720 ABCDE 700..1100 TC AB820 ABCDE... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

9. Shell Programming and Scripting

Parsing C Data Tipes from Input File

Im really beginner in this case, maybe someone can help me find the answer: if my input file like this: void main(int a, int b){ int x; double y; printf("file"); } and i want output like this: int a int b int x double y A awk script that can parse only data tipe, im confused. what... (2 Replies)
Discussion started by: radynaraya
2 Replies

10. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

LEARN ABOUT SUSE

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 08:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy