Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to extract lines from a file to different file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to extract lines from a file to different file
# 1  
Old 12-29-2009
How to extract lines from a file to different file
Hi All,
I have a file which is look like this
Code:
<st>
<dt>200912240100Z</dt>
<gp>900</gp>
<cts>resendfail</cts>
<cts>resendsuccessful</cts>
<cts>resendpending</cts>
<info>
<tp>bs=1,slot=1</tp>
<v>1</v>
<v>100</v>
<v>1</0>
</info>
<info>
<tp>bs=1,slot=2</tp>
<v>0</v>
<v>103</v>
<v>9</0>
</info>
<info>
<tp>bs=1,slot=3</tp>
<v>1</v>
<v>100</v>
<v>1</0>
</info>
<info>
<tp>bs=1,slot=4</tp>
<v>0</v>
<v>90</v>
<v>1</0>
</info>
<info>
<tp>bs=1,slot=5</tp>
<v>0</v>
<v>10</v>
<v>0</0>
</info>
<info>
<tp>bs=1,slot=6</tp>
<v>0</v>
<v>150</v>
<v>2</0>
</info>
</st>

The <cts> is the header and the <v> is the value of the header.
In every <info> tag,
The first <cts> will have the value of the first <r>
the second<cts> will have the value of the second<r>.
I need to output each cts and its value into different files.
File looks like:
Code:
<dt>200912240100Z</dt>
<gp>900</gp>
<cts>resendfail</cts>
<tp>bs=1,slot=1</tp>
<v>1</v>
<tp>bs=1,slot=2</tp>
<v>0</v>
<tp>bs=1,slot=3</tp>
<v>1</v>
<tp>bs=1,slot=4</tp>
<v>0</v>
<tp>bs=1,slot=5</tp>
<v>0</v>
<tp>bs=1,slot=6</tp>
<v>0</v>

-->file1

Code:
<dt>200912240100Z</dt>
<gp>900</gp>
<cts>resendsuccessful</cts>
<tp>bs=1,slot=1</tp>
<v>100</v>
<tp>bs=1,slot=2</tp>
<v>103</v>
<tp>bs=1,slot=3</tp>
<v>100</v>
<tp>bs=1,slot=4</tp>
<v>90</v>
<tp>bs=1,slot=5</tp>
<v>10</v>
<tp>bs=1,slot=6</tp>
<v>150</v>

-->file2

Code:
<dt>200912240100Z</dt>
<gp>900</gp>
<cts>resendpending</cts>
<tp>bs=1,slot=1</tp>
<v>1</0>
<tp>bs=1,slot=2</tp>
<v>9</0>
<tp>bs=1,slot=3</tp>
<v>1</0>
<tp>bs=1,slot=4</tp>
<v>1</0>
<tp>bs=1,slot=5</tp>
<v>0</0>
<tp>bs=1,slot=6</tp>
<v>2</0>

-->file3

I have used the sed.

Code:
#!/bin/bash
sed -n -e '2,3p' -n -e '4p' -n  -e '/^<info>/ {n;p;n;p;}' file>file1
sed -n -e '2,3p' -n -e '5p' -n -e '/^<info>/ {n;p;n;n;p;}' file>file2
sed -n -e '2,3p' -n -e '6p' -n  -e '/^<info>/ {n;p;n;n;n;p;}' file>file3

May I know is there any other method to do this because some raw files have more than 200 <cts>?Smilie

Appreciate your help!
Last edited by natalie23; 12-29-2009 at 04:37 AM.. Reason: code tags, please...
# 2  
Old 12-30-2009
Code:
awk '/^<dt>/{dt=$0}
     /^<gp>/{gp=$0}
     /^<cts>/{cts[++i]=$0} 
     /^<tp>/{tp[++j]=$0;k=0}
     /^<v>/{v[++k,j]=$0}
     END { for (o=1;o<=i;o++)
              {printf "%s\n%s\n%s\n",dt,gp,cts[o]  > (fn ="file"o".txt") ;
              for (m=1;m<=j;m++)
                 {printf "%s\n%s\n",tp[m],v[o,m] >fn } }
         }' file

# 3  
Old 12-30-2009
HI rdcwayx,

I have errors when
Code:
 
/^<r>/{v[++k,j]=$0}

and
Code:
{printf "%s\n%s\n",tp[m],v[o,m] >fn }

# 4  
Old 12-30-2009
Quote:
Originally Posted by natalie23
HI rdcwayx,

I have errors when
Code:
 
/^<r>/{v[++k,j]=$0}

and
Code:
{printf "%s\n%s\n",tp[m],v[o,m] >fn }

Use nawk or /usr/xpg4/bin/awk on Solaris.
# 5  
Old 12-30-2009
Hi rdcwayx & Franklin,

Can explain the code to me?

Besides I have encountered
Code:
nawk:file21.txt too many open file

is it I need to close the file

Code:
nawk '/^<dt>/{dt=$0}
     /^<gp>/{gp=$0}
     /^<cts>/{cts[++i]=$0} 
     /^<tp>/{tp[++j]=$0;k=0}
     /^<v>/{v[++k,j]=$0}
     END { for (o=1;o<=i;o++)
              {printf "%s\n%s\n%s\n",dt,gp,cts[o]  > (fn ="file"o".txt") ;
              for (m=1;m<=j;m++)
                 {printf "%s\n%s\n",tp[m],v[o,m] >fn } 
            close fn}
         }' file

Thanks for your help! Really appreaciate!

Happy New Year!
# 6  
Old 12-30-2009
To open 21 files should be no problem on any system, could you please run below command and put the output here?
Code:
ulimit -a

Or attach your real data, I can try on my Solaris box.

---------- Post updated at 10:04 PM ---------- Previous update was at 09:59 PM ----------
Quote:
Originally Posted by natalie23
Hi rdcwayx & Franklin,

Can explain the code to me?
Code:
awk '/^<dt>/{dt=$0}   # collect all data which need be exported and save into var or arrays.
     /^<gp>/{gp=$0}
     /^<cts>/{cts[++i]=$0} 
     /^<tp>/{tp[++j]=$0;k=0}
     /^<v>/{v[++k,j]=$0}
     END { for (o=1;o<=i;o++)
              {printf "%s\n%s\n%s\n",dt,gp,cts[o]  > (fn ="file"o".txt") ;  # print the titles to files
              for (m=1;m<=j;m++)
                 {printf "%s\n%s\n",tp[m],v[o,m] >fn } }  # print tp and v to above files.
         }' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Extract lines that have entries in VI file

Dears experts i have UNIX file that contain 4 million lines , i need to extract all lines that have entiries saved in VI file , i have below comand but it takes tooooo long time : for i in `cat file1.csv`; do cat dump | grep -i $i >> file2.csv; done where : file1.csv = VI file that... (12 Replies)
Discussion started by: is2_egypt
12 Replies

2. Shell Programming and Scripting

Extract lines from a file

Hi all; Here is my file which contains a list of files (recent versions of files are in red). This file is dynamic, files versions can change at any time (versions can increment) filename ------------------------------------------------------- ... (8 Replies)
Discussion started by: chercheur111
8 Replies

3. Shell Programming and Scripting

How to extract certain lines from a file?

Hi guys I have a several thousands line file in the following format: n817 -------------------------------------------------- n842 -------------------------------------------------- n877 -------------------------------------------------- n513 /bb/data/rmt2db.lrl:JBSKDB 31915 75... (4 Replies)
Discussion started by: aoussenko
4 Replies

4. Shell Programming and Scripting

Extract particular lines from a file

Hi all, I have a file with many records with information as given below ID A16L2_HUMAN Reviewed; 619 AA. AC Q8NAA4; A5PL30; B2RPK5; Q658V4; Q6PID3; Q8NBG0; DT 20-MAY-2008, integrated into UniProtKB/Swiss-Prot. DT 20-MAY-2008, sequence version 2. DT ... (1 Reply)
Discussion started by: kaav06
1 Replies

5. Shell Programming and Scripting

Extract some lines from one file and add those lines to current file

hi, i have two files. file1.sh echo "unix" echo "linux" file2.sh echo "unix linux forums" now the output i need is $./file2.sh unix linux forums (3 Replies)
Discussion started by: snreddy_gopu
3 Replies

6. Shell Programming and Scripting

Extract specific lines from a file

Hi, I have a file which contains DDL statements- CREATE TABLE, CREATE INDEX, ALTER TABLE etc. I have to only pick CREATE TABLE statements from the file- Source : ---------------------------------------------- --DDL for table abc -------------------------------------------- CREATE TABLE... (4 Replies)
Discussion started by: newb
4 Replies

7. Shell Programming and Scripting

To extract the required lines from a file

I have a files in a directory in this format data data data ---BEGIN CERT----- data data data ---END CERT ----- Now, I want to extract the lines starting from --BEGIN CERT-- and write the contents till the end of file into a new file.How can I do this for all the files in the... (1 Reply)
Discussion started by: sureshg
1 Replies

8. UNIX for Dummies Questions & Answers

need to extract few lines from a file and put it in another file

Hi all, I have a file which contains comments,dotted lines(-------),actual records etc.I need to fetch only the actual records and put into a new file neglecting/deleting other rows.I have a header record too which contains the column names for the actual records.i need to discard this too.how... (5 Replies)
Discussion started by: ganesh_248
5 Replies

9. Shell Programming and Scripting

how to extract a range of lines from a file

I am reading a file that contains over 5000 lines and I want to assign it to a shell variable array (which has a restriction of 1024 rows). I had an idea that if I could grab 1000 record hunks of the file, and pipe the records out, that I could perform a loop until I got to the end and process 1000... (5 Replies)
Discussion started by: beilstwh
5 Replies

10. UNIX for Dummies Questions & Answers

extract specific lines from file

hi, how would i extract a range of lines in a file by using the line number? ex: file contains: 1 title 2 i want 3 this part 4 to be taken out 5 from this file 6 and sent to 7 another file 8 not needed 9 end of file In this case, i want to copy line number 2 to 7 on a new... (2 Replies)
Discussion started by: apalex
2 Replies
Login or Register to Ask a Question