Perl or Awk script to copy a part of text file.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Perl or Awk script to copy a part of text file.
# 1  
Old 10-30-2009
Perl or Awk script to copy a part of text file.

Hi Gurus,
I'm a total newbie to Perl and Awk scripting. Let me explain the scenario, there is a DB2 table with 5 columns and one of the column is a CLOB datatype containing XML. We need all the 4 columns but only a portion of string from the XML column.
We decided to export DB2 table to a .del file and process it using Perl or Awk script. I need a script to process the .del file so that I have column1, column2, column3 and in column 4 which is XML, we just need the string which is in between <text> and </text> (there may be multiple occurrence of this so they can be seperated by number) plus the column 5.
I know it will be piece of cake for the experts.

Thanks,
# 2  
Old 10-30-2009
using grep

try this
Code:
I=0
while read LINE
do
    TEXT[$I]=$(echo $LINE | grep -o '<text>.*</text>' | sed -e 's/<text>//' -e 's/<\/text>//'
    (( I ++ ))
done < $FILENAME

That creates an array where each item is the text contained between <text> and </text>.
Hope this helps
This User Gave Thanks to frans For This Post:
# 3  
Old 10-30-2009
Quote:
Originally Posted by frans
try this
Code:
I=0
while read LINE
do
    TEXT[$I]=$(echo $LINE | grep -o '<text>.*</text>' | sed -e 's/<text>//' -e 's/<\/text>//'
    (( I ++ ))
done < $FILENAME

That creates an array where each item is the text contained between <text> and </text>.
Hope this helps
Thanks Frans for your prompt reply but I've a question which may sound stupid to you but still.
1) What is the extension of the file I should save? Is it .awk or .ksh?
2) Where do I replace the input and output file name in the code?

Thanks,
# 4  
Old 10-30-2009
Quote:
Originally Posted by asandy1234
Thanks Frans for your prompt reply but I've a question which may sound stupid to you but still.
1) What is the extension of the file I should save? Is it .awk or .ksh?
2) Where do I replace the input and output file name in the code?

Thanks,
It's bash scripting so on the first line of the script you write #!/bin/bash If the path of your shell is /bin/bash, of course.
The full code :
Code:
#!/bin/bash
while read LINE
do
    echo $LINE | grep -o '<text>.*</text>' | sed -e 's/<text>//' -e 's/<\/text>//'
    (( I ++ ))
done < INPUTFILE > OUTPUTFILE

You could use variables for the input and output files if they have to be re-used later, else just replace 'INPUTFILE' and 'OUTPUTFILE' by your own file names.

P.S. no matter the extension ! make it executable with chmod +x and go.

Last edited by frans; 10-30-2009 at 06:52 PM.. Reason: P.S.
This User Gave Thanks to frans For This Post:
# 5  
Old 11-02-2009
Thank you very much Frans, I'll test and let you know.

---------- Post updated at 12:46 PM ---------- Previous update was at 12:05 PM ----------

Hi Frans,
There is no directory #!/bin/bash in the unix box, but there is directory for #/usr/bin/bsh. Are these 2 same? I tried to run the script but I get an error as below

grep: Not a recognized flag: o
Usage: grep [-r] [-R] [-H] [-L] [-E|-F] [-c|-l|-q] [-insvxbhwy] [-p[parasep]] -e pattern_list...
[-f pattern_file...] [file...]
Please clarify.

Thanks.
# 6  
Old 11-02-2009
bsh is Bourne SHell, bash is Bourne Again SHell, so the options of the grep command doesn't seem to be the same.
the -o option tells grep to output only the matching part of the line, it's helpful.
Tell me what's the output when you use grep without option.
I believe that there's only one occurence of the <text>....</text> in each line.
To go faster in trying, don't redirect to the output file ( > OUTPUTFILE ) so you directly see what happens.
a possibility is to use the extract like
Code:
LINE="lbla bla bla jcjfd<text>what i want</text>flh%(j blablabla" # for testing
LINE=$(echo $LINE | grep <text>*</text>) # Returns every line containing the match.
echo $LINE
LINE=${LINE##*<text>} # Deletes the matching from the beginning
echo $LINE
LINE=${LINE%%</text>*} # Deletes the matching from the end
echo $LINE

and see what happens
# 7  
Old 11-02-2009
Quote:
Originally Posted by frans
bsh is Bourne SHell, bash is Bourne Again SHell, so the options of the grep command doesn't seem to be the same.
the -o option tells grep to output only the matching part of the line, it's helpful.
Tell me what's the output when you use grep without option.
I believe that there's only one occurence of the <text>....</text> in each line.
To go faster in trying, don't redirect to the output file ( > OUTPUTFILE ) so you directly see what happens.
a possibility is to use the extract like
Code:
LINE="lbla bla bla jcjfd<text>what i want</text>flh%(j blablabla" # for testing
LINE=$(echo $LINE | grep <text>*</text>) # Returns every line containing the match.
echo $LINE
LINE=${LINE##*<text>} # Deletes the matching from the beginning
echo $LINE
LINE=${LINE%%</text>*} # Deletes the matching from the end
echo $LINE

and see what happens
No there are multiple occurence of this pattern in the record
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help in UNIX shell to copy part of file name to new file name

Hi, I want to do the following in a Unix shell script and wonder if someone could assist me? I want to take files in a specific directory that start with the name pxpur012 and copy them to the same directory with the file name not containg pxpur012. For example, I have files like... (4 Replies)
Discussion started by: lnemitz
4 Replies

2. Shell Programming and Scripting

Not able to copy the file in perl cgi script

Hello experts, I am facing an very typical problem and hope the issue can be solved. I have a page download.cgi in /cgi-bin folder. use CGI; use CGI::Carp qw ( fatalsToBrowser ); use File::Copy copy("C:\\Program Files\\Apache Software... (8 Replies)
Discussion started by: scriptscript
8 Replies

3. Shell Programming and Scripting

Copy a file from local host to a list of remote hosts --- perl script

Hi friends, i need to prepare a script ( in perl) i have a file called "demo.exe" in my local unix host. i have a list of remote hosts in a file "hosts.txt" now i need to push "demo.exe" file to all the hosts in "hosts.txt" file. for this i need to prepare a script(in perl, but shell... (5 Replies)
Discussion started by: siva kumar
5 Replies

4. Shell Programming and Scripting

Copy part of file between two strings to another

I am a newbie to shell scripting I have a large log file , i need to work on the part of the log file for a particular date. Is there a way to find the first occurance of the date string and last occurance of the next day date date string and move this section to a new file. to explain it... (3 Replies)
Discussion started by: swayam123
3 Replies

5. UNIX for Dummies Questions & Answers

Copy the last part since the file has been updated

Input File1 constatntly running and growing in size. My Program Erorr ddmmyy hh:mm:ss My Program Error **Port 123 terminated **ID PIN 12345 Comamnd Successful Command Terminated Command Successful Command Terminated **My Program Erorr ddmmyy hh:mm:ss My Program Error **Port 345... (3 Replies)
Discussion started by: eurouno
3 Replies

6. UNIX for Dummies Questions & Answers

Copy a part of file

Hi, I want to copy text between expressions ">bcr1" and ">bcr2" to another file. Any simple solutions? Thanks (4 Replies)
Discussion started by: alpesh
4 Replies

7. Shell Programming and Scripting

search needed part in text file (awk?)

Hello! I have text file: From aaa@bbb Fri Jun 1 10:04:29 2010 --____OSPHWOJQGRPHNTTXKYGR____ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline My code '234565'. ... (2 Replies)
Discussion started by: candyme
2 Replies

8. Shell Programming and Scripting

shell script to take input from a text file and perform check on each servers and copy files

HI all, I want to script where all the server names will be in a text file like server1 server2 server3 . and the script should take servernames from a text file and perform copy of files if the files are not present on those servers.after which it should take next servername till the end of... (0 Replies)
Discussion started by: joseph.dmello
0 Replies

9. Shell Programming and Scripting

awk, perl Script for processing a single line text file

I need a script to process a huge single line text file: The sample of the text is: "forward_inline_item": "Inline", "options_region_Australia": "Australia", "server_event_err_msg": "There was an error attempting to save", "Token": "Yes", "family": "Family","pwd_login_tab": "Enter Your... (1 Reply)
Discussion started by: hmsadiq
1 Replies

10. UNIX for Dummies Questions & Answers

Shell script to search for text in a file and copy file

Compete noob question.... I need a script to search through a directory and find files containing text string abcde1234 for example and then copy that file with that text string to another directory help please :eek: (9 Replies)
Discussion started by: imeadows
9 Replies
Login or Register to Ask a Question