Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help in File Processing Script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help in File Processing Script
# 1  
Old 07-27-2012
Need help in File Processing Script
I made one script in shell, but it is not running in my server. So i am trying to make perl for the same. Can anyone help me?

Code:
#!/bin/sh
p=`date '+20%y%m%d'`
cp Process_*"$p"*.txt /tmp/
cat /tmp/number.txt | egrep -v "Prefix|---|" | sed -e 's/ //g' -e '/^$/d' | nawk -F"->" 'BEGIN{FS="->";OFS=","} {print $1,$2}' > /tmp/numberserieslookup.txt
cat /tmp/numberserieslookup.txt | nawk 'BEGIN{FS=OFS=","} { print substr($1,3,14),$2}' > /tmp/lookup.txt
for i in `cat  /tmp/lookup.txt  |awk -F, '{print $2}' |sort -u`; 
do 
    grep $i /tmp/lookup.txt > /tmp/$i.txt ;
done


Steps:

1.Copying the Process_currentdate.txt file in to tmp path

2.Making one file numberserieslookup.txt from input file number.txt

number.txt will be like this

Code:
Prefix

----------

7777777 -> PPPP

8888888 -> QQQQ

I dont want that Prefix and ----. so using sed i am removing those Prefix,---- and blank space. This gives output numberserieslookup.txt as

numberseries.txt

Code:
7777777,PPPP

8888888,QQQQ

Using numberseries.txt, removing first 2 digit of first field.
lookup.txt

Code:
77777,PPPP

88888,QQQQ

4.for loop is used to make different files as per 2nd field.

Filename will be PPPP.txt inside that file

Eg:

Code:
PPPP.txt

77777,PPPP

QQQQ.txt

88888,QQQQ

can any one help me to make perl for this?
Moderator's Comments:
Mod Comment Code tags for code, please.
Last edited by Corona688; 07-27-2012 at 02:16 PM..
# 2  
Old 07-27-2012
I cannot write perl, but we could do this a heck of a lot better in sh and/or awk.

Code:
mute@ovh:~/temp/lecria$ awk -F' *-> *' 'NF==2{print substr($1,3),$2 > $2 ".txt"}' OFS=, number.txt

Code:
number.txt:Prefix
number.txt:----------
number.txt:7777777 -> PPPP
number.txt:8888888 -> QQQQ
PPPP.txt:77777,PPPP
QQQQ.txt:88888,QQQQ

# 3  
Old 07-27-2012
I bet we can make a script that works without resorting to perl. Your script could use a lot of improvement in any case.

Do your data files really have all those blank lines in them?
# 4  
Old 07-27-2012
@neutronscott: Thanls alot. Shell is not running my server, i am getting some error, if i run perl script then it is running.
# 5  
Old 07-27-2012
I'm sure it's more worthwhile to fix whatever your shell problem is than to do this in perl. My code used only nawk, the shell should not matter.
Please copy and paste exactly what you tried and the errors it produced.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to log file processing details to database table usnig UNIX shell script?

we are getting files on daily basis.we need to process these files. i need a unix shell script where we can count 1-The no of files processed 2-No of data/record processed for each files. The script should log these details into a database table. If there is any error while file... (3 Replies)
Discussion started by: Atul kumar
3 Replies

2. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

3. Shell Programming and Scripting

File Processing in shell script

i have a file with following type of data abcd : gggggg gggggg ; 1234 gggggg ; 5678 gggggg ; 3434 gggggg ; 6565 gggggg ; 1231 1234 ; vvvv ;Eng=Myfirstname 5678 ; xyzf ;Eng=Mysecondname 3434 ; xyzf ;Eng=Mythirdname 6565 ; xyzf ;Eng=Mysfourthname 1231 ; xyzf ;Eng=Mysfifthname... (7 Replies)
Discussion started by: telangmadhuri
7 Replies

4. Shell Programming and Scripting

2 file processing script in C shell

I want to use an awk for the following scenario but not sure if it will work or not. I have two input file: F1 and F2 F1 02 05 08 F2 00 01 02 03 04 05 06 07 08 09 10 (1 Reply)
Discussion started by: jclanc8
1 Replies

5. Shell Programming and Scripting

File processing using script needs help

Dear all, Need help to write the shell script to process the file. My requirement is script to process the file line by line and count the ids belong to the server name mentioned as in first field. if IN-S-BA1 is there count 4 lines and add the end of count echo the first filed and... (2 Replies)
Discussion started by: nmadhuhb
2 Replies

6. Shell Programming and Scripting

Simple Script needed for Processing CSV file perl

I am new to perl, and need a script to pull a CSV file, strip out 2 leading columns, and 2 ending columns, and resave the file in a new location. The file is basic and has less than 10,000 records. Secondly, can I schedule perl scripts to run daily? Can someone provide the basic script to... (1 Reply)
Discussion started by: cobbjob
1 Replies

7. Shell Programming and Scripting

awk, perl Script for processing a single line text file

I need a script to process a huge single line text file: The sample of the text is: "forward_inline_item": "Inline", "options_region_Australia": "Australia", "server_event_err_msg": "There was an error attempting to save", "Token": "Yes", "family": "Family","pwd_login_tab": "Enter Your... (1 Reply)
Discussion started by: hmsadiq
1 Replies

8. Shell Programming and Scripting

KSH script -text file processing NULL issues

I'm trying to strip any garbage that may be at the end of my text file and that part is working. The problem only seems to be with the really long lines in the file. When the head command is executed I am directing the output to a new file. The new file always get a null in the 4096 position but... (2 Replies)
Discussion started by: geauxsaints
2 Replies

9. Shell Programming and Scripting

perl script for file processing

Aim: To scan a file and ignore all characters that has an ASCII value from 0 to 31 and 127 to 255 and accept only those characters having an ASCII between 32 and 126. Script: #!/usr/local/bin/perl $filename = "$ARGV"; if (-e $filename) { open(OUT, "${filename}") || die "can't... (10 Replies)
Discussion started by: SEEHTAS
10 Replies

10. Shell Programming and Scripting

Have a shell script check for a file to exist before processing another file

I have a shell script that runs all the time looking for a certain type of file and then it processes the file through a series of other scripts. The script is watching a directory that has files uploaded to it via SFTP. It already checks the size of the file to make sure that it is not still... (3 Replies)
Discussion started by: heprox
3 Replies
Login or Register to Ask a Question