Shell script to manipulate files


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Shell script to manipulate files
# 1  
Old 04-17-2013
Shell script to manipulate files

My requirement is explained below:

list of files available in server 1 in path /home/xxx/src are:
Code:
XX_SRC_20130417.txt
XX_SRC_20130417.dat

Code:
$cat XX_SRC_20130417.txt
col1=ABC
col2=
col3=xyz

a sequence file name is maintained which is in the path /ab_app/xx/seq
Code:
$cd /ab_app/xx/seq

Code:
$cat val.txt

Code:
xm|20130104|001|0
xy|20130704|002|1 
xx|20130405|002|1
xz|20121213|001|1

Rule: Get the latest sequence value from the file for xx ie, 002.increment it by 1 (it will be 003) and store it in a variable.Make another copy of the file in /home/xxx/src renaming the file like below:

Code:
XX_SRC_20130417_003.txt
XX_SRC_20130417_003.dat

003 -> is the incremented number here.

Also, update the file XX_SRC_20130417_003.txt content with the incremented value

Code:
col1=ABC
col2=003
col3=xyz

sftp the newly created file.
If sftp is successful (used scp command to transfer files) for all the newly created two files sent( sent to server 2 , path: /extract/src/xx) then only update the content of val.txt with the incremented value.

After update, It should be something like:

Code:
xm|20130104|001|0
xy|20130704|002|1 
xx|20130405|003|1             <---------------------------value updated here 
xz|20121213|001|1

Remove the files below from /home/xxx/src(server1) no matter whether sftp is succeed or failed

Code:
XX_SRC_20130417_003.txt
XX_SRC_20130417_003.dat

Need your help and guidance to get the desired output.

Thanks in advance.

---------- Post updated at 03:19 PM ---------- Previous update was at 02:24 PM ----------

I have made a draft of my approach something like below:
Code:
export source_dir=/home/xxx/src
file_from_list=`ls  $source_dir/$file_name`
new_file_list=`ls  $source_dir/$new_file_name`

send_sftp(){

    scp $file_to_move $conn_string
    if [[ $? -ne 0 ]]
    then
        print "++++ File Transfer failed at `date +'%Y%m%d %H:%M%S'` " ;
        print "++++ File transfer failed at `date +'%Y%m%d %H:%M%S'`"
        exit 14
    fi

    rm $file_to_move ## irrespective of success or failure of sftp remove the new files
    if [[ $? -ne 0 ]]
    then
        print "++++ ERROR: File Transfer to  failed  at `date +'%Y%m%d %H:%M%S'` " ;
        print "++++    Transferred file could not be removed"
        exit 15
    fi
  


}
if [[ $application_id = "2" ]]
then
    print "Entering here..."
    conn_string=xxx@myserver2:/home/xxx/src
    file_name=xx_*
fi

value=grep "xx" val.txt| cut -d"|" -f2 
get_seq_val=$value+1
if [[ $application_id = "2" ]]
then
for list_files in $file_from_list
do
   fl_name=`basename list_files`
   fl=`echo $fl_name|cut -d'.' -f1`
   extn=`echo .${fl_name#*.}`
   new_file_name=$fl"_"$get_seq_val$extn   
   cp $list_files $new_file_name
done

for new_files in $new_file_list
do
   file_to_move=$new_files
   send_sftp()
   ### if all the files trasferred successfully then update the value in value.txt file ?? should we use sed to update the file? Can we track if all the files
   # are sftp ed successfully?
done
fi

I am new to unix world. I need your help here.

Last edited by vedanta; 04-17-2013 at 04:30 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Manipulate the columns of 2 files

Hello, I have two files to be treated. First file: col1 col2 col3 col4 Second file: colbis - I try to add the unique column of the file 2 towards the file 1. - To obtain the following result with a shell script ksh: col1 col2 col3 col4 colbis (4 Replies)
Discussion started by: khalidou13
4 Replies

2. Shell Programming and Scripting

Manipulate XML File Continous STRING by each Order Line using SHELL

heres sample File: <?xml version="1.0"?> <!DOCTYPE cXML SYSTEM "www"><cXML.............................................. <OrderRequest>USE UNIX.com</Extrinsic><Extrinsic name="UniqueName">Peter@UNIX.com</Extrinsic><Extrinsic name="ContractingEntity">UNIX... (3 Replies)
Discussion started by: Pete.kriya
3 Replies

3. UNIX for Dummies Questions & Answers

Writing a for loop to manipulate multiple files

Hi, I have 1000 text files in a folder that are labeled data1.txt all the way to data1000.txt. I want to write a small script that manipulates the text files in this way: (1) cut the 2nd and 9th columns of the text files (2) sort by the numerical value in the 9th column (3) then save the rows... (3 Replies)
Discussion started by: evelibertine
3 Replies

4. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

5. Shell Programming and Scripting

Manipulate string in shell script

I am writing a shell script for some purpose. I have a variable of the form -- var1 = "policy=set policy" Now I need to manipulate the variable var to get the string after index =. that is i should have "set polcy". Also I need to to this for many other variables where the value of "=" is not... (3 Replies)
Discussion started by: Dev_Sharma987
3 Replies

6. Shell Programming and Scripting

Shell script to manipulate a file

Hello, I have a file with following contents : WSL SRVGRP=LISTENER SRVID=2 CLOPT="-A -t -- -n 0x0002aa050a03cc65 " RQPERM=0660 REPLYQ=Y RPPERM=0660 MIN=1 MAX=1 CONV=N I need to print only the value in Hex i.e.... (2 Replies)
Discussion started by: deo_kaustubh
2 Replies

7. UNIX for Dummies Questions & Answers

using sed to manipulate text in files

Hi, I have a slight problem in trying to manipulate the text within a file using the "sed" command in that the text i need changed has "/" slashes in. I have a .sh script that scans the "/db/sybbackup/" directories for any .dmp file older than 2 days and then to >> the information to a file called... (3 Replies)
Discussion started by: Jefferson333
3 Replies

8. Shell Programming and Scripting

Shell script to manipulate a message count for the same IP @ diff session

I have a file as like below, 10:20:30.45 START 10.20.30.40 10:20:31.46 HELLO 10.20.30.40 10:20:32.46 START 10.20.30.41 10:20:33.44 END 10.20.30.40 10:20:35.44 HELLO 10.20.30.41 10:20:36.56 HELLO 10.20.30.41 10:20:37.78 HELLO 10.20.30.41 10:20:38.99 START 10.20.30.40... (1 Reply)
Discussion started by: gobinath
1 Replies

9. Shell Programming and Scripting

Manipulate files

Hi everybody: I have a problem. I have a output files which have this pattern: number1 --space block1a - 7rows/10columns/65elements --space block1b - 7rows/10columns/65elements --space block1c - 7rows/10columns/65elements --space number2 --space block2a - 7rows/10columns/65elements... (0 Replies)
Discussion started by: tonet
0 Replies
Login or Register to Ask a Question