Need to ftp some files, and check the number of rows in the transferd file


 
Thread Tools Search this Thread
Special Forums UNIX Desktop Questions & Answers Need to ftp some files, and check the number of rows in the transferd file
# 1  
Old 01-31-2011
Need to ftp some files, and check the number of rows in the transferd file

1)I need to write a script which ftps 3 files to a unix box,
2)once the files are ftped i need to check the number of rows in each file and compare it with the data (no of rows) coming in a manifest file, if the number of rows in each file matches the data coming in manifest file, then i need to send a email saying ftp is successful.
3)These files come in with a timestamp I need to rename these files by removing the time stamp.

I am new to unix can anyone help me with this please.

Thanks in advance
# 2  
Old 01-31-2011
Does it have to be ftp? Well, you can use wget with ftp and make it pretty script painless, as good as scp, just less secure and no compression option. That takes the pain out of #1. #2 is pretty simple, depending on the structure of the manifest file. wc -l will count your lines. What do you do for bad files? Renaming is no big deal, but what is the structure of the name before/after?
# 3  
Old 01-31-2011
Hi
Thanks for you early reply,
Actually its SFTP.
here is the script i came up with to ftp and rename the file., i need to add the manifest and email logic to it...

Please find below he script....

Code:
#!/usr/bin/ksh

ftpFrom="/JANUS_IMS_health_Test/MarketRx"
ftpTo="/coe/informatica/v712_OMJ/OMJ_Proherant/SrcFiles"
host="Hostname.com"
user="aimran79"


echo "sftp start"
cd $ftpTo
sftp $user@$host <<EOF
cd /JANUS_IMS_health_Test/MarketRx
mget *.*
bye
EOF

for file in `ls *.csv`
do
 	mv $file `echo "$file" | sed 's/_[^_]*\././'`
done

echo "Program Ended Successfully"


Last edited by pludi; 01-31-2011 at 05:17 PM.. Reason: code tags, please...
# 4  
Old 01-31-2011
Look into the ksh trimmers like ${#} and ${%%}, since "${file%_*}.extn" is like your sed. My memory mnemonic is pound # on the nose, get your % in the end. Double is aggressive wc, single is lazy wildcard. Saves fork, exec per.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to check if any files are being FTP'ed currently ?

how to check if any files are being FTP'ed currently ? both FTP in and FTP out. And is there any system log for all the FTP activities ? (2 Replies)
Discussion started by: i4ismail
2 Replies

2. Shell Programming and Scripting

Split File based on number of rows

Hi I have a requirement, where i will receive multiple files in a folder (say: /fol1/fol2/). There will be at least 14 to 16 files. The size of the files will different, some may be 80GB or 90GB, some may be less than 5 GB (and the size of the files are very unpredictable). But the names of the... (10 Replies)
Discussion started by: kpk_ds
10 Replies

3. Shell Programming and Scripting

How to check whether files are transferred or not using ftp?

Hi, i want to execute a shell script which transfers files from one server to another using ftp in unix. How can i check whether the ftp is successful or not.(i.e files are transferred to destination server). because if i am checking the return code of ftp, it always shows 0 (denoting ftp is... (5 Replies)
Discussion started by: Little
5 Replies

4. UNIX for Dummies Questions & Answers

Write the total number of rows in multiple files into another file

Hello Friends, I know you all are busy and inteligent too... I am stuck with one small issue if you can help me then it will be really great. My problem is I am having some files i.e. Input.txt1 Input.txt2 Input.txt3 Now my task is I need to check the total number of rows in... (4 Replies)
Discussion started by: malaya kumar
4 Replies

5. UNIX for Dummies Questions & Answers

Merge two files with common IDs but unequal number of rows

Hi, I have two files that I would like to merge and think that there should be a solution using awk. The files look something like this: file 1 IDX1 IDY1 IDX2 IDY2 IDX3 IDY3 file 2 IDY1 dataA data1 IDY2 dataB data2 IDY3 dataC data3 Desired output IDX1 IDY1 dataA data1 IDX2 ... (5 Replies)
Discussion started by: katie8856
5 Replies

6. UNIX for Dummies Questions & Answers

How to separate a single column file into files of the same size (i.e. number of rows)?

I have a text file with 1,000,000 rows (It is a single column text file of numbers). I would like to separate the text file into 100 files of equal size (i.e. number of rows). The first file will contain the first 10,000 rows, the second row will contain the second 10,000 rows (rows 10,001-20,000)... (2 Replies)
Discussion started by: evelibertine
2 Replies

7. Shell Programming and Scripting

awk number rows from 1 to 3 in file

I have a file with the following format, i need to change de field9 each 3 rows to renumber field9 with gpo1, gpo2, gpo3. I need to use awk Original file field1 field2 field3 field4 field5 field6 field7 field8 gpo3 field1 field2 field3 field4 field5 ... (3 Replies)
Discussion started by: robonet
3 Replies

8. UNIX for Advanced & Expert Users

Using FTP to check whether file is completely FTP... plz find the description below

Hi, We have some clients who will place huge files in to one of the remote server. And the shell script written in our local server to retrieve client files (using FTP) placed on one of the remote server of ours by clients. My question Is there any FTP command/script to check from my local... (1 Reply)
Discussion started by: nmsrao
1 Replies

9. Shell Programming and Scripting

Splitting file based on number of rows

Hi, I'm, new to shell scripting, I have a requirement where I have to split an incoming file into separate files each containing a maximum of 3 million rows. For e.g: if my incoming file say In.txt has 8 mn rows then I need to create 3 files, in which two will 3 mn rows and one will contain 2... (2 Replies)
Discussion started by: wahi80
2 Replies

10. UNIX for Dummies Questions & Answers

Number Grouped Rows in File

I have a file containing 750,000 records and have managed to sort them by related columns and now i'd like to add an ID number to the front of each line of the records that are grouped together. Which probably makes no sense so i have provided some example data and desired result. Given data.txt... (2 Replies)
Discussion started by: RacerX
2 Replies
Login or Register to Ask a Question