Help with SFTP and Number of lines reconcillation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with SFTP and Number of lines reconcillation
# 1  
Old 03-09-2011
Help with SFTP and Number of lines reconcillation

Hi,

I have a remote server from where i have to sftp 4 files.
Say today is 03/09/2011. Then i will recevie file with extension *_03082011.txt today

i.e i will receive today, files with yesterday's date.

And on monday i will get files with date of friday.

I am currently using mget *.* which is pulling all older files.

I tried using
Code:
sftp xyz@abc
cd directory
 ls -altr | tail 
 
ls -altr | tail -4 says invalid command.

Looks like ls -altr | tail -4 will not work in sftp session.

I need help on how to pick the last 4 files.

-------------------------------------------------------------------

2nd Issue:

Once this files are ftped from remote server to local server.

Out of these 4 files one file will contain details of the number of records in the other 3 files. And we need to use this file for reconcilation purpose.


The format is as follows:
Quote:
FileName|Number of records
XYZ|123
ABC|25
CDE|34
How do i implement this.

Appreciate help.
# 2  
Old 03-10-2011
Setup a batchfile with ls -atr in it. Now:
Code:
filelist=$(sftp -b myscript  user@host | tail -4)


To load record counts:
Code:
awk -F\| 'FNR==1 {k=$2 ~ "Number of records"; next} k {print $1,$2}' $filelist | while read file count
do
    .... your code deals with $file and $count here [eg testing $(wc -l $file) -eq  $count ]
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Print number of lines for files in directory, also print number of unique lines

I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with: wc -l *|sort 15263 Image.txt 16401 reference.txt 40459 richtexteditor.txt How can I also print the number of unique lines in each file? 15263 1401 Image.txt 16401... (15 Replies)
Discussion started by: spacegoose
15 Replies

2. UNIX for Beginners Questions & Answers

Number of lines in vi

Is there a limit on the number of lines that can be copied into vi editor from another file? When I copy some data items from a file into the vi, I see a limit of 1780 lines can be pasted. After that there seems to be a limit. How can this be increased? As an example:, I open an excel file which... (3 Replies)
Discussion started by: ggayathri
3 Replies

3. Shell Programming and Scripting

Grep lines for number greater than given number

Hello, I am newbie to bash scripting. Could someone help me with the following. I have log file with output as shown below **************************LOG************************* 11/20/2013 9:11:23.64 Pinging xx.xx.xx.xx with 32 bytes of data: 11/20/2013 9:11:23.64 Reply from xx.xx.xx.xx:... (4 Replies)
Discussion started by: meena_2013
4 Replies

4. Shell Programming and Scripting

Sftp large number of files

Want to sftp large number of files ... approx 150 files will come to server every minute. (AIX box) Also need make sure file has been sftped successfully... Please let me know : 1. What is the best / faster way to transfer files? 2. should I use batch option -b so that connectivity will be... (3 Replies)
Discussion started by: vegasluxor
3 Replies

5. AIX

Limitation for SFTP on AIX number of sessions

Hello. I am using AIX 6 and If wish to receive more than 500 files via SFTP, I get some time out errors. Could you please advise where is the limit for number of concurrent transfers setup in AIX Box or what is the limit and can that be changed? Many Thanks (3 Replies)
Discussion started by: panchpan
3 Replies

6. Shell Programming and Scripting

count the number of lines that start with the number

I have a file with contents similar to this. abcd 1234 4567 7666 jdjdjd 89289 9382 92 jksdj 9823 298 I want to write a shell script which count the number of lines that start with the number (disregard the lines starting with alphabets) (1 Reply)
Discussion started by: grajp002
1 Replies

7. Shell Programming and Scripting

Number lines of file and assign variable to each number

I have a file with a list of config files numbered on the lefthand side 1-300. I need to have bash read each lines number and assign it to a variable so it can be chosen by the user called by the script later. Ex. 1 some data 2 something else 3 more stuff which number do you... (1 Reply)
Discussion started by: glev2005
1 Replies

8. UNIX for Dummies Questions & Answers

deleting lines by number

Is it possible to delete lines by their number? Also, I'd like to delete the last 3 rows of a file too. So from the front and back. Thanks. (9 Replies)
Discussion started by: cary530
9 Replies

9. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies

10. Linux

counting the number of lines

Hello, I have afile which begins with a few urls on multiple lines and then there is listing of some information on separate lines. The listing begins with the word Name on a given line followed by teh actual list. I want to count the number of lines in this file after the line having... (6 Replies)
Discussion started by: nayeemmz
6 Replies
Login or Register to Ask a Question