Query on scp


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Query on scp
# 1  
Old 07-21-2017
Query on scp

Hi ,
I am new to Shell script.

I got a requirement to get the filenames and timestamp into an array like below.
Code:
array=(Jul 19 06:10 file1-en-us_US-20170718T160150Z.json,Jul 19 06:10 file1-en-us_US-20170718T150157Z.json)

So as the first step I have downloaded the required files from remote server using scp .To preserve the server time have added -p flag.

Code
Code:
scp -p   username@hostname:/folder1/folder2 /home/username/json

Output
Code:
-rw-r--r-- 1 username domain users    17382 Jul 19 06:10 file1-en-us_US-20170718T160150Z.json
-rw-r--r-- 1 username domain users    20463 Jul 19 06:10 file1-en-us_US-20170718T150157Z.json
-rw-r--r-- 1 username domain users   493244 Jul 19 06:10 file1-en-us_US-20170718T163218Z.json
-rw-r--r-- 1 username domain users    36499 Jul 19 06:10 file1-en-us_US-20170718T170151Z.json

Following to this I tried by getting the required fields by awk command as follows
Code:
awk ' { print $7,$8,$9,$10,$11 } ' > /home/usrname/input.txt.

This works fine by shell commands.Is it possible to club the awk in the scp command and redirect the output to a file.I tried by piping but it is not giving any result.
Another way is to change to the required directory and put awk command.But cd is not working in shell script.
Please suggest a way to resolve this.

Moderator's Comments:
Mod Comment
Please wrap all code, files, input & output.errors in CODE tags.
It makes it easier to read and preserves multiple spaces for indenting or fixed-width data.

Last edited by rbatte1; 07-21-2017 at 01:06 PM..
# 2  
Old 07-21-2017
Having got the files, would a loop using stat and it's various flags be suitable? You don't tell us you OS, so I'm hoping it is available to you by default. The risk with trying to read the output of something like ls -l is that group names may contain multiple spaces, but maybe none, so that would cause a mismatch on your field counts.

Looking at my manual page for stat, I would use the -c flag with parameters to get the values like this:-
Code:
$ stat -c '%y' /etc/profile
2017-03-21 21:56:32.000000000 +0000

You could build on this with a loop, e.g.:-
Code:
for filename in *
do
   timestamp=$(stat -c %y $filename)
   ## Trim the timestamp as required or adjust it to your required format
   array_val="${array_val},${timestamp} ${filename}"
done

array_val="${array_val#,}"          # Trim off leading comma
array=("${array_val}")

Does that help? The timestamps from stat would be numeric so depending on your need, you might have to convert them. You might be better using %Y which will give you the value in seconds since the Epoch (1970-01-01 00:00:00) and then convert to the time format you want, perhaps with date if your system supports the -d flag.


Have a try and let us know if that helps or where you get stuck.




Kind regards,
Robin
# 3  
Old 07-25-2017
Query on scp

Thanks for the approach, how can i get the array elements, say array[1],array[2] etc
When i gave echo ${array[1]}
it prints nothing.
# 4  
Old 07-25-2017
Ah, I should not have quoted the line array=("${array_val}")

Try is with array=(${array_val}) for the last line instead. Does that help?


You might need to work with line 6 too, perhaps this would be better overall:-
Code:
array_val=""
for filename in *
do
   timestamp=$(stat -c %Y $filename)
   ## Trim the timestamp as required or adjust it to your required format
   array_val="${array_val} ${timestamp},${filename}"
done
array=(${array_val})

for item in 0 1 2 3 4 5
do
   echo "Value for item $item is ${array[$item]}"
done


The date/time value is in seconds from the Epoch, else the array would split up. is that a problem? We can work on fixing it if we need to.



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to use scp?

How to copy multiple directories using single command on solaris 10 from server A to server B. I tried scp but its working only one directory at atime How to acheive this with simple and short solution????? (6 Replies)
Discussion started by: buzzme
6 Replies

2. Shell Programming and Scripting

Shell Script to execute Oracle query taking input from a file to form query

Hi, I need to query Oracle database for 100 users. I have these 100 users in a file. I need a shell script which would read this User file (one user at a time) & query database. For instance: USER CITY --------- ---------- A CITY_A B CITY_B C ... (2 Replies)
Discussion started by: DevendraG
2 Replies

3. Shell Programming and Scripting

Query Oracle tables and return values to shell script that calls the query

Hi, I have a requirement as below which needs to be done viz UNIX shell script (1) I have to connect to an Oracle database (2) Exexute "SELECT field_status from table 1" query on one of the tables. (3) Based on the result that I get from point (2), I have to update another table in the... (6 Replies)
Discussion started by: balaeswari
6 Replies

4. Shell Programming and Scripting

add the output of a query to a variable to be used in another query

I would like to use the result of a query in another query. How do I redirect/add the output to another variable? $result = odbc_exec($connect, $query); while ($row = odbc_fetch_array($result)) { echo $row,"\n"; } odbc_close($connect); ?> This will output hostnames: host1... (0 Replies)
Discussion started by: hazno
0 Replies

5. Shell Programming and Scripting

Is this possible with SCP?

I normally download a directory recursively using: scp -r <name>@host:<path> . This has worked fine. As everyone knows this will download all of the directory named in <path> and all of the sub directories. I would like to know if it is possible to not download a particular file if it... (5 Replies)
Discussion started by: cpabrego
5 Replies

6. Shell Programming and Scripting

Query related to scp command

Hi Friends, I need to execute a scp command to transfer some files from source to target server. Unfortunately the ftp is not working in my case. This scp command needs to be executed via Unix script. I need to know the complete scp command which includes the user-id and password of the... (2 Replies)
Discussion started by: sureshg_sampat
2 Replies

7. Shell Programming and Scripting

scp

hi can any one pls tell me how to copy a file from a remote host to the same remote host with a timestamp, i need to use only scp. thnks (4 Replies)
Discussion started by: bkan77
4 Replies

8. Shell Programming and Scripting

What is scp-ed over?

Hi all, i have a directory in server A. the directory path is /home/kevin. I need to scp the directory to another server B. i would like to ask, when i do a scp of the /home/kevin , i can expect all the files from A to go B. However, how about the hidden files? for example the ssh keys in the... (4 Replies)
Discussion started by: new2ss
4 Replies
Login or Register to Ask a Question