Bash to append array value to file before copying


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Bash to append array value to file before copying
# 15  
Old 10-18-2019
Directory structure:
Code:
/path/to/${ID}*/path/to/folder (always this format)

The ${ID} will always be unique and have some random text after it that the * is used for. At the last folder in the path there is a .png always named cn_results that I am trying to append the unique ${ID} on front of ${ID}-cn_results.png, then scp that .png to xxxx@xxx.xx.xxx.xx:/path/to/destination/${ID}-cn_results.png)

There are two values in array, but only the first is printed:

Code:
# Set an array in one step in bash
array=( $(awk -F '\n' -v RS="" -v ref="$s5" '$0 ~ ref {d=split($0, val, " "); for(i=2;i<d;i+=2) printf "%s ",val[i]; printf "\n"}' list | tr -d '\r') )  ## match run with samples
echo "This is array" "$array" > /tmp/out

Contents of out
Code:
This is array ID1

I added a loop to iterate though each ID in array, but only the second value is printed:
Code:
# Set an array in one step in bash
array=( $(awk -F '\n' -v RS="" -v ref="$s5" '$0 ~ ref {d=split($0, val, " "); for(i=2;i<d;i+=2) printf "%s ",val[i]; printf "\n"}' "$run_dir"/rna.barcodes | tr -d '\r') )  ## match run with samples
   for file in "${array[@]}" ; do
    echo "This is array" "$file" > /tmp/loop.out
done

Contents of loop.out
Code:
This is array ID2

In this example array has two values in it ID1 and ID2, but there may be more or less it just depends on the run which the values for array come from. Thank you very much Smilie

Last edited by cmccabe; 10-18-2019 at 11:54 AM.. Reason: fixed format
# 16  
Old 10-18-2019
Show your actual directory structure EXPLICITLY, like
Code:
tree -d
.
├── D1
│   ├── D2
│   │   └── L1
│   ├── D3
│   │   └── L2
│   └── D4
│       ├── L1
│       ├── L2
│       └── L3
└── test
    └── D1
        └── D3
            ├── L1
            └── L2

14 directories

If your system doesn't have tree, use e.g. an editor to compose it.
This User Gave Thanks to RudiC For This Post:
# 17  
Old 10-18-2019
Hopefully this is better. Thank you Smilie.

Code:
├──/path/to/
│   ├── ${ID}*   --- unique ${ID}* represents random text ---
│   │   └── /this/ID/folder
│   ├── ${ID}*   --- unique ${ID}* represents random text ---
│   │   └── /this/ID/folder

# 18  
Old 10-18-2019
No, it's not. How do you expect people to bug hunt if they don't see what is actually going wrong? Show the directory listing from your target system. If that is confidential, build a dummy one but adapt your code to reproduce the error.
This User Gave Thanks to RudiC For This Post:
# 19  
Old 10-18-2019
Hopefully, this is better and helps:
Code:
# Set run array
array=( $(awk -F '\n' -v RS="" -v ref="$s5" '$0 ~ ref {d=split($0, val, " "); for(i=2;i<d;i+=2) printf "%s ",val[i]; printf "\n"}' list | tr -d '\r') )  ## match run in rna.barcode and extract patient

# iterate through array
for ID in "${array[*]}" ; do
echo "$ID" > /tmp/out.txt
       echo "( cd /path/to/${ID}*/path/to/folder && xec sshpass -f file.txt scp -- *.png* xxx@xxx.xx.xx.xxx:/path/to/destination/${ID}-cn_results)"
       sshpass -f out.txt ssh -o strictHostKeyChecking=no -t xxx@xxx.xx.xx.xx
done

Contents of out --- This is array with all samples one one line ---
Code:
ID1 ID2

echo to terminal
Code:
( cd /path/to/ID1 ID2*/*/path/to/folder && exec sshpass -f file.txt scp -- *.png* xxx@xxx.xx.xx.xxx:/path/to/destination/ID1 ID2-cn_results.png)

Code:
├──/path/to/   ---- common path after ssh ---
│   ├── ID1*   --- unique ${ID}* represents random text ---
│   │   └── /this/ID1/folder
│   ├── ID2*   --- unique ${ID}* represents random text ---
│   │   └── /this/ID2/folder
Description:

After ssh to the common path on the server, each unique ID is used to further navigate to folder. In each folder there is a png (cn_results), that the unique ID is append to (ID-cn_results) and this 
append file is scp to xxx@xxx.xx.xx.xxx:/path/to/destination. It seems that since array is one line both are being used and that is causing the error?

# 20  
Old 10-18-2019
Well that's not going to work. You're just printing the statements to the screen and not giving anything to sshpass at all.

Does your script change the variable IFS? Strings are supposed to split when you do ARRAY=( 1 2 3 ) and not doing so is very weird unless IFS was changed.

That's why you need to be careful to do

Code:
OLDIFS="$IFS"
IFS="."

...

IFS="$OLDIFS"

...whenever you alter IFS for anything. If you don't put it back when you're done, that changes how all splitting works later.
This User Gave Thanks to Corona688 For This Post:
# 21  
Old 10-22-2019
I only printed the array to see what values were there and didn't change IFS (I don't think anyway). Thank you Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. OS X (Apple)

Create a bash array from a flat file of whitespaces only.

Hi guys and gals... MacBook Pro. OSX 10.13.2, default bash terminal. I have a flat file 1920 bytes in size of whitespaces only. I need to put every single whitespace character into a bash array cell. Below are two methods that work, but both are seriously ugly. The first one requires that I... (7 Replies)
Discussion started by: wisecracker
7 Replies

2. Shell Programming and Scripting

Bash append values if keywords are present in the file

Hi Team, i have a web ui where user will be passing values and the output will be saved to a file say test with the following contents . These below mentioned values will change according to the user_input Just gave here one example Contents of file test is given below Gateway... (7 Replies)
Discussion started by: venkitesh
7 Replies

3. Shell Programming and Scripting

Ftp with bash, append file where left off

I'm working on a bash script to finish uploading a file. I need a way to get $filesize so that "restart $filesize" will work. Here is my script: ftp -n -v <<END_SCRIPT open ftp.$domain user $user@$domain $password size $file restart $filesize put $file quit END_SCRIPTWayne Sallee... (9 Replies)
Discussion started by: WayneSallee
9 Replies

4. Shell Programming and Scripting

Append awk results into file or array

for a in {1..100} do awk '{ sum+=$a} END {print sum}' a=$a file1 > file2 done I know I will get only one number if following the code above, how can I get 100 sum numbers in file2? (2 Replies)
Discussion started by: wanliushao
2 Replies

5. UNIX for Dummies Questions & Answers

Passing values from file into array in Bash

Hi, I'm trying to write a bash script that takes a file and passes each line from the file into an array with elements separated by column. For example: Sample file "file1.txt": 1 name1 a first 2 name2 b second 3 name3 c third and have arrays such as: line1 = ( "1" "name1" "a"... (3 Replies)
Discussion started by: ShiGua
3 Replies

6. Shell Programming and Scripting

Bash Script Issues (If statement for file copying)

Writing a bash script for use with Geektool, pulls the battery info, and shuffles images around so that an Image geeklet can display the correct expression as the desktop background. (Eventually I intend to make it more intricate, based on more variables, and add more expressions) I'm extremely... (1 Reply)
Discussion started by: The_Ardly374
1 Replies

7. Shell Programming and Scripting

Bash Scripting - How to grep a file into an array

I have figured out how to grep the file like this: echo `grep $(date +'%Y-%m-%d') Cos-01.csv | cut -d',' -f1` The above line does echo the correct information from the lines in which my search criteria is found. Now I am trying to get that information (Yes, just one column of every line) into... (6 Replies)
Discussion started by: TwelveDays
6 Replies

8. Shell Programming and Scripting

Noob's 1st...bash-script for copying one file into many

I have one file "file.a.b.c-d.r" that I would like to use to spawn 4 other files: "file.a.b.1-A.r" "file.a.b.1-B.r" "file.a.b.1-C.r" "file.a.b.1-D.r" where the field "c-d" changes into my 1 and A-D. I was doing this manually at the prompt with > cp "file.a.b.c-d.r" "file.a.b.1-A.r" >... (13 Replies)
Discussion started by: WSUToad
13 Replies

9. Shell Programming and Scripting

Most reliable way to store file contents in an array in bash

Hi Guys, I have a file which has numbers in it separated by newlines as follows: 1.113 1.456 0.556 0.021 -0.541 -0.444 I am using the following code to store these in an array in bash: FILE14=data.txt ARRAY14=(`awk '{print}' $FILE14`) (6 Replies)
Discussion started by: npatwardhan
6 Replies

10. Shell Programming and Scripting

[bash] redirect (save) array to a file

Hi, I'm trying to write a function that redirects the contents of an array to a file. The array contains the lines of a data file with white space. The function seems to preserve all white space when redirected except that it seems to ignore newlines. As a consequence, the elements of the... (7 Replies)
Discussion started by: ASGR
7 Replies
Login or Register to Ask a Question