06-25-2015
A few comments may lead you in the right direction but can't replace an in depth reading of e.g. man awk plus a lot of experimenting with samples.
Arrays don't represent files, nor do the field variables $3, $1 etc. The latter hold the single fields of the actual line read from the data stream which in turn can consist of several files. Most arrays are user defined and created on first reference.
So - the trick is to find out which file is operated on when. There's the FILENAME system variable that changes with the actual file. And - there's the NR and FNR variables. NR is the record (= line) No. in the entire stream, FNR the same but in the actual file, reset to 1 when the actual file changes. So NR==FNR only for the first file, and Scrutinizer uses this to load the A (and B) arrays. If this is no more true, we must have left the first file, and we can check if e.g. $1 (of the second file) can be found as an index into the A array, and, if yes, modify the actual line as requested.
These 3 Users Gave Thanks to RudiC For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I have a number of tab delimited data files consists of two columns. Like that:
File1
800.000000 0.002744
799.000000 0.002517
798.000000 0.002836
797.000000 0.002553
FIle2
800.000000 0.000261
799.000000 0.000001
798.000000 0.000551
797.000000 0.000275
File3... (19 Replies)
Discussion started by: erden
19 Replies
2. UNIX for Dummies Questions & Answers
Hi there,
I'm trying to merge two files and make a third file.
However, two of the columns need to match exactly in both files AND I want everything from both files in the output if the two columns match in that row.
First file looks like this:
chr1 10001980 T A
Second... (12 Replies)
Discussion started by: infiniteabyss
12 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I have two text files that I would like to merge/join. I would like to join them if the first columns of both text files match and the second column of the first text file matches the third column of the second text file.
Example input:
First file:
1334 10 0 0 1 5.2
1334 12 0 0 1 4.5... (4 Replies)
Discussion started by: evelibertine
4 Replies
4. Shell Programming and Scripting
Dear List,
I have a file of csv data which has a different line per compliance check per host. I do not want any omissions from this csv data file which looks like this:
date,hostname,status,color,check
02-03-2012,COMP1,FAIL,Yellow,auth_pass_change... (3 Replies)
Discussion started by: landossa
3 Replies
5. Shell Programming and Scripting
I have two files.
FileA.txt
30910 rs7468327
36587 rs10814410
91857 rs9408752
105797 rs1133715
146659 rs2262038
152695 rs2810979
181843 rs3008128
182129 rs3008131
192118 rs3008170
FileB.txt
30910 1.9415219673 0
36431 1.3351312477 0.0107191428
36587 1.3169171182... (2 Replies)
Discussion started by: genehunter
2 Replies
6. Shell Programming and Scripting
Hi.
I have 2 files of below format.
File1
AA~1~STEVE~3.1~4.1~5.1
AA~2~DANIEL~3.2~4.2~5.2
BB~3~STEVE~3.3~4.3~5.3
BB~4~TIM~3.4~4.4~5.4
File 2
AA~STEVE~AA STEVE WORKS at AUTO COMPANY
AA~DANIEL~AA DANIEL IS A ELECTRICIAN
BB~STEVE~BB STEVE IS A COOK
I want to match 1st and 3rd... (2 Replies)
Discussion started by: crypto87
2 Replies
7. Shell Programming and Scripting
Hello guys,
I have 2 CSV files which goes like this:
CSV1:
Breaking.csv:
UTF-8
"Name","Description","Occupation","Email"
"Walter White","","Chemistry Teacher","w.w@bb.com"
"Jessie Pinkman","","Junkie","j.p@bb.com"
"Hank Schrader","","DEA Agent","h.s@bb.com"
CSV2:
Bad.csv... (7 Replies)
Discussion started by: jeffreybsu
7 Replies
8. Shell Programming and Scripting
Respected Members.
Hello.
This is my first post in the forum. I will try to follow all the rules as prescribed by the forum. In case of non-compliance, I request you to kindly give me some more time to understand and abide by them.
I am working on two files. I wish to merge the two files... (1 Reply)
Discussion started by: manojmalhotra
1 Replies
9. Shell Programming and Scripting
Respected Members.
Hello.
This is my first post in the forum. I will try to follow all the rules as prescribed by the forum. In case of non-compliance, I request you to kindly give me some more time to understand and abide by them.
I am working on two files. I wish to merge the two files... (6 Replies)
Discussion started by: manojmalhotra
6 Replies
10. UNIX for Beginners Questions & Answers
Hi,
i have 2 files , the data i need to match is in masterfile and i need to pull out column 3 from master if column 1 and 2 match and output entire row to new file
I have tried with join and awk and i keep getting blank outputs or same file
is there an easier way than what i am... (4 Replies)
Discussion started by: axis88
4 Replies
LEARN ABOUT DEBIAN
parallel-slurp
PARALLEL-SLURP(1) PARALLEL-SLURP(1)
NAME
parallel-slurp - copy files from listed hosts
SYNOPSIS
parallel-slurp [OPTIONS] -h hosts.txt -L destdir remote local
DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
homogenously-configured hosts.
parallel-slurp gathers specified files from hosts you listed.
OPTIONS
-r --recursive
recusively copy directories (OPTIONAL)
-L --localdir
output directory for remote file copies
-h --hosts
hosts file (each line "host[:port] [user]")
-l --user
username (OPTIONAL)
-p --par
max number of parallel threads (OPTIONAL)
-o --outdir
output directory for stdout files (OPTIONAL)
-e --errdir
output directory for stderr files (OPTIONAL)
-t --timeout
timeout (secs) (-1 = no timeout) per host (OPTIONAL)
-O --options
SSH options (OPTIONAL)
-v --verbose
turn on warning and diagnostic messages (OPTIONAL)
EXAMPLE
An example to copy /home/irb2/foo.txt from each host. Files gathered will be stored in /tmp/outdir/hostname/foo.txt.
# prallel-slurp -h hosts.txt -L /tmp/outdir -l irb2
/home/irb2/foo.txt foo.txt
ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables:
o PSSH_HOSTS
o PSSH_USER
o PSSH_PAR
o PSSH_OUTDIR
o PSSH_VERBOSE
o PSSH_OPTIONS
SEE ALSO
parallel-ssh(1), parallel-scp(1), parallel-nuke(1), parallel-rsync(1), ssh(1)
AUTHOR
Brent N. Chun <bnc@theether.org>
COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun
NOTES
1. bnc@theether.org
mailto:bnc@theether.org
03/30/2009 PARALLEL-SLURP(1)