Sponsored Content
Top Forums Shell Programming and Scripting Complex comparison of 3 files Post 302850549 by MR.bean on Thursday 5th of September 2013 03:02:20 AM
Old 09-05-2013
Another one

Code:
bash-3.2$ cat f1
547680
575210
804270
123989
623989
221209
bash-3.2$ cat f2
1|4501892|547680|1|2|30|73491|12|34|1
2|4788930|575210|1|2|30|73472|12|34|1
3|6793773|804270|1|2|30|73420
4|6673724|123989|1|2|30|73001|12|34|1
5|8099821|333722|1|30|73473|10|34|1
6|7889200|623989|1|2|30|73001|12|45|1
7|8882662|221209|1|2|30|83002|12|34|1
bash-3.2$ 
bash-3.2$ cat f3
1|2|30|73472|12|34|1
1|2|30|73001|12|34|1
1|2|30|83002|12|34|1
bash-3.2$ 
bash-3.2$ awk 'NR==FNR {  x[$0]++; next; } { m=0; for(i in x) { regexp=gensub(/\|/, "\\\\\\|", "g", i); if(match($0, regexp)) { m++; }} printf "%s", $0; print m ? ",C" : ",D" } ' f3 <(join -t'|' -2 3 f1 f2 | cut -d'|' -f 1,4-)
547680|1|2|30|73491|12|34|1,D
575210|1|2|30|73472|12|34|1,C
804270|1|2|30|73420,D
123989|1|2|30|73001|12|34|1,C
623989|1|2|30|73001|12|45|1,D
221209|1|2|30|83002|12|34|1,C


Last edited by MR.bean; 09-05-2013 at 04:11 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with complex merg of files with common field

Please help, I am new to shell Programming. I have three files each containg a unique text (key) field (e.g. ABCDEF, XCDUD as shown below), line return followed by some data of which there can be more then one instance. In addition, in some cases there may be no data but only a key field. Please... (18 Replies)
Discussion started by: gugs
18 Replies

2. Shell Programming and Scripting

comparison of 2 files

Kindly help on follows. I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Discussion started by: kamal_418
2 Replies

3. Shell Programming and Scripting

sed: How to modify files in a complex way

Hello, I am new to sed and hope that someone can help me with the following task. I need to modify a txt file which has format like this: xy=CreateDB|head.queue|head.source|head.definition|rtf.edit|rtf.task|rft.cut abc|source|divine|line4|5|true into something like: head.queue=abc... (19 Replies)
Discussion started by: pinkypunky
19 Replies

4. Shell Programming and Scripting

Joining files in a complex way

if input1 1st row labels (S1or S2 or S3 or any (actually so many in original text file)) are similar to 1st column of input2 i.e "ID" merge them together based on input1 1st row labels. for example take S1..... input1 "aphab" "S1" "S2" "S3" "a" "A/A" "A/A" "A/A" "b" ... (19 Replies)
Discussion started by: stateperl
19 Replies

5. Shell Programming and Scripting

Comparison of two files (sh)

Hi, I have a problem with comparison of two files file1 20100101 20090101 20080101 20071001 20121229 file2 19990112 12 456 7 20011131 19 20100101 2 567 1 987 17890709 123 555 and, sh script needs to compare of these two files and give out to me result: 20100101 2 567 1 987 it... (5 Replies)
Discussion started by: shizik
5 Replies

6. Shell Programming and Scripting

mass renaming files with complex filenames

Hi, I've got files with names like this : _Some_Name_178_HD_.mp4 _Some_Name_-_496_Vost_SD_(720x400_XviD_MP3).avi Goffytofansub_Some name 483_HD.avi And iam trying to rename it with a regular pattern. My gola is this : Ep 178.mp4 Ep 496.avi Ep 483.avi I've tried using sed with... (8 Replies)
Discussion started by: VLaw
8 Replies

7. Shell Programming and Scripting

Complex renaming then moving files

I am a biologist and using an program on a computer cluster that generates a lot of data. The program creates a directory named ExperimentX (where X is a number) that contains files "out.pdb" and "log.txt". I would like to create a script that renames the out.pdb file to out_ExperimentX.pdb (or... (1 Reply)
Discussion started by: yaledocker
1 Replies

8. Shell Programming and Scripting

Complex data sorting in excel files or text files

Dear all, I have a complex data file shown below,,,,, A_ABCD_13208 0 0 4.16735 141044 902449 1293900 168919 C_ABCD_13208 0 0 4.16735 141044 902449 1293900 168919 A_ABCDEF715 52410.9 18598.2 10611 10754.7 122535 252426 36631.4 C_DBCDI_1353 0... (19 Replies)
Discussion started by: AAWT
19 Replies

9. Shell Programming and Scripting

Comparison of two files

Hi all I have two files which I have to compare that whetehr there is soemthing common or not body, div, table, thead, tbody, tfoot, tr, th, td, p { font-family: "Liberation Sans"; font-size: x-small; } body, div, table, thead, tbody, tfoot,... (2 Replies)
Discussion started by: manigrover
2 Replies

10. Shell Programming and Scripting

Comparison of files

I have the requirement I have two files cat fileA something anythg nothing everythg cat fileB everythg anythg Now i shld use fileB and compare every line at fileA and get the output as something nothing (3 Replies)
Discussion started by: Priya Amaresh
3 Replies
PMDABASH(1)						      General Commands Manual						       PMDABASH(1)

NAME
pmdabash - Bourne-Again SHell trace performance metrics domain agent SYNOPSIS
$PCP_PMDAS_DIR/bash/pmdabash [-C] [-d domain] [-l logfile] [-I interval] [-t timeout] [-U username] configfile DESCRIPTION
pmdabash is an experimental Performance Metrics Domain Agent (PMDA) which exports "xtrace" events from a traced bash(1) process. This includes the command execution information that would usually be sent to standard error with the set -x option to the shell. Event metrics are exported showing each command executed, the function name and line number in the script, and a timestamp. Additionally, the process identifier for the shell and its parent process are exported. This requires bash version 4 or later. A brief description of the pmdabash command line options follows: -d It is absolutely crucial that the performance metrics domain number specified here is unique and consistent. That is, domain should be different for every PMDA on the one host, and the same domain number should be used for the same PMDA on all hosts. -l Location of the log file. By default, a log file named bash.log is written in the current directory of pmcd(1) when pmdabash is started, i.e. $PCP_LOG_DIR/pmcd. If the log file cannot be created or is not writable, output is written to the standard error instead. -s Amount of time (in seconds) between subsequent evaluations of the shell trace file descriptor(s). The default is 2 seconds. -m Maximum amount of memory to be allowed for each event queue (one per traced process). The default is 2 megabytes. -U User account under which to run the agent. The default is the unprivileged "pcp" account in current versions of PCP, but in older versions the superuser account ("root") was used by default. INSTALLATION
In order for a host to export the names, help text and values for the bash performance metrics, do the following as root: # cd $PCP_PMDAS_DIR/bash # ./Install As soon as an instrumented shell script (see INSTRUMENTATION selection below) is run, with tracing enabled, new metric values will appear - no further setup of the agent is required. If you want to undo the installation, do the following as root: # cd $PCP_PMDAS_DIR/bash # ./Remove pmdabash is launched by pmcd(1) and should never be executed directly. The Install and Remove scripts notify pmcd(1) when the agent is installed or removed. INSTRUMENTATION
In order to allow the flow of event data between a bash(1) script and pmdabash, the script should take the following actions: #!/bin/sh source $PCP_DIR/etc/pcp.sh pcp_trace on $@ # enable tracing echo "awoke, $count" pcp_trace off # disable tracing The tracing can be enabled and disabled any number of times by the script. On successful installation of the agent, several metrics will be available: $ pminfo bash bash.xtrace.numclients bash.xtrace.maxmem bash.xtrace.queuemem bash.xtrace.count bash.xtrace.records bash.xtrace.parameters.pid bash.xtrace.parameters.parent bash.xtrace.parameters.lineno bash.xtrace.parameters.function bash.xtrace.parameters.command When an instrumented script is running, the generation of event records can be verified using the pmevent(1) command, as follows: $ pmevent -t 1 -x '' bash.xtrace.records host: localhost samples: all bash.xtrace.records["4538 ./test-trace.sh 1 2 3"]: 5 event records 10:00:05.000 --- event record [0] flags 0x19 (point,id,parent) --- bash.xtrace.parameters.pid 4538 bash.xtrace.parameters.parent 4432 bash.xtrace.parameters.lineno 43 bash.xtrace.parameters.command "true" 10:00:05.000 --- event record [1] flags 0x19 (point,id,parent) --- bash.xtrace.parameters.pid 4538 bash.xtrace.parameters.parent 4432 bash.xtrace.parameters.lineno 45 bash.xtrace.parameters.command "(( count++ ))" 10:00:05.000 --- event record [2] flags 0x19 (point,id,parent) --- bash.xtrace.parameters.pid 4538 bash.xtrace.parameters.parent 4432 bash.xtrace.parameters.lineno 46 bash.xtrace.parameters.command "echo 'awoke, 3'" 10:00:05.000 --- event record [3] flags 0x19 (point,id,parent) --- bash.xtrace.parameters.pid 4538 bash.xtrace.parameters.parent 4432 bash.xtrace.parameters.lineno 47 bash.xtrace.parameters.command "tired 2" 10:00:05.000 --- event record [4] flags 0x19 (point,id,parent) --- bash.xtrace.parameters.pid 4538 bash.xtrace.parameters.parent 4432 bash.xtrace.parameters.lineno 38 bash.xtrace.parameters.function "tired" bash.xtrace.parameters.command "sleep 2" FILES
$PCP_PMCDCONF_PATH command line options used to launch pmdabash $PCP_PMDAS_DIR/bash/help default help text file for the bash metrics $PCP_PMDAS_DIR/bash/Install installation script for the pmdabash agent $PCP_PMDAS_DIR/bash/Remove undo installation script for pmdabash $PCP_LOG_DIR/pmcd/bash.log default log file for error messages and other information from pmdabash PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to parameterize the file and directory names used by PCP. On each installation, the file /etc/pcp.conf contains the local values for these variables. The $PCP_CONF variable may be used to specify an alternative configura- tion file, as described in pcp.conf(5). SEE ALSO
bash(1), pmevent(1) and pmcd(1). Performance Co-Pilot PCP PMDABASH(1)
All times are GMT -4. The time now is 06:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy