Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

If contents of A are in B then move the common contents to C

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting If contents of A are in B then move the common contents to C
# 1  
Old 05-07-2015
If contents of A are in B then move the common contents to C
Hallo Team,

I have 2 .csv files file A has 47600 lines and file B has 67000 lines

FILEA
Code:
SD0o9rb01-1d320ddbcc8d220f572739ebed5f58d1-v300g00
SD8bt0101-a0810bfe0e3396060126ec51b30dac0a-v300g00
SD05sce01-cb056af347ed4651f29eb3c3e9addbd6-v300g00
SDp9vv101-4179e27012ee1a250339025b6e7dd628-v300g00
SDud4hd01-ecbf99d937c192b48c96f5daca515769-v300g00
SD2jjk601-ac387dbd7f651c35c772e927d5f73a5f-v300g00
SD7k6g801-50114e1d12e0a1065fd8e26c8e2b7fd3-v300g00
SD5j0fc01-bc6daaf345754791d93acf70f19ec7b8-v300g00
SDal08c01-1c7d08b47eac029b590e3575cb66c661-v300g00
SD23ova01-a3861f173d70f18b0d955a8d9cc8e812-v300g00
SD4lmjf01-8eeeecccb17b4137ec3332cc115dc8d0-v300g00

FILEB

Code:
"SD0op6201-1430204309fffd58c09b1f76a0a6dc25-v300g00","3/1/2015","09:07:24","Unavailable","0873546275","087","5","0.0175","0.21"
"SDpfja101-b65120fdb4c27a155c203efb6126c0e1-v300g00","3/1/2015","13:55:39","0114358297","0873546275","087","5","0.0175","0.21"
"SDgnf6301-0f28db45ad48c4aab02a8dde712cc8f8-v300g00","3/1/2015","13:11:02","0114358297","0873546275","087","5","0.0175","0.21"
"SDifra101-344addc6f19f978b7155d06a0367542c-v300g00","3/1/2015","10:19:41","Unavailable","0873546275","087","14","0.049","0.21"
"SDg7o2201-be65aab507cc26f5aea10af8afa9851b-v300g00","3/1/2015","10:16:38","Anonymous@Anonymous.invalid","0873529097","087","76","0.266","0.21"
"SDg7je101-b2282f1118686ee39719d0a53c00c299-v300g00","3/1/2015","13:39:08","0114358297","0873546275","087","14","0.049","0.21"
"SDn7kq101-b04a267da7bb1b8f29f327fe1423538f-v300g00","3/1/2015","16:52:14","0114358297","0873546275","087","6","0.021","0.21"
"SD0gb4301-695accdb4ff6a9e26be6fc520fa893c3-v300g00","3/1/2015","15:56:57","0114358297","0873546275","087","6","0.021","0.21"
"SDv6n8201-2634518bc8d5df434f4582835a81e27f-v300g00","3/1/2015","09:30:52","Unavailable","0873534205","087","3","0.0105","0.21"
"SDivio201-647a46bc70eda17d0726a8b008eb1156-v300g00","3/1/2015","15:14:03","0114358311","0873546275","087","13","0.0455","0.21"
"SDb7cm101-33fcd3ab871cca2b9f421767af460877-v300g00","3/1/2015","14:42:47","0114358297","0873546275","087","8","0.028","0.21"
"SDkvqc101-3ad4713540402023523dd81da8503eac-v300g00","3/1/2015","10:51:56","Unavailable","0873546275","087","14","0.049","0.21"
"SD20ie201-4d596b87b1684d10d0d15673395c6f6f-v300g00","3/1/2015","16:00:10","0114358311","0873546275","087","11","0.0385","0.21"
"SD20ps001-3a730e8cf102a792219141497c85b797-v300g00","3/1/2015","12:30:07","0114358297","0873546275","087","5","0.0175","0.21"
"SDl7fk101-5d5374409dce12e79d6dee0f40ff329d-v300g00","3/1/2015","17:17:50","0114358311","0873546275","087","14","0.049","0.21"
"SD57hg201-98196ea08a3fac94aa7029767b85155a-v300g00","3/1/2015","13:03:17","0114358311","0873546275","087","14","0.049","0.21"
"SDenea301-8242e990ea455b06fe8779d581a0f0fe-v300g00","3/1/2015","14:06:55","0114358311","0873546275","087","10","0.035","0.21"
"SDumga101-2d768591eb96b411949838bfeca18eea-v300g00","3/1/2015","15:17:46","0114358297","0873546275","087","14","0.049","0.21"
"SDmfhc201-84d4a6e39b5410bc64d556419fdc7bfa-v300g00","3/1/2015","13:51:18","0114358311","0873546275","087","14","0.049","0.21"
"SD20bk101-3a3884a936427fb9f039e803d976fdc5-v300g00","3/1/2015","15:59:20","0114358297","0873546275","087","9","0.0315","0.21"
"SD7serd01-6afc6a5202347b24cdfad11c89507cd3-v300g00","3/1/2015","07:38:15","0219188500","0214881822","D850","194","0.679","0.21"
"SD7k95d01-532a804796ec110cc14adb01aa93dba3-v300g00","3/1/2015","08:34:13","0219188500","0214881822","D850","187","0.6545","0.21"
"SDhpd5401-9ece59dc2c72e47bb50ab8067d21bac2-v300g00","3/1/2015","08:48:06","0874901777","0112157000","D850","114","0.399","0.21"

This is the logic that i want my output to follow.

Everything that is in File A and matches FileB should be placed in file C. File C should have all the columns that are in File B.

I hope this makes sense.

Regards,

Paxley
Last edited by vgersh99; 05-07-2015 at 09:06 AM.. Reason: fixed code tags
# 2  
Old 05-07-2015
Frequently asked question, frequently given answer:
Code:
fgrep -f FILEA FILEB > FILEC

Faster and more precise: search only in the first column:
Code:
awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' FILEA FILEB > FILEC

# 3  
Old 05-07-2015
Thank you MadeInGermany,

Code:
fgrep -f FILEA FILEB > FILEC

-- somehow this does not work

Code:
awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' FILEA FILEB > FILEC

-- this is not that fast i have been waiting for the command to complete for the last 5 minutes and the command is loves memory between 80-90% CPU usage.

Code:
awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' mweb_mediation.csv mweb_rev1.csv > mweb_inbound_march2015.csv

-- that is how I ran my command
Last edited by rbatte1; 05-07-2015 at 01:27 PM.. Reason: code tags, please!
# 4  
Old 05-07-2015
awk 'FNR==NR {fa[$1];next} $2 in fa' FILEA FS='"' FILEB > FILEC
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Best way to move the contents of a folder to another one

what is the best way to move the contents of a folder to another one without deleting the structure of the first one. the contents could include subfolder too. both folder, the source-folder and the target-folder are on the same host. any idea is appreciated . (7 Replies)
Discussion started by: andy2000
7 Replies

2. Shell Programming and Scripting

Folder contents getting appended as strings while redirecting file contents to a variable

Hi one of the output of the command is as below # sed -n "/CCM-ResourceHealthCheck:/,/---------/{/CCM-ResourceHealthCheck:/d;/---------/d;p;}" Automation.OutputZ$zoneCounter | sed 's/$/<br>/' Resource List : <br> *************************** 1. row ***************************<br> ... (2 Replies)
Discussion started by: vivek d r
2 Replies

3. Shell Programming and Scripting

Move parent folder if contents match a parameter

Hi all, Hoping someone can help. I'm looking to be able to search a particular folder path for folders that contain certain files / formats and then move the parent folder. eg. /foo/bar/folder1/file.exe /foo/bar/folder2/file.exe and then move the folder1/2 tp another folder. So if... (1 Reply)
Discussion started by: springs2
1 Replies

4. UNIX for Dummies Questions & Answers

Move Directory & Contents Between Two Machines

Hi All, I have a large amount of files that I need to copy from one server to another server using SFTP. Can comeone please help me with the command I would use here? Here is what I am thinking, but being new at this I know this is probably wrong: Login to the destination host using... (1 Reply)
Discussion started by: SalientAnimal
1 Replies

5. Shell Programming and Scripting

finding common numbers (contents) across 2 or 3 files

I have 3 files which are tab delimited and have numbers in it. file 1 1 2 3 4 5 6 7 File 2 3 5 7 8 File 3 1 (4 Replies)
Discussion started by: Lucky Ali
4 Replies

6. Shell Programming and Scripting

move contents from one file to another based on line number or content

I want a script that will move everything beyond a certain line number or beyond a certain content word into another file. For example, if file A has this: first line second line third line forth line fifth line sixth line I want to run a script that will move everything beyond the third... (4 Replies)
Discussion started by: robp2175
4 Replies

7. Shell Programming and Scripting

common contents of two files

I have two files: file a with contents 1 2 3 4 5 file b with contents 6 3 5 8 9 10 i want go get file c which has the common contents of both files so file c should have contents 3 5 (9 Replies)
Discussion started by: tomjones
9 Replies

8. UNIX for Dummies Questions & Answers

compare 2 file contents , if same delete 2nd file contents

Give shell script....which takes two file names as input and compares the contents, is both are same delete second file's contents..... I try with "diff"...... but confusion how to use "diff" with if ---else Thanking you (5 Replies)
Discussion started by: krishnampkkm
5 Replies

9. UNIX for Dummies Questions & Answers

Move contents of a directory into one file for e-mail distribution ...

Hello, Here is what I am trying to accomplish. I am going to have one directory in which there will be files of varying types (Excel, Word, PPT, and possible others), and I need to be able to be bundle however many files there are in there together in to one file to be used as an e-mail... (3 Replies)
Discussion started by: rip73
3 Replies

10. Shell Programming and Scripting

Join file contents via common field

I have 2 files with a common parm - Jobname File 1 0507 1202 JOBA 0507 1302 JOBB 0507 1452 JOBC 0507 1552 JOBA 0507 1553 JOBA File2 JOBA abcdefg server4 JOBB defghij server22 JOBC vwxyz12 server55 I would like to take each line from File1 and match the jobname with the jobname... (8 Replies)
Discussion started by: Northerner
8 Replies
Login or Register to Ask a Question