Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filtering files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filtering files
# 8  
Old 05-08-2012
Code:
awk 'NR==FNR { A[$1]++; O[++L]=$1; next }; $1 in A { A[$1]=$0 }; END { for(N=1; N<=L; N++) print O[N], A[O[N]]; }' names.txt masterfile.txt > output.txt

This User Gave Thanks to Corona688 For This Post:
alecapo
# 9  
Old 05-08-2012
Thanks @Corona688!. Amazing..
# 10  
Old 06-08-2012
Hi guys, I need your help once again
So I have this master.txt file with thousands of markers (tab separated cells):

Code:
A1  11   33   13   13   11       
A2  11   11   11   11   11       
A3  11   11   11   11   11        
A4    0   12   12   12   12       
A5  33   13   13   13   13       
A6   11   13   13   11   13       
A7   11   11   11   13   11       
A8   13   11   11   11   11       
A9   11   11   11   11   11       
A10  11   12   12   22   11       
A11  13   33   13   11   11       
A12  11   11   13   11   11

And I want to remove some markers from this list i.e having this file with their names: names.txt
Code:
A1
A2
A3

So the final.txt should be

Code:
 A4    0   12   12   12   12       
A5  33   13   13   13   13       
A6   11   13   13   11   13       
A7   11   11   11   13   11       
A8   13   11   11   11   11       
A9   11   11   11   11   11       
A10  11   12   12   22   11       
A11  13   33   13   11   11       
A12  11   11   13   11   11

Thanks!
# 11  
Old 06-08-2012
That's much easier:
Code:
awk -F"\t" 'NR==FNR { A[$1]++; next }; !($1 in A)' names.txt master.txt

This User Gave Thanks to Corona688 For This Post:
alecapo
# 12  
Old 06-08-2012
Works great, thanks man!

---------- Post updated at 02:02 PM ---------- Previous update was at 01:54 PM ----------

Sorry to bother you again, but from the previous code to separate by names, Is is possible to edit it so it wont leave the blank spaces in those markers that were not found?
At the beginning I thought it was Ok but its becoming very time consuming to remove all those blank cells.
# 13  
Old 06-08-2012
Try this:
Code:
awk 'NR==FNR { A[$1]++; O[++L]=$1; next }; $1 in A { A[$1]=$0 }; END { for(N=1; N<=L; N++) if(O[N] in A) print O[N], A[O[N]]; }' names.txt masterfile.txt > output.txt

# 14  
Old 06-08-2012
Mm no it didnt work. still looks the same.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help on filtering

Hi experts, I have a file image.csv as below: COMPUTERNAME,23/07/2013,22/07/2013,21/07/2013,20/07/2013,19/07/2013,18/07/2013,17/07/2013 AED03852180,3,3,3,3,3,3,3 AED03852181,3,3,3,3,3,3,1 AED09020382,3,0,3,0,3,3,3 AED09020383,1,3,3,3,2,1,3 AED09020386,3,3,0,3,3,0,3 ... (4 Replies)
Discussion started by: zaq1xsw2
4 Replies

2. Shell Programming and Scripting

Reading 2 CSV files and filtering data based on group

I have two CSV files in the following format: First file: GroupID, PID:TID, IP, Port Sample data: 0,1000:11,127.0.0.1,445 0,-1:-1,127.0.0.1,800 1,1000:11,127.0.0.1,445 1,-1:-1,127.0.0.1,900 2,1000:11,127.0.0.1,445 2,-1:-1,180.0.0.3,900 Second file: IP,Port,PID Sample data... (6 Replies)
Discussion started by: rakesh_arxmind
6 Replies

3. Shell Programming and Scripting

Filtering files

Hi all, I have some files with different extensions. I want to list the files that doesnt end with particular extension for eg .txt. I want to list all files except .txt. How can I do the same? Thanks Ananth (2 Replies)
Discussion started by: Ananthdoss
2 Replies

4. Programming

need help with shell script filtering files and sort! newbie question?

Hi folks, I would like to get familiar with shell script programing. The first task is: write a shell script that: scans your home-folder + sub-directory for all txt-files that all users of your group are allowed to read and write then output these files sorted by date of last... (4 Replies)
Discussion started by: rollinator
4 Replies

5. Shell Programming and Scripting

Please help me to do some filtering

I have to grep a pattern. scenario is like :- Suppose "/etc/sec/one" is a string, i need to check if this string contains "one" using any utility something like if /etc/sec/one | grep ; then Thanks in advance Renjesh Raju (3 Replies)
Discussion started by: Renjesh
3 Replies

6. Shell Programming and Scripting

Filtering the yesterdays date from log files via script.

hi All, I have this sample text file - access.log: Jan 18 21:34:29 root 209.151.232.70 Jan 18 21:34:40 root 209.151.232.70 Jan 18 21:34:43 root 209.151.232.70 Jan 18 21:34:56 root 209.151.232.70 Jan 18 21:35:10 root 209.151.232.70 Jan 18 21:35:23 root 209.151.232.70 Jan 18 21:36:04 root... (2 Replies)
Discussion started by: linuxgeek
2 Replies

7. Shell Programming and Scripting

Filtering multiple files with variables

Hi, I spend few hours already searching this forum, but cannot find the solution matching exactly my case. I have multiple log files, I need to filter them so I get info about certain event. So we have files: LOGA.txt LOGB.txt LOGC.txt LOGD.txt LOGE.txt 1. I need to grep lines in... (10 Replies)
Discussion started by: Vitoriung
10 Replies

8. Shell Programming and Scripting

Indexing or Filtering code- Pattern Search by comparing two files

So here is goes to the Gurus of shell programming......I have tried a lot of different ways and its a very challenging code to write but i am enjoying it as i troubleshoot and hopefully someone can provide me a better option....Thank you in advance for your time and support....Much appreciated... ... (12 Replies)
Discussion started by: aavam
12 Replies

9. UNIX for Dummies Questions & Answers

Filtering pcap files

Hi, I am new at UNIX and programing in general and only have a basic knowledge of C++. I am helping out with some research at a college and was given the task to sort through captured packets via IP addresses. I was wondering if anyone could help me with writing a code which filters through pcap... (1 Reply)
Discussion started by: hershey101
1 Replies

10. Shell Programming and Scripting

Merging files with AWK filtering and counting lines

Hi there, I have a couple of files I need to merge. I can do a simple merge by concatenating them into one larger file. But then I need to filter the file to get a desired result. The output looks like this: TRNH 0000000010941 ORDH OADR OADR ORDL ENDT 1116399 000000003... (2 Replies)
Discussion started by: Meert
2 Replies
Login or Register to Ask a Question