01-20-2011
Finding and Extracting uniq data in multiple files
Hi,
I have several files that look like this:
File1.txt
HTML Code:
Data1
Data2
Data20
File2.txt
HTML Code:
Data1
Data5
Data10
File3.txt
HTML Code:
Data1
Data2
Data17
File4.txt
HTML Code:
Data2
Data5
Data12
Data30
What I need is a piece of code (bash/awk) that will check and compare the data of each file and will return each file only with the UNIQ data. For example, for the above data the expected output will be:
File1.txt
File2.txt
File3.txt
File4.txt
If there are multiple UNIQ data in a file, it will only give the first UNIQ data. As 'File4.txt' has two UNIQ data: 'Data12 and Data30', but it'll give the first one as :
File4.txt
Thanks in advance for your help.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
frnds,
I m having prob woth doing some 2-3 task simultaneously...
what I want is...
I have lots ( lacs ) of files in a dir...
I want.. these info from arround 2-3 months files
filename convention is - abc20080403sdas.xyz ( for todays files )
I want
1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies
2. Shell Programming and Scripting
Hello,
I have a log file that has following output as below.
LAP.sun5 CC
LAP.sun5 CQ
perl.sun5 CC
perl.sun5 CQ
TSLogger.sun5 CC
TSLogger.sun5 CQ
TSLogger.sun5 KR
WAS.sun5 CC
WAS.sun5 MT
WAS.sun5 CQ
I want to output to be in the way below, i tried using awk but could not do it. ... (12 Replies)
Discussion started by: asirohi
12 Replies
3. UNIX for Dummies Questions & Answers
I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies
4. Shell Programming and Scripting
Hi Everyone,
I'm really new to all this so I'm really hoping someone can help. I have a directory with ~1000 lists from which I want to extract lines from and write to new files. For simplicity lets say they are shopping lists and I want to write out the lines corresponding to apples to a new... (2 Replies)
Discussion started by: born2phase
2 Replies
5. UNIX for Dummies Questions & Answers
I need help extracting specific lines in a text file. The file looks like this:
POSITION TOTAL-FORCE (eV/Angst)
-----------------------------------------------------------------------------------
1.86126 1.86973 1.86972 ... (14 Replies)
Discussion started by: captainalright
14 Replies
6. Shell Programming and Scripting
Hey,
I have number of .tgz files and want to extract the file with the ending *results.txt from each one.
I have tried
for file in *.tgz; do tar --wildcards -zxf $file *results.txt; doneas well as
list=$(ls *.tgz)
for i in $list; do tar --wildcards -zxvf $i *.results.txt; done... (1 Reply)
Discussion started by: jfern
1 Replies
7. Shell Programming and Scripting
File 1
ID Name Po1 Po2
DD134 DD134_4A_1 NN-1 L_0_1
DD134 DD134_4B_1 NN-2 L_1_1
DD134 DD134_4C_1 NN-3 L_2_1
DD142 DD142_4A_1 NN-1 L_0_1
DD142 DD142_4B_1 NN-2 L_1_1
DD142 DD142_4C_1 NN-3 L_2_1
DD142 DD142_3A_1 NN-41 L_3_1
DD142 DD142_3A_1 NN-42 L_3_2
File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies
8. Shell Programming and Scripting
Hi All,
I am stuck in one step..
I have one file named file.txt having content:
And SGMT.perd_id = (SELECT cal.fiscal_perd_id FROM $ODS_TARGT.TIM_DT_CAL_D CAL
FROM $ODS_TARGT.GL_COA_SEGMNT_XREF_A SGMT
SGMT.COA_XREF_TYP_IDN In (SEL COA_XREF_TYP_IDN From... (4 Replies)
Discussion started by: Shilpi Gupta
4 Replies
9. Shell Programming and Scripting
I have a series of csv files in the following format
eg file1
Experiment Name,XYZ_07/28/15,
Specimen Name,Specimen_001,
Tube Name, Control,
Record Date,7/28/2015 14:50,
$OP,XYZYZ,
GUID,abc,
Population,#Events,%Parent
All Events,10500,
P1,10071,95.9
Early Apoptosis,1113,11.1
Late... (6 Replies)
Discussion started by: pawannoel
6 Replies
10. Shell Programming and Scripting
Hi All,
I have log files as below.
log1.txt
<table name="content_analyzer" primary-key="id">
<type="global" />
</table>
<table name="content_analyzer2" primary-key="id">
<type="global" />
</table>
Time taken: 1.008 seconds
ID = gd54321bbvbvbcvb
<table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies