Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Finding and Extracting uniq data in multiple files Post 302489262 by Fahmida on Thursday 20th of January 2011 03:10:33 AM
Old 01-20-2011
Finding and Extracting uniq data in multiple files

Hi,

I have several files that look like this:
File1.txt
HTML Code:
Data1
Data2
Data20
File2.txt
HTML Code:
Data1
Data5
Data10
File3.txt
HTML Code:
Data1
Data2
Data17
File4.txt
HTML Code:
Data2
Data5
Data12
Data30
What I need is a piece of code (bash/awk) that will check and compare the data of each file and will return each file only with the UNIQ data. For example, for the above data the expected output will be:
File1.txt
HTML Code:
Data20
File2.txt
HTML Code:
Data10
File3.txt
HTML Code:
Data17
File4.txt
HTML Code:
Data12
If there are multiple UNIQ data in a file, it will only give the first UNIQ data. As 'File4.txt' has two UNIQ data: 'Data12 and Data30', but it'll give the first one as :
File4.txt
HTML Code:
 Data12
Thanks in advance for your help.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extracting data from files..

frnds, I m having prob woth doing some 2-3 task simultaneously... what I want is... I have lots ( lacs ) of files in a dir... I want.. these info from arround 2-3 months files filename convention is - abc20080403sdas.xyz ( for todays files ) I want 1. total no of files for 1 dec... (1 Reply)
Discussion started by: clx
1 Replies

2. Shell Programming and Scripting

Modify log files to get uniq data

Hello, I have a log file that has following output as below. LAP.sun5 CC LAP.sun5 CQ perl.sun5 CC perl.sun5 CQ TSLogger.sun5 CC TSLogger.sun5 CQ TSLogger.sun5 KR WAS.sun5 CC WAS.sun5 MT WAS.sun5 CQ I want to output to be in the way below, i tried using awk but could not do it. ... (12 Replies)
Discussion started by: asirohi
12 Replies

3. UNIX for Dummies Questions & Answers

Extracting data from many compressed files

I have a large number (50,000) of pretty large compressed files and I need only certain lines of data from them (each relevant line contains a certain key word). Each file contains 300 such lines. The individual file names are indexed by file number (file_name.1, file_name.2, ... ,... (1 Reply)
Discussion started by: Boltzmann
1 Replies

4. Shell Programming and Scripting

Extracting/condensing text from multiple files to multiples files

Hi Everyone, I'm really new to all this so I'm really hoping someone can help. I have a directory with ~1000 lists from which I want to extract lines from and write to new files. For simplicity lets say they are shopping lists and I want to write out the lines corresponding to apples to a new... (2 Replies)
Discussion started by: born2phase
2 Replies

5. UNIX for Dummies Questions & Answers

Extracting data between specific lines, multiple times

I need help extracting specific lines in a text file. The file looks like this: POSITION TOTAL-FORCE (eV/Angst) ----------------------------------------------------------------------------------- 1.86126 1.86973 1.86972 ... (14 Replies)
Discussion started by: captainalright
14 Replies

6. Shell Programming and Scripting

Extracting specific files from multiple .tgz files

Hey, I have number of .tgz files and want to extract the file with the ending *results.txt from each one. I have tried for file in *.tgz; do tar --wildcards -zxf $file *results.txt; doneas well as list=$(ls *.tgz) for i in $list; do tar --wildcards -zxvf $i *.results.txt; done... (1 Reply)
Discussion started by: jfern
1 Replies

7. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

8. Shell Programming and Scripting

Extracting data from multiple lines

Hi All, I am stuck in one step.. I have one file named file.txt having content: And SGMT.perd_id = (SELECT cal.fiscal_perd_id FROM $ODS_TARGT.TIM_DT_CAL_D CAL FROM $ODS_TARGT.GL_COA_SEGMNT_XREF_A SGMT SGMT.COA_XREF_TYP_IDN In (SEL COA_XREF_TYP_IDN From... (4 Replies)
Discussion started by: Shilpi Gupta
4 Replies

9. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

10. Shell Programming and Scripting

Extracting part of data from files

Hi All, I have log files as below. log1.txt <table name="content_analyzer" primary-key="id"> <type="global" /> </table> <table name="content_analyzer2" primary-key="id"> <type="global" /> </table> Time taken: 1.008 seconds ID = gd54321bbvbvbcvb <table name="content_analyzer"... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies
aha(1)								 Ansi HTML Adapter							    aha(1)

NAME
aha - Ansi HTML Adapter SYNOPSIS
aha [options] [-f datei] DESCRIPTION
aha takes SGR-colored Input and prints W3C conform HTML-Code. aha reads the Input from a file or stdin and writes HTML-Code to stdout. OPTIONS
--help , -h , -? A help like this --black , -b Black Background and white "standard color" --pink , -p Pink Background --iso X , -i X Uses ISO 8859-X instead of utf-8. X must be 1..16 --title X , -t X Gives the html output the title --line-fix , -l Uses a fix for inputs using control sequences to change the cursor position like htop. It's a hot fix, it may not work with any pro- gram like htop. (See EXAMPLE) EXAMPLE
aha --help | aha --black --title "the awesome aha help"> aha-help.htm Creates an HTML file with the help of aha with black background colordiff oldfile.c newfile.c | aha > colordiff.htm Creates an HTML file with a colorful diff-output of two files "oldfile.c" and "newfile.c" with white background ls --color=always | aha --pink > ls.htm Creates an HTML file with a colorful ls-output with pink background. echo a | htop | aha --black --line-fix > htop.htm Creates an HTML file with the output of htop. You have to use --line-fix due the other new-line-commands htop uses. AUTHOR
Copyleft Alexander Matthes aka Ziz 2011 zizsdl@googlemail.com SEE ALSO
http://ziz.delphigl.com/tool_aha.php August 31, 2011 aha(1)
All times are GMT -4. The time now is 03:23 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy