04-05-2008
Bash uniq/ diff/ and other I cant figure it out
First off thank you for any help.
Here is the problem. I have two text files that fit the same format. The first I created using an ls -d command and then with the help of the forums ran awk resulting in the fallowing output.
W00CHZ0103345-I1CZ44
W00E6S1016722-I01JW159
W00E6S1016722-I01JW160
W00E6S1016722-I01JW161
W00EGS10125151-I01JW176
W00EGS10125151-I01JW177
W00EGS10125151-I01JW178
W00EGS10125151-I01JW179
W00EGS10125151-I01JW180
W00EGS1012593-I00EGS1017114
I have a second text file whose format i was trying to successfully managed to match. Here it is.
W00CHZ0103345-I1CZ44
W00EGS1016051-I00EGS1016053
W00EGS1016054-I00EGS1016056
W00EGS1016057-I00EGS1016059
W00EGS1016060-I00EGS1016062
W00EGS1016181-I1PD10388
W00EGS1016199-I00EGS1016201
W00EGS1016202-I1GS65488
W00EGS1016210-I00EGS1016212
W00EGS1016213-I00EGS1016216
Now these lists are nearly 10,000 lines long and what i need to do is compare them and see what only occurs on the second list. I have tried sorting and using uniq and diff and everything else that i can think of but was unable to generate a list of the lines that only appear in txt2. That being said if there were lines that appeared in the first text and not the second i would want to know but i would need to know from which list it came.
Please help if you can.
Thank you
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3.
For example:
file 1
123
1234
123456
file 2
123
2345
23456
file 3 should have
2345
23456
I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies
2. Shell Programming and Scripting
Moderator, please, delete this topic (1 Reply)
Discussion started by: optik77
1 Replies
3. Shell Programming and Scripting
I just put together a script for work that will essentially automate the migration of our Windows fileserver to my newly created Debian based SAMBA server.
My script will create the necessary directories then copy the data over to my new server, after that it will set the ACL's by using... (3 Replies)
Discussion started by: binary-ninja
3 Replies
4. Shell Programming and Scripting
Hi,
I am new to shell scripting.
please help me to find out the solution.
I need a script where we need to read the text file(consists of all file names) and get the file names one by one
and append the date suffix for each file name as 'yyyymmdd' .
Then search each file if exists... (1 Reply)
Discussion started by: Lucky123
1 Replies
5. Shell Programming and Scripting
I have an output file .dat. From this file i have to do a distinct of the ID using the sort uniq command in bash script. How can i do it? i found :
sort -u ${FILEOUT_DAT}
but i don't think is my solution because the id isn't specified.. is there other solution? (7 Replies)
Discussion started by: punticci
7 Replies
6. Shell Programming and Scripting
hi all
kindly i want to subtract to time stamps
like:
t1=08:35:20
t2=08:36:58
diff=00:01:38 <<<<
or
diff=01:38 <<<<
using bash
thanks alot (9 Replies)
Discussion started by: teefa
9 Replies
7. Shell Programming and Scripting
Hi,
Below is my input file:
Long list of significant figure
1.757E-4
7.51E-3
5.634E-5
.
.
.
Desired output file:
0.0001757
0.00751
0.00005634
.
.
. (10 Replies)
Discussion started by: perl_beginner
10 Replies
8. Shell Programming and Scripting
Guys i have 3 files,
but i want to compare and diff only the 2nd column
path=`/home/whois/doms`
for i in `cat domain.tx`
do
whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant
whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies
9. Shell Programming and Scripting
Hi,
I am quite scripting illiterate and have been trying to write a bash script to compare to two files which i have populated in two seperate arrays as below and confirmed that all the files are loaded into the array.
IFS=$'\n'
filea=($(find /var/tmp/dir1 -type f -follow -print))... (12 Replies)
Discussion started by: jlykke
12 Replies
10. UNIX for Beginners Questions & Answers
Hi everyone,
I've an issue trying to soustracte two dates, e.g:
d1=$(date -d "Nov 18, 2017" +%s)
d2=$(date +%s) # Today we are 2017-11-16
echo "$(( (d1 - d2) / 86400 ))"
Output:
1
I don't understand why it doesn't work. for me, it should give "18 - 16 = 2".
Much appreciated... (1 Reply)
Discussion started by: Arnaudh78
1 Replies
uniq(1) General Commands Manual uniq(1)
Name
uniq - report repeated lines in a file
Syntax
uniq [-udc[+n][-n]] [input[output]]
Description
The command reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are
removed; the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found. For further infor-
mation, see
Options
The n arguments specify skipping an initial portion of each line in the comparison:
-n Skips specified number of fields. A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
neighbors.
+n Skips specified number of characters in addition to fields. Fields are skipped before characters.
-c Displays number of repetitions, if any, for each line.
-d Displays only lines that were repeated.
-u Displays only unique (nonrepeated) lines.
If the -u flag is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of
just the repeated lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
See Also
comm(1), sort(1)
uniq(1)