Removing duplicate files from list with different path
I have a list which contains all the jar files shipped with the product I am involved with. Now, in this list I have some jar files which appear again and again. But these jar files are present in different folders.
My input file looks like this
Now I need to remove the duplicate entries i.e. remove the extra a.jar, c.jar and others like wise.
Final list would be like this:
This is the script I have so far..
But with this, the main functionality of the script is not working. i.e. the if condition is not working as required. Could be the sed problem or the logic.
I am getting the jarnew.txt as good as the jar.txt
Generally scripts are written based on pattern change. In your requirement on input, only c.jar is taken from path/2/. I have simulated your input to required output.
And, your input and output is not being generallized so that script is given with using speicific filenames.
I dont think sort is a possible soution. The path length varies i.e. the directory structure is different for files. Some of them have a depth of 3.. others a depth of more than 3.
Hello
I have been trying to remove a row from a file which has the same first three columns as another row - I have tried lots of different combinations of suggestion on this forum but can't get it exactly right.
what I have is
900 - 1000 = 0
900 - 1000 = 2562
1000 - 1100 = 0
1000 - 1100... (7 Replies)
Hello,
I have a huge directory (with millions of files) and need to find out duplicates based on BOTH file name and File size.
I know fdupes but it calculates MD5 which is very time-consuming and especially it takes forever as I have millions of files.
Can anyone please suggest a script or... (7 Replies)
Gents,
I have a file like this.
1 1
1 2
2 3
2 4
2 5
3 6
3 7
4 8
5 9
I would like to get something like it
1 1 2
2 3 4 5
3 6 7
Thanks in advance for your support :b: (8 Replies)
Hi,
I need a script/command to list out all the files in current path and also the files in folder and subfolders.
Ex: My files are like below
$ ls -lrt
total 8
-rw-r--r-- 1 abc users 419 May 25 10:27 abcd.xml
drwxr-xr-x 3 abc users 4096 May 25 10:28 TEST
$
Under TEST, there are... (2 Replies)
Hi All,
I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need.
File1.csv:
RAJAK,ACTIVE,1
VIJAY,ACTIVE,2
TAHA,ACTIVE,3... (6 Replies)
I have this piece of code
printf '%s\n' $pth*.msf | tr ' ' '\n' | sort -t '-' -k7 -k6r \
| awk -F- '{c=($6$7!=p&&FNR!=1)?ORS:"";p=$6$7}{printf("%c%s\n",c,$0)}'
When I run it I get
/home/chrisd/tatsh/branches/terr0.50/darwin/n02-z30-dsr65-terr0.50-dc0.002-8x6drw-csq.msf... (8 Replies)
Can anyone help me to removing duplicate records from 2 separate files in UNIX?
Please find the sample records for both the files
cat Monday.dat
3FAHP0JA1AR319226MOHMED ATEK 966504453742 SAU2010DE
3LNHL2GC6AR636361HEA DEUK CHOI 821057314531 KOR2010LE
3MEHM0JG7AR652083MUTLAB NAL-NAFISAH... (4 Replies)
Hi all,
How to save file full name to a file. I tried the following but don't know to include path name.
$ ls -l | awk '{print $9}' > outputfile.dat
$ cat outputfile.dat
fifth.txt
first.txt
fourth.txt
second.txt
third.txt
My wanted result is ie:
... (3 Replies)
Hi
I am writing a script to find the list of files in dir1 and my script is place in dir2
while doing ls of files dir1 it is displaying with path. I would like to omit the path and display the only file name so that I can pass it to my script as arguments.
for filename in ... (2 Replies)
How can i list every single file on a sun solaris server running 2.8 starting from '/' with the full path included in it?
example.
/
...
...
...
/etc/inetd.conf
/etc/passwd
/etc/shadow
...
...
...
/var/adm/messages
/var/adm/messages.0
/var/adm/messages.1
...
...
...... (4 Replies)