10 More Discussions You Might Find Interesting
1. UNIX for Beginners Questions & Answers
Hello all,
I need to filter a dataframe composed of several columns of data to remove the duplicates according to one of the columns. I did it with pandas. In the main time, I need that the last column that contains all different data ( not redundant) is conserved in the output like this:
A ... (5 Replies)
Discussion started by: pedro88
5 Replies
2. UNIX for Beginners Questions & Answers
I have /tmp dir with filename as:
010020001_S-FOR-Sort-SYEXC_20160229_2212101.marker
010020001_S-FOR-Sort-SYEXC_20160229_2212102.marker
010020001-S-XOR-Sort-SYEXC_20160229_2212104.marker
010020001-S-XOR-Sort-SYEXC_20160229_2212105.marker
010020001_S-ZOR-Sort-SYEXC_20160229_2212106.marker... (4 Replies)
Discussion started by: gnnsprapa
4 Replies
3. Shell Programming and Scripting
Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker...
Column #1 is a simple ID, which is used to identify the duplicate.
Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
4. Shell Programming and Scripting
Hello,
I have a single text file with three columns like:
2 1 10
3 6 11
4 4 12
5 5 13
6 8 14
I was wondering how I can run a shell script to (do loop) to read the first point in the first column , compare it with all the points in the second column and if they are the same,... (12 Replies)
Discussion started by: Paul Moghadam
12 Replies
5. Shell Programming and Scripting
Hi team,
I have 20 columns csv files. i want to find the duplicates in that file based on the column1 column10 column4 column6 coulnn8 coulunm2 . if those columns have same values . then it should be a duplicate record.
can one help me on finding the duplicates,
Thanks in advance.
... (2 Replies)
Discussion started by: baskivs
2 Replies
6. Shell Programming and Scripting
Hi Everyone,
I have a file which contains the autosys (job scheduler) JIL (Job Information Language) Information.
For Example:
/* ----------------- TS_QDB_DATA_MASK_RUN ----------------- */
insert_job: TS_QDB_DATA_MASK_RUN job_type: c
command: ${DATA_BIN}/hrt_data_mask.sh
machine:... (5 Replies)
Discussion started by: filter
5 Replies
7. Shell Programming and Scripting
Hi,
I am unable to search the duplicates in a file based on the 1st,2nd,4th,5th columns in a file and also remove the duplicates in the same file.
Source filename: Filename.csv
"1","ccc","information","5000","temp","concept","new"
"1","ddd","information","6000","temp","concept","new"... (2 Replies)
Discussion started by: onesuri
2 Replies
8. Shell Programming and Scripting
Hi All,
I needs to fetch unique records based on a keycolumn(ie., first column1) and also I needs to get the records which are having max value on column2 in sorted manner... and duplicates have to store in another output file.
Input :
Input.txt
1234,0,x
1234,1,y
5678,10,z
9999,10,k... (7 Replies)
Discussion started by: kmsekhar
7 Replies
9. Shell Programming and Scripting
I am trying to figure out how to scan a file like so:
1 ralphs office","555-555-5555","ralph@mail.com","www.ralph.com
2 margies office","555-555-5555","ralph@mail.com","www.ralph.com
3 kims office","555-555-5555","kims@mail.com","www.ralph.com
4 tims... (17 Replies)
Discussion started by: totus
17 Replies
10. UNIX for Dummies Questions & Answers
Is it possible to search through a large file full of rows and columns of text and retrieve only the rows that contain duplicates fields,
searchiing for duplicates on col4 & col6
Sample below
Col1 col2 col3 col4 col5 col6
G405H SURG FERGUSON ... (2 Replies)
Discussion started by: Gerry405
2 Replies