10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all,
how can delete duplicate files in file form, e.g.
$cat file1
aaa 123 234 345 456
bbb 345 345 657 568
ccc 345 768 897 456
aaa 123 234 345 456
ddd 786 784 234 263
ccc 345 768 897 456
aaa 123 234 345 456
ccc 345 768 897 456
then i need ouput file1 some, (4 Replies)
Discussion started by: aav1307
4 Replies
2. Shell Programming and Scripting
Hi All,
I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,).
SEARCH_CRITERIA = "REJECT, DUPLICATE"
Input File:
ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies
3. Shell Programming and Scripting
Hi,
I have a csv file with old data..i need to have only last 30 days from the current dateof data in the file.The fourth field in the file is a date field.i need to write a script to delete the old data by comparing the the fourth field with the (current date -30).I need to delete the rows in... (2 Replies)
Discussion started by: pals70423
2 Replies
4. Shell Programming and Scripting
I am a newbie to shell scripting ..
I have a .csv file. It has 1000 some rows and about 7 columns...
but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type...
example below..
column 1 ... (2 Replies)
Discussion started by: mitr
2 Replies
5. Shell Programming and Scripting
I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records.
contents of file looks like
30002157,40342424,OTC,mart_rec,100, ,0
30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Discussion started by: mukeshguliao
6 Replies
6. Shell Programming and Scripting
Hi all!
A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want.
For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies
7. UNIX for Dummies Questions & Answers
Hey all, a relative bash/script newbie trying solve a problem.
I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like
2007-11-03... (3 Replies)
Discussion started by: mattv
3 Replies
8. Shell Programming and Scripting
hii i have a huge amt of data stored in a file.Here in this file i need to remove duplicates rows in such a way that the last column has different data & i must check for greatest among last colmn data & print the largest data along with other entries but just one of other duplicate entries is... (16 Replies)
Discussion started by: reva
16 Replies
9. Shell Programming and Scripting
I'm trying to remove lines of data that contain duplicate data in a specific column.
For example.
apple 12345
apple 54321
apple 14234
orange 55656
orange 88989
orange 99898
I only want to see
apple 12345
orange 55656
How would i go about doing this? (5 Replies)
Discussion started by: spartan22
5 Replies
10. Shell Programming and Scripting
For example suppose I have a file which contains data as:
$cat data
800,2
100,9
700,3
100,9
200,8
100,3
Now I want the output as
200,8
700,3
800,2
Key is first three characters, I don't want any reords which are having duplicate keys.
Like sort +0.0 -0.3 data can we use... (9 Replies)
Discussion started by: sumitc
9 Replies