10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code:
awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies
2. Shell Programming and Scripting
I need to be able to search for a string in the first column and if that string exists than replace the nth column with "-9.99".
AW12000012012 2.38 1.51 3.01 1.66 0.90 0.91 1.22 0.82 0.57 1.67 2.31 3.63 0.00
AW12000012013 1.52 0.90 1.20 1.34 1.21 0.67 ... (14 Replies)
Discussion started by: ncwxpanther
14 Replies
3. Shell Programming and Scripting
I'm looking to remove duplicate rows from a CSV file with a twist.
The first row is a header.
There are 31 columns. I want to remove duplicates when the first 29 rows are identical ignoring row 30 and 31 BUT the duplicate that is kept should have the shortest total character length in rows 30... (6 Replies)
Discussion started by: Michael Stora
6 Replies
4. Shell Programming and Scripting
I need to take one column of data and put it into the following format:
1st line,2nd line
3rd line,4th line
5th line,6th line
...
Thanks! (6 Replies)
Discussion started by: batcho
6 Replies
5. Shell Programming and Scripting
Hi All,
Can you please guide me to search a string in a particular column of file and return the line number of the line where it was found using awk. As an example :
abc.txt
7000,john,2,1,0,1,6
7001,elen,2,2,0,1,7
7002,sami,2,3,0,1,6
7003,mike,1,4,0,2,1
8001,nike,1,5,0,1,8... (3 Replies)
Discussion started by: arunshankar.c
3 Replies
6. Shell Programming and Scripting
Hello,
I'm trying to remove an arbitrary number of semicolons at the end of each line in the input file.
Input:
44;I;1000031;;;B;0137;0;;01.02.2008;03.02.2009;;;;;;;;;;;;;0028-101746;;;
45;I;1000031;;;B;0137;0;;01.02.2008;03.02.2009;;;;;;;;;;;;;0028-101746;;;;;
... (6 Replies)
Discussion started by: uioreanu
6 Replies
7. Shell Programming and Scripting
Dear All,
i want to search particular string and want to replance next line value.
following is the test file.
search string is
tmp,???
,10:1 "???" may contain any 3 character it should remain the same and next line replace with ,10:50
tmp,123 --- if match tmp,??? then... (3 Replies)
Discussion started by: arvindng
3 Replies
8. Shell Programming and Scripting
Hi,
Could you please help me in removal of newline chracter present in between the double quotes and replacing it with space.
For example ...
Every field is wrapped with double quotes with comma delimiter, so I need to travese from first double quote occerence to till second double... (7 Replies)
Discussion started by: vsairam
7 Replies
9. Shell Programming and Scripting
Hello.
I've been banging my head against walls trying to search a comma delimited file, using awk. I'm trying to search a "column" for a specific parameter, if it matches, then I'd like to print the whole line.
I've read in multiple texts:
awk -F, '{ if ($4 == "string") print $0 }'... (2 Replies)
Discussion started by: Matthias03
2 Replies
10. Shell Programming and Scripting
I have a file with thousands of lines. I need to search for a specific value in a specific field and delete the lines that match.
example.
abcdXX1234567
abcdXY1234567
abcdXX1234567
abcdXX1234567
If there is an XY in position 5 and 6 then remove that line.
Any suggestions would... (4 Replies)
Discussion started by: thudak
4 Replies