how to strip rows from a text file?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting how to strip rows from a text file?
# 1  
Old 10-16-2007
how to strip rows from a text file?

Can an expert kindly write an efficient Linux ksh script that will strip rows with no numbers from a text file?
Supposing there are three rows that text file called text.txt :

"field1","field2","field3",11,22,33,44
"field1","field2","field3",1,2,3,4
"field1","field2","field3",,,,

The above rows are delimited by commas.

I want to strip out the last row that does not have numbers in the last 4 fields. So the new file text2.txt will only have two rows.
"field1","field2","field3",11,22,33,44
"field1","field2","field3",1,2,3,4

Thanks in advance.

(I suppose an awk command will do the job, but am not sure how to write it. here is a start:
awk '{ if (index($0,"\"\",")==4) {print > "text2.txt" } }' bigfile
Smilie
# 2  
Old 10-16-2007
Code:
grep -v ',,' bigfile

# 3  
Old 10-16-2007
cool!! now why didn't I think of that?

Thanks!! Now last question:
How can I integrate this grep with the following awk statement which breaks my bigfile into 2 smaller files?

Here again are the two ksh commands:

grep -v ',,,,' bigfile

awk '{ if (index($0,"\"TG\",")==1) {print > "TG.txt" } else {print > "NoTG.txt"} }' bigfile

Somehow in my awk command above, I want to strip out rows that have the pattern ',,,,' before I print it out to TG.txt or to NoTG.txt. How can I achieve that?
Thanks in advance to all you shell geniuses out there!!
# 4  
Old 10-16-2007
break your code up...makes your code easier to read.could be something like this if i get you right
Code:
awk '!/,,,,$/{ if (index($0,"\"TG\",")==1) 
                     {print > "TG.txt" 
                  } 
                else {
                       print > "NoTG.txt"
                } 
               }' bigfile


Last edited by ghostdog74; 10-16-2007 at 10:17 PM..
# 5  
Old 10-16-2007
I lied, can we improve on grep ',,,,' bigfile?

I lied when I said that was my last question. Smilie
Sorry to bother you gentlemen again... can we improve on grep ',,,,' bigfile?

You see, grep ',,,,' bigfile will strip out the first row in this table:
,,,,,22,33,44
"field1","field2","field3",1,2,3,4
"field1","field2","field3",,,,

which is not what I am after. I only want to strip out the rows with ',,,,' at the end of the row. So I only want to search for pattern ',,,,' in the last four chars of the row. Is there a way to grep only the last four characters?

I owe you all a favor!!
Smilie
# 6  
Old 10-16-2007
I believe ghostdog74's solution will do what you want. Care to try it out?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Strip some text and format with new delimited

Sample input 19:08:12.172; Cat1 74598; Cat2 1366; Cat3 227; Cat4 389; Cat5 572; Cat6 2228; Cat7 1039; Cat8 25; 19:08:22.173; Cat1 75589; Cat2 1388; Cat3 233; Cat4 393; Cat5 582; Cat6 2253; Cat7 1055; Cat8 25; 19:08:32.173; Cat1 76518; Cat2 1404; Cat3 238; Cat4 397; Cat5 592; Cat6 2270; Cat7... (5 Replies)
Discussion started by: before4
5 Replies

2. Shell Programming and Scripting

Removing rows and chars from text file

Dear community, maybe I'm asking the moon :rolleyes:, but I'm scratching my head to find a solution for it. :wall: I have a file called query.out (coming from Oracle query), the file is like this: ADDR TOTAL -------------------- ---------- TGROUPAGGR... (16 Replies)
Discussion started by: Lord Spectre
16 Replies

3. UNIX Desktop Questions & Answers

Calculate average for rows in a text file

Dear Gurus, I have tab-delimited text files with matrix containing values. The first column is a identifier and other columns have the corresponding values. I would like to calculate the average value (total number/number of entries) for all entries from 2nd column to the last column in row... (3 Replies)
Discussion started by: Unilearn
3 Replies

4. UNIX for Dummies Questions & Answers

Deleting specific rows from a text file

How do I go about deleting specific rows from a text file (given row number)? (5 Replies)
Discussion started by: evelibertine
5 Replies

5. Shell Programming and Scripting

Insert rows of text into a file after pattern

SHELL=bash OS=rhel I have a file1 that contains text in sentences like so: file1 this is a sentence this is a sentence this is a sentence I also have a file2 like so: file2 command=" here is some text here is more text again we have some text " I wish to echo the text from... (4 Replies)
Discussion started by: jaysunn
4 Replies

6. UNIX for Dummies Questions & Answers

Help with deleting specific rows from a text file

I know this is a complicated question but I will try to illustrate it with some data. I have a data file that looks like the following: 1341 NA06985 0 0 2 46.6432798439 1341 NA06991 NA06993 NA06985 2 48.8478948517 1341 NA06993 0 0 1 45.8022601455 1340 NA06994 0 0 1 48.780669145 1340... (1 Reply)
Discussion started by: evelibertine
1 Replies

7. Shell Programming and Scripting

Duplicate rows in a text file

notes: i am using cygwin and notepad++ only for checking this and my OS is XP. #!/bin/bash typeset -i totalvalue=(wc -w /cygdrive/c/cygwinfiles/database.txt) typeset -i totallines=(wc -l /cygdrive/c/cygwinfiles/database.txt) typeset -i columnlines=`expr $totalvalue / $totallines` awk -F' ' -v... (5 Replies)
Discussion started by: whitecross
5 Replies

8. Linux

Splitting a Text File by Rows

Hello, Please help me. I have hundreds of text files composed of several rows of information and I need to separate each row into a new text file. I was trying to figure out how to split the text file into different text files, based on each row of text in the original text file. Here is an... (2 Replies)
Discussion started by: dvdrevilla
2 Replies

9. Shell Programming and Scripting

Delete rows in text file

Hi I do have a text file with 1000's of lines with 1 row and column with a specific pattern. 1102 1 1 1 1 1234 1 1 1 1 1009 1 1 1 1 1056 1 (3 Replies)
Discussion started by: Lucky Ali
3 Replies

10. Shell Programming and Scripting

How do I strip and add tabbed fields to a long text file?

I was wondering if there was a way to modify a tab delimited text file, up to 185,000 lines long by adding a repeated field to a block of 20 data , then repeat the process until the file is processed. The current file looks like: ... (3 Replies)
Discussion started by: saint65
3 Replies
Login or Register to Ask a Question