Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removing rows and chars from text file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing rows and chars from text file
# 1  
Old 09-09-2012
Error Removing rows and chars from text file
Dear community,
maybe I'm asking the moon Smilie, but I'm scratching my head to find a solution for it. Smilie
I have a file called query.out (coming from Oracle query), the file is like this:
Code:
ADDR                 TOTAL
-------------------- ----------
TGROUPAGGR
--------------------------------------------------------------------------------------------------------------
1234567890                    1
9876[9876=1]

7634567890                    1
3456[3456=1]

8934567890                    4
0123[0123=1];1234[4567=3]


ADDR                 TOTAL
-------------------- ----------
TGROUPAGGR
--------------------------------------------------------------------------------------------------------------
5534567890                    1
0001[0101=1]

7834567890                    1
3256[3256=1]

4534567890                    1
7560[7560=1]


ADDR                 TOTAL
-------------------- ----------
TGROUPAGGR
--------------------------------------------------------------------------------------------------------------
1334567890                    1
0900[0900=1]

0034567890                    2
9899[9899=2]

What I would like to do is remove unneeded rows and chars (like CRLF) and have the following output to simplify my post analysis:
Code:
1234567890 1 9876[9876=1]
7634567890 1 3456[3456=1]
8934567890 4 0123[0123=1];1234[4567=3]
5534567890 1 0001[0101=1]
7834567890 1 3256[3256=1]
4534567890 1 7560[7560=1]
1334567890 1 900[0900=1]
0034567890 2 9899[9899=2]

So, in the final output file I would like to have two lines reported in one row and other rows deleted (like --------------).

Is it possibile to do that?
# 2  
Old 09-09-2012
Code:
awk '/^[0-9]/{a++;gsub(/ +/," ");if(a%2) printf "%s ",$0; else print}' file

This User Gave Thanks to elixir_sinari For This Post:
Lord Spectre
# 3  
Old 09-09-2012
Assuming there are no extra spaces..

Code:
awk '/^[0-9]/' file | xargs -n 3

# 4  
Old 09-09-2012
Quote:
Originally Posted by pamu
Assuming there are no extra spaces..
Extra spaces won't matter (provided these are not within the values which seems improbable looking at the sample) Smilie
# 5  
Old 09-09-2012
elixir sinari, I already said in another post, you're the master! Smilie

SmilieSmilieSmilie

May I ask you other related things?
I'm using the simple redirect to another file, like:
Code:
awk '/^[0-9]/{a++;gsub(/ +/," ");if(a%2) printf "%s ",$0; else print}' query.out > query.tmp

Is it possibile to edit the original file to avoid another file creation?

Then, my first post analysis will be the follwoing:
Code:
cat query.tmp | while read line; do
echo $line | awk '{printf $2}'; echo
done

Is it possibile to print $2 only when it's greater then a specific number (like 10)?

Many many Thanks!
# 6  
Old 09-09-2012
Quote:
Originally Posted by elixir_sinari
Extra spaces won't matter (provided these are not within the values which seems improbable looking at the sample) Smilie
Yes. that's why this condition...Smilie

Ok another one...Smilie

Code:
 awk '/^[0-9]/{ if(! (NR%2)){s=s" "$0;print s;s=""}else{s=$0}}' file

# 7  
Old 09-09-2012
Thanks pamu, but the latest command will remove the extra rows and CRLF chars, but leave the output in two rows. Smilie

Code:
1234567890 1 
9876[9876=1]
7634567890 1 
3456[3456=1]
8934567890 4 
0123[0123=1];1234[4567=3]
....
....

Edit, my mistake, it put in one rows, but it will report back also the first rows (so duplicate rows):
Code:
1234567890 1 
1234567890 1 9876[9876=1]
7634567890 1 
7634567890 1 3456[3456=1]
8934567890 4
8934567890 4  0123[0123=1];1234[4567=3]
....
....
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing top few and bottom few rows in a file

Hi, I have a requirement where I need to delete given number of top and bottom rows in a flat file which has new line as its delimiter. For ex: if top_rows=2 & bottom_rows=1 Then in a given file which looks like: New York DC LA London Tokyo Prague Paris Bombay Sydney... (7 Replies)
Discussion started by: calredd
7 Replies

2. UNIX for Advanced & Expert Users

Removing special chars from file and maintain field separator

Running SunOs 5.6. Solaris. I've been able to remove all special characters from a fixed length file which appear in the first column but as a result all subsequent columns have shifted to the left by the amount of characters deleted. It is a space separated file. Line 1 in input file is... (6 Replies)
Discussion started by: iffy290
6 Replies

3. Shell Programming and Scripting

Removing Duplicate Rows in a file

Hello I have a file with contents like this... Part1 Field2 Field3 Field4 (line1) Part2 Field2 Field3 Field4 (line2) Part3 Field2 Field3 Field4 (line3) Part1 Field2 Field3 Field4 (line4) Part4 Field2 Field3 Field4 (line5) Part5 Field2 Field3 Field4 (line6) Part2 Field2 Field3 Field4... (7 Replies)
Discussion started by: ekbaazigar
7 Replies

4. Shell Programming and Scripting

Bash script help - removing certain rows from .csv file

Hello Everyone, I am trying to find a way to take a .csv file with 7 columns and a ton of rows (over 600,000) and remove the entire row if the cell in forth column is blank. Just to give you a little background on why I am doing this (just in case there is an easier way), I am pulling... (3 Replies)
Discussion started by: MrTuxor
3 Replies

5. Shell Programming and Scripting

Removing unknow chars from file names.

I'm trying to move a large folder to an external drive but some files have these weird chars that the external drive won't accept. Does anyone know any command of any bash script that will look through a given folder and remove any weird chars? (4 Replies)
Discussion started by: Joktaa
4 Replies

6. Shell Programming and Scripting

removing rows from text file older than certain date

Hi I need a way of removing rows from a txt file that are older than 30 days from today, going by the date in column 2, below is an example from my file. I have tried awk but don't have enough knowledge. I would really appreciate some help. 41982,15/07/2010,H833AB/0,JZ,288... (6 Replies)
Discussion started by: firefox2k2
6 Replies

7. Shell Programming and Scripting

Removing rows based on a different file (ignore my earlier post - there was a mistake).

Sorry I made a mistake in my last post (output is suppose to be the opposite). Here is a revised post. Hi, I am not sure if this has already been asked (I tried the search but the search was too broad). Basically I want to remove rows based on another file. So file1 looks like this (tab... (3 Replies)
Discussion started by: kylle345
3 Replies

8. Shell Programming and Scripting

Removing rows based on a another file

Hi, I am not sure if this has already been asked (I tried the search but the search was too broad). Basically I want to remove rows based on another file. So file1 looks like this (tab seperated): HHN 3 5 5 HUJ 2 2 1 JJJ 3 1 1 JUN 2 1 3 I have another file (file2)... (2 Replies)
Discussion started by: kylle345
2 Replies

9. Shell Programming and Scripting

Removing rows from a file

I have a file like below and want to use awk to solve this problem. The record separator is ">". I want to look at each record section enclosed within ">". Find the row with the 2nd and 3rd columns being 0, such as 10 0 0 I need to take the first number which in this case is 10. Then... (15 Replies)
Discussion started by: kristinu
15 Replies

10. Shell Programming and Scripting

Removing rows from a file based on date comparison

I have a '|' delimited file and want to remove all the records from the file if the date is greater than a year from sysdate. The layout of the file is as below - xxxxxxxxxxxxxx|yyyyyy|zzzzzz|2009-12-27-00:00| 000000000|N xxxxxxxxxxxxxx|yyyyyy|zzzzzz|2010-01-03-00:00| 000000000|N... (4 Replies)
Discussion started by: Max_2503
4 Replies
Login or Register to Ask a Question