Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Replace char between chars - help needed
Top Forums Shell Programming and Scripting Replace char between chars - help needed Post 302667759 by go0ogl3 on Saturday 7th of July 2012 09:19:43 AM
Old 07-07-2012
Lightbulb
Thank you Pikk45! One problem solved!

Using
Code:
awk -F'\^\|' '{while(NF<4 && getline p)$0=$0 p}1'

on the file I can remove max 4 new lines on the 4th field.
The delimiter used is ^| which is at the end of the 4th field.

Now I must replace the "|" between the "^"...

---------- Post updated at 12:15 PM ---------- Previous update was at 09:47 AM ----------

I have come to the:

Code:
awk ' {for (i=1;i<=NF;i++) if (i!=NF) {printf (5<i && i<(NF-1))?$i"-":$i"|"} print $NF} ' FS='|' OFS='^'

but this replaces all "|" till the end... how can I tell awk to stop replacing at first "^|"?

---------- Post updated at 04:19 PM ---------- Previous update was at 12:15 PM ----------

Hello,

I finally found the answer. It's not so pretty, but it works and is extremely fast (a few seconds) on a almost 2Mb file:

Code:
cat csv_in_file | awk -F'\^\|' '{while(NF<4 && getline p)$0=$0 p}1' \
| sed 's/%/_procent_/g'\
| awk '{for (i=1;i<=NF;i++) if (i!=NF) {printf (3<i && i<(NF-1))?$i"!":$i"|"} print $NF}' FS='|' OFS='^' \
| perl -npe 's/^(.*)!(.*)!(.*)!(.*)!(.*)!(.*)!(.*)!(.*)!(.*)!(.*)!(.*)$/$1|$2|$3|$4|$5|$6|$7|$8|$9|$10|$11/' \
| sed 's/_procent_/%/g' > csv_out_file

I'll try to explain what every command does:
- first awk sets the field delimiter to "^|" and deletes at most 3 new lines (I've discovered I have at most 4 lines of data in the 4th record = 3 new lines)
- the sed replaces the "%" with "_precent_" because I found that it breaks somehow the next awk command...
- the second awk command replaces all the "|" symbols beginning with the 4th field with "!" (I'm sure there will never be "!" in my last fields)
- the perl command replaces the last "!" with "|"
- the last sed replaces back the "%"

The result is a csv file with pipe as a delimiter wich I can use...

Maybe there is a more elegant solution but this works and is damn fast compared to the windows excel.
Hope this will help others too.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

replace ascii chars without loosing it.

Hi, Can some one tell, how to replace ascii non printable TAB from the while to something, then later on replace it back to TAB. Basciallz we do bulk data processing, our processin treats TAB as new field , So I thought we can replace it with something and later on revert it. TIA (4 Replies)
Discussion started by: braindrain
4 Replies

2. Shell Programming and Scripting

replace chars,

:rolleyes: Hi, I want to replace the particular text in the middle of the line. Ex. In the line 40-50 wanted to replace the char as 'x' (7 Replies)
Discussion started by: Jairaj
7 Replies

3. Shell Programming and Scripting

Replace Junk chars (Sed)

I know this has been asked previously on this forum...But I think I have a different scenario to present. I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers) BB7118450 6004718 BIANCALANA =HEI BZ5842819 ... (4 Replies)
Discussion started by: alfredo123
4 Replies

4. Shell Programming and Scripting

How to replace any char with newline char.

Hi, How to replace any character in a file with a newline character using sed .. Ex: To replace ',' with newline Input: abcd,efgh,ijkl,mnop Output: abcd efgh ijkl mnop Thnx in advance. Regards, Sasidhar (5 Replies)
Discussion started by: mightysam
5 Replies

5. Shell Programming and Scripting

Help needed to remove a char from a specified sub-string

Hi! I'm having trouble usind sed to remove the char ' from within a database's varchar register's sql. For example, on the following sql: INSERT INTO patrimonio_municipal.patrimonio_municipal_airc_tmp Values('|Estação Elevatória|',|16723|,'|Grandes Reparações|', '|2010-03-26... (3 Replies)
Discussion started by: cremat0rio
3 Replies

6. Shell Programming and Scripting

Replace char on text file

Hi , I have problem on replace char on text file #!/bin/bash echo "Enter Third value : " read a sed '2 s/192.160.1.1/cut -d"." -f3/$a/g' tcpip.txt > a.txt I want replace line 2 on tcpip.txt with value a that I input but it's not run Please help me (2 Replies)
Discussion started by: phillipss
2 Replies

7. Shell Programming and Scripting

Find and replace all extended char.

Hi Guys, I wand find and replace all Extended ASCII Codes from all my log files. My Log files: /home/Kalr/PPool/Output i have logs file in sub dir. /home/Kalr/PPool/Output/X /home/Kalr/PPool/Output/Y /home/Kalr/PPool/Output/Z My Abc.log file input: Extended ASCII Codes :– ... (4 Replies)
Discussion started by: asavaliya
4 Replies

8. UNIX for Dummies Questions & Answers

Replacing multiple special chars with single char

Hi I've a string . And i need to replace set of characters with a single character Means .. or . or ... and so on should be replaced with single % character Irrespective of number of dots in between the characters , those should be replaced with single % All the above strings should be... (3 Replies)
Discussion started by: smile689
3 Replies

9. UNIX for Dummies Questions & Answers

Grep or sed to search, replace/insert chars!

HI All Im trying to come up with an approach to finding a string, using a portion of that string to insert it on lines starting with the value "GOTO" appending to end of line after removing PT's ( See example below! ) EXAMPLE: 1. I would like to search for the line that starts with "TLAXIS/"... (7 Replies)
Discussion started by: graymj
7 Replies

10. UNIX for Advanced & Expert Users

Replace \n char in Data

File is pipe delimited with 17 fields. We may get \n char (1 or more \n in one field or multi fileds) in data in any field. Need to replace \n in data with space and not the Ture \n that is line separator. I tried below awk command it did not work as expected. awk '(NR-1)%2{$1=$1}1' RS=\|... (9 Replies)
Discussion started by: rajeshkumare
9 Replies

LEARN ABOUT OPENSOLARIS

igawk

IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME

       igawk - gawk with include files

SYNOPSIS

       igawk [ all gawk options ] -f program-file [ -- ] file ...
       igawk [ all gawk options ] [ -- ] program-text file ...

DESCRIPTION

       Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).

       AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like

	      @include getopt.awk

       in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.

OPTIONS

       See gawk(1) for a full description of the AWK language and the options that gawk supports.

EXAMPLES

       cat << EOF > test.awk
       @include getopt.awk

       BEGIN {
	    while (getopt(ARGC, ARGV, "am:q") != -1)
		 ...
       }
       EOF

       igawk -f test.awk

SEE ALSO

       gawk(1)

       Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.

AUTHOR

       Arnold Robbins (arnold@skeeve.com).

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +--------------------+-----------------+
       |  ATTRIBUTE TYPE    | ATTRIBUTE VALUE |
       +--------------------+-----------------+
       |Availability	    | SUNWgawk	      |
       +--------------------+-----------------+
       |Interface Stability | Volatile	      |
       +--------------------+-----------------+
NOTES

       Source for gawk is available on http://opensolaris.org.

Free Software Foundation					    Nov 3 1999								  IGAWK(1)
All times are GMT -4. The time now is 05:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy