09-10-2007
Retaining spaces between words
Retaining Spaces within a word
--------------------------------------------------------------------------------
Hi Experts,
I have a 2 GB flat file which have unicode field, some of them are blanks and its size is 4000 character. In the existing system SED command removes the spaces. Because of this field itself....it is taking almost three days to complete the file processing. I removed sed and used tr command...it worked in less than a minute. Now the challenging part is the character fields have more than one space, I am tr -s ' ' '' to remove the spaces, but it is removing the spaces inbetween the characters which is more than one space.
My sample record is this:
262774372|58959454 | Rajiv Rajiv | tuerueeu | | erueirei
647585858|784783434 | Ramesha Ramesha| tyuu5u4o| | ruieieiei
Earlier following is the command used to remove spaces:
sed s/[[:space]]*|/|/g; s/[ \t]*$//g < File1 > File2
Output was:
262774372|58959454|Rajiv Rajiv|tuerueeu||erueirei
647585858|784783434|Ramesha Ramesha|tyuu5u4o||ruieieiei
Time taken to process file was 3.5 days
Later I added tr command before the sed to remove spaces faster by adding the following
tr -s ' ' '' < File1 > File2
sed 's/[[:space]]*|/|/g; s/[ \t]*$//g;s/^[ \t]*//g;' < File 2 > File3
Output was:
262774372|58959454|Rajiv Rajiv|tuerueeu||erueirei
647585858|784783434| Ramesha Ramesha|tyuu5u4o||ruieieiei
Time taken to process file was less than a minute, since the big spaces are translated faster.
I am not able to retain the spaces between the characters as is, since tr -s will squeeze the space to one space.
The value | Rajiv Rajiv | -> changed to |Rajiv Rajiv|
I have to retain the space..... ie., |Rajiv Rajiv|
Please let me know if you have any workaround...
Thanks,
Rajiv
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I need to merge data from more than one file and I am using
while read line_record
do
field1=`echo $line_record | awk -F "," '{ print $1 }'`
echo $line_record >> $outFile
if
then
while read new_linerec
do
echo $new_linerec... (3 Replies)
Discussion started by: skrakesh
3 Replies
2. UNIX for Dummies Questions & Answers
Hi Experts,
I have a 2 GB flat file which have unicode field, some of them are blanks and its size is 4000 character. In the existing system SED command removes the spaces. Because of this field itself....it is taking almost three days to complete the file processing. I removed sed and... (0 Replies)
Discussion started by: RcR
0 Replies
3. Programming
I am very new to C programming.
How could I write a C program that could count the characters, words, spaces, and punctuations in a text file?
Any help will be really appreciated. I am doing this as part of my C learning exercise.
Thanks,
Ajay (4 Replies)
Discussion started by: ajay41aj
4 Replies
4. Shell Programming and Scripting
hello,
i 'd like your help about a bash script which:
1. finds inside the html file (it is attached with my post) the code number of the Latest Stable Kernel,
2.finds the link which leads to the download location of the Latest Stable Kernel version,
(the right link should lead to the file... (3 Replies)
Discussion started by: alex83
3 Replies
5. Shell Programming and Scripting
Hey all,
Fist post, so be kind... I have written an expect script which logs into a terminal and gathers several screens of information. Unfortunately the log file gives me all the special escape and control characters from the terminal. I am hoping to use a combination of shell scripting, sed,... (1 Reply)
Discussion started by: mpacer
1 Replies
6. Shell Programming and Scripting
Hi All,
I have written a C program to solve this problem but I am eager to know whether the same output can be obtained using sed or awk?
This is the input:
star
ferry
computer
symbol
prime
time
This is the output:
starferry
ferrycomputer
computersymbol
symbolprime
primetime (7 Replies)
Discussion started by: shoaibjameel123
7 Replies
7. Shell Programming and Scripting
I have a file that contains the schedule for a tournament with 41 teams. The team names have spaces in them. I would like to search for each teams schedule and then save that to that teams file
For example
Team name: "Team Two"
I would like to search for all the games for "Team Two" and... (8 Replies)
Discussion started by: knijjar
8 Replies
8. Shell Programming and Scripting
Hi all,
Is there a sed/awk cmd that will remove blank space from between words in a particular field, replacing with a single space?
Field containing 'E's in the example below:
Example input file:
AAAAA AA|BBBB|CCCCCCC|DDDDDD |EEEE EEEEEE| FFF FFFFF|
... (6 Replies)
Discussion started by: dendright
6 Replies
9. Shell Programming and Scripting
Hi
I have strings like these :
Vengeance mitt
Men Vengeance gloves
Women Quatro Windstopper Etip gloves
Quatro Windstopper Etip gloves
Girls Thermobite hooded jacket
Thermobite Triclimate snow jacket
Boys Thermobite Triclimate snow jacket
and I would like to get the lower case words at... (2 Replies)
Discussion started by: louisJ
2 Replies
10. Shell Programming and Scripting
Hi All,
I need one help to replace particular words in file based on if finds another words in that file .
i.e.
my self is peter@king.
i am staying at north sydney.
we all are peter@king.
How to replace peter to sham if it finds @king in any line of that file.
Please help me... (8 Replies)
Discussion started by: Rajib Podder
8 Replies
LEARN ABOUT DEBIAN
sql::reservedwords::postgresql
SQL::ReservedWords::PostgreSQL(3pm) User Contributed Perl Documentation SQL::ReservedWords::PostgreSQL(3pm)
NAME
SQL::ReservedWords::PostgreSQL - Reserved SQL words by PostgreSQL
SYNOPSIS
if ( SQL::ReservedWords::PostgreSQL->is_reserved( $word ) ) {
print "$word is a reserved PostgreSQL word!";
}
DESCRIPTION
Determine if words are reserved by PostgreSQL.
METHODS
is_reserved( $word )
Returns a boolean indicating if $word is reserved by either PostgreSQL 7.3, 7.4, 8.0 or 8.1.
is_reserved_by_postgresql7( $word )
Returns a boolean indicating if $word is reserved by either PostgreSQL 7.3 or 7.4.
is_reserved_by_postgresql8( $word )
Returns a boolean indicating if $word is reserved by either PostgreSQL 8.0 or 8.1.
reserved_by( $word )
Returns a list with PostgreSQL versions that reserves $word.
words
Returns a list with all reserved words.
EXPORTS
Nothing by default. Following subroutines can be exported:
is_reserved
is_reserved_by_postgresql7
is_reserved_by_postgresql8
reserved_by
words
SEE ALSO
SQL::ReservedWords
<http://www.postgresql.org/docs/manuals/>
AUTHOR
Christian Hansen "chansen@cpan.org"
COPYRIGHT
This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.8.8 2008-03-28 SQL::ReservedWords::PostgreSQL(3pm)