Sponsored Content
Top Forums Shell Programming and Scripting Removing White spaces from a huge file Post 302990011 by amvip on Friday 20th of January 2017 02:16:10 PM
Old 01-20-2017
Removing White spaces from a huge file

I am trying to remove whitespaces from a file containing sample data as:
Code:
457 <EOFD> Mar  1 2007 12:00:00:000AM   <EOFD> Mar 31 2007 12:00:00:000AM   <EOFD>  system  <EORD> 458 <EOFD>    Mar  1 2007 12:00:00:000AM<EOFD>agf <EOFD> Apr 20 2007  9:10:56:036PM    <EOFD>  prodiws<EORD>

. Basically these files are delimited extracted files from a database with the delimiters as <EOFD>
I am using the below command to remove the whitespace
perl -pi -e 's/[[:space:]]*\<EOFD\>[[:space:]]*/\<EOFD\>/g' sample.dat

The above command is part of shell scripts t.ksh which is being invoked on the command line and internally runs the perl command.
It is working fine for moderately huge files but once the file reaches a conderable huge size, say 2.2 -3 GB, it empties the file content and the scripts exist with the below error:

./t.ksh[249]: 700492 Memory fault(coredump)
Server Message: <my unix machine name>- Msg 208, Level 16, State 1:


[249] - this points to my perl command. Also, I see a huge coredump file getting created in the directory from where I am invoking the shell script t.ksh. The file system has adequate space.

Can someone advise whats wrong here or any workaround?

Regards

Last edited by Scrutinizer; 01-22-2017 at 06:59 AM.. Reason: icode -> code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

deleting white spaces in a file

Hello Guys, I am a newbie to unix. I am having a requirement. Please help me for finding a solution for this, I am having a file as mentioned below: $ cat shank ackca acackac akcajc akcjkcja akcj ckcklc I want to delete all the white spaces in this file, I tried... (2 Replies)
Discussion started by: mraghunandanan
2 Replies

2. Shell Programming and Scripting

ksh: removing all white spaces

'String' file contains the following contents, D11, D31, D92, D29, D24, using ksh, I want to remove all white spaces between characters no matter how long the string is. Would you please give me some help? (1 Reply)
Discussion started by: yoonius
1 Replies

3. Solaris

removing special characters, white spaces from a field in a file

what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code:: Code: awk -F":"... (2 Replies)
Discussion started by: priyanka3006
2 Replies

4. Shell Programming and Scripting

Removing blank spaces, tab spaces from file

Hello All, I am trying to remove all tabspaces and all blankspaces from my file using sed & awk, but not getting proper code. Please help me out. My file is like this (<b> means one blank space, <t> means one tab space)- $ cat file NARESH<b><b><b>KUMAR<t><t>PRADHAN... (3 Replies)
Discussion started by: NARESH1302
3 Replies

5. UNIX for Dummies Questions & Answers

[Solved] Help with using tr - Removing white spaces

Hi, I have a file that contains whitespaces with spaces and spaces and tabs on each line and am wanting to remove the whitespaces. My version of sed is one that does not recognize \t etc. The sed and awk one-liners below that I found via Google both does not work. So my next best... (3 Replies)
Discussion started by: newbie_01
3 Replies

6. Post Here to Contact Site Administrators and Moderators

Want a tcl script to compare a string in a file ignoring white spaces

Hi , I want a tcl script to search a string ignoring whitespaces in a .log file . It should correctly match . The string are as follows "Output-Maps 1 1 0 0 0" 1 and Active Intermediate-Maps 0 0 0 ... (1 Reply)
Discussion started by: kulua
1 Replies

7. UNIX for Advanced & Expert Users

Performance problem with removing duplicates in a huge file (50+ GB)

I'm trying to remove duplicate data from an input file with unsorted data which is of size >50GB and write the unique records to a new file. I'm trying and already tried out a variety of options posted in similar threads/forums. But no luck so far.. Any suggestions please ? Thanks !! (9 Replies)
Discussion started by: Kannan K
9 Replies

8. Shell Programming and Scripting

Remove white spaces from flat file generated from Oracle table...

I have to export data from table into flat file with | delimited. In the ksh file, I am adding below to do this activity. $DBSTRING contains the sqlplus command and $SQL_STRING contains the SQL query. File is created properly with the data as per SQL command. I am getting white spaces in the... (1 Reply)
Discussion started by: mgpatil31
1 Replies

9. Shell Programming and Scripting

Removing blank/white spaces and special characters

Hello All , 1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :- ;05/12/1990 ;31/03/2014 ; Output:- ;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Discussion started by: himanshu sood
6 Replies

10. Shell Programming and Scripting

List and Delete Files which are older than 7 days, but have white spaces in file name

I need to list and delete all files in current older which are olderthan 7 days. But my file names have white spaces. Before deleting I want to list all the files, so that I can verify.find . -type f -mtime +7 | xargs ls -l {} But the ls command is the working on the files which have white... (16 Replies)
Discussion started by: karumudi7
16 Replies
ppmtosixel(1)						      General Commands Manual						     ppmtosixel(1)

NAME
ppmtosixel - convert a portable pixmap into DEC sixel format SYNOPSIS
ppmtosixel [-raw] [-margin] [ppmfile] DESCRIPTION
Reads a portable pixmap as input. Produces sixel commands (SIX) as output. The output is formatted for color printing, e.g. for a DEC LJ250 color inkjet printer. If RGB values from the PPM file do not have maxval=100, the RGB values are rescaled. A printer control header and a color assignment table begin the SIX file. Image data is written in a compressed format by default. A printer control footer ends the image file. OPTIONS
-raw If specified, each pixel will be explicitly described in the image file. If -raw is not specified, output will default to com- pressed format in which identical adjacent pixels are replaced by "repeat pixel" commands. A raw file is often an order of magni- tude larger than a compressed file and prints much slower. -margin If -margin is not specified, the image will be start at the left margin (of the window, paper, or whatever). If -margin is speci- fied, a 1.5 inch left margin will offset the image. PRINTING
Generally, sixel files must reach the printer unfiltered. Use the lpr -x option or cat filename > /dev/tty0?. BUGS
Upon rescaling, truncation of the least significant bits of RGB values may result in poor color conversion. If the original PPM maxval was greater than 100, rescaling also reduces the image depth. While the actual RGB values from the ppm file are more or less retained, the color palette of the LJ250 may not match the colors on your screen. This seems to be a printer limitation. SEE ALSO
ppm(5) AUTHOR
Copyright (C) 1991 by Rick Vinci. 26 April 1991 ppmtosixel(1)
All times are GMT -4. The time now is 04:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy