Sponsored Content
Top Forums Shell Programming and Scripting Removing White spaces from a huge file Post 302990011 by amvip on Friday 20th of January 2017 02:16:10 PM
Old 01-20-2017
Removing White spaces from a huge file

I am trying to remove whitespaces from a file containing sample data as:
Code:
457 <EOFD> Mar  1 2007 12:00:00:000AM   <EOFD> Mar 31 2007 12:00:00:000AM   <EOFD>  system  <EORD> 458 <EOFD>    Mar  1 2007 12:00:00:000AM<EOFD>agf <EOFD> Apr 20 2007  9:10:56:036PM    <EOFD>  prodiws<EORD>

. Basically these files are delimited extracted files from a database with the delimiters as <EOFD>
I am using the below command to remove the whitespace
perl -pi -e 's/[[:space:]]*\<EOFD\>[[:space:]]*/\<EOFD\>/g' sample.dat

The above command is part of shell scripts t.ksh which is being invoked on the command line and internally runs the perl command.
It is working fine for moderately huge files but once the file reaches a conderable huge size, say 2.2 -3 GB, it empties the file content and the scripts exist with the below error:

./t.ksh[249]: 700492 Memory fault(coredump)
Server Message: <my unix machine name>- Msg 208, Level 16, State 1:


[249] - this points to my perl command. Also, I see a huge coredump file getting created in the directory from where I am invoking the shell script t.ksh. The file system has adequate space.

Can someone advise whats wrong here or any workaround?

Regards

Last edited by Scrutinizer; 01-22-2017 at 06:59 AM.. Reason: icode -> code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

deleting white spaces in a file

Hello Guys, I am a newbie to unix. I am having a requirement. Please help me for finding a solution for this, I am having a file as mentioned below: $ cat shank ackca acackac akcajc akcjkcja akcj ckcklc I want to delete all the white spaces in this file, I tried... (2 Replies)
Discussion started by: mraghunandanan
2 Replies

2. Shell Programming and Scripting

ksh: removing all white spaces

'String' file contains the following contents, D11, D31, D92, D29, D24, using ksh, I want to remove all white spaces between characters no matter how long the string is. Would you please give me some help? (1 Reply)
Discussion started by: yoonius
1 Replies

3. Solaris

removing special characters, white spaces from a field in a file

what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code:: Code: awk -F":"... (2 Replies)
Discussion started by: priyanka3006
2 Replies

4. Shell Programming and Scripting

Removing blank spaces, tab spaces from file

Hello All, I am trying to remove all tabspaces and all blankspaces from my file using sed & awk, but not getting proper code. Please help me out. My file is like this (<b> means one blank space, <t> means one tab space)- $ cat file NARESH<b><b><b>KUMAR<t><t>PRADHAN... (3 Replies)
Discussion started by: NARESH1302
3 Replies

5. UNIX for Dummies Questions & Answers

[Solved] Help with using tr - Removing white spaces

Hi, I have a file that contains whitespaces with spaces and spaces and tabs on each line and am wanting to remove the whitespaces. My version of sed is one that does not recognize \t etc. The sed and awk one-liners below that I found via Google both does not work. So my next best... (3 Replies)
Discussion started by: newbie_01
3 Replies

6. Post Here to Contact Site Administrators and Moderators

Want a tcl script to compare a string in a file ignoring white spaces

Hi , I want a tcl script to search a string ignoring whitespaces in a .log file . It should correctly match . The string are as follows "Output-Maps 1 1 0 0 0" 1 and Active Intermediate-Maps 0 0 0 ... (1 Reply)
Discussion started by: kulua
1 Replies

7. UNIX for Advanced & Expert Users

Performance problem with removing duplicates in a huge file (50+ GB)

I'm trying to remove duplicate data from an input file with unsorted data which is of size >50GB and write the unique records to a new file. I'm trying and already tried out a variety of options posted in similar threads/forums. But no luck so far.. Any suggestions please ? Thanks !! (9 Replies)
Discussion started by: Kannan K
9 Replies

8. Shell Programming and Scripting

Remove white spaces from flat file generated from Oracle table...

I have to export data from table into flat file with | delimited. In the ksh file, I am adding below to do this activity. $DBSTRING contains the sqlplus command and $SQL_STRING contains the SQL query. File is created properly with the data as per SQL command. I am getting white spaces in the... (1 Reply)
Discussion started by: mgpatil31
1 Replies

9. Shell Programming and Scripting

Removing blank/white spaces and special characters

Hello All , 1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :- ;05/12/1990 ;31/03/2014 ; Output:- ;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Discussion started by: himanshu sood
6 Replies

10. Shell Programming and Scripting

List and Delete Files which are older than 7 days, but have white spaces in file name

I need to list and delete all files in current older which are olderthan 7 days. But my file names have white spaces. Before deleting I want to list all the files, so that I can verify.find . -type f -mtime +7 | xargs ls -l {} But the ls command is the working on the files which have white... (16 Replies)
Discussion started by: karumudi7
16 Replies
Graphics::Primitive::Insets(3pm)			User Contributed Perl Documentation			  Graphics::Primitive::Insets(3pm)

NAME
Graphics::Primitive::Insets - Space between things DESCRIPTION
Graphics::Primitive::Insets represents the amount of space that surrounds something. This object can be used to represent either padding or margins (in the CSS sense, one being inside the bounding box, the other being outside) SYNOPSIS
use Graphics::Primitive::Insets; my $insets = Graphics::Primitive::Insets->new({ top => 5, bottom => 5, left => 5, right => 5 }); METHODS
Constructor new Creates a new Graphics::Primitive::Insets. Instance Methods as_array Return these insets as an array in the form of top, right, bottom and left. bottom Set/Get the inset from the bottom. equal_to Determine if these Insets are equal to another. left Set/Get the inset from the left. right Set/Get the inset from the right. top Set/Get the inset from the top. zero Sets all the insets (top, left, bottom, right) to 0. AUTHOR
Cory Watson, "<gphat@cpan.org>" SEE ALSO
perl(1) COPYRIGHT &; LICENSE Copyright 2008-2010 by Cory G Watson. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.12.3 2010-08-21 Graphics::Primitive::Insets(3pm)
All times are GMT -4. The time now is 04:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy