The UNIX and Linux Forums  


Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
to find header in Mp3 file and retrieve data shashi High Level Programming 2 09-12-2008 04:03 AM
How to extract data from a huge file? srsahu75 Shell Programming and Scripting 5 01-18-2008 05:06 AM
search and grab data from a huge file ting123 UNIX for Dummies Questions & Answers 1 06-06-2006 10:41 PM
sed, insert data from a file to another? ctcuser Shell Programming and Scripting 4 05-03-2005 02:43 PM
Insert a line as the first line into a very huge file shriek UNIX for Advanced & Expert Users 3 03-09-2005 01:22 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 02-10-2009
deepaktanna deepaktanna is offline
Registered User
  
 

Join Date: Feb 2009
Posts: 4
Question insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract

if i use sed, i have to redirect the output to a seperate file like

sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat

the same is true for awk

and in its simplist form i could say
echo 'H|PayerDataExtract' > ExtractDataFileWithHeader.dat
cat ExtractDataFile.dat >> ExtractDataFileWithHeader.dat
mv ExtractDataFileWithHeader.dat ExtractDataFile.dat

but in all of the above an extra file is created. If i were to do this in vi manually the extra file could be avoided.

Is there a way to avoid the extra file while still not having to manually use vi in a interactive manner? This is even more neccessary if the file uses over 50% filespace and an extra file will just double my usage , though temporarily
  #2 (permalink)  
Old 02-10-2009
quirkasaurus's Avatar
quirkasaurus quirkasaurus is offline
Registered User
  
 

Join Date: Jan 2009
Location: canton, michigan
Posts: 388
This is crazy, I agree.

Typically, in situations like this, I employ an "original_file_name.info" file instead.

In the .info file I'll put all the information necessary for another program/user
to verify the contents of the huge file i created.

There are other options . . . . like creating a dummy header record in
the file creation program . . . and then using fseek() to hop back to
the beginning and overwrite the header info....

but you may not have this much control over how this file is created.
  #3 (permalink)  
Old 02-10-2009
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
  
 

Join Date: Feb 2004
Location: NM
Posts: 5,800
There are 'edit in place' features for GNU sed and for perl (perl -pie ' perl script here').
However they do use tmp files behind the scenes
  #4 (permalink)  
Old 02-10-2009
ddreggors ddreggors is offline
Registered User
  
 

Join Date: Aug 2008
Posts: 91
You can even use the sed 'insert' command along with -i (inline) option to place the the "header" at a specific line number (like line 1).

Something like this should work:

Code:
sed -i '1 i \Some Header Text Here' ExtractDataFile.dat

  #5 (permalink)  
Old 02-11-2009
angheloko's Avatar
angheloko angheloko is offline
Registered User
  
 

Join Date: Jul 2008
Location: Philippines
Posts: 125
Why not create an ed script instead? And then use 'patch'? But that would still require a temp file in the background though you wouldn't notice.
  #6 (permalink)  
Old 02-18-2009
deepaktanna deepaktanna is offline
Registered User
  
 

Join Date: Feb 2009
Posts: 4
Unhappy

hi ddreggors/jim

i tried this - and it fails with the below error

sed -i '1 i \HeaderText' ExtractDataFile.txt
sed: illegal option -- i

The system is a SunOS ussun1l 5.8 Generic_117350-60 sun4u sparc SUNW,Sun-Fire-15000

the perl option is out for me as the client may not want to install any piece of software without business cause, however as long as the tmp file is created in the /tmp space and not my file-dir i think i'm ok - any more ideas anyone?
  #7 (permalink)  
Old 02-18-2009
ddreggors ddreggors is offline
Registered User
  
 

Join Date: Aug 2008
Posts: 91
try this:

Code:
sed 1'i\HeaderText' ExtractDataFile.txt

If this outputs to the screen as expected (this does not update the file) then the inner 'i' (insert command) is working fine.

Next try:


Code:
sed -i 1'i\HeaderText' ExtractDataFile.txt

If either gives an error then you may have to upgrade sed to get the use of the 'inline' flag (-i) or the insert command ('i\text') to be able to do this without writing to another file first and specifying a line number.
Closed Thread

Bookmarks

Tags
non-interactive vi

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 05:11 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0