|
|||||||
| Forums | Search Forums | Register | Forum Rules | Man Pages | Albums | FAQ | Members | Calendar | Search | Today's Posts | Mark Forums Read |
| UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !! |
|
|
|
Thread Tools | Search this Thread | Display Modes |
|
#1
|
|||
|
|||
|
Large file data handling issue
I have a single record large file, semicolon ';' and pipe '|' separated. I am doing a vi on the file. It is throwing an error "File to long" I need to actually remove the last | symbol from this file. Code:
sed -e 's/\|*$//' filename is working fine for small files. But not working on this big file. The file size is "614077" Last edited by Scrutinizer; 11-14-2012 at 03:40 PM.. Reason: code tags |
| Sponsored Links | ||
|
|
#2
|
|||
|
|||
|
614077 what? 600K isn't very large at all, sed should be able to handle it fine.
Even if it's a large file, sed will still work with it, it will just take longer. What might be a problem are really really long lines... |
| Sponsored Links | ||
|
|
#3
|
|||
|
|||
|
hmm, yes this is a 600K file and as already said it is a single record. How can i handle the file manupulation?
|
|
#4
|
||||
|
||||
|
Quote:
Paste the first and last 100 bytes of the file. Also try it this way: Code:
sed 's/|$//' filename |
| Sponsored Links | |
|
|
#5
|
|||
|
|||
|
Thanks for the reply.. but your command also didnt work.
With the small files i can see record values and the record getting trimmed off with the trailling '|'. When I use the same command with my actual file there is nothing that is coming on screen even if I echo it into a newfile nothing with get populated in the new file. Starting bytes 001;04.5.1;2012-10-25 08:47:41;ABCDE||3;1351169231;1351169261;;;1351169256;1351169256;1351169261;;;;;;;;;;;00:00:05; Ending bytes. 255;8;1;incoming;;;;;2;0;;;0;;10;0;0;0;0;;abc.com;x1-6-00-1c-fb-2f-8e-14.XXXX.xc.xr.com;;;;;0;000000;;Singh Gurkamal;1;;;this is a very big file;;;||1000;2012-10-25 08:49:03| |
| Sponsored Links | |
|
|
#6
|
|||
|
|||
|
Is this "file" actually a "stream" with no record terminators?
|
| Sponsored Links | |
|
|
#7
|
|||
|
|||
|
yes.. I am getting from where u r coming.. do u mean the line feed??
|
| Sponsored Links | ||
|
![]() |
| Thread Tools | Search this Thread |
| Display Modes | |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| UNIX file handling issue | Gurkamal83 | Shell Programming and Scripting | 3 | 11-14-2012 03:28 AM |
| Advice regarding filesystems handling large number of files | shoaibjameel123 | Red Hat | 2 | 09-20-2011 06:02 AM |
| Severe performance issue while 'grep'ing on large volume of data | Souvik | Shell Programming and Scripting | 6 | 04-04-2011 04:47 AM |
| UNIX File handling -Issue in reading a file | KuldeepSinghTCS | Shell Programming and Scripting | 1 | 08-16-2010 11:47 AM |
| Performance issue in UNIX while generating .dat file from large text file | KRAMA | Shell Programming and Scripting | 19 | 05-20-2009 10:55 AM |
|
|