|
|
|
google unix.com
|
|||||||
| Forums | Register | Forum Rules | Links | Albums | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| SUN Solaris The Solaris Operating System, usually known simply as Solaris, is a free Unix-based operating system introduced by Sun Microsystems . |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Splitting huge XML Files into fixsized wellformed parts | Malapha | Shell Programming and Scripting | 0 | 03-17-2008 11:35 AM |
| How to remove FIRST Line of huge text file on Solaris | madoatz | UNIX for Dummies Questions & Answers | 5 | 06-23-2007 02:19 PM |
| comparing Huge Files - Performance is very bad | madhukalyan | UNIX for Dummies Questions & Answers | 5 | 10-10-2006 11:58 PM |
| text files, ASCII files, binary files and ftp transfers | Perderabo | Answers to Frequently Asked Questions | 0 | 04-08-2004 05:25 PM |
| grep multiple text files in folder into 1 text file? | coppertone | UNIX for Dummies Questions & Answers | 7 | 08-23-2002 03:50 PM |
Powered by 
|
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
07-16-2004
|
||||
|
||||
|
Huge (repeated Entry) text files
Somebody HELP!
I have a huge log file (TEXT) 76298035 bytes. It's a logfile of IMEIs and IMSIS that I get from my EIR node. Here is how the contents of the file look like: 000000, 1 33016382000913 652020100423994 1 33016382002353 652020100430743 1 33017035101003 652020100441736 .... .... .... 235800, 1 35725620987678 652020100545862 Problem is, the file is to some degree made huge by repeated entries ( repeated lines - non consecutive). I have tried this code to eliminate the repeated entries: cat myfile | sed -n 'G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P' | tee mynewfile | wc -l but it takes forever and stops midway, at 024000 instead of 235800. |
| Bookmarks |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
Threaded Mode
