Nice with the three times. It could also be written as
Code:
awk 'NR==FNR{next} FNR*3>NR{exit}1' file file
-v h="$(wc -l file)" is a nicer alternative to remove the leading space (by awk , rather than shell), although there was no determinable difference in speed..
In testing (using the second result, with file in read cache):
Seems to be the fastest solution with 100001 lines..
Code:
$ time awk -v h="$(wc -l file)" 'NR>h/2{exit}1' file > /dev/null
real 0m0.055s
$ time awk 'NR!=FNR {if (FNR*3>NR) exit; print}' file file > /dev/null
real 0m0.113s
$ time awk 'FNR==NR{lines++;next}FNR>lines/2{exit}1' file file >/dev/null
real 0m0.119s
$ time awk 'FNR==NR{lines++;next}FNR<=lines/2' file file >/dev/null
real 0m0.150s
$ time awk '{x[++xc] = $0}END{for(i = 1; i <= xc/2; i++) print x[i]}' file > /dev/null
real 0m0.204s
$ time awk '{i=NR/2} i in A{print A[i]; delete A[i]}{A[NR]=$0}' file > /dev/null
real 0m0.310s
$ time awk -F"\n" -v OFS="\n" '{X=X$0"\n"}END{$0=X;NF=NR/2;print}' file > /dev/null
real 0m11.353s
The latter did not seem to work properly with BSD awk, since it produced 10,002 lines (an additional line due to an extra newline) and in mawk it produced mawk: program limit exceeded: maximum number of fields size=32767
With gawk though it worked really well though:
Code:
$ time gawk -F"\n" -v OFS="\n" '{X=X$0"\n"}END{$0=X;NF=NR/2;print}' file > /dev/null
real 0m0.067s
As is most often the case, mawk was fastest:
Code:
$ time mawk -v h="$(wc -l file)" 'NR>h/2{exit}1' file > /dev/null
real 0m0.021s
In fact when using mawk, it became less relevant, since it performed so fast that there was a negligible difference between the fastest solutions
gawk 4 (which uses some of the techniques of mawk) now also often performs better than gawk 3 and BSD awk..
I need to split the incoming source file in to multiple files using awk.
Split position is (6,13) : 8 positions
All the records that are greater than 20170101 and less than or equal to 20181231 should go in a split file with file name as source... (11 Replies)
HI Guys,
Can you please let me know the procedure to reestablish the SRDF which is half split, as you can see from the below O/P that one of the device is synchronized and other devices are in split mode
Source (R1) View Target (R2) View MODES... (2 Replies)
Im stumped on this one. Id like to echo into a .txt file all names for an xml feed in a huge folder. Can that be done?? Id need to echo <name>This name</name> in client.xml files.
$path="/mnt/windows/path"
echo 'recording names'
cd "$path"
for names in $path
than Im stuck on... (2 Replies)
Hi all,
I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example:
... (2 Replies)
Hi!
I have a file like this:
a,b,c,12,d,e
a,b,c,13,d,e
a,b,c,14,d,e
a,b,c,15,d,e
a,b,c,16,d,e
a,b,c,17,d,e
I need to split that file in two:
If field 4 is equal or higher than 14 that row goes to one file and if it is equal or higher than 15 to another.
Can anyone point me in the... (2 Replies)
I did a lot of search on this forum on spiting file; found a lot, but my requirement is a bit different, please guide.
Master file:
x:start:5
line1:23
line2:12
2:90
x:end:5
x:start:2
45:56
22:90
x:end:2
x:start:3
line1:23
line2:12
x:end:3
x:start:2
line5:23 (1 Reply)
I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this.
For example:
split -l 3000000 filename.txt
This is very slow and it splits the file with 3 million records in each... (10 Replies)
I am trying to read a file and split the file into multiple files. I need to create new files with different set of lines from the original file. ie, the first output file may contain 10 lines and the second 100 lines and so on. The criteria is to get the lines between two lines starting with some... (8 Replies)
07-21-2014
