calculation on a large file

calculation on a large file
# 1
08-18-2009
calculation on a large file

Trying to do some simple maths on a large file. Excel works fine but I have 1 million entries:

If the difference between a number in column 2 and the one above it is more than 100 the insert a new line

8 4001 4100
8 4101 4200
8 4201 4300
8 15901 16000
8 15910 16100

For column 2
if 4101-4001 is > than 100 then insert a new line
else 'move onto next number'
end if
exit loop

The result for the example given should be:
8 4001 4100
8 4101 4200
8 4201 4300

8 15901 16000
8 15910 16100

Thanks for any help

L
 dr_sabz View Public Profile for dr_sabz Find all posts by dr_sabz
# 2
08-18-2009

Output:

 rubin View Public Profile for rubin Find all posts by rubin

Test Your Knowledge in Computers #219
Difficulty: Easy
When a data packet comes in to a port on a router, the router reads the network address information in the packet footer to determine the ultimate destination.
True or False?

Help with awk percentage calculation from a file

i have a file say test with the below mentioned details Folder Name Total space Space used /test/test1 500.1GB 112.0 GB /test/test2 3.2 TB 5TB /test/test3 3TB 100GB i need to calculate percentage of each row based on total space and space used and copy...

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +...

How to get full width of screen, when using large calculation with bc ?

Experts, When doing large calculation the digits in the screen are limiting to 68 digit and then with a \ ( backspace) next line comes. example: ubuntu# echo "123456789 ^ 50 " | bc 37648602365872212683379005814670372328125515868188009630652959693316\...

How do I include the file being compared into calculation?

nawk -F, 'NR==FNR{file=FILENAME;a++;next} a{if(FILENAME~file)next;b++;} END{ for(i in a){if(a && !b){print "NEW: "i}} for(i in b){if(b)print i"\t\t"b}}' OFS=, 123.csv *.csv I need to include 123.csv into the equation for the total output currently it compares whatever is on 123.csv against...

math calculation for a txt file

Hi All, I have a text file which is only one column. I want to multiply all these values in file1.txt by 0.01 and get the output.txt file1.txt 65 85 90 ... output.txt 0.65 0.85 0.90 ... Thanks. Please use code tags when posting data and code samples!

Calculation from a file

Hi , I have a file with below format 6/1/200618:00:011348797339524635352499218493964052971 6/1/200617:00:011348672070311735349025833693964052971 6/1/200616:00:011348546711070035345679259193964052971 I want a output as below 6/1/200618:00:011348797339524635352499218493964052971...

KSH SHELL: problem calculation number of lines inside compressed file

Hi Gurus, I am working with a korn shell script to simplify some operations of calculation number of lines inside compressed file. The called function (inside a cycle) is the following: ######################################### # F.ne: CheckCount #########################################...

File Size calculation with AWK

Hello Friends, Im calculating file sizes with below AWK script. I do this before some spesific files are transferred. I run the script it works but after several running it stuck with a limit of 2147483647 (2 Gbytes -1 byte) and cant exceed this. Something is wrong and I can't proceed, would...

Performance issue in UNIX while generating .dat file from large text file

Hello Gurus, We are facing some performance issue in UNIX. If someone had faced such kind of issue in past please provide your suggestions on this . Problem Definition: /Few of load processes of our Finance Application are facing issue in UNIX when they uses a shell script having below...

Help In Calculation of large values in loop

Hi Gurus, I am writing a shell script in which i need to strip out the numbers from file the values are unknown i. e. the range cannot be predicted.. and in my current program the sum of values is not coming as desired i think the value of calculation is crossing the range i.e. after some...