thanks for the quick answer but the output is not the expected:
I want to remove just the lines where the 4th column have consecutive values not all the others.
I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records.
contents of file looks like
30002157,40342424,OTC,mart_rec,100, ,0
30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Trying to cut down the size of some log files. Now that I write this out it looks more dificult than i thought it would be.
Need a bash script or command that goes sequentially through all lines of a file, and does this:
if field1 (space separated) is the number 2012 print the entire line. Do... (7 Replies)
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Hi,
I have a file with fields like below:
A;XYZ;102345;222
B;XYZ;123243;333
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
desired output:
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
ie, if the 4rth field is a duplicate.. i need only those records where... (5 Replies)
Hi All,
i have input file like below...
CA009156;20091003;M;AWBKCA72;123;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;321;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;;
CA009156;20091003;M;AWBKCA72;231;;CANADIAN... (2 Replies)
Hello,
I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Hi all,
I've got a file that has 12 fields. I've merged 2 files and there will be some duplicates in the following:
FILE:
1. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, 100
2. ABC, 12345, TEST1, BILLING, GV, 20/10/2012, C, 8, 100, AA, TT, (EMPTY)
3. CDC, 54321, TEST3,... (4 Replies)
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT DEBIAN
battery-stats
BATTERY-STATS(5) File Formats Manual BATTERY-STATS(5)NAME
battery-stats - collected battery statistics
DESCRIPTION
The battery-stats contains statistics about battery charge over time, as collected by the battery-stats-collector (8) daemon.
FORMAT
Each line in the file represents one sample and is of the form:
<seconds> <charge%> <powermode> <UTC-date> <UTC-time>
separated by spaces and terminated by a newline.
Where:
seconds
is the number of seconds since 1st Jan 1970 00:00:00 UTC
charge%
is the battery charge a number between 0 and 100
powermode
indicates the power mode:
0 A/C off
1 On battery
2 On backup power
UTC-date
a human-readable date in the format YYYY/MM/DD. Note that this is in UTC, not the local or system timezone.
UTC-time
a human-readable time in the format HH:MM:SS (using a 24-hour clock). Note that this is in UTC, not the local or system timezone.
minutes-left
is the BIOS estimate of how many minutes of running time the battery would provide. This may be unreliable, depending on the BIOS
through which it was collected.
EXAMPLE
A battery under charge might result in the following samples:
1032651245 27 1 2002/09/21 23:34:05 94
1032651275 28 1 2002/09/21 23:34:35 97
1032651305 29 1 2002/09/21 23:35:05 100
1032651335 30 1 2002/09/21 23:35:35 103
1032651365 30 1 2002/09/21 23:36:05 106
FILES
The default set-up is to save battery statistics in /var/log and rotate the logs weekly, which results in this set of files:
/var/log/battery-stats - current (most recent) statistics
/var/log/battery-stats.[0-9]+ - less recent statistics
/var/log/battery-stats.[0-9]+.gz - ancient statistics
SEE ALSO battery-graph(1), battery-stats-collector(8).
AUTHOR
Karl E. Jorgensen <karl@jorgensen.com>
September 23, 2002 BATTERY-STATS(5)