Visit Our UNIX and Linux User Community


best practice please


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting best practice please
# 1  
Old 06-10-2009
Power best practice please

I have a data file that I tail and dump the data into a new file at midnight.
To work I had to use tail -n2 livefile.csv >> storefile.csv. This is ging me 2 or 3 entries at a time into the storefile.csv.
if I continue this then I will have to run a second pass on the file to remove the duplicates from the csv file. all the files start with the date.
I only need one entry per night.

What is the best way to do this ? I have 6 files to do this to each night.

Thank you

Mike

i.e.

06/06/2009,23:56:04,22.8
06/06/2009,23:56:09,22.8
06/06/2009,23:56:14,22.8
06/07/2009,23:56:09,8.5
06/07/2009,23:56:14,8.5
06/08/2009,23:56:09,-2.5
06/08/2009,23:56:14,-2.5
06/09/2009,23:56:09,-6.9
06/09/2009,23:56:14,-6.9

should be
06/06/2009,23:56:14,22.8
06/07/2009,23:56:14,8.5
06/08/2009,23:56:14,-2.5
06/09/2009,23:56:14,-6.9
# 2  
Old 06-10-2009
this will do
Code:
awk -F"," '{A[$1]=$0}END{for ( i in A) {print A[i]}}' filename

# 3  
Old 06-10-2009
Thank you It worked but ...

It reversed the order of all the dates in the file.

Now they are all in there starting yesterday and ending in begining.
Its kind of funny looking at the graphs backwards.

-----Post Update-----

I'm not sure if its right or not but this does work

sort /var/www/solar_month3.csv >> /var/www/solar_month2.csv
tail -n2 /var/www/rec_month.csv >> /var/www/rec_month2.csv
awk -F"," '{A[$1]=$0}END{for ( i in A) {print A[i]}}' /var/www/rec_month2.csv >> /var/www/rec_month3.csv
rm /var/www/rec_month2.csv
sort /var/www/rec_month3.csv >> /var/www/rec_month2.csv
#
# 4  
Old 06-10-2009
awk's associative arrays neither are stored in any particular order or accessed in any particular order. Here's the trick 'round this - assuming your input file comes sorted by date/time:

Code:
awk -F"," '!($1 in A) {key[++key[0]]=$1} {A[$1]=$0}END{for (i=1;i<=key[0];i++){print A[key[i]]}}' filename


Last edited by vgersh99; 06-10-2009 at 07:05 PM..
# 5  
Old 06-10-2009
Bug Works great and much cleaner Thank you

Took out all of my extras that weren't needed

Previous Thread | Next Thread
Test Your Knowledge in Computers #743
Difficulty: Medium
The AMD Optimizing C/C++ Compiler (AOCC) is a free, open source, optimizing compiler from AMD.
True or False?

10 More Discussions You Might Find Interesting

1. HP-UX

LVM PE. Best practice

Hi everyone. Unfortunately I can't find advice's about LVM Physical Extent for HPUX. As far as I know we can set for 1PE size from 1MB to 256 MB. But in what cases it should be different? Maybe exist any strict rules, requirements or any best practice? (2 Replies)
Discussion started by: jess_t03
2 Replies

2. AIX

AIX practice

Dears, Kindly as preparing to get AIX certification I'd like to have your guide & support where can I find dump or exam forms for below :- Test 000-103: AIX 6.1 Basic Operations Test 000-221: AIX 7 Administration I want to look at real questions and try to solve it , is it possible to... (1 Reply)
Discussion started by: arm
1 Replies

3. UNIX for Dummies Questions & Answers

practice scripting

Is there any sites out there that have examples of things they want you to go and script for practice? I have looked around but have came up empty handed. I have been writing scripts for things I need on my nix box but have am wanting more challenging things to keep me learning. I would prefer... (3 Replies)
Discussion started by: linuxn00b
3 Replies

4. UNIX for Dummies Questions & Answers

hp ux practice

hi i am new in unix.i have installed linux in my pc,but i want to practice and be expatise in hp ux.but i have no hp server for do it.how could i practice hp ux?please help and give suggestion. (1 Reply)
Discussion started by: dipanjan123
1 Replies

5. Programming

how to practice my c++ skill

Hi all, I'v already read some basic books about c++ such as <c++ primer>,<The C++ standard library> etc.,and as we know,knowledge is easy to forget without practicing,so i wanna know some ways to practice my skill with programming.First,i know some ACM sites,however,solving problems on those sites... (4 Replies)
Discussion started by: homeboy
4 Replies

6. Shell Programming and Scripting

Best practice for bracket comparisons?

So, I have no formal higher education in programming at all and am self taught. I am now wondering what would be considered best practices? Like should I hard code a variable, then compare it to what I want to know or achieve, or should I just put the commands with in the brackets? Example, a... (5 Replies)
Discussion started by: tlarkin
5 Replies

7. UNIX for Dummies Questions & Answers

Newbie Wants Hardware For Practice?

I want a cheap dedicated machine to install a flavour of unix on and try to learn this OS. I know nothing about unix, just gone out and bought some books etc. Should I just get the cheapest old laptop off Ebay to install it on? What type of unix OS should I install?...a version of... (5 Replies)
Discussion started by: IBMPBC
5 Replies

8. Cybersecurity

best practice for logging options

Hi We just had an auditor tell us to formally review our logging parameters on the server and implement best practice for logging, based on a risk assessment. Phew! Before my time here, there was no reasoning behind what we chose to log or not log. Any ideas where such a "best practice"... (1 Reply)
Discussion started by: theonewhowas
1 Replies

9. Shell Programming and Scripting

What is the best practice?

I'm playing the shell game of moving and deleting files to make room for current DB backup. A set of files gets moved to serverB when serverA threshold is exceeded. I created two scripts one for serverA and one for serverB. Both scripts are run by cron entry. There is a difference of 12 minutes... (4 Replies)
Discussion started by: BigSky
4 Replies

10. UNIX for Dummies Questions & Answers

Samba Practice

I have a dual boot machine with both "Windoze ME" and Redhat Linux 7.2. I want to know if it is possible to practice with the Samba Suite on a dual boot machine if I was able to incorporate using IP addresses. I was thinking if your able to ping yourself why wouldn't you be able to administer... (3 Replies)
Discussion started by: bilal_aa
3 Replies

Featured Tech Videos