Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Rearange fields within a csv

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rearange fields within a csv
# 1  
Old 12-21-2010
Rearange fields within a csv
Hello with sed and tr I have converted my "quite big" cronlist into the below csv file
Here is a sample:
Code:
05,15,15,25,35,45,55;*;*;*;*;/SCRIPT1;/SCRIPT1.log 
10;6,16;*;*;*;/SCRIPT2;/SCRIPT2.log 
30;*;*;*;*;/SCRIPT3;/SCRIPT3.cfg;/SCRIPT3.log  
50;8;*;*;*;/SCRIPT4;-1;/SCRIPT4.log 
30;5;*;*;*;/SCRIPT5;0;-1;/SCRIPT5.log

Unfortunatelly some scripts run with parameters, and some with configuration files
I need to find a way to add extra ;s so I can properly inport the columns into excel
Like:
Code:
05,15,15,25,35,45,55;*;*;*;*;/SCRIPT1;;;;/SCRIPT1.log 
10;6,16;*;*;*;/SCRIPT2;;;;/SCRIPT2.log 
30;*;*;*;*;/SCRIPT3;/SCRIPT3.cfg;;;/SCRIPT3.log  
50;8;*;*;*;/SCRIPT4;-1;;;;/SCRIPT4.log 
30;5;*;*;*;/SCRIPT5;;0;-1;/SCRIPT5.log

What will be the best way to implement this?
I am thinking storing into array with awk.
Any recommendatins ?
Thank you
# 2  
Old 12-21-2010
One wonders why, initially, but I'm sure there is a reason. The file would have to be imported into Excel as a delimited text file if you are using ; as the separator.

It looks like you want the format of the time controls first, followed by the first script and then up to four more ; separated fields, kind of right-justified, which seems a little odd. Left justified would be easier:-
Code:
crontab -l|while read mins hours day_of_mon month day_of_week f1 f2 f3 f4 f5
do
   echo "${mins};${hours};${day_of_mon month};${day_of_week};${f1};${f2};${f3};${f4};${f5}"
done > crontab.txt

To get right justified, you would need to adjust the values of f2 to f5, so:-
Code:
crontab -l|while read mins hours day_of_mon month day_of_week f1 f2 f3 f4 f5
do
   if [ "$f2" != "" ]
   then
      while [ "$f5" = "" ]
      do
         f5="${f4}"
         f4="${f3}"
         f3="${f2}"
         f2=""
      done
   fi
   echo "${mins};${hours};${day_of_mon month};${day_of_week};${f1};${f2};${f3};${f4};${f5}"
done > crontab.txt

That should shuffle items (except the first script) to the right.



Does this acheive the results you are after? If not, please write back and we'll have another look.



Robin
Liverpool/Blackburn
UK
This User Gave Thanks to rbatte1 For This Post:
drbiloukos
# 3  
Old 12-21-2010
I want to build

Code:
mins;hours;day_of_mon;month;day_of_week;script;conf_file;param1;param2;logfile

but some lines do not use parameters or conf files, so the output will be:

Code:
mins;hours;day_of_mon;month;day_of_week;script;;;;logfile

# 4  
Old 12-21-2010
So, how do you know which records have them and which do not?
In either case, the second chunk of code should do it. I'm still wondering why you need this, but......

If you are not getting the output you need, can you paste in the output of crontab -l and then the output you would want for a sample of the lines.




Cheers,
Robin
This User Gave Thanks to rbatte1 For This Post:
drbiloukos
# 5  
Old 12-21-2010
Maybe I've missed the point here, but if you want to import your crontab into Excel, why not just do it with the raw file using white space as the delimiter?
Each crontab line has 6 entries - minute, hour, day, month, day of week, command, so each will appear in a different column. Commands will be split across columns 6 to last, so you can process those at will in the spreadsheet
This User Gave Thanks to JerryHone For This Post:
drbiloukos
# 6  
Old 12-21-2010
Sample from my cron:

Code:
00 05  *  *  * (. ~/.profile ; timex /Data/bin/load_data.sh /Data/bin/load.cfg ) >> /Data/logs/load_data.log 2>&1
00 07 *  *  * (. ~/.profile ; timex /Data/bin/process -1 0 )   >> /Data/logs/processs.log 2>&1
#30 15,00 *  *  3,6     (. ~/.profile ; timex /Data/bin/analyze ) >> /Data/logs/analyze.log 2>&1

As you can see some script use cfg files and some use parameters.
I need to have ;; if no parameter is specified so it will load in the right column
# 7  
Old 12-30-2010
A correction required because of a typo in the second script offerrin:-

For the output statement, it should read
Code:
   echo "${mins};${hours};${day_of_mon};${month};${day_of_week};${f1};${f2};${f3};${f4};${f5}"

I had missed the };${ out between day_of_month and month and it threw a wobbler. Smilie

Correcting it and running against your sample data gave me:-
Code:
00;05;*;*;*;(.;~/.profile;;;timex;/Data/bin/load_data.sh /Data/bin/load.cfg ) >> /Data/logs/load_data.log 2>&1
00;07;*;*;*;(.;~/.profile;;;timex;/Data/bin/process -1 0 )   >> /Data/logs/processs.log 2>&1
#30;15,00;*;*;3,6;(.;~/.profile;;;timex;/Data/bin/analyze ) >> /Data/logs/analyze.log 2>&1


Is that near to what is required? It looks rather messy to me, probably because the input has lots of extra spaces in it. If this represents all of the input from crontab -l then you will need to recode my suggestion to cater for that. It is picking up the date bit okay, but the next "field" I'm calling f1 is selected as just (. then f2 gets .profile and f3 is ; which makes no sense.



Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

2. Emergency UNIX and Linux Support

How to convert the following characters in some fields in a csv file?

I have a csv file which is produced out of a SED command sed 's/|/","/g; s/^/"/; s/$/"/' A4.txt > A5.csv and I need either an addition to the SED command or a separate command to convert the following characters which occur within the fields in multiple lines 1) "=" to =" and 2)""~ to " (4 Replies)
Discussion started by: etldev
4 Replies

3. Shell Programming and Scripting

Merging fields in CSV

Hi experts, I have a csv file which has one field (ID) repeated multiple times with corresponding other field values. I need to convert this file in a format where for a ID all other values has to be present in single field. For Eg : Here in below file ID 1 is repeated 3 times with different... (7 Replies)
Discussion started by: bharathbangalor
7 Replies

4. Shell Programming and Scripting

Inserting new fields to a csv file

hi I have a csv file with few rows > cat job_stat 1,jobname1,somthing,somthing 2,jobname2,somthing,somthing 3,jobname3,somthing,somthing 4,jobname4,somthing,somthing I want to add few columns after the 2nd column and then append rest of the columns after the 3rd newly added... (3 Replies)
Discussion started by: midhun19
3 Replies

5. Shell Programming and Scripting

compare 2 CSV fields from same diff output

Attached is a file called diff.txt It is the output from this command: diff -y --suppress-common-lines --width=5000 1.txt 2.txt > diff.txt I have also attached 1.txt and 2.txt for your convenience. Both 1.txt and 2.txt contain one very long CSV string. File 1.txt is a CSV dump of... (0 Replies)
Discussion started by: gvolpini
0 Replies

6. Shell Programming and Scripting

pulling different fields from a csv file

Hi, I have a requirment where I need to pull different columns from a .csv file. Here is the sample of the csv file. account,item,flag1,flag2,flag3,flag4,flag5,......feed,tran I will be have a config.txt file which will have the following information. item,flag5,flag10,feed,tran... (2 Replies)
Discussion started by: akdevula
2 Replies

7. Shell Programming and Scripting

How to (n)awk lines of CSV with certain number of fields?

I have a CSV file with a variable number of fields per record. How do I print lines of a certain number of fields only? Several permutations of the following (including the use of escape characters) have failed to retrieve the line I'm after (1,2,3,4)... $ cat myfile 1,2,3,4 1,2,3 $ # Print... (1 Reply)
Discussion started by: cs03dmj
1 Replies

8. Shell Programming and Scripting

awk - rearange data

Dear All, once again I am encountering a problem with awk. The file looks like this: BroadLeaves 43.6 clc2006 37.6 Conifers 8.3 edge100 5.1 dem30sec 3.9 aspect 1.5 slope 0 dem30sec 58.3 Conifers 28.5 clc2006 7.3 edge100 3 slope 2.4 BroadLeaves 0.4 aspect 0.1 .... ... .. My... (9 Replies)
Discussion started by: creamcheese
9 Replies

9. UNIX for Dummies Questions & Answers

Fields in csv files using sed

Hi, I am working right now with a csv file and I want to insert an excel formula say to the 6th column. sample csv file: 1234,lag,0,77,544,,7 1234,lag,222,0,7,,7 at first i used a simple command: sed 's/^\(.\{17\}\)/\1word/' file.csv but the result is this: ... (2 Replies)
Discussion started by: paoie
2 Replies

10. UNIX for Dummies Questions & Answers

How to work with fields of a csv file?

I need to insert data into a perticular field of a csv file, lets say second field. Can any one help me to do this? I found that we can do it with sed. can any one guide me to accomplish this? thanks in advance. (12 Replies)
Discussion started by: praveen_b744
12 Replies
Login or Register to Ask a Question