Visit Our UNIX and Linux User Community


Save output of updated csv file as csv file itself, part 2


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Save output of updated csv file as csv file itself, part 2
# 1  
Old 04-11-2015
Save output of updated csv file as csv file itself, part 2

Hi,

I have another problem. I want to sort another csv file by the first field.
result.csv
Code:
SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw
/home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 
/home/intannf/foto5/2015_0313_090323_155.JPG,0.,-7.77224,110.37312,30.73,996.46,148.76,181.01,181.92,63.82 
/home/intannf/foto5/2015_0313_090142_124.JPG,0.,-7.77224,110.37312,30.73,996.46,148.75,181.13,182.06,63.87 
/home/intannf/foto5/2015_0313_085929_083.JPG,0.,-7.77224,110.37312,30.72,996.46,148.74,180.93,182.12,63.64 
/home/intannf/foto5/2015_0313_090710_225.JPG,0.,-7.77224,110.37312,30.72,996.46,148.77,181.09,181.86,63.78 
/home/intannf/foto5/2015_0313_090628_212.JPG,0.,-7.77223,110.37310,30.72,996.47,148.67,181.09,181.91,63.87 
/home/intannf/foto5/2015_0313_085942_087.JPG,0.,-7.77219,110.37317,30.76,996.47,148.71,181.12,182.17,63.78 
/home/intannf/foto5/2015_0313_090717_227.JPG,0.,-7.77217,110.37315,30.77,996.48,148.66,181.06,182.21,63.87

Output required:
Code:
SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw
/home/intannf/foto5/2015_0313_085929_083.JPG,0.,-7.77224,110.37312,30.72,996.46,148.74,180.93,182.12,63.64 
/home/intannf/foto5/2015_0313_085942_087.JPG,0.,-7.77219,110.37317,30.76,996.47,148.71,181.12,182.17,63.78 
/home/intannf/foto5/2015_0313_090142_124.JPG,0.,-7.77224,110.37312,30.73,996.46,148.75,181.13,182.06,63.87 
/home/intannf/foto5/2015_0313_090323_155.JPG,0.,-7.77224,110.37312,30.73,996.46,148.76,181.01,181.92,63.82 
/home/intannf/foto5/2015_0313_090628_212.JPG,0.,-7.77223,110.37310,30.72,996.47,148.67,181.09,181.91,63.87 
/home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 
/home/intannf/foto5/2015_0313_090710_225.JPG,0.,-7.77224,110.37312,30.72,996.46,148.77,181.09,181.86,63.78 
/home/intannf/foto5/2015_0313_090717_227.JPG,0.,-7.77217,110.37315,30.77,996.48,148.66,181.06,182.21,63.87

I have tried to sort them by sort -n -k2.1,1 -o result.csv -t, result.csv and sort -k2.1,1 -o result.csv -t, result.csv. But what i got that the heading line is always writen below the data. Please help me to figure it out. Thanks in advance.

Regards,
Intan
# 2  
Old 04-11-2015
Quote:
Originally Posted by refrain
Hi,

I have another problem. I want to sort another csv file by the first field.
result.csv
Code:
SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw
/home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 
/home/intannf/foto5/2015_0313_090323_155.JPG,0.,-7.77224,110.37312,30.73,996.46,148.76,181.01,181.92,63.82 
/home/intannf/foto5/2015_0313_090142_124.JPG,0.,-7.77224,110.37312,30.73,996.46,148.75,181.13,182.06,63.87 
/home/intannf/foto5/2015_0313_085929_083.JPG,0.,-7.77224,110.37312,30.72,996.46,148.74,180.93,182.12,63.64 
/home/intannf/foto5/2015_0313_090710_225.JPG,0.,-7.77224,110.37312,30.72,996.46,148.77,181.09,181.86,63.78 
/home/intannf/foto5/2015_0313_090628_212.JPG,0.,-7.77223,110.37310,30.72,996.47,148.67,181.09,181.91,63.87 
/home/intannf/foto5/2015_0313_085942_087.JPG,0.,-7.77219,110.37317,30.76,996.47,148.71,181.12,182.17,63.78 
/home/intannf/foto5/2015_0313_090717_227.JPG,0.,-7.77217,110.37315,30.77,996.48,148.66,181.06,182.21,63.87

Output required:
Code:
SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw
/home/intannf/foto5/2015_0313_085929_083.JPG,0.,-7.77224,110.37312,30.72,996.46,148.74,180.93,182.12,63.64 
/home/intannf/foto5/2015_0313_085942_087.JPG,0.,-7.77219,110.37317,30.76,996.47,148.71,181.12,182.17,63.78 
/home/intannf/foto5/2015_0313_090142_124.JPG,0.,-7.77224,110.37312,30.73,996.46,148.75,181.13,182.06,63.87 
/home/intannf/foto5/2015_0313_090323_155.JPG,0.,-7.77224,110.37312,30.73,996.46,148.76,181.01,181.92,63.82 
/home/intannf/foto5/2015_0313_090628_212.JPG,0.,-7.77223,110.37310,30.72,996.47,148.67,181.09,181.91,63.87 
/home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 
/home/intannf/foto5/2015_0313_090710_225.JPG,0.,-7.77224,110.37312,30.72,996.46,148.77,181.09,181.86,63.78 
/home/intannf/foto5/2015_0313_090717_227.JPG,0.,-7.77217,110.37315,30.77,996.48,148.66,181.06,182.21,63.87

I have tried to sort them by sort -n -k2.1,1 -o result.csv -t, result.csv and sort -k2.1,1 -o result.csv -t, result.csv. But what i got that the heading line is always writen below the data. Please help me to figure it out. Thanks in advance.

Regards,
Intan
Moderator's Comments:
Mod Comment This is a new problem. It has been moved to a new thread.

In the future, please start a new thread on your own when you have a new problem. Including multiple problems in a single thread makes it hard to follow what is being discussed.


In your previous thread (Save output of updated cvs file as csv file itself), you were given two methods to perform a numeric sort on a field and keep the file's header in the correct spot. And, you were given lengthy descriptions explaining why both of those methods worked for the input you provided. It looks like you completely ignored those descriptions and tried to apply the tricks provided there (that work when sorting numeric fields with positive values and a single heading line containing alphabetic titles) to sorting completely alphanumeric fields (starting with non-numeric characters).

You say you want to sort your file using the 1st field as your sort key. Your 1st field appears to be absolute pathnames, but you are sorting them as numeric fields when none of the data in that field stats with a number. And, the sort key -k2.1,1 specifies a sort key starting with the 1st character in the second field and ending in the 1st field. The standards do not specify what happens if the end of the sort key comes before the start of the sort key. And, with an alphanumeric sort based on an ASCII codes, / will sort before any alphabetic characters in the heading that you want to remain as the 1st line in your output.

So, to get the output you say you want, you'll have to use the method RudiC suggested in your previous thread to extract the heading from your data and print it, and then sort the rest of your file after throwing away the heading:
Code:
(head -1 result.csv && tail -n+2 result.csv | sort) > $$.csv && cp $$.csv result.csv; rm -f $$.csv

# 3  
Old 04-11-2015
Hi, Don Cragun

Sorry for posting my new problem in my previous thread. I have tried your coding. But, it doesn't work with my whole program/script. The program doesn't want to execute those sorting part, but i don't get any errors. When i tried to run those script in a new program, it works well. I just don't know why. Is there other way to sort them?
and my problem is getting more complicated now. I need your help. Please take a look at my problem in my new thread here. Thanks in advance.

Regards,
Intan

Previous Thread | Next Thread
Test Your Knowledge in Computers #629
Difficulty: Easy
In PHP, False == 0;
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Print particular string in a field of csv file - part 2

Hi, all I need your help and suggestions. I want to print particular strings in a field of a csv file and show them in terminal. Here is an example of the csv file. SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw... (7 Replies)
Discussion started by: refrain
7 Replies

2. Shell Programming and Scripting

Save output of updated csv file as csv file itself

Hi, all I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file. test.csv SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21 /home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Discussion started by: refrain
10 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

5. Shell Programming and Scripting

FILE_ID extraction from file name and save it in CSV file after looping through each folders

FILE_ID extraction from file name and save it in CSV file after looping through each folders My files are located in UNIX Server, i want to extract file_id and file_name from each file .and save it in a CSV file. How do I do that? I have folders in unix environment, directory structure is... (15 Replies)
Discussion started by: princetd001
15 Replies

6. Shell Programming and Scripting

need to save the space when converting to CSV file

Hi, I have a text file with the following format. Some of the fields are blank. 1234 3456 23 45464 327837283232 343434 5654353 34 34343 3434345 434242 .... .... .... I need to convert this file to a CSV file, like 1234, ,23, ... (3 Replies)
Discussion started by: wintersnow2011
3 Replies

7. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

8. Shell Programming and Scripting

select data from oracle table and save the output as csv file

Hi I need to execute a select statement in a solaris environment with oracle database. The select statement returns number of rows of data. I need the data to be inserted into a CSV file with proper format. For that we normally use "You have to select all your columns as one big string,... (2 Replies)
Discussion started by: rdhanek
2 Replies

9. Shell Programming and Scripting

Data fetched from text file and save in a csv file

Hi i have wriiten a script which fetches the data from text file, and saves in the output in a text file itself, but i want that the output should save in different columns. I have the output like: For Channel:response_time__24.txt 1547 data points 0.339 0.299 0.448 0.581 7.380 ... (1 Reply)
Discussion started by: rohitkalia
1 Replies

Featured Tech Videos