Manipulate data in CSV file using Shell script


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Answers to Frequently Asked Questions Manipulate data in CSV file using Shell script
# 1  
Old 04-12-2020
Manipulate data in CSV file using Shell script

HI,
Can you guys please help me in creating Pivot table for below data using shell script?

Code:
Location_id| Effective_date| Picking_limit     
------- | ---------------- | ----------:
2000| 04/04/2020| 120
2001|  04/04/2020| 130
2002| 04/04/2020 | 150
2003| 04/04/2020| 100
2001| 04/04/2020| 140

I want output as below

Code:
Location_id| Picking_Limit    
------- | ---------------- :
2000| 120
2001|  170
2002| 150
2003| 100

Basically i prepared a pivot table for location id. Where ever location id is same i am adding value of picking limit and wanted to write in a new csv file.

Thanks,
Swetha. G

Last edited by vbe; 04-12-2020 at 10:34 AM.. Reason: code tags
# 2  
Old 04-13-2020
Pivot table for data in CSV file using shell script

Code:
IFS=","
while read f1 f2
do
awk -F, '{ A[$1]+=$2 } END { OFS=","; for (x in A) print x,A[x]; }'

# 3  
Old 04-13-2020
Any attempts / ideas / thoughts from your side? Did you search these fora?
# 4  
Old 04-13-2020
I could resolve my issue using awk

Code:
IFS=","
while read f1 f2
do
awk -F, '{ A[$1]+=$2 } END { OFS=","; for (x in A) print x,A[x]; }' > /home/ec2-user/slotslatest.csv
done < Storewiseslots.csv

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Is there a way to handle commas inside the data when generating a csv file from shell script?

I am extracting data via sql query and some of the data has commas. Output File must be csv and I cannot update the data in the db (as it is used by other application). Example table FavoriteThings Person VARCHAR2(25), Favorite VARCHAR2(100) Sample Data Greta rain drop on... (12 Replies)
Discussion started by: patk625
12 Replies

2. Shell Programming and Scripting

Manipulate a CSV File

Hello, How do i manipulate .csv file to this format? Thank you very much. Source: john,5 marco,7 john,4 paul,3 marco,8 Output: john,9 marco,15 (5 Replies)
Discussion started by: tara123
5 Replies

3. Shell Programming and Scripting

Shell script to extract data from csv file

Hi everyone, I have a csv file which has data with different heading and column names as below. Static Data Ingested ,,,,,,,,,,,,Known Explained Rejections Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Discussion started by: Vivekit82
14 Replies

4. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values. Name, Address,... (7 Replies)
Discussion started by: Vivekit82
7 Replies

5. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file based on certain conditions

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
Discussion started by: Vivekit82
1 Replies

6. Shell Programming and Scripting

Read data from .csv file through shell script & modify

I need to read data from a file called "test.csv" through shell script where the file contains values like name,price,descriptor etc. There are rows where descriptor (& in some rows name) are written as string & other characters like "car_+" OR "bike*" etc where it should contains strings like... (3 Replies)
Discussion started by: raj100
3 Replies

7. Shell Programming and Scripting

Shell script to manipulate a file

Hello, I have a file with following contents : WSL SRVGRP=LISTENER SRVID=2 CLOPT="-A -t -- -n 0x0002aa050a03cc65 " RQPERM=0660 REPLYQ=Y RPPERM=0660 MIN=1 MAX=1 CONV=N I need to print only the value in Hex i.e.... (2 Replies)
Discussion started by: deo_kaustubh
2 Replies

8. Shell Programming and Scripting

Exporting data as a CSV file from Unix shell script

Friends...This is the first time i am trying the report generation using shell script... any suggestions are welcome. Is there a way to set the font size & color when i am exporting the data from unix shell script as a CSV file ? The following sample data is saved as a .csv file in the... (2 Replies)
Discussion started by: appu2176
2 Replies

9. Filesystems, Disks and Memory

manipulate csv file to add columns

Hi, I have a csv file with a key composed by 3 columns and some other numeric fields and I need to obtain the partial amounts by some part of the key. This may be some difficult to understand, so better see an example, where my input file is: name,surname,department,y2004,y2005,y2006... (6 Replies)
Discussion started by: oscarmon
6 Replies

10. Shell Programming and Scripting

Shell Script to Load data into the database using a .csv file and .ctl file

Since i'm new to scripting i'm findind it difficult to code a script. The script has to be an executable with 2 paramters passed to it.The Parameters are 1. The Control file name(.ctl file) 2. The Data file name(.csv file) Does anybody have an idea about it? :confused: (3 Replies)
Discussion started by: Csmani
3 Replies
Login or Register to Ask a Question