Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Unix Script to parse a CSV
Top Forums Shell Programming and Scripting Unix Script to parse a CSV Post 302234434 by RJ17 on Tuesday 9th of September 2008 04:37:27 PM
Old 09-09-2008
Unix Script to parse a CSV
I am writing a unix script that will parse a CSV and edit the values. My CSV looks like this
0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0
10,11,7,0,4,12,2,3,7,0,11,3,12,4,0,5,5,4,5,0,8,6,12,0,9,3,3,0,2,7,8
19,11,7,0,4,14,16,10,8,2,13,7,15,6,0,76,6,4,10,0,18,10,17,1,11,3,3,0,9,9,8
22,11,13,1,5,14,16,10,9,10,13,7,16,6,0,59,6,4,10,0,18,13,17,1,11,3,3,0,12,9,10
22,11,13,1,5,14,16,10,9,10,13,7,16,6,22,90,6,4,10,0,18,13,17,1,11,3,4,0,12,9,10
41,18,27,9,27,41,59,20,27,54,63,34,28,43,40,131,7,8,19,0,62,16,30,23,25,3,4,9,24,12,19
42,18,27,9,27,41,59,20,27,55,68,36,28,46,41,132,7,8,19,13,64,16,31,25,25,3,4,9,24,12,19
125,124,78,62,97,87,145,70,87,119,150,124,99,95,41,175,85,58,57,88,142,83,92,102,107,80,45,64,64,94, 89
125,126,78,62,99,87,145,70,87,119,161,124,99,95,41,175,85,58,58,88,142,84,112,103,108,80,68,64,65,98 ,89
189,254,164,153,192,153,230,132,188,163,210,210,167,198,93,235,146,110,97,130,211,107,181,140,151,11 9,105,105,178,126,165
189,324,168,192,194,159,233,132,192,169,244,210,167,201,103,235,147,152,180,181,213,107,192,190,212, 119,119,126,195,126,166
189,324,168,255,194,225,233,141,192,230,244,260,167,201,172,283,181,206,217,216,261,107,192,235,212, 119,169,197,264,189,229
366,438,315,319,382,287,398,320,416,382,407,397,342,448,276,392,297,368,237,347,336,332,384,405,412, 284,329,350,396,326,356


This script would run every hour on the hour deleting the first value on a specific line and adding a new record to the end of that same line. So for instance at 8:00am I might delete the 0 from the first line and add a 10 at the end. I also have a version of the CSV that has the time as the first value of each row. I have tried both awk and sed and I can't figure out how to replace a value at a specfic location. I think I can delete the first value in row 1 using this code snippet, but I dont know how to add the value to the end of the same row. Is there a way to parse the csv into a two dimesional array and then output it back into a csv at the end? It probably isn't the most efficient way but it would work.
I hope this isn't too confusing. Please let me know if you need more information. Thanks

Code:
 
sed '1s/^[^,]*,//' file.csv

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to Parse a CSV file into a Different Format

Hi I have a CSV file with me in this format Currency, USD, EUR, USD, 1.00, 1.32, EUR, 0.66, 1.00, How do I transpose the file to get to the format below. currency, currency, rate USD, USD, 1.00 USD, EUR, 1.32 EUR, USD, 0.66 EUR, EUR, 1.00 Thanks for your help We are using... (2 Replies)
Discussion started by: cdesiks
2 Replies

2. Shell Programming and Scripting

CSV File parse help in Perl

Folks, I have a bit of an issue trying to obtain some data from a csv file using PERL. I can sort the file and remove any duplicates leaving only 4 or 5 rows containing data. My problem is that the data contained in the original file contains a lot more columns and when I try ro run this script... (13 Replies)
Discussion started by: lodey
13 Replies

3. Shell Programming and Scripting

Unix shell script to parse the contents of comma-separated file

Dear All, I have a comma-separated file. 1. The first line of the file(header) should have 4 commas(5 fields). 2. The last line of the file should have 1 comma(2 fields). Pls help me in checking this condition in a shell script. And the number of lines between the first line and last... (11 Replies)
Discussion started by: KrishnaSaran
11 Replies

4. Shell Programming and Scripting

Need help in creating a Unix Script to parse xml file

Hi All, My requirement is create an unix script to parse the xml file and display the values of the Elements/value between the tags on console. Like say, I would like to fetch the value of errorCode from the below xml which is 'U007' and display it. Can we use SED command for this? I have tried... (10 Replies)
Discussion started by: Anil.Wmg
10 Replies

5. Shell Programming and Scripting

Parse csv file

Hi, Our requirement is to parse the input file(.csv format). The each column in the file is delimited with comma. We need to take each column and apply some business validation rule. If data itself contains comma, then those fields are enclosed with double quotes ("). We can see this double... (7 Replies)
Discussion started by: vfrg
7 Replies

6. Shell Programming and Scripting

How to parse csv format?

Hi, I have a file with 3 fields in csv format: /tmp/foo/,MODIFY,bar/toto "/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"I would like to split fields in order to obtain the following: Line1: /tmp/foo/ MODIFY bar/totoLine2: /tmp/foo, bar/ ATTRIB,ISDIR toto, tata/foobarCan't find my way... (11 Replies)
Discussion started by: chebarbudo
11 Replies

7. UNIX for Dummies Questions & Answers

Help to parse csv file with shell script

Hello ! I am very aware that this is not the first time this question is asked here, because I have already read a lot of previous answers, but none of them worked, so... As said in the title, I want to read a csv file with a bash script. Here is a sample of the file: ... (4 Replies)
Discussion started by: Grhyll
4 Replies

8. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

9. Shell Programming and Scripting

Parse csv files by their names

HI all I have multiple csv files with the names VAR1_VAR2_VAR3_VAR4.csv All the files have the same structure inside just values change. I am trying to retrieve data from those files by fixing at each time one or more VAR. I tried to write a script but I have 2 problems: 2-... (1 Reply)
Discussion started by: Jhon.c
1 Replies

10. Shell Programming and Scripting

Need a script to parse data and output to csv

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement. So I have the raw data that gets updated to a log as shown below StudentInfo: FullInfo = { Address = Newark Age = 20 Name= John } StudentInfo:... (2 Replies)
Discussion started by: sidnow
2 Replies

LEARN ABOUT CENTOS

cp1251

CP1251(7)						     Linux Programmer's Manual							 CP1251(7)

NAME

       cp1251 - CP 1251 character set encoded in octal, decimal, and hexadecimal

DESCRIPTION

       The  Windows Code Pages include several 8-bit extensions to the ASCII character set (also known as ISO 646-IRV).  CP 1251 encodes the char-
       acters used in Cyrillic scripts.  The following table displays the characters in CP 1251, which are printable and unlisted in the  ascii(7)
       manual page.  The fourth column will only show the proper glyphs in an environment configured for CP 1251.

       Oct  Dec  Hex  Char  Description
       ----------------------------------------------------------------------
       200  128  80	    CYRILLIC CAPITAL LETTER DJE
       201  129  81	    CYRILLIC CAPITAL LETTER GJE
       202  130  82    ,    SINGLE LOW-9 QUOTATION MARK
       203  131  83	    CYRILLIC SMALL LETTER GJE
       204  132  84    ,,   DOUBLE LOW-9 QUOTATION MARK
       205  133  85	    HORIZONTAL ELLIPSIS
       206  134  86    -    DAGGER
       207  135  87    =    DOUBLE DAGGER
       210  136  88   EUR   EURO SIGN
       211  137  89    %o   PER MILLE SIGN
       212  138  8A	    CYRILLIC CAPITAL LETTER LJE
       213  139  8B    <    SINGLE LEFT-POINTING ANGLE QUOTATION MARK
       214  140  8C	    CYRILLIC CAPITAL LETTER NJE
       215  141  8D	    CYRILLIC CAPITAL LETTER KJE
       216  142  8E	    CYRILLIC CAPITAL LETTER TSHE
       217  143  8F	    CYRILLIC CAPITAL LETTER DZHE
       220  144  90	    CYRILLIC SMALL LETTER DJE
       221  145  91    '    LEFT SINGLE QUOTATION MARK
       222  146  92    '    RIGHT SINGLE QUOTATION MARK
       223  147  93    "    LEFT DOUBLE QUOTATION MARK
       224  148  94    "    RIGHT DOUBLE QUOTATION MARK
       225  149  95    o    BULLET
       226  150  96    -    EN DASH
       227  151  97    --   EM DASH
       230  152  98	    UNDEFINED
       231  153  99    tm   TRADE MARK SIGN
       232  154  9A	    CYRILLIC SMALL LETTER LJE
       233  155  9B    >    SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
       234  156  9C	    CYRILLIC SMALL LETTER NJE
       235  157  9D	    CYRILLIC SMALL LETTER KJE
       236  158  9E	    CYRILLIC SMALL LETTER TSHE
       237  159  9F	    CYRILLIC SMALL LETTER DZHE
       240  160  A0	    NO-BREAK SPACE
       241  161  A1	    CYRILLIC CAPITAL LETTER SHORT U
       242  162  A2	    CYRILLIC SMALL LETTER SHORT U
       243  163  A3	    CYRILLIC CAPITAL LETTER JE
       244  164  A4    x    CURRENCY SIGN
       245  165  A5	    CYRILLIC CAPITAL LETTER GHE WITH UPTURN
       246  166  A6    |    BROKEN BAR
       247  167  A7    S    SECTION SIGN
       250  168  A8	    CYRILLIC CAPITAL LETTER IO
       251  169  A9   (C)   COPYRIGHT SIGN
       252  170  AA	    CYRILLIC CAPITAL LETTER UKRAINIAN IE
       253  171  AB    <<   LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
       254  172  AC    ~    NOT SIGN
       255  173  AD	    SOFT HYPHEN
       256  174  AE   (R)   REGISTERED SIGN
       257  175  AF	    CYRILLIC CAPITAL LETTER YI
       260  176  B0    o    DEGREE SIGN
       261  177  B1    +-   PLUS-MINUS SIGN
       262  178  B2	    CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I

       263  179  B3	    CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
       264  180  B4	    CYRILLIC SMALL LETTER GHE WITH UPTURN
       265  181  B5    u    MICRO SIGN
       266  182  B6    9|   PILCROW SIGN
       267  183  B7    .    MIDDLE DOT
       270  184  B8	    CYRILLIC SMALL LETTER IO
       271  185  B9	    NUMERO SIGN
       272  186  BA	    CYRILLIC SMALL LETTER UKRAINIAN IE
       273  187  BB    >>   RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
       274  188  BC	    CYRILLIC SMALL LETTER JE
       275  189  BD	    CYRILLIC CAPITAL LETTER DZE
       276  190  BE	    CYRILLIC SMALL LETTER DZE
       277  191  BF	    CYRILLIC SMALL LETTER YI
       300  192  C0	    CYRILLIC CAPITAL LETTER A
       301  193  C1	    CYRILLIC CAPITAL LETTER BE
       302  194  C2	    CYRILLIC CAPITAL LETTER VE
       303  195  C3	    CYRILLIC CAPITAL LETTER GHE
       304  196  C4	    CYRILLIC CAPITAL LETTER DE
       305  197  C5	    CYRILLIC CAPITAL LETTER IE
       306  198  C6	    CYRILLIC CAPITAL LETTER ZHE
       307  199  C7    v    CYRILLIC CAPITAL LETTER ZE
       310  200  C8	    CYRILLIC CAPITAL LETTER I
       311  201  C9	    CYRILLIC CAPITAL LETTER SHORT I
       312  202  CA	    CYRILLIC CAPITAL LETTER KA
       313  203  CB	    CYRILLIC CAPITAL LETTER EL
       314  204  CC	    CYRILLIC CAPITAL LETTER EM
       315  205  CD	    CYRILLIC CAPITAL LETTER EN
       316  206  CE	    CYRILLIC CAPITAL LETTER O
       317  207  CF	    CYRILLIC CAPITAL LETTER PE
       320  208  D0	    CYRILLIC CAPITAL LETTER ER
       321  209  D1	    CYRILLIC CAPITAL LETTER ES
       322  210  D2	    CYRILLIC CAPITAL LETTER TE
       323  211  D3	    CYRILLIC CAPITAL LETTER U
       324  212  D4	    CYRILLIC CAPITAL LETTER EF
       325  213  D5	    CYRILLIC CAPITAL LETTER HA
       326  214  D6	    CYRILLIC CAPITAL LETTER TSE
       327  215  D7	    CYRILLIC CAPITAL LETTER CHE
       330  216  D8	    CYRILLIC CAPITAL LETTER SHA
       331  217  D9	    CYRILLIC CAPITAL LETTER SHCHA
       332  218  DA	    CYRILLIC CAPITAL LETTER HARD SIGN
       333  219  DB	    CYRILLIC CAPITAL LETTER YERU
       334  220  DC	    CYRILLIC CAPITAL LETTER SOFT SIGN
       335  221  DD	    CYRILLIC CAPITAL LETTER E
       336  222  DE	    CYRILLIC CAPITAL LETTER YU
       337  223  DF	    CYRILLIC CAPITAL LETTER YA
       340  224  E0	    CYRILLIC SMALL LETTER A
       341  225  E1	    CYRILLIC SMALL LETTER BE
       342  226  E2	    CYRILLIC SMALL LETTER VE
       343  227  E3	    CYRILLIC SMALL LETTER GHE
       344  228  E4	    CYRILLIC SMALL LETTER DE
       345  229  E5	    CYRILLIC SMALL LETTER IE
       346  230  E6	    CYRILLIC SMALL LETTER ZHE
       347  231  E7	    CYRILLIC SMALL LETTER ZE
       350  232  E8	    CYRILLIC SMALL LETTER I
       351  233  E9	    CYRILLIC SMALL LETTER SHORT I
       352  234  EA	    CYRILLIC SMALL LETTER KA
       353  235  EB	    CYRILLIC SMALL LETTER EL
       354  236  EC	    CYRILLIC SMALL LETTER EM
       355  237  ED	    CYRILLIC SMALL LETTER EN
       356  238  EE	    CYRILLIC SMALL LETTER O
       357  239  EF	    CYRILLIC SMALL LETTER PE
       360  240  F0	    CYRILLIC SMALL LETTER ER
       361  241  F1	    CYRILLIC SMALL LETTER ES
       362  242  F2	    CYRILLIC SMALL LETTER TE
       363  243  F3	    CYRILLIC SMALL LETTER U

       364  244  F4	    CYRILLIC SMALL LETTER EF
       365  245  F5	    CYRILLIC SMALL LETTER HA
       366  246  F6	    CYRILLIC SMALL LETTER TSE
       367  247  F7	    CYRILLIC SMALL LETTER CHE
       370  248  F8	    CYRILLIC SMALL LETTER SHA
       371  249  F9	    CYRILLIC SMALL LETTER SHCHA
       372  250  FA	    CYRILLIC SMALL LETTER HARD SIGN
       373  251  FB	    CYRILLIC SMALL LETTER YERU
       374  252  FC	    CYRILLIC SMALL LETTER SOFT SIGN
       375  253  FD	    CYRILLIC SMALL LETTER E
       376  254  FE	    CYRILLIC SMALL LETTER YU
       377  255  FF	    CYRILLIC SMALL LETTER YA

NOTES

       CP 1251 is also known as Windows Cyrillic.

SEE ALSO

       ascii(7)

COLOPHON

       This  page is part of release 3.53 of the Linux man-pages project.  A description of the project, and information about reporting bugs, can
       be found at http://www.kernel.org/doc/man-pages/.

Linux								    2010-09-20								 CP1251(7)
All times are GMT -4. The time now is 12:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy