Shell Programming and Scripting

Hi, I would like to change my CSV file by adding " and : and moving some of the information around. the CSV file looks as follows:

Code:

501254424;500440257;PE PACKS;300467279;PREP;;276476070;655031001867176;Two Block;Olga;25/12/2015 00:00:00;Olga

I would like to move the field 7 to the front " and followed by the : and putting the rest of the CSV information in "". How can I do this with a shell script?

The desired output should look like this:

Code:

"276476070": "501254424;500440257;PE PACKS;300467279;PREP;;276476070;655031001867176;Two Block;Olga;25/12/2015 00:00:00;Olga"

Any attempts/ideas/thoughts from your side?

I tried the following:

Code:

awk 'BEGIN { FS=";"; OFS=";"; } {print $7,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13}' file

however this does not work and how to add the " and : I have not found a solution.

How about

Code:

awk -F";" -vDQ='"' '{print DQ $7 DQ ": " DQ $0 DQ}' file4
"276476070": "501254424;500440257;PE PACKS;300467279;PREP;;276476070;655031001867176;Two Block;Olga;25/12/2015 00:00:00;Olga"

Thank you very much that is amazing, it works.

When I echo the line and run the filter all is working as expected, however if I run the statement against a file, the last Double Quote is not present, any idea why?

Here is what I use:

Code:

 awk -F";" -vDQ='"' '{print DQ "264" $7 DQ ": " DQ $0 DQ}' /archive/ftp/SUBSDATA_001_20161118235001.txt

---------- Post updated at 09:53 PM ---------- Previous update was at 09:35 PM ----------

I have it now like this in the script:

Code:

#!/bin/bash
file="/archive/ftp/TNM_SUBSDATA_001_20161118235001.txt"
while IFS= read -r line
do
 awk -F";" -vDQ='"' '{print DQ "264" $7 DQ ": " DQ $0 DQ }' >> /tmp/output.txt
done <"$file"

However the last " behind the text is not present in the output file?

Hi,

When i try the command RudiC ( including your addition of 264 in latest post ) it works as expected.

Code:

cat file
501254424;500440257;PE PACKS;300467279;PREP;;276476070;655031001867176;Two Block;Olga;25/12/2015 00:00:00;Olga
501254424;500440257;PE PACKS;300467279;PREP;;27647xxxx;655031001867176;Two Block;Olga;25/12/2015 00:00:00;awk

Code:

awk -F";" -vDQ='"' '{print DQ "264" $7 DQ ": " DQ $0 DQ}' file > output

Gives output as follows:

I notice that you have "ftp" in your path and my wild guess is that you are doing ftp from one OS to another and that is messing up your end-of-line characters, due to which you don't see the last character.

Do you observe the missing character after an ftp of "output.txt" from Unix/Linux to Windows?

First thing to check is that the "output.txt" is formed correctly in your Unix/Linux box. Run your script on a small data file (maybe 3 or 4 lines) and once "output.txt" is created, run the following command:

Code:

od -bc output.txt

You should see "\n" as the end-of-line character in Unix/Linux.
If you see "\r\n" as the end-of-line character, then that file is in DOS (or Windows) format.
If you are doing an ftp of the file, ensure that you do a binary ftp and not ascii. The ftp program usually takes care of end-of-line character unless you override it.

If the assumptions above are not true, then your problem might lie somewhere else.
You could start with showing us the output of "od -bc output.txt".
Also, what exactly do you do after the "output.txt" is created to find that the last character is missing?
Do you open it in vim/emacs/nano etc? Do you transfer it somewhere and then open it in some editor?