Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help me! Format output file using shell program

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help me! Format output file using shell program
# 1  
Old 11-07-2010
Help me! Format output file using shell program
Hi,
I have following input file. I want to generate output file in specific format using shell program. The input file has atleast few thousands of lines, the below are some sample lines.

Input file:
Code:
"ORDER NO"|"ORDER AMT"|"LINE ITEM"|"LINE AMT"|"SALES COMMISION %"
ORD3456|5000|LIN345|30|25%
ORD3456|5000
|LIN223|20|15%
0RD3456|5000|LIN221|
10|10
%
ORD3456|
5000|20%
LIN222|5
ORD3456|5000|LIN224|25|30%|ORD2233|6000|LIN0011|24
|100%

Output File:
Code:
"ORDER NO"|"ORDER AMT"|"LINE ITEM"|"LINE AMT"|"SALES COMMISION %"
ORD3456|5000|LIN345|30|25%
ORD3456|5000|LIN223|20|15%
0RD3456|5000|LIN221|10|10%
ORD3456|5000|LIN222|5|20%
ORD3456|5000|LIN224|25|30%
ORD2233|6000|LIN0011|24|100%

Thanks,
Subha
Last edited by Scott; 11-08-2010 at 03:24 AM.. Reason: Please use code tags
# 2  
Old 11-07-2010
There is a real problem with this part of you example:
Code:
0RD3456|5000|LIN221|
10|10
%
ORD3456|

There is no way to tell the end of a record, see how % must be joined up to previous line but ORD3456 dosn't. This is because the record seperator was a newline, but extra new lines have been added to your file.

Can we use some sort of knowledge about the data, eg does each order number start with the string "ORD"?

---Edit---
Below is even worse, the fields have swapped order! A program may be able to sort this out, but it must make assumptions about the data from it's format and determine where it belongs.

Code:
ORD3456|
5000|20%
LIN222|5

Can you give some rules that the data must follow?
Last edited by Chubler_XL; 11-07-2010 at 11:33 PM..
# 3  
Old 11-08-2010
Hi,
Please check the input file and output file. I have added rules to consider. Somehow in previous post the posted blank new lines disappeared.

Input file:
Code:
"ORDER NO"|"ORDER AMT"|"LINE ITEM"|"LINE AMT"|"SALES COMMISION %"
ORD3456|5000|LIN345|30|25%
 
ORD3456|5000
|LIN223|20|15%
 
0RD3456|5000|LIN221|
10|10
%
 
ORD3456|
5000|LIN222|5|20%
 
ORD3456|5000|LIN224|25|30%|ORD2233|6000|LIN0011|24|100%

Output file:
Code:
"ORDER NO"|"ORDER AMT"|"LINE ITEM"|"LINE AMT"|"SALES COMMISION %"
ORD3456|5000|LIN345|30|25%
ORD3456|5000|LIN223|20|15%
0RD3456|5000|LIN221|10|10%
ORD3456|5000|LIN222|5|20%
ORD3456|5000|LIN224|25|30%
ORD2233|6000|LIN0011|24|100%

Rule:
1. If delimiter '|' occurrances is 4 then it should be treated as seperate line
2. For wrapped lines new line exists at start of line and end of line.
3. For concatenated lines if delimiter '|' occurances exceeds 4 then it should be broken to new line starting from 5 occurance.
Last edited by Scott; 11-08-2010 at 03:25 AM.. Reason: Code tags
# 4  
Old 11-08-2010
This should do what you want, just make sure you have a blank line between your header row and the first data line (demo input file was missing this).

Code:
sed 's/^$/|/' infile | tr -d '\n' | awk -F \| '{for (i=1;i<=NF;i++) {if ($i!="") printf (++j%5)?$i FS:$i "\n"}}'

Last edited by Chubler_XL; 11-08-2010 at 12:56 AM..
# 5  
Old 11-08-2010
Try this,
Code:
awk 'NR==1{print}NR>1 && ! /^$/ {if(/^ORD.*%$/) {print} else if (/.*%$/){printf "%s\n",$0} else {printf $0}}' inputfile| sed 's/%|/%\n/g'

# 6  
Old 11-08-2010
Code:
local $/="%";
while(<DATA>){
  if($.==1){
   print $_,'"',"\n";
   next;
  }
  s/\n//g;
  s/^["|]//;
  print $_,"\n";
}
__DATA__

# 7  
Old 11-08-2010
Code:
awk 'NR==1{RS="%"}{gsub(ORS,x);$1=$1;sub(/^\|/,x)}NR>1&&NF>1{$0=$0 RS}1' infile

Code:
awk 'NR==1{RS="%"}{gsub(ORS,x);sub(/^[| ]/,x)}NR>1&&NF>1{$0=$0 RS}1' infile
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to convert any shell command output to JSON format?

Hi All, I am new to shell scripting, Need your help in creating a shell script which converts any unix command output to JSON format output. example: sample df -h command ouput : Filesystem size used avail capacity Mounted /dev/dsk/c1t0d0s0 8.1G 4.0G 4.0G 50% /... (13 Replies)
Discussion started by: balu1234
13 Replies

2. Programming

Python or Shell script to Grep strings from input file and output in csv format

Hi Experts, I am writing a python script to grep string from file and display output in csv file as in attached screenshot https://drive.google.com/file/d/1gfUUdfmQma33tz65NskThYDhkZUGQO0H/view Input file(result_EPFT_config_device) Below is the python script i have prepared as of... (1 Reply)
Discussion started by: as7951
1 Replies

3. Shell Programming and Scripting

Shell output format like table

Hi, OS: Redhat 7.5 shell: Bash Wrote below script to login into oracle via shell script and trying to reset locked account..It works as expected. But I need specific output << EOF should go to target terminal not all out put running below script from ansible command line.. #!/bin/bash... (1 Reply)
Discussion started by: onenessboy
1 Replies

4. Shell Programming and Scripting

Script to generate Excel file or to SQL output data to Excel format/tabular format

Hi , i am generating some data by firing sql query with connecting to the database by my solaris box. The below one should be the header line of my excel ,here its coming in separate row. TO_CHAR(C. CURR_EMP_NO ---------- --------------- LST_NM... (6 Replies)
Discussion started by: dani1234
6 Replies

5. Shell Programming and Scripting

UNIX shell script to format output report

i am new to unix shell scripting, could someone please help me. i was asked to develop a unix script and requirement of the script is as follows: 1. In source directory, if any new files are being dropped by external system then an email should be sent out with a message saying "files are... (3 Replies)
Discussion started by: crefi1545
3 Replies

6. Shell Programming and Scripting

Passing multiple C program output into a shell

Hi I have the following C code # cat test.c #include <stdio.h> main() { printf ("The output is : Power\n"); printf ("The output is : No Power\n"); } The output of this C code is # ./test The output is : Power The output is : No Power Now i need to pass this outputs into a shell... (4 Replies)
Discussion started by: Priya Amaresh
4 Replies

7. Shell Programming and Scripting

Format options while Redirecting output of sql to a file in shell

In my korn shell - I have a sql with say 6 columns whose output i am redirecting to a file and attaching this file while sending a mail. While all this happens correctly, i face issues in the format of this file. my intended output is Column_1 Column_2 Column_3 Column_4 ... (7 Replies)
Discussion started by: qwertyu
7 Replies

8. Shell Programming and Scripting

shell script to format command output

Hello team, I am running below command which is giving following output. bash-3.00$ ps -eo pid,pcpu,args | sort +1n | grep -i java 12 0.0 grep -i java 8804 0.0 /opt/app/ccr/home/ccr/WebSphere/AppServer/java/bin/sparcv9/java -XX:+UnlockDiag 9241 0.0... (7 Replies)
Discussion started by: coolguyamy
7 Replies

9. Shell Programming and Scripting

Dynamic output file generation using a input text file with predefined output format

Hi, I have two files , one file with data file with attributes that need to be sent to another file to generate a predefined format. Example: File.txt AP|{SSHA}VEEg42CNCghUnGhCVg== APVG3|{SSHA}XK|"password" AP3|{SSHA}XK|"This is test" .... etc --------- test.sh has... (1 Reply)
Discussion started by: hudson03051nh
1 Replies

10. UNIX for Advanced & Expert Users

how do collect shell output in a C program

i use the system command to execute a shell command... ca i collect the out put in the form of a string or something using the same C program? (5 Replies)
Discussion started by: damn_bkb
5 Replies
Login or Register to Ask a Question