Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Break output file into three files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Break output file into three files
# 1  
Old 11-14-2016
Break output file into three files
Help! Smilie

I am getting an output file that looks similar to below.

Code:
EMAIL_ADDR
-----------------------------------------------------------------------------------
user@gmail.com
DATABASENAME
-----------------------------------------------------------------------------------
db1
db2
db3
db4

RoleName
----------------------------------------------------------------------------------
dbrole1
dbrole2
dbrole3
dbrole4
dbrole5

I would love to get three files:
file1.txt - with contents being:
Code:
user@gmail.com

file2.txt - with contents being:
Code:
db1
db2
db3
db4

file3.txt - with contents being:
Code:
dbrole1
dbrole2
dbrole3
dbrole4
dbrole5


I know how to cut off the top line of the file but I am not sure how to parse the files to make them look like this.

ideas?
Last edited by Don Cragun; 11-15-2016 at 12:19 AM.. Reason: Add CODE and ICODE tags.
# 2  
Old 11-15-2016
Could be a start..

Code:
awk -v x=0 '/---/{x="F"++i;next}{print > x;}' infile

# 3  
Old 11-15-2016
Not knowing how much of your example is real, here's a generic attempt based on the structure shown.

Save as process.pl
Run as perl process.pl cpolikowsky.input

Code:
#!/usr/bin/perl
#
use strict;
use warnings;

my $ver;
my @keep;

while(<>) {
   my $current = $_;
   if(/^-+$/) {
       pop @keep;
       $current = "";
   }

   if($current !~ /^$/) {
       push @keep, $current;
   }
   elsif(@keep) {
       write_out(++$ver, @keep);
       @keep=();
   }
}
write_out(++$ver, @keep);

sub write_out {
    my ($n, @lines) = @_;

    open my $fw, '>', "file$n.txt" || die;
    for my $line (@lines) {
        print $fw $line;
    }
    close $fw;
}


Output:
Code:
for f in file*.txt; do echo "$f"; cat "$f"; echo; done
file1.txt
user@gmail.com

file2.txt
db1
db2
db3
db4

file3.txt
dbrole1
dbrole2
dbrole3
dbrole4
dbrole5

Last edited by Aia; 11-15-2016 at 01:37 AM..
# 4  
Old 11-15-2016
Try also
Code:
awk '
/^---/  {FN = LLINE ".txt"
         LLINE = ""
         next
        }

LLINE   {print LLINE  >  FN
        }

        {LLINE = $0
        }

END     {print LLINE  >  FN
        }
' file

# 5  
Old 11-15-2016
Hi.

Command csplit was designed for situations like this. The core of this solution is the single line:
Code:
csplit -b'%d.txt' -f'file' -k -z -q --suppress-matched $FILE '/^-----/' '{*}'

A sample script might be:
Code:
#!/usr/bin/env bash

# @(#) s1       Demonstrate splitting by context.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C dixf csplit
pe
dixf csplit

FILE=${1-data1}
E=expected-output.txt

pl " Input data file $FILE:"
cat $FILE

pl " Expected output:"
grep file. expected-output.txt

pl " Results:"
csplit -b'%d.txt' -f'file' -k -z -q --suppress-matched $FILE '/^-----/' '{*}'
head file*

exit 0

producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie) 
bash GNU bash 4.3.30
dixf (local) 1.12
csplit (GNU coreutils) 8.23

csplit  split a file into sections determined by context lines (man)
Path    : /usr/bin/csplit
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYSV ...)
Help    : probably available with --help

-----
 Input data file data1:
EMAIL_ADDR
-----------------------------------------------------------------------------------
user@gmail.com
DATABASENAME
-----------------------------------------------------------------------------------
db1
db2
db3
db4

RoleName
----------------------------------------------------------------------------------
dbrole1
dbrole2
dbrole3
dbrole4
dbrole5

-----
 Expected output:
file1.txt - with contents being:
file2.txt - with contents being:
file3.txt - with contents being:

-----
 Results:
==> file0.txt <==
EMAIL_ADDR

==> file1.txt <==
user@gmail.com
DATABASENAME

==> file2.txt <==
db1
db2
db3
db4

RoleName

==> file3.txt <==
dbrole1
dbrole2
dbrole3
dbrole4
dbrole5

Some earlier versions of csplit might need a slightly different scheme for repeating the pattern. See your man page for details.

Best wishes ... cheers, drl
# 6  
Old 11-15-2016
@drl: I considered csplit as well, but noticed that column headers will be part of the next file, then.
This User Gave Thanks to RudiC For This Post:
drl
# 7  
Old 11-15-2016
Hi, RudiC.
Quote:
Originally Posted by RudiC
@drl: I considered csplit as well, but noticed that column headers will be part of the next file, then.
Yes, I agree.

My take on many problems is that the user often does not have awk, perl, etc., expertise in order to craft a solution. However, doing many operations with shell can often lead to good, but approximate results. If cleaner results are desired, one can interactively edit the results (for one-off problems), or use a few well-placed simple grep or sed commands to help obtain the desired results.

Sometimes even the input might be able to be cleaned up by the source, or perhaps pre-process as an intermediate step before the approximating command(s).

However, in general, I agree with you about the csplit output in this case.

Thanks for the comments. I think discussions like this help users to see how we all approach problems.

cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Output file name and file contents of multiple files to a single file

I am trying to consolidate multiple information files (<hostname>.Linux.nfslist) into one file so that I can import it into Excel. I can get the file contents with cat *Linux.nfslist >> nfslist.txt. I need each line prefaced with the hostname. I am unsure how to do this. --- Post updated at... (5 Replies)
Discussion started by: Kentlee65
5 Replies

2. Shell Programming and Scripting

Line break in sqlplus output through ksh script

Hi, I am new to shell script programming. I have written a ksh script to run the sql File placed in server directory and spool the output in destination directory. Below Command: $ORACLE_HOME/bin/sqlplus -s $ora_uid @$sqlfile_loc$testquery.sql > $opfiledirectory It is generating the output... (6 Replies)
Discussion started by: Sumit Arora
6 Replies

3. Shell Programming and Scripting

Output result with break line

Hello, I am coding a script to check mysql databases using bash script, I would like to check if the status of a table is not 'OK', will return the table name and do some more actions: check.log table1 OK table2 Some error here table3 ... (5 Replies)
Discussion started by: Abu Rayane
5 Replies

4. UNIX for Dummies Questions & Answers

To compare two files,Output into a new file

Hi Please help me to compare two files and output into a new file file1.txt 15114933 |4001 15291649 |933502 15764675 |4316 15764678 |4316 15761974 |282501 15673104 |933505 15673577 |933505 15673098 |933505 15673096 |933505 15673092 |933505 15760705 ... (13 Replies)
Discussion started by: Ankita Talukdar
13 Replies

5. UNIX for Dummies Questions & Answers

Script to break up file (write new files) in bash

Hello experts, I need help writing individual files from a data matrix, with each new file being written every time there is a blank line: From this cat file.txt col1 col2 col3 6661 7771 8881 6661 7771 8881 6661 7771 8881 col1 col2 col3 3451 2221 1221... (6 Replies)
Discussion started by: torchij
6 Replies

6. UNIX for Dummies Questions & Answers

Compare two files and output to new file

Hi, Please help How to compare two files- Any mismatches 2nd and 3rd column's values corresponding to 1st column. file1 15294024|Not Allowed|null 15291398|Not Allowed|null 15303292|Dropship (standard)|N 15303291|Dropship (standard)|N 15275561|Store Only|Y 15275560|Store Only|Y... (2 Replies)
Discussion started by: Ankita Talukdar
2 Replies

7. Shell Programming and Scripting

join lines on line break in files

i had a file where lines appear to be broken when they shouldn't eg Line 1. kerl abc sdskd sdsjkdlsd sdsdksd \ Line 2. ksdkks sdnjs djsdjsd i can do a shift join to combine the lines but i there are plenty of files with this issue Line 1. kerl abc sdskd sdsjkdlsd sdsdksd ksdkks sdnjs... (6 Replies)
Discussion started by: mad_man12
6 Replies

8. UNIX for Dummies Questions & Answers

to break a file into 2 files after matching a pattern.

Hi, i need to break a file into 2 files afetr matching a pattern for ex. there is a fil, file .txt which contains here i need to look for mat $ demon if it matches then i need to transfer the data into another file till the line in which a "d6s" comes,and i have to delete tat line... (3 Replies)
Discussion started by: manit
3 Replies

9. Solaris

Output to 1 file from more than 2 files

I am currently in FTP. I want to transfer two files and need the out put in One file. Can you help me.... (3 Replies)
Discussion started by: manish.s
3 Replies

10. Shell Programming and Scripting

Break a file into separate files

Hello I am facing a scenario where I have a file with XML content and I am running shell script over it. But the problem is the XML is getting updated with new services. In the below scenario, my script takes values from the xml file from one service name say ABCD. Since there are multiple, it is... (8 Replies)
Discussion started by: chiru_h
8 Replies
Login or Register to Ask a Question