Bash Command To Delete Number from Array | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Bash Command To Delete Number from Array

Shell Programming and Scripting


Tags
bash arrays, csplit, grep

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 09-12-2013
pmurray21 pmurray21 is offline
Registered User
 
Join Date: Sep 2013
Last Activity: 12 September 2013, 9:37 AM EDT
Posts: 3
Thanks: 1
Thanked 0 Times in 0 Posts
Bash Command To Delete Number from Array

Hi,

I am writing a script to split a log file - the log could contain multiple days worth of logs. The second line of the log contains the string "Version ". In my test log which comprises of two days worth of logs, this string appears twice - once each day.

Essentially I would like to split on the line before this line, unfortunately there is nothing unique in it. The script thus far gets the line number that "Version " is on and populates an array. My problem is....I am not sure how to subtract one from each array element. The main part of my code looks like this:


Code:
IFS=$'\n'
arr=($(grep -n "Version " $filename))
arr=("${arr[@]%%:*}")
unset IFS
for i in "${arr[@]}"
do
  echo "${arr[a]}"
  csplit -f $filename $filename $i &>/dev/null
  rm -f $filename'00'
done
exit 1;

I delete the file ending in 00 currently as a hack because it currently splits on line 2, so that log actually only contains one line. And the result from the echo is:

Code:
2 1177299

So in this instance from my one log, I want to create two logs - the second log will begin on line 1177298

Thoughts on how to do this, or a better way to achieve it?

Cheers
Sponsored Links
    #2  
Old 09-12-2013
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
 
Join Date: Feb 2004
Last Activity: 30 July 2014, 10:30 AM EDT
Location: NM
Posts: 10,159
Thanks: 269
Thanked 781 Times in 731 Posts
Your code is "interesting" syntactically. We can deal with that later.
Example:
Code:
exit 1

means return failure, so your script "always fails" as far as the caller is concerned.

Could you please post sample input.
Sponsored Links
    #3  
Old 09-12-2013
pmurray21 pmurray21 is offline
Registered User
 
Join Date: Sep 2013
Last Activity: 12 September 2013, 9:37 AM EDT
Posts: 3
Thanks: 1
Thanked 0 Times in 0 Posts
This is a trimmed example of the input:


Code:
Info: ****************************************************************
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 10 10:30:04 2013
Info: Hostname:           Server1
Info: Configuration file: App
Info: blah
Info: blah
Info: blah
Info: blah
Info: blah
Info: blah
Info: blah
Info: ****************************************************************
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 11 10:30:03 2013
Info: Hostname:           Server1
Info: Configuration file: App
Info: blah
Info: blah
Info: blah
Info: blah
Info: blah
Info: blah

Note there are config options in the application to have the date and or time at the start of each line. In this example I would want two files created from this.....one containing lines 1-13 and the second containing line 14-end

Cheers
    #4  
Old 09-12-2013
drl's Avatar
drl drl is offline Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 30 July 2014, 8:39 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 1,663
Thanks: 34
Thanked 186 Times in 170 Posts
Hi.

Addressing the splitting of the file with standard utility csplit:

Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate splitting of file at /pattern/+-count, csplit.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C csplit

FILE=${1-data1}

# Remove debris from previous runs.
rm -f xx??

pl " Input data file $FILE:"
cat $FILE

pl " Current situation of resultant xx?? files (expecting none):"
ls xx??

pl " Results, counts of split file in characters:"
csplit -k -z $FILE '/Version/-1' '{*}'

pl " Current situation, and content of files:"
ls -lgG xx??
pe
head xx??

exit 0

producing:

Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
csplit (GNU coreutils) 6.10

-----
 Input data file data1:
Unrelated stuff
Version something
No discernible pattern
Version otherwise

-----
 Current situation of resultant xx?? files (expecting none):
ls: cannot access xx??: No such file or directory

-----
 Results, counts of split file in characters:
34
41

-----
 Current situation, and content of files:
-rw-r--r-- 1 34 Sep 12 06:33 xx00
-rw-r--r-- 1 41 Sep 12 06:33 xx01

==> xx00 <==
Unrelated stuff
Version something

==> xx01 <==
No discernible pattern
Version otherwise

See man pages for details.

Best wshes ... cheers, drl
The Following User Says Thank You to drl For This Useful Post:
pmurray21 (09-12-2013)
Sponsored Links
    #5  
Old 09-12-2013
pmurray21 pmurray21 is offline
Registered User
 
Join Date: Sep 2013
Last Activity: 12 September 2013, 9:37 AM EDT
Posts: 3
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks for that!

How can i modify it, so that the output filename contains the date from the log line "Started at"?

The second output file in this case might be called output.log_10Sep13_10:30:04 for example to signify the date it was started on.
Sponsored Links
    #6  
Old 09-12-2013
drl's Avatar
drl drl is offline Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 30 July 2014, 8:39 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 1,663
Thanks: 34
Thanked 186 Times in 170 Posts
Hi.

Here's one method with perl:

Code:
#!/usr/bin/env bash

# @(#) s2	Demonstrate splitting of file at /pattern/+-count, csplit.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C csplit perl

FILE=${1-data2}

# Remove debris from previous runs.
rm -f xx?? output*

pl " Input data file $FILE:"
cat $FILE

pl " Current situation of resultant xx?? files (expecting none):"
ls xx??

pl " Results, counts of split file in characters:"
csplit -k -z $FILE '/Version/-1' '{*}'

pl " Current situation, and content of files:"
ls -lgG xx??
pe
head xx??

pl " Renaming script, sample in perl:"
cat p1

# Rename split file based on content of file.
# Sample data line:
# Info: Started at:         Tue Sep 10 10:30:04 2013
# New file name from fragments of data line:
# output.log_10Sep13_10:30:04
#
# extract line "Info: started at" from file
# split into array, create string from fields 758_6
# close, issue rename file to output.log_<string>
for file in xx*
do
  ./p1 $file
done

pl " Renamed files:"
ls -1 output*

exit 0

producing:

Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
csplit (GNU coreutils) 6.10
perl 5.10.0

-----
 Input data file data2:
Unrelated stuff
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 10 10:30:04 2013
No discernible pattern
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 11 10:30:03 2013

-----
 Current situation of resultant xx?? files (expecting none):
ls: cannot access xx??: No such file or directory

-----
 Results, counts of split file in characters:
181
188

-----
 Current situation, and content of files:
-rw-r--r-- 1 181 Sep 12 10:14 xx00
-rw-r--r-- 1 188 Sep 12 10:14 xx01

==> xx00 <==
Unrelated stuff
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 10 10:30:04 2013

==> xx01 <==
No discernible pattern
Info: App:  Version 2.0.1  Date 2013/09/01
Info: ****************************************************************
Info: Started at:         Tue Sep 11 10:30:03 2013

-----
 Renaming script, sample in perl:
#!/usr/bin/env perl

# @(#) p1	Demonstrate manipulation to rename file.

# Info: Started at:         Tue Sep 10 10:30:04 2013
# 0     1       2           3   4   5  6        7
# output.log_10Sep13_10:30:04
#            5 4  7  6
# extract line "Info: started at" from file
# split into array, create string from fields 758_6 (but i-1 # array)
# close, issue rename file to output.log_<string>

use strict;
use warnings;

my ( $file, $f, @pieces, $newname );
$file = shift || die " Expected filename, got nothing.\n";
open( $f, "<", $file ) || die " Cannot open file $file\.";
while (<$f>) {
  chomp;
  next if not /Info: Started at:/;
  @pieces  = split(/\s+/);
  $newname = $pieces[5] . $pieces[4] . $pieces[7] . "_" . $pieces[6];
  close $f || die " Oops, cannot close $file\n";
  rename $file, "output_$newname" || die " Cannot rename $file\n";
  last;
}

exit(0);

-----
 Renamed files:
output_10Sep2013_10:30:04
output_11Sep2013_10:30:03

To avoid making the perl script too complex, the file loop is in the shell script. One optimization that could be done is to pull that loop into the perl script.

Best wishes ... cheers, drl
Sponsored Links
    #7  
Old 09-12-2013
RudiC RudiC is online now Forum Advisor  
Registered User
 
Join Date: Jul 2012
Last Activity: 30 July 2014, 11:01 AM EDT
Location: Aachen, Germany
Posts: 3,926
Thanks: 63
Thanked 933 Times in 885 Posts
How about using this to assign to your array and then continue with your script:
Code:
arr=($(grep -B1 -n "Version " file | grep -Eo "^[0-9]+-"))

You might want to use the minus sign a IFS char.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Bash 3.2 - Array / Regex - IF 3rd member in array ends in 5 digits then do somthing... briandanielz Shell Programming and Scripting 4 08-26-2013 01:28 PM
How to delete lines above a certin line number in bash shell learninguser235 UNIX for Dummies Questions & Answers 5 05-10-2012 12:17 PM
How to delete lines above a certin line number in bash shell learninguser235 UNIX for Dummies Questions & Answers 2 05-10-2012 11:03 AM
Bash script to delete file input on command line animesharma UNIX for Dummies Questions & Answers 1 07-21-2011 04:27 AM
display the actual number of bash command interpreter nguyendu0102 Shell Programming and Scripting 3 05-09-2011 02:56 PM



All times are GMT -4. The time now is 11:10 AM.