Sponsored Content
Top Forums Shell Programming and Scripting Awk script to create new file based on previous line Post 302420912 by m1xram on Wednesday 12th of May 2010 11:45:25 PM
Old 05-13-2010
gawk fun

That wasn't much fun. Took me a while to clue into the fact that arrays are associative and not numeric. Sorting was a pain until I made the element name a combo of the the last field and a record number to guarantee uniqueness.

Without GAWK the function asorti() is unavailable so I'm hoping you have GAWK. I'm a little unsure what you describe about where the records end up but I think you mean.. All the records that match the last field go into unique files.

Code:
cat input13.txt | gawk -F '|' '
BEGIN {
  i = 1;
}
{
  idx = $16 i;
  l[idx] = $0;
  ++i;
}
END {
  n = asorti(l, ni);
  prev = "";
  for (j = 1; j <= n; ++j) {
    outline = l[ni[j]];
    x = split(outline, al, "|");
    
    file = "out_" al[x] ".txt";
    if (al[x] != prev) {
      print "Unique " al[x];
      print outline > file;
      prev = al[x];
    } else {
      print "Same   " al[x];
      print outline >> file;
    }
  }
}'

Your data generates the following files...

Code:
out_IMM_20090131.txt
out_IMM_20090228.txt
out_IMM_20090330.txt
out_IMM_20090630.txt

Each file contains records that match the last field.

There's an easier way to do this and you can skip the sort. Here's the pseudo code...

Code:
# Remove all out_ files.
# Foreach record of input
#    File is record "out_" field[last] ".txt"
#    Append record to file

...which turns into...

Code:
rm out_*.txt > /dev/null 2>&1
cat input13.txt | gawk -F '|' '
{
  file = "out_" $16 ".txt";
  print $0 >> file
}'

This User Gave Thanks to m1xram For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk;sed appending line to previous line....

I know this has been asked before but I just can't parse the syntax as explained. I have a set of files that has user information spread out over two lines that I wish to merge into one: User1NameLast User1NameFirst User1Address E-Mail:User1email User2NameLast User2NameFirst User2Address... (11 Replies)
Discussion started by: walkerwheeler
11 Replies

2. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

3. Shell Programming and Scripting

Printing previous line based on pattern using sed

Hi, I have a written a shell script to get the previous line based on the pattern. For example if a file has below lines: ---------------------------------------------- #UNBLOCK_As _per #As per 205.162.42.92 #BLOCK_As_per #----------------------- #input checks abc.com... (5 Replies)
Discussion started by: Anjan1
5 Replies

4. Shell Programming and Scripting

awk get matched line's previous line

hi everyone, a b in c d e f in g output is: a e so awk search for "in", then print out the matched line's previuos line. Please advice. (11 Replies)
Discussion started by: jimmy_y
11 Replies

5. Shell Programming and Scripting

Improve script made to calculate value based on present and previous line

Hi all, I have made at small script to make a simple calculation on a file which is formatted in this way: I want to create a new file in which the value of particular line minus the previous line is printed. So my wanted output is: I have made the following program to do the job... (5 Replies)
Discussion started by: s052866
5 Replies

6. Programming

create a spool file based on values passed from korn shell to sql script

this is my issue. 4 parameters are passed from korn shell to sql script. parameter_1= varchar2 datatype or no value entered my user. parameter_2= number datatype or no value entered my user. parameter_3= number datatype or no value entered my user. parameter_4= number datatype or no... (5 Replies)
Discussion started by: megha2525
5 Replies

7. Shell Programming and Scripting

Perl: Conditional replace based on previous and current value in a line

I need to read the contents of a file. Then I need to grep for a keyword and replace part of the grepped line based on the condition of previous and present line. Example input file: V { port1 = P; port2 = 0; shift_port = P0; /* if next shift_port is P0 I need... (9 Replies)
Discussion started by: naveen@
9 Replies

8. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

9. Shell Programming and Scripting

awk script -print line when $2 > $2 of previous line

Hi all, From a while loop I am reading a sorted file where I want to print only the lines that have $1 match and $2 only when the difference from $2 from the previous line is > 30. Input would be like ... AN237 010 193019 0502 1 CSU Amoxycillin AN237 080 ... (2 Replies)
Discussion started by: gafoleyo73
2 Replies

10. Shell Programming and Scripting

Returning multiple outputs of a single line based on previous repeated lines

Hello, I am trying to return a time multiple times from a file that has varying output just before the time instance, i.e. cat jumped cat jumped cat jumped time = 1.1 cat jumped cat jumped time = 1.2 cat jumped cat jumped time = 1.3 In this case i would like to output a time.txt... (6 Replies)
Discussion started by: ryddner
6 Replies
jrunscript(1)						      General Commands Manual						     jrunscript(1)

NAME
jrunscript - command line script shell SYNOPSIS
jrunscript [ options ] [ arguments... ] PARAMETERS
options Options, if used, should follow immediately after the command name. arguments Arguments, if used, should follow immediately after options or command name. DESCRIPTION
jrunscript is a command line script shell. jrunscript supports both an interactive (read-eval-print) mode and a batch (-f option) mode of script execution. This is a scripting language independent shell. By default, JavaScript is the language used, but the -l option can be used to specify a different language. Through Java to scripting language communication, jrunscript supports "exploratory programming" style. NOTE: This tool is experimental and may not be available in future versions of the JDK. OPTIONS
-classpath path Specify where to find the user's .class files that are accessed by the script. -cp path This is a synonym for -classpath path -Dname=value Set a Java system property. -Jflag Pass flag directly to the Java virtual machine on which jrunscript is run. -l language Use the specified scripting language. By default, JavaScript is used. Note that to use other scripting languages, you also need to spec- ify the corresponding script engine's jar file using -cp or -classpath option. -e script Evaluate the given script. This option can be used to run "one liner" scripts specified completely on the command line. -encoding encoding Specify the character encoding used while reading script files. -f script-file Evaluate the given script file (batch mode). -f - Read and evaluate a script from standard input (interactive mode). -help Output help message and exit. -? Output help message and exit. -q List all script engines available and exit. ARGUMENTS
If [arguments...] are present and if no -e or -f option is used, then the first argument is the script file and the rest of the arguments, if any, are passed as script arguments. If [arguments..] and -e or -f option are used, then all [arguments..] are passed as script argu- ments. If [arguments..], -e and -f are missing, interactive mode is used. Script arguments are available to a script in an engine variable named "arguments" of type String array. EXAMPLES
Executing inline scripts jrunscript -e "print('hello world')" jrunscript -e "cat('http://java.sun.com')" Use specified language and evaluate given script file jrunscript -l js -f test.js Interactive mode jrunscript js>print('hello world'); hello world js>34 + 55 89 js> thread(function() { print('hello world'); } hello world js> Run script file with script arguments jrunscript test.js arg1 arg2 arg3 test.js is script file to execute and arg1, arg2 and arg3 are passed to script as script arguments. Script can access these using "argu- ments" array. SEE ALSO
If JavaScript is used, then before evaluating any user defined script, jrunscript initializes certain built-in functions and objects. These JavaScript built-ins are documented in jsdocs. 06 Aug 2006 jrunscript(1)
All times are GMT -4. The time now is 01:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy