Sponsored Content
Top Forums Shell Programming and Scripting Append specific lines to a previous line based on sequential search criteria Post 302345926 by jesse on Thursday 20th of August 2009 02:58:51 PM
Old 08-20-2009
Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear.

I have large text files that contain data as such:

Code:
143593502  09-08-20 09:02:13 xxxxxxxxxxx          xxxxxxxxxxx          09-08-20 09:02:11 N     line 1 test
line 2 test
line 3 test
143593503  09-08-20 09:02:13 xxxxxxxxxxx          xxxxxxxxxxx          09-08-20 09:02:10 N     another message

Every line in the file that starts with a 9 digit number (followed by a date / time and so on) is a unique message. In the example the first 3 lines are really 1 message (with 2 newlines in it).

This first 9 digit number increments sequentially.

What I want to do is get each message in it's entirety onto 1 line. So I *want* the file to look like:

Code:
143593502  09-08-20 09:02:13 xxxxxxxxxxx          xxxxxxxxxxx          09-08-20 09:02:11 N     line 1 test line 2 test line 3 test
143593503  09-08-20 09:02:13 xxxxxxxxxxx          xxxxxxxxxxx          09-08-20 09:02:10 N     another message

Note that I'd like there to be a space between the additional lines in a single message.

My first idea was to remove ALL newlines from the file and replace them with spaces, and then work through that data inserting a newline after each of the sequence numbers.

I believe this will solve the problem but unfortunately I don't have the chops to pull it off. I'm sure there are also other, potentially better, ways of solving the problem.

One potential issue, I suppose, would be if one of the "extra" lines in a single message was miraculously the next 9 digit number in the sequence itself. I believe the chances of this would be pretty slim, probably to the extent of making this a moot concern for me at this point... but nonetheless it's something to consider.

Ideally I would like to do this with either perl or bash.

Thanks.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies

2. Shell Programming and Scripting

How to use sed to search for string and Print previous two lines and current line

Hello, Can anybody help me to correct my sed syntax to find the string and print previous two lines and current line and next one line. i am using string as "testing" netstat -v | sed -n -e '/test/{x;2!p;g;$!N;p;D;}' -e h i am able to get the previous line current line next line but... (1 Reply)
Discussion started by: nmadhuhb
1 Replies

3. Shell Programming and Scripting

Delete new lines based on search criteria

Hi all! A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want. For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies

4. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

5. Shell Programming and Scripting

Merging Lines based on criteria

Hello, Need help with following scenario. A file contains following text: {beginning of file} New: This is a new record and it is not on same line. Since I have lost touch with script take this challenge and bring all this in one line. New: Hello losttouch. You seem to be struggling... (4 Replies)
Discussion started by: losttouch
4 Replies

6. Shell Programming and Scripting

Need To Delete Lines Based On Search Criteria

Hi All, I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,). SEARCH_CRITERIA = "REJECT, DUPLICATE" Input File: ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies

7. Shell Programming and Scripting

Append next line to previous lines when NF is less than 0

Hi All, This is very urgent, I've a data file with 1.7 millions rows in the file and the delimiter is cedilla and I need to format the data in such a way that if the NF in the next row is less than 1, it will append that value to previous line. Any help will be appricated. Thanks,... (17 Replies)
Discussion started by: cumeh1624
17 Replies

8. Shell Programming and Scripting

Copying section of file based on search criteria

Hi Guru's, I am new to unix scripting. I have a huge file with user details in it(file2) and I have another file with a list of users(file1). Script has to search a user from file1 and get all the associated lines from file2. Example: fiel1: cn=abc cn=DEF cn=xyx File 2: dn:... (10 Replies)
Discussion started by: Samingla
10 Replies

9. Shell Programming and Scripting

Returning multiple outputs of a single line based on previous repeated lines

Hello, I am trying to return a time multiple times from a file that has varying output just before the time instance, i.e. cat jumped cat jumped cat jumped time = 1.1 cat jumped cat jumped time = 1.2 cat jumped cat jumped time = 1.3 In this case i would like to output a time.txt... (6 Replies)
Discussion started by: ryddner
6 Replies

10. Shell Programming and Scripting

awk to print specific line in file based on criteria

In the file below I am trying to extract a specific instance of path, if the adjacent plugin": "/rundb/api/v1/plugin/49/. Thank you :). file "path": "/results/analysis/output/Home/Auto_user_S5-00580-4-Medexome_65_028/plugin_out/FileExporter_out.52", "plugin": "/rundb/api/v1/plugin/49/",... (8 Replies)
Discussion started by: cmccabe
8 Replies
MORE(1) 						    BSD General Commands Manual 						   MORE(1)

NAME
more -- file perusal filter for crt viewing SYNOPSIS
more [-dlfpcsu] [-num] [+/ pattern] [+ linenum] [file ...] DESCRIPTION
More is a filter for paging through text one screenful at a time. This version is especially primitve. Users should realize that less(1) provides more(1) emulation and extensive enhancements. OPTIONS
Command line options are described below. Options are also taken from the environment variable MORE (make sure to precede them with a dash (``-'')) but command line options will override them. -num This option specifies an integer which is the screen size (in lines). -d more will prompt the user with the message "[Press space to continue, 'q' to quit.]" and will display "[Press 'h' for instructions.]" instead of ringing the bell when an illegal key is pressed. -l more usually treats ^L (form feed) as a special character, and will pause after any line that contains a form feed. The -l option will prevent this behavior. -f Causes more to count logical, rather than screen lines (i.e., long lines are not folded). -p Do not scroll. Instead, clear the whole screen and then display the text. -c Do not scroll. Instead, paint each screen from the top, clearing the remainder of each line as it is displayed. -s Squeeze multiple blank lines into one. -u Suppress underlining. +/ The +/ option specifies a string that will be searched for before each file is displayed. +num Start at line number num. COMMANDS
Interactive commands for more are based on vi(1). Some commands may be preceded by a decimal number, called k in the descriptions below. In the following descriptions, ^X means control-X. h or ? Help: display a summary of these commands. If you forget all the other commands, remember this one. SPACE Display next k lines of text. Defaults to current screen size. z Display next k lines of text. Defaults to current screen size. Argument becomes new default. RETURN Display next k lines of text. Defaults to 1. Argument becomes new default. d or ^D Scroll k lines. Default is current scroll size, initially 11. Argument becomes new default. q or Q or INTERRUPT Exit. s Skip forward k lines of text. Defaults to 1. f Skip forward k screenfuls of text. Defaults to 1. b or ^B Skip backwards k screenfuls of text. Defaults to 1. Only works with files, not pipes. ' Go to place where previous search started. = Display current line number. /pattern Search for kth occurrence of regular expression. Defaults to 1. n Search for kth occurrence of last r.e. Defaults to 1. !<cmd> or :!<cmd> Execute <cmd> in a subshell v Start up an editor at current line. The editor is taken from the environment variable VISUAL if defined, or EDITOR if VISUAL is not defined, or defaults to "vi" if neither VISUAL nor EDITOR is defined. ^L Redraw screen :n Go to kth next file. Defaults to 1. :p Go to kth previous file. Defaults to 1. :f Display current file name and line number . Repeat previous command ENVIRONMENT
More utilizes the following environment variables, if they exist: MORE This variable may be set with favored options to more. SHELL Current shell in use (normally set by the shell at login time). TERM Specifies terminal type, used by more to get the terminal characteristics necessary to manipulate the screen. SEE ALSO
vi(1) less(1) AUTHORS
Eric Shienbrood, UC Berkeley Modified by Geoff Peck, UCB to add underlining, single spacing Modified by John Foderaro, UCB to add -c and MORE environment variable HISTORY
The more command appeared in 3.0BSD. This man page documents more version 5.19 (Berkeley 6/29/88), which is currently in use in the Linux community. Documentation was produced using several other versions of the man page, and extensive inspection of the source code. Linux 0.98 December 25, 1992 Linux 0.98
All times are GMT -4. The time now is 05:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy