Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Grabbing a sub section of a file between 2 specific values Post 303044813 by Big_Jeffrey on Wednesday 4th of March 2020 11:38:19 AM
Old 03-04-2020
Grabbing a sub section of a file between 2 specific values

Hi. I have a file that contains one continuous line of output which is made up of multiple repeating sections of code with just some of the values being unique. No spaces and no carriage returns.

Effectively the file could be divided up into multiple repeating blocks of info. I need to grab the sub section of the line around one of these values.
So, I have unique value for HostName (test12-42213.devserver.com) and I need all the info from that value until the first occurrence of a non-unique delimiter called "EndTime".

I have tried the following but it's not filtering anything out. It only seems to work on simple strings:
Code:
cat example.out | sed 's/.*"test12-42213.devserver.com"\(.*\)"EndTime"/\1/'

An example of the file with one continuous line would be (no spaces and no carriage returns):
Code:
Translation,occurs,if,-d,is,not,given,and,both,SET1,and,SET2,appear.-tmaybeusedonlywhentranslating.SET2,is,extended,to,length,of"SET1",
by,repeating,its,last,character:as:necessary,"HostName":"test12-42213.devserver.com",Excess_characters_of,SET2,are,ignored.Only[:lower:]and[:upper:]are,guaranteed,
to,expand,in,ascending:order;"EndTime":null}Translation,occurs,if,-d,is,not,given,and,both,SET1,and,SET2,appear.-tmaybeusedonlywhentranslating.
SET2,is,extended,to,length,of"SET1",by,repeating,its,last,character:as:necessary,"HostName":"test99-9999.devserver.com",Excess_characters_of,SET2,are,ignored.
Only[:lower:]and[:upper:]are,guaranteed,to,expand,in,ascending:order;"EndTime":null}

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

finding specific values in a within a file

Hi everyone, Can anyone guide me on how to search through a huge file and look on specific column and if it finds a discrepancy on that column that does not conform to the specified criteria, ie (1) Numeric and (3) alpha chars F123 or G333..etc, etc! then idientify it and redirect... (3 Replies)
Discussion started by: Gerry405
3 Replies

2. Shell Programming and Scripting

How to read a specific section and modify within

Hi, I am n00b to shell scripting and I am learning Ksh, sed and awk. I have a requirement and need your help. 1) How to read a specific section of a file. I have a file and I want to read the contents between say "Page Number:1" to "End of Page 1" 2) Within the section of the file that was... (2 Replies)
Discussion started by: kn.naresh
2 Replies

3. Shell Programming and Scripting

grabbing specific column perl

Alright, I'm new to Perl so be gentle. Given the following script: ---- open(file, "<file.txt"); @lines = <file>; close(file); $var = print $lines; ---- So I'm printing line 18 of the file "file.txt". I now want the 5th column, minus the forward slash. The line looks like this: ... (2 Replies)
Discussion started by: wxornot
2 Replies

4. Shell Programming and Scripting

to extract specific values twice in a file

Hi Friends, I have a file with the following values.. xyz.txt,12345.xml abc.txt,04567.xml cde.txt,12134.xml I would like to extract all the 2nd column values twice as shown in the example like 12345,12345.xml 04567,04567.xml 12134,12134.xml Please advice!! In the formus one of... (7 Replies)
Discussion started by: techmoris
7 Replies

5. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

6. Shell Programming and Scripting

parsing filename and grabbing specific string patterns

Hi guys...Wow I just composed a huge post and it got erased as I was logged out automatically Anyways I hope someone can help me out here. So the task I'm working on is like this I have a bunch of files that I care about sitting in a directory say $HOME/files Now my job is to go and loop... (6 Replies)
Discussion started by: rukasetsuna
6 Replies

7. Shell Programming and Scripting

using awk to get specific section of lines in logs

i have a log file that has the date and time that looks like this: Wed Jun 28 15:46:21 2012 test failed tailed passed passed not error panic what we want to focus on is the first 5 columns because they contain the date and time. the date and time can be anywhere on the line. in this... (6 Replies)
Discussion started by: SkySmart
6 Replies

8. Shell Programming and Scripting

Adding a lines to specific section of the file.

Hello, I have to a add 2 lines to /etc/sudoers file under this section below, can someone please suggest script to add these two lines when execute this remotely on to a multiple servers. before ## Allow root to run any commands anywhere root ALL=(ALL) ALL After ## Allow root... (2 Replies)
Discussion started by: bobby320
2 Replies

9. AIX

Vmstat fault section all values are 0

Hi all, Recently I facing problem with my AIX server. we experience slowness on performance. there are some application installed in this server such as : Oracle 10g database, control-m client agent, and some monitoring tools. when we're facing the problem we're noticing that vmstat value a... (7 Replies)
Discussion started by: Arief Winanto
7 Replies

10. Shell Programming and Scripting

Get specific values from a file, help

Dear all, I have a specific problem that i cannot solve and I hope someone here can help me. :) I have two text files with one column of values. Example: File1: 67 94 95 . . File2 0.1 0.003 0.5 . . (3 Replies)
Discussion started by: Higgo
3 Replies
regcomp(3)						     Library Functions Manual							regcomp(3)

NAME
regcomp, regerror, regexec, regfree - Compares string to regular expression LIBRARY
Standard C Library (libc.so, libc.a) SYNOPSIS
#include <sys/types.h> #include <regex.h> int regcomp( regex_t *preg, const char *pattern, int cflags); size_t regerror( int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size); int regexec( const regex_t *preg, const char *string, size_t nmatch, regmatch_t *pmatch, int eflags); void regfree( regex_t *preg); STANDARDS
Interfaces documented on this reference page conform to industry standards as follows: regcomp(), regexec(), regerror(), regfree(): POSIX.2, XPG4, XPG4-UNIX Refer to the standards(5) reference page for more information about industry standards and associated tags. PARAMETERS
Specifies the flags for regcomp(). The cflags parameter is the bitwise inclusive OR of zero or more of the following flags, which are defined in the /usr/include/regex.h file. Uses extended regular expressions. Ignores case in match. Reports only success or failure in regexec(); does not report subexpressions. Treats newline as a special character marking the end and beginning of lines. Contains the basic or extended regular expression to be compiled by regcomp(). The structure that contains the compiled basic or extended regular expression. Identifies the error code. Points to the buffer where regerror() stores the message text. Specifies the size of the errbuf buffer. Contains the data to be matched. Contains the number of subexpressions to match. Contains the array of offsets into the string parameter that match the corresponding subexpression in the preg parameter. Specifies the flags controlling the customizable behavior of the regexec function. The eflags parameter modifies the interpretation of the contents of the string parameter. The value for this param- eter is formed by bitwise inclusive ORing zero or more of the following flags, which are defined in the /usr/include/regex.h file. The first character of the string pointed to by the string parameter is not the beginning of the line. Therefore, the circumflex character ^ (circumflex), when taken as a special character, does not match the beginning of the string parameter. The last character of the string pointed to by the string parameter is not the end of the line. Therefore, the $ (dollar sign), when taken as a special character, does not match the end of the string parameter. DESCRIPTION
The regcomp(), regerror(), regexec(), and regfree() functions perform regular expression matching. The regcomp() function compiles a regu- lar expression and the regexec() function compares the compiled regular expression to a string. The regerror() function returns text asso- ciated with an error condition encountered by regcomp() or regexec(). The regfree() function frees the internal storage allocated for the compiled regular expression. The regcomp() function compiles the basic or extended regular expression specified by the pattern parameter and places the output in the preg structure. The default regular expression type for the pattern parameter is a basic regular expression. An application can specify extended regular expressions with the REG_EXTENDED flag. If the REG_NOSUB flag is not set in cflags, the regcomp() function sets the number of parenthetic subexpressions (delimited by ( and ) in basic regular expressions or by () in extended regular expressions) to the number found in pattern. The regexec() function compares the null-terminated string in the string parameter against the compiled basic or extended regular expres- sion in the preg parameter. If a match is found, the regexec() function returns a value of 0 (zero). The regexec() function returns REG_NOMATCH if there is no match. Any other nonzero value returned indicates an error. If the value of the nmatch parameter is 0 (zero) or if the REG_NOSUB flag was set on the call to the regcomp() function, the regexec() function ignores the pmatch parameter. Otherwise, the pmatch parameter points to an array of at least the number of elements specified by the nmatch parameter. The regexec() function fills in the elements of the array pointed to by the pmatch parameter with offsets of the sub- strings of the string parameter. The elements of the pmatch array correspond to the parenthetic subexpressions of the original pattern parameter that was specified to the regcomp() function. The pmatch[i].rm_so structure is the byte offset of the beginning of the substring, and the pmatch[i].rm_eo structure is one greater than the byte offset of the end of the substring. Subexpression i begins at the ith matched open parenthesis, counting from 1. The 0 (zero) element of the array corresponds to the entire pattern. Unused elements of the pmatch parameter, up to the value pmatch[nmatch-1], are filled with -1. If the number of subexpressions exceeds the number specified by the nmatch parameter (the pattern parameter itself counts as a subexpression), only the first nmatch-1 are recorded. When matching a basic or extended regular expression, any given parenthetic subexpression of the pattern parameter can participate in the match of several different substrings of the string parameter; however, it may not match any substring even though the pattern as a whole did match. The following rules are used to determine which substrings to report in the pmatch parameter when matching regular expressions: If a subexpression in a regular expression participated in the match several times, the offset of the last matching substring is reported in the pmatch parameter. If a subexpression did not participate in a match, the byte offset in the pmatch parameter is a value of -1. If a subexpression is contained in a subexpression, the data in the pmatch parameter refers to the last such subexpression. If a subexpres- sion is contained in a subexpression and the byte offsets in the pmatch parameter have a value of -1, the pointers in the pmatch parameter also have a value of -1. If a subexpression matched a zero-length string, the offsets in the pmatch parameter refer to the byte immedi- ately following the matching string. If the REG_NOSUB flag was set in the cflags parameter in the call to the regcomp() function and the nmatch parameter is not equal to 0 (zero) in the call to the regexec function, the content of the pmatch array is unspecified. If the REG_NEWLINE flag was not set in the cflags parameter when the regcomp() function was called, a newline character in the pattern or string parameter is treated as an ordinary character. If the REG_NEWLINE flag was set when the regcomp() function was called, the newline character is treated as an ordinary character, except as follows: A newline character in the string parameter is not matched by a . (dot) outside of a bracket expression or by any form of a nonmatching list. A ^ (circumflex) in the pattern parameter, when used to specify expression anchoring, matches the zero-length string immediately after a newline character in the string parameter, regardless of the set- ting of the REG_NOTBOL flag. A $ (dollar sign) in the pattern parameter, when used to specify expression anchoring, matches the zero- length string immediately before a newline character in the string parameter, regardless of the setting of the REG_NOTEOL flag. The regerror() function returns the text associated with the specified error code. If the regcomp() or regexec() function fails, it returns a nonzero error code. If this return value is assigned to the errcode parameter, the regerror() function returns the text of the associated message. If the errbuf_size parameter is not 0, regerror() places the generated string into the buffer size errbuf_size bytes pointed to by errbuf. If the string (including the terminating null) cannot fit in the buffer, regerror() truncates the string and null-terminates the result. If errbuf_size is 0, regerror() ignores the errbuf parameter and returns the size of the buffer needed to hold the generated string. The regfree() function frees any memory allocated by the regcomp() function associated with the preg parameter. An expression defined by the preg parameter is no longer treated as a compiled basic or extended regular expression after it is given to the regfree() function. EXAMPLES
The following example demonstrates how the REG_NOTBOL flag can be used with the regexec() function to find all substrings in a line that match a pattern supplied by a user. The main() function in the example accepts two input strings from the user. The match() function in the example uses regcomp() and regexec() to search for matches. #include <sys/types.h> #include <regex.h> #include <locale.h> #include <stdio.h> #include <string.h> #include <nl_types.h> #include "reg_example.h" #define SLENGTH 128 main() { char patt[SLENGTH], strng[SLENGTH]; char *eol; nl_catd catd; (void)setlocale(LC_ALL, ""); catd = catopen("reg_example.cat", NL_CAT_LOCALE); printf(catgets(catd,SET1,INPUT, "Enter a regular expression:")); fgets(patt, SLENGTH, stdin); if ((eol = strchr(patt, ' ')) != NULL) *eol = ''; /* Replace newline with null */ else return; /* Line entered too long */ printf(catgets(catd,SET1,COMPARE, "Enter string to compare String: ")); fgets(strng, SLENGTH, stdin); if ((eol = strchr(strng, ' ')) != NULL) *eol = ''; /* Replace newline with null */ else return; /* Line entered too long */ match(patt, strng); } int match(char *pattern, char *string) { char message[SLENGTH]; char *start_search; int error, msize, count; regex_t preg; regmatch_t pmatch; error = regcomp(&preg, pattern, REG_ICASE | REG_EXTENDED); if (error) { msize = regerror(error, &preg, message, SLENGTH); printf("%s ", message); if (msize > SLENGTH) printf(catgets(catd,SET1,LOST,"Additional text lost ")); return; } error = regexec(&preg, string, 1, &pmatch, 0); if (error == REG_NOMATCH) { printf(catgets(catd,SET1,NO_MATCH, "No matches in string ")); return; } else if (error != 0) { msize = regerror(error, &preg, message, SLENGTH); printf("%s ", message); if (msize > SLENGTH) printf(catgets(catd,SET1,LOST, "Additional text lost ")); return; }; count = 1; start_search = string + pmatch.rm_eo; while (error == 0) { error = regexec(&preg, start_search, 1, &pmatch, REG_NOTBOL); start_search = start_search + pmatch.rm_eo; count++; }; count--; printf(catgets(catd,SET1,MATCH, "There are %i matches "), count); regfree(&preg); catclose(catd); } The following example finds out which subexpressions in the regular expression have matches in the string. This example uses the same main() program as the preceding example. This example does not specify REG_EXTENDED in the call to regcomp() and, consequently, uses basic regular expressions, not extended regular expressions. #define MAX_MATCH 10 int match(char *pattern, char *string) { char message[SLENGTH]; char *start_search; int error, msize, count, matches_tocheck; regex_t preg; regmatch_t pmatch[MAX_MATCH]; error = regcomp(&preg, pattern, REG_ICASE); if (error) { msize = regerror(error, &preg, message, SLENGTH); printf("regcomp: %s ", message); if (msize > SLENGTH) printf(catgets(catd,SET1,LOST, "Additional text lost ")); return; } if (preg.re_nsub > MAX_MATCH) { printf(catgets(catd,SET1,SUBEXPR, "There are %1$i subexpressions, checking %2$i "), preg.re_nsub, MAX_MATCH); matches_tocheck = MAX_MATCH; } else { printf(catgets(catd,SET1,SUB_EXPR_NUM, "There are %i subexpressions in the regular expression "), preg.re_nsub); matches_tocheck = preg.re_nsub; } error = regexec(&preg, string, MAX_MATCH, &pmatch[0], 0); if (error == REG_NOMATCH) { printf(catgets(catd,SET1,NO_MATCH_ENT, "String did not contain match for entire regular expression ")); return; } else if (error != 0) { msize = regerror(error, &preg, message, SLENGTH); printf("regexe: %s ", message); if (msize > SLENGTH) printf(catgets(catd,SET1,LOST, "Additional text lost ")); return; } else printf(catgets(catd,SET1,MATCH_ENT, "String contained match for the entire regular expression ")); for (count = 0; count <= matches_tocheck; count++) { if (pmatch[count].rm_so != -1) { printf(catgets(catd,SET1,SUB_EXPR_MATCH "Subexpression %i matched in string "),count); printf(catgets(catd,SET1,MATCH_WHERE, "Match starts at %1$i. Byte after match is %2$i "), pmatch[count].rm_so, pmatch[count].rm_eo); } else printf(catgets(catd,SET1,NO_MATCH_SUB, "Subexpression %i had NO match "), count); } regfree(&preg); catclose(catd); } RETURN VALUES
Upon successful completion, the regcomp() function returns a value of 0 (zero). Otherwise, regcomp() returns an integer value indicating an error as described below, and the contents of the preg parameter is undefined. If the regcomp() function detects an illegal basic or extended regular expression, it returns REG_BADPAT or an error code that more precisely describes the error. If the regexec() function finds a match, the function returns a value of 0 (zero). Otherwise, it returns REG_NOMATCH to indicate no match or REG_NOSYS to indicate that the function is not supported. Upon successful completion, the regerror() function returns the number of bytes needed to hold the entire generated string. This value may be greater than the value of the errbuf_size parameter. If regerror fails, it returns 0 (zero) to indicate that the function is not imple- mented. The regfree() function returns no value. The following constants are defined as error return values: The contents within the pair { and } are invalid: not a number, number too large, more than two numbers, or first number larger than second. The pattern contains an invalid regular expression. The ?, *, or + sym- bols are not preceded by a valid regular expression. The use of a pair of { and } or {} is unbalanced. The use of [] is unbalanced. An invalid collating element was referenced. An invalid character class type was referenced. The pattern contains a trailing (backslash). The function is unsupported. The use of a pair of ( and ) or () is unbalanced. An endpoint in the range expression is invalid. Insuf- ficient memory space is available. The number in digit is invalid or in error. The pattern contains too many parenthetic subexpressions. The regexec() function did not find a match. ERRORS
These functions do not set errno to indicate an error. RELATED INFORMATION
Commands: grep(1) Standards: standards(5) delim off regcomp(3)
All times are GMT -4. The time now is 05:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy