Sponsored Content
Top Forums Shell Programming and Scripting Log4j combining lines to single line Post 303026131 by wahi80 on Tuesday 20th of November 2018 02:57:12 PM
Old 11-20-2018
Log4j combining lines to single line

Hi,
Our log4j file contents look like this:
Code:
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP1
 AS SELECT name,dept
 from employee
  Where employee='Jeff'
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP2
 AS SELECT name,dept
 from employee
  Where employee='Yung'
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182

As you can see the create statement is across many lines, and the number of lines can vary.
I need to have only one line per entry.
My output should look like this:
Code:
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP1 AS SELECT name,dept from employee  Where employee='Jeff'
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c181
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Executing command(queryId=hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182): CREATE TABLE RESULTS.E7014485_ALL_HMS_CAP2 AS SELECT name,dept from employee  Where employee='Yung'
2018-11-20T00:06:58,888  INFO [HiveServer2-Background-Pool: Thread-21912] ql.Driver: Query ID = hive_20181120000656_49af4ad0-1d37-4312-872c-a247ed80c182

Any idea on how to achieve this?

I was trying sed and some regex patterns, but was unable to make it work

Thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Break lines up into single lines after each space in every line

It sounds a bit confusing but what I have is a text file like the example below (without the Line1, Line2, Line3 etc. of course) and I want to move every group of characters into a new line after each space. Example of text file; line1 .digg-widget-theme2 ul { background: rgb(0, 0, 0) none... (7 Replies)
Discussion started by: lewk
7 Replies

2. Shell Programming and Scripting

split single line into two line or three lines

Dear All, I want to split single line into two line or three lines wherever “|” separated values comes using Input line test,DEMTEMPUT20100404010012,,,,,,,,|0070086|0070087, output shoule be test,DEMTEMPUT20100404010012,,,,,,,,0070086, test,DEMTEMPUT20100404010012,,,,,,,,0070087, (14 Replies)
Discussion started by: arvindng
14 Replies

3. Shell Programming and Scripting

Multiple lines in a single column to be merged as a single line for a record

Hi, I have a requirement with, No~Dt~Notes 1~2011/08/1~"aaa bbb ccc ddd eee fff ggg hhh" Single column alone got splitted into multiple lines. I require the output as No~Dt~Notes 1~2011/08/1~"aaa<>bbb<>ccc<>ddd<>eee<>fff<>ggg<>hhh" mean to say those new lines to be... (1 Reply)
Discussion started by: Bhuvaneswari
1 Replies

4. Shell Programming and Scripting

Combining 2 lines in a file into 1 line

Hi all, I have a file with lot of lines with repeating pattern. ( TABLE_NAME line followed by Total line). I would like combine these two lines into one line seperated by cama and create a new file. Is there a simple way to do this. Current Format ( just a sample 4 lines ) TABLE_NAME:... (10 Replies)
Discussion started by: MKNENI
10 Replies

5. Programming

PERL:Combining multiple lines to single line

Hi All I need a small help for the below format in making a small script in Perl or Shell. I have a file in which a single line entries are broken into three line entries. Eg: I have a pen and notebook. All i want is to capture in a single line in a separate file. eg: I have a pen and... (4 Replies)
Discussion started by: Kalaiela
4 Replies

6. Shell Programming and Scripting

Combining lines in to one line

Hi Friends, I have a file1.txt 1001 jkilo yrhfm 200056 jhdf rjhwjkrh 3+u8jk5h3 uru ehjk 1002 jkfhk hfjkd 2748395 fdjksfh hefjkh 3hdfk ejkh kjhjke In the above if you see the firt charcter of each line mentioned in red has a pattern . I need to create another file where , the... (6 Replies)
Discussion started by: i150371485
6 Replies

7. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

8. Shell Programming and Scripting

Combining multiple block of lines in one comma separated line

Hi Everyone, On my Linux box I have a text file having block of few lines and this block lines separated by one blank line. I would like to format and print these lines in such a way that this entire block of lines will come as single comma separated line & again next block of lines in next... (7 Replies)
Discussion started by: gr8_usk
7 Replies

9. Shell Programming and Scripting

Combining lines into a single line

i have a file (where the column values are separated by ' and the text can be enclosed in ~) which contains data in form of 4461,2,~Basic: 2 Years/Unlimited Miles Drivetrain: Gas Engine 2 Years/Unlimited Miles Duramax Engine 3 Years/Unlimited... (2 Replies)
Discussion started by: rahulchandak
2 Replies

10. UNIX for Beginners Questions & Answers

Combining lines in one line

Hi below is the input file snippet. here i want that all the line which is coming after 1 shoud be in one line. so for exanple if after 1 there is two lines which is starting with 2 should be combine in one line. input file content 1,8091012,BATCH_1430903_01,21,T,2,808738,,,,21121:87:01,... (19 Replies)
Discussion started by: scriptor
19 Replies
CREATE 
SCHEMA(7) SQL Commands CREATE SCHEMA(7) NAME
CREATE SCHEMA - define a new schema SYNOPSIS
CREATE SCHEMA schemaname [ AUTHORIZATION username ] [ schema_element [ ... ] ] CREATE SCHEMA AUTHORIZATION username [ schema_element [ ... ] ] DESCRIPTION
CREATE SCHEMA enters a new schema into the current database. The schema name must be distinct from the name of any existing schema in the current database. A schema is essentially a namespace: it contains named objects (tables, data types, functions, and operators) whose names can duplicate those of other objects existing in other schemas. Named objects are accessed either by ``qualifying'' their names with the schema name as a prefix, or by setting a search path that includes the desired schema(s). A CREATE command specifying an unqualified object name creates the object in the current schema (the one at the front of the search path, which can be determined with the function current_schema). Optionally, CREATE SCHEMA can include subcommands to create objects within the new schema. The subcommands are treated essentially the same as separate commands issued after creating the schema, except that if the AUTHORIZATION clause is used, all the created objects will be owned by that user. PARAMETERS
schemaname The name of a schema to be created. If this is omitted, the user name is used as the schema name. The name cannot begin with pg_, as such names are reserved for system schemas. username The name of the user who will own the schema. If omitted, defaults to the user executing the command. Only superusers can create schemas owned by users other than themselves. schema_element An SQL statement defining an object to be created within the schema. Currently, only CREATE TABLE, CREATE VIEW, CREATE INDEX, CREATE SEQUENCE, CREATE TRIGGER and GRANT are accepted as clauses within CREATE SCHEMA. Other kinds of objects may be created in separate commands after the schema is created. NOTES
To create a schema, the invoking user must have the CREATE privilege for the current database. (Of course, superusers bypass this check.) EXAMPLES
Create a schema: CREATE SCHEMA myschema; Create a schema for user joe; the schema will also be named joe: CREATE SCHEMA AUTHORIZATION joe; Create a schema and create a table and view within it: CREATE SCHEMA hollywood CREATE TABLE films (title text, release date, awards text[]) CREATE VIEW winners AS SELECT title, release FROM films WHERE awards IS NOT NULL; Notice that the individual subcommands do not end with semicolons. The following is an equivalent way of accomplishing the same result: CREATE SCHEMA hollywood; CREATE TABLE hollywood.films (title text, release date, awards text[]); CREATE VIEW hollywood.winners AS SELECT title, release FROM hollywood.films WHERE awards IS NOT NULL; COMPATIBILITY
The SQL standard allows a DEFAULT CHARACTER SET clause in CREATE SCHEMA, as well as more subcommand types than are presently accepted by PostgreSQL. The SQL standard specifies that the subcommands in CREATE SCHEMA can appear in any order. The present PostgreSQL implementation does not handle all cases of forward references in subcommands; it might sometimes be necessary to reorder the subcommands in order to avoid forward references. According to the SQL standard, the owner of a schema always owns all objects within it. PostgreSQL allows schemas to contain objects owned by users other than the schema owner. This can happen only if the schema owner grants the CREATE privilege on his schema to someone else. SEE ALSO
ALTER SCHEMA [alter_schema(7)], DROP SCHEMA [drop_schema(7)] SQL - Language Statements 2010-05-14 CREATE SCHEMA(7)
All times are GMT -4. The time now is 08:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy