Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Shell script to split data with a delimiter having chars and special chars

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Shell script to split data with a delimiter having chars and special chars
# 1  
Old 08-23-2019
Shell script to split data with a delimiter having chars and special chars
Hi Team,

I have a file a1.txt with data as follows.
Code:
dfjakjf...asdfkasj</EnableQuotedIDs><SQL><SelectStatement modified='1' type='string'><![CDATA[ SELECT

The delimiter string:
Code:
<SelectStatement modified='1' type='string'><![CDATA[

Code:
dlm="<SelectStatement modified='1' type='string'><![CDATA["
head -1 a1.txt | awk -F"$dlm" '{print $2}'

The above command is not working if we have multiple chars + special chars as delimiter.
Expected output is as follows.
Code:
SELECT

Can anyone please me to fix this issue?

Thanks
Krishna


Moderator's Comments:
Mod Comment Please, please, USE CODE TAGS !!!
Last edited by Scrutinizer; 08-23-2019 at 05:16 PM.. Reason: code tags !
# 2  
Old 08-23-2019
WHAT "is not working"? Any error messages?


If it's about awk not being happy with the field separator try escaping the square brackets:
Code:
dlm="<SelectStatement modified='1' type='string'><\!\\[CDATA\\["
awk -F"$dlm" '{print $2}' file
 SELECT

# 3  
Old 08-23-2019
Yes Sir. Still not working. Here's the exec msg's.

Code:
-sh-4.2$ dlm="<SelectStatement modified='1' type='string'><\!\\[CDATA\\["
-sh-4.2$ head -1 T24CustAuthSignerRlshpToXfmLoad.sql.txt
<?xml version='1.0' encoding='UTF-16'?><Properties version='1.1'><Common><Context type='int'>1</Context><![CDATA[0]]></EnableQuotedIDs><SQL><SelectStatement modified='1' type='string'><![CDATA[SELECT
-sh-4.2$ head -1 T24CustAuthSignerRlshpToXfmLoad.sql.txt | awk -F"${dlm}" '{print $2}'
awk: warning: escape sequence `\!' treated as plain `!'
awk: warning: escape sequence `\[' treated as plain `['
awk: fatal: Unmatched [ or [^: /<SelectStatement modified='1' type='string'><![CDATA[/
-sh-4.2$
-sh-4.2$ uname -a
Linux  3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
-sh-4.2$

Last edited by Scrutinizer; 08-23-2019 at 05:49 PM.. Reason: Changing code tags angular brackets to square brackets
# 4  
Old 08-23-2019
Code:
dlm="<SelectStatement modified='1' type='string'><![CDATA["
awk 'NR==1 && index($0, dlm) {print substr($0, index($0, dlm) + length(dlm))}' dlm="$dlm" a1.txt

Last edited by rdrtx1; 02-18-2020 at 08:25 PM..
# 5  
Old 08-26-2019
Thank you for your reply rdxtr1. It still did not work.

Here's the error.

Code:
-sh-4.2$ dlm="<SelectStatement modified='1' type='string'><![CDATA["
-sh: ![CDATA[": event not found
-sh-4.2$

# 6  
Old 08-26-2019
In bash - when used interactively - you need to turn off history expansion/substitution:
Code:
set +H

to keep the shell from interpreting the ! character within double quotes
This User Gave Thanks to Scrutinizer For This Post:
kmanivan82
# 7  
Old 08-26-2019
Thank you Scrutinizer. It worked.

Similarly I have another scenario. The last line of the record is as follows.


Code:
  FROM XYZ.[dbo].[STG_PRQ_UVW] ]]><ReadStatementFromFile type='bool'><![CDATA[0]]></ReadStatementFromFile><Tables collapsed='1'></Tables><Parameters collapsed='1'></Parameters><Columns collapsed='1'></Columns></SelectStatement><EnablePartitioning collapsed='1' type='bool'><![CDATA[0]]></EnablePartitioning></SQL><Transaction><RecordCount modified='1' type='int'><![CDATA[20000]]></RecordCount><EndOfWave collapsed='1' type='int'><![CDATA[0]]></EndOfWave></Transaction><Session><IsolationLevel type='int'><![CDATA[1]]></IsolationLevel><AutocommitMode type='int'><![CDATA[0]]></AutocommitMode><ArraySize modified='1' type='int'><![CDATA[20000]]></ArraySize><SchemaReconciliation><FailOnSizeMismatch type='bool'><![CDATA[1]]></FailOnSizeMismatch><FailOnTypeMismatch type='bool'><![CDATA[1]]></FailOnTypeMismatch><FailOnCodePageMismatch type='bool'><![CDATA[0]]></FailOnCodePageMismatch></SchemaReconciliation><PassLobLocator collapsed='1' type='bool'><![CDATA[0]]></PassLobLocator><CodePage collapsed='1' type='int'><![CDATA[0]]></CodePage></Session><BeforeAfter collapsed='1' type='bool'><![CDATA[0]]></BeforeAfter><LimitRows collapsed='1' type='bool'><![CDATA[0]]></LimitRows></Usage></Properties >

The output should be as follows.
Code:
  FROM XYZ.[dbo].[STG_PRQ_UVW]

We need to look for the code snippet "]]><ReadStatementFromFile type" and strip out the text before the snippet.

I have tried tweaking the suggested awk command, it did not work. Can you please help me out?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell scripting to determine special chars in file

Hi, I need all your help to achieve the below functionality. I have a big 2 GB file and inside the file we need to identify, whether having a comma(,) or pipe(|) or tab or fixed position or semicolon(;) delimiter. If any of those delimiter found need to replace the file with pipe(|)... (1 Reply)
Discussion started by: lkeswar
1 Replies

2. UNIX for Advanced & Expert Users

Inserting delimiter after a specific number of chars

Hello guys, I have a problem where I need to add a delimiter, that can be | for example, after each 28000 chars. The problem is that sometimes 1 row, which should contain 28000 chars is split in 2, so I want to put the delimiter after each 28000 so I will know the end of each row. Please... (2 Replies)
Discussion started by: Diogo R Jesus
2 Replies

3. Shell Programming and Scripting

If condition matching with special chars

Hi, I have file #cat drivers.txt fcs0 fcs1 vscsi1 vscsi2 In this i need to check the availabality of "fcs" or "vscsi" alone not vscsi0,fcs1 I tried with "if condition" but it is not working. cat drivers.txt| while read ADAP do echo "Checking for $ADAP" if ;then echo "FC... (9 Replies)
Discussion started by: ksgnathan
9 Replies

4. Shell Programming and Scripting

All strings within two special chars

I have a file with multiple lines. From each line I want to get all strings that starts with '+' and ends with '/'. Then I want the strings to be separated by ' + ' Example input: +$A$/NOUN+At/NSUFF_FEM_PL+K/CASE_INDEF_ACC Sample output: $A$ + At + K (20 Replies)
Discussion started by: Viernes
20 Replies

5. UNIX for Dummies Questions & Answers

Strings with Special chars in IF condition

I was trying to run a code to check if a fax number is empty or not. for that, I've written the following code which is throwing an error. #!/bin/ksh fax= "999-999-9999" if ; then fax_no="000-000-0000" else fax_no=$fax fi echo $fax_no And I get the... (7 Replies)
Discussion started by: hooaamai
7 Replies

6. Shell Programming and Scripting

find 4 chars on 2nd line, 44 chars over

I know this should be simple, but I've been manning sed awk grep and find and am stupidly stumped :( I'm trying to use sed (or awk, find, etc) to find 4 characters on the second line of a file.txt 44-47 characters in. I can find lots of sed things for lines, but not characters. (4 Replies)
Discussion started by: unclecameron
4 Replies

7. Shell Programming and Scripting

How to convert C source from 8bit chars to 16bit chars?

I was using the following bash command inside the emacs compile command to search C++ source code: grep -inr --include='*.h' --include='*.cpp' '"' * | sed "/include/d" | sed "/_T/d" | sed '/^ *\/\//d' | sed '/extern/d' Emacs will then position me in the correct file and at the correct line... (0 Replies)
Discussion started by: siegfried
0 Replies

8. Shell Programming and Scripting

special chars arrangement in code

here is my simple script to show process and owners except me: ps `-ef |grep xterm |grep -v aucar` | while read a1 a2 a3 a4 a5 a6 a7 a8 do echo KILL..\($a1\).. $a2 |more done how can I pass values from command "ps -ef |grep xterm|grep -v aucar" to ? because above command... (2 Replies)
Discussion started by: xramm
2 Replies

9. Shell Programming and Scripting

treating special chars

Hi, I need some advise on treating non printable chars over ascii value 126 Case 1 : On some fields in the text , I need to retiain then 'as-is' and load to a database.I understand it also depends on database codepage. but i just wanna know how do i ensure it do not change while loading... (1 Reply)
Discussion started by: braindrain
1 Replies

10. UNIX for Advanced & Expert Users

Supress special chars in vi

Hi, One of our application is producing log files. But if we open the log file in vi or less or view mode, it shows all the special characters in it. The 'cat' shows correctly but it shows only last page. If I do 'cat' <file_name> | more, then again it shows special characters. ... (1 Reply)
Discussion started by: divakarp
1 Replies
Login or Register to Ask a Question