Use awk to turn character to newline then scan output


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Use awk to turn character to newline then scan output
# 1  
Old 03-31-2017
Use awk to turn character to newline then scan output

i have a datafile (data.txt) that has the following data:

Code:
#Beginner`echo '`8iYwyW8CG44LTbmUw1RX77t3wj9uqLn/fpTafgWZ4ObrYhP11IAe0uVBZLov7dlq`KegQ8JmUsvuhKZ167LbrfJB6MobPjv97p1mTNF4Nd6PN//aaNVB/c7QuNYHILf94`PiHk4K8YieO+hBTd5uwBYu2WOe9FCKyKNwNnOA5mno6xr6C2qsG2peYOS9q8v3Go`Ri/bXPaanLh0ZosIEHzmG4PRMRvO7ZorGbmsLhUawqEEqtFbiv4XvkkGk2A3PC0I`slPXK0Ifu+BMl/oVQisjMWthJq6ScRS33SmhskFZ9/U9zjZbnGgs8L8Q91CcDDDt`HxpuljzeCjSP4hiCpuDWGsopIjRyRHg/5EcnvImpiKgyE9iE4xCMZFsT1mo2vfDj`sT6MJoZjrBEeU3IzPzCeMG3fYpnqLM5l/wK3GixAMKy8kxfJqmkHaRaYYb5kdLon`V3UWNyn/tVbo54HgH233Xx9+OLM3/9el4vEJFVokBoc9QTBzeD05OI88CU1nhNwo`HuHv7AMsH877ID5GYYG0zQHqW7tTAY2UaLYvsE5BP+xWQfg2vdnqui/NEylPnR1t`/9TlseQGbbdm9zNCcyx/ElfbVLu49vuVLNawTr15WaIN400gWBg/pRMCsP7fhiaa`eNlrTNhMkhOmQmTfoko3NRJNfyj6/P8cJn0V10ssFXBG9RZz+9cW3zWhQjac4LZi`mbl207sUVVXa3zkGf6k7GXX9euq56ED4p+p6jAbJwJGEgPFQpdbI+zDLrYdB5MdC`eYtKD6+3Y8toaKKq3pXLVWARbHA4ALtkR0ONPj06SUCmzdkh4zLWGmm74F4BtqWl`GfGyp69lN9J/ScFpFKUXMd390wO2yhujpfrxLU9fwB9CbJpRhIYvSCO9n/qUWJoA`UpFRcXdVT9TuUkNxPsqnGnyy7RmHfGH2odkEPIq/oOw2LCVEbcMhdtUCoM5Ag6bk`xE0lKJb45VOg96uFFyuYN5sQc7Gn4RSY+oYta3NpBJfycIS3aKYfgX9eTHGg6VbG`yG35cmIoyH68QvCpXSkW0sfO0kxAJbZOFkLRKg5lsNX1qbtBCXwB0wRn9V7Gg7EU`rCe3ztcXqo8EDw1pPhqrPkdG0+qZpMVUtNPMKIh3UO0n5Of/JbxR7GisneETVL73`uMUkw9LLJx0PW7pyD0g7fe+RQltXUXRBeZCVDlbKXDPgHcjYj1BrlXTmjeCLNG8Z`Zr//JBsnKaX/Uu1QUNYJaifWsUyJyaoDQmQoNXxc1tAe41J9U/gAuoTMOnVc37IQ`C2OwJ7e0sVqiOLiv1wo99VxgnCGFSo4qHcArIszAqJnHnnXcoM9SpmuZzPt7W5Xi`NDokBtBFjjpeHsIpPCbQS+1624jvvtWHKqG/59OgVJxZlCJYbgoLzqADJq+xei9W`Pm1C6311wIQaAe4Q5CujwLUI08JtJp8wYcaXcUV0wR3mdpNR0AdBZxmO9a+xYECw`pmTr+PJruLXvYPLCX/jqMXom2BxyfNiLPsrSfhPr1pBM2MgIKQBGYVo8D3mw+b3S`tejP70nLybnGbXgYpfsj4sOLW3SSIntUsZAKh7LFhos0chVq7IOvMhph59EkvUjF`rMS9AHjINV3zWOcDP9XgXBKuD93JpfMB+k1nVfcfS3Qoa0uMUfqx/pkIjYo5BBzs`J/eYnCcDkRiJ0svlf2sAjctAANMylVT3vd4u0aVWBOtdD887UqPbqWd+0KszrJeJ`3cdRU8qvpcY8LkkoxSJfmY0kZTJGbSHKiwKXPOEnNNMT5j9R6rJIXvIpMb+MpOOn`vNRbDdCPXNAOhcFTRb8uhrGfl9sR06ko4WELTvo4fjAOPrILAiXGphFozR0l6OoZ`qhTFxlg5RCNL2d4HFyqwWWCGGDSNxtvUkhHEVMIkmhoUNHloTfsxEHDxMO/M9gbl`pq7t5NcakKNDwER/E8hfYOU+NdxS9ssbCSU5+LEdRvKSKo6iK8oY34Pp2+L/SNcz`CEbR4GGmMQK5F+BGwORgRTZrPe5Qg3pHsQGWA0e4SGee32OLrWSTO2JhQ1o3H5Rv`RqKJsPhU1HQXzo35e2kJW2vEWK6Tc7e3qAXqXJxNnB+TQrlYeLNt4+7xJF7HVjfs`LM+z/EJcu/mifC6HJEmhNyrSW6SZjwxAmJvDUwBajIsSBQSQp6nS53cTQeG/MrbC`3kodQxCGQiQ3CRwUKjii1fBF74j8v9QzpPlhatTZ+TOe1sY03Oyu6pm/XNNE8+24`YLuvauZp6T7y/96xNWhErKDoaoGU3On242kSVISWQr/VGFp5GlOykt5SSo0CjTrD`BegOCxF11cZRMZ6QxDy0Brqf7sp7k6YEwunWxlAErQLM8d1oq/T6O94DChV/6l0U`Ai5TMyEO7V1DGt3eVZz7OVBXYlE28045tdMDreVZJ+QEb/WIq5JlP8TB1oxLniXi`PXtc8jjBzwa8jpBZdox2mAzIr38B0Zmo/wlQPxjrD0XwPgS9HtwwVBzjO/6aBb1G`T6V2cir/zk5b/UEbQXe//tXJXsJv61d9Y7eppMK5v9wumdUsDlbbgVAOunSTHpB1`VIJjMMbN0Jjih1WeR2fUTFb238DY6sGzVR0xfdsiJvKgVq6L4ingzZXRg6EjTGjA`z9c3Uk2aP/v9PHBVu/nzM0VYaeJ1I8mCRLw0Q5WL5VIOHzb+aONfQnl5KnR6GXHv`mgOtWHeKQOCyH2o0fOpPovsAz9HJjZgcXSobJIoQGI8RDM1imeN5dEAHzbsAdX3f`0s17q7mTeBEQIiflJZDmdt69ep+oGAW/6VOYtyUF6ADoVx+Ct7hoBKuq5PwZWWzB`bpJ+kWlU7EOEPJJC2+abpkiIs/qc7/mWlDH4Y7RWdNrOT+4udGFqoHip7D+MQ4oF`v4aDxdQmTSNqv2c6GpWbSJla5C6ATimUvTlxEGKKDR1L0xchgV59t6ofOnpR7ZAF`HLHty2E17r9WYHKr9I47BGqlttMceH5vPs0CeM7sFPoOVcxsSDLvc1B20uBZX3Ty`CiJMwzAgqQ+BKTryvJ0rXZ9DqDg17vLJDY66+fxgPzAIM02r4IFfjYH9pZy1XgeH`zb8Hwha2ms0y8rqaGcodYWJOhPZX/TJqMTYc6bwdxPTHC+bc24+GJ54qVSNpKoVJ`p2ob57uiTpg6tZO6Lchfxpk7V24m9/ECJaU0FgLmTuaV9TA06/3zlWQCWs6WSCRb`' | openssl blah blah`#RRB`#Feb`#Ender`

below is the code im currently using to get what I want:

Code:
tr '`' '\n' < data.txt | sed -n '/^#Beginner/,/^#Ender/p' | egrep -v "openssl|RRB|#Feb|echo"

This code works for my purposes. However, i want to do the same exact thing with awk. and hopefully avoid havin to make 3 external calls to system utilities.

basically, i want awk to:

1. turn all instances of backticks to new lines
2. then from the output of step 1, grab only the text between the lines #Beginner and #Ender
3. then i want it to exclude all lines it finds that contain any of the following strings openssl|RRB|#Feb|echo

Last edited by SkySmart; 03-31-2017 at 12:22 PM..
# 2  
Old 03-31-2017
Incredible ... there's no #Beginner nor #Ender string in the sample data. How do you expect people to come up with a reasonable proposal if they can't seriously test it?

EDIT: Nor is any of the to-be-suppressed strings in there.
# 3  
Old 03-31-2017
sorry about that.

I have just updated my original post with an accurate data.txt file.

please review.

thank you
# 4  
Old 03-31-2017
If you don't care about your post, why should anybody else?

Howsoever, try
Code:
awk -vRS='`' '/^#Beginner/,/^#Ender/ {if (!/openssl|RRB|#Feb|echo/) print}' file

This User Gave Thanks to RudiC For This Post:
# 5  
Old 03-31-2017
Are you building elaborate rube goldberg schemes in the hope of deterring people from using your code again? Unless the 'openssl blah blah' is run on some other machine - somewhere beyond your clients' control - this is entirely pointless, as all they have to do is read it to see what's going on.

Last edited by Corona688; 03-31-2017 at 04:24 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies

2. Shell Programming and Scripting

Remove last newline character..

Hi all.. I have a text file which looks like below: abcd efgh ijkl (blank space) I need to remove only the last (blank space) from the file. When I try wc -l the file name,the number of lines coming is 3 only, however blank space is there in the file. I have tried options like... (14 Replies)
Discussion started by: Sathya83aa
14 Replies

3. Shell Programming and Scripting

replacing by newline character

I have a file (pema)with a single long record which i have to break up into multiple lines Input s1aaaaaaaaaaaaaaaaaaaaaaas1bbbbbbbbbbs1cccccccccc Output s1aaaaaaaaaaaaaaaaaaaaaaa s1bbbbbbbbbb s1cccccccccc m planning to do it by replacing s1 by \ns1 \n is the new line character i... (5 Replies)
Discussion started by: pema.yozer
5 Replies

4. Shell Programming and Scripting

Error awk: 0602-592 cannot contain a newline character

Hi guys, I'am new at this forum as well as with unix. currently iam trying to create a unix script that will write certain string of words into a file depending on certain conditions apparently i am having problems in terms of writing into the file using the awk. it returns an error ... (0 Replies)
Discussion started by: jihmantiquilla
0 Replies

5. UNIX for Dummies Questions & Answers

newline character in a variable

variable="unix\nlinux" echo $variable expected output: unix linux :wall: can i do that ?? thanks in advance!! (3 Replies)
Discussion started by: sathish92
3 Replies

6. Shell Programming and Scripting

Why SED can't see the last newline character?

Removed. My question does not make sense. and SED does see the last newline character. But I still have a question: How to remove the last newline character(the newline character at the end of last line) using SED? ---------- Post updated 05-01-11 at 10:51 AM ---------- Previous update was... (7 Replies)
Discussion started by: kevintse
7 Replies

7. Shell Programming and Scripting

any savant ? using AWK/SED to remove newline character between two strings : conditional removal

I'd like to remove (do a pattern or precise replacement - this I can handle in SED using Regex ) ---AFTER THE 1ST Occurrence ( i.e. on the 2nd occurrence - from the 2nd to fourth occurance ) of a specific string : type 1 -- After the 1st occurrence of 1 string1 till the 1st occurrence of... (4 Replies)
Discussion started by: sieger007
4 Replies

8. UNIX for Dummies Questions & Answers

echo without newline character

hi, I have a for loop where in I write some file name to another file. I want to write all the filenames to another without any newlines. how can i avoid getting new lines with echo? Thanks, Srilaxmi (2 Replies)
Discussion started by: srilaxmi
2 Replies

9. UNIX for Dummies Questions & Answers

newline character

hi, I want to print the below lines "Message from bac logistics The Confirmation File has not been received." When i give like this in the code "Message from bac logistics\n The Confirmation File has not been received." It is giving only Message from bac logistics\n The... (9 Replies)
Discussion started by: trichyselva
9 Replies

10. UNIX for Dummies Questions & Answers

How can I replace newline character?

Hi, I am trying to write a script to prepare some text for use as web content. What is happening is that all the newlines in the textfile are ignored, so I want to be able to replace/add a few characters so that for a file containg: This is line 1. This is line two. This is line four.... (1 Reply)
Discussion started by: ghoti
1 Replies
Login or Register to Ask a Question