How to remove answers and explanation parts from a text file with lots of questions.?


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove answers and explanation parts from a text file with lots of questions.?
# 1  
How to remove answers and explanation parts from a text file with lots of questions.?

Hi,
I have a text file with thousands of questions in it. Each question (multiple lines) with multiple choice options, Answer and Explanation (optional). I need to delete Answer & explanation parts for all Questions and insert a blank line before net question. Each question starts with NO.

I tried to do that with egrep as shown below, it mostly works except for extracting only the first line (as egrep matches only first line of each question) of Q. Please advise, thanks!

Code:
root@TESTBOX:~/# cat test.txt
NO.13 Darius is analysing IDS logs. During the investigation, he noticed that there was nothing
suspicious found and an alert was triggered on normal web application traffic. He can mark this alert
as:
A. False-Negative
B. False-Positive
C. True-Positive
D. False-Signature
Answer: A
NO.14 What is the proper response for a NULL scan if the port is closed?
A. SYN
B. ACK
C. FIN
D. PSH
E. RST
F. No response
Answer: E
NO.15 The Open Web Application Security Project (OWASP) is the worldwide not-for-profit
charitable organization focused on improving the security of software. What item is the primary
concern on OWASP's Top Ten Project Most Critical Web Application Security Risks?
A. Injection
B. Cross Site Scripting
C. Cross Site Request Forgery
D. Path disclosure
Answer: A
Explanation
IT Certification Guaranteed, The Easy Way!
4
The top item of the OWASP 2013 OWASP's Top Ten Project Most Critical Web Application Security
Risks is injection.
Injection flaws, such as SQL, OS, and LDAP injection occur when untrusted data is sent to an
interpreter as part of a command or query. The attacker's hostile data can trick the interpreter into
executing unintended commands or accessing data without proper authorization.
References: https://www.owasp.org/index.php/Top_10_2013-Top_10
NO.16 A recent security audit revealed that there were indeed several occasions that the company's
network was breached. After investigating, you discover that your IDS is not configured properly and
therefore is unable to trigger alarms when needed. What type of alert is the IDS giving?
A. True Positive
B. False Negative
C. False Positive
D. False Positive
Answer: B
Explanation
New questions
NO.17 A Network Administrator was recently promoted to Chief Security Officer at a local university.
One of employee's new responsibilities is to manage the implementation of an RFID card access
system to a new server room on campus. The server room will house student enrollment information
that is securely backed up to an off-site location.
During a meeting with an outside consultant, the Chief Security Officer explains that he is concerned
that the existing security controls have not been designed properly. Currently, the Network
Administrator is responsible for approving and issuing RFID card access to the server room, as well as
reviewing the electronic access logs on a weekly basis.
Which of the following is an issue with the situation?
A. Segregation of duties
B. Undue influence
C. Lack of experience
D. Inadequate disaster recovery plan
Answer: A

What I tried...
Code:
root@TESTBOX:~/# egrep -E '^A\.|^B\.|^C\.|^D\.|^E\.|^F\.|^G\.|^NO\.' test.txt
NO.13 Darius is analysing IDS logs. During the investigation, he noticed that there was nothing
A. False-Negative
B. False-Positive
C. True-Positive
D. False-Signature
NO.14 What is the proper response for a NULL scan if the port is closed?
A. SYN
B. ACK
C. FIN
D. PSH
E. RST
F. No response
NO.15 The Open Web Application Security Project (OWASP) is the worldwide not-for-profit
A. Injection
B. Cross Site Scripting
C. Cross Site Request Forgery
D. Path disclosure
NO.16 A recent security audit revealed that there were indeed several occasions that the company's
A. True Positive
B. False Negative
C. False Positive
D. False Positive
NO.17 A Network Administrator was recently promoted to Chief Security Officer at a local university.
A. Segregation of duties
B. Undue influence
C. Lack of experience
D. Inadequate disaster recovery plan
root@TESTBOX:~/#

# 2  
The problem I have is trying to match multiple lines using sed or egrep, so I did not quickly find a way to filter out the Explanation text. The rest was easy (plus I removed the stray number and the "New questions" phrase):

Code:
sed 's/NO./\nNO./' test.txt |egrep -v '^Answer|^Explanation|^New questions|^[0-9]'

Leaving an incomplete solution because of the requirement for a multiline regex match:

Code:
ubuntu# sed 's/NO./\nNO./' text.txt |egrep -v '^Answer|^Explanation|^New questions|^[0-9]'

NO.13 Darius is analysing IDS logs. During the investigation, he noticed that there was nothing
suspicious found and an alert was triggered on normal web application traffic. He can mark this alert
as:
A. False-Negative
B. False-Positive
C. True-Positive
D. False-Signature

NO.14 What is the proper response for a NULL scan if the port is closed?
A. SYN
B. ACK
C. FIN
D. PSH
E. RST
F. No response

NO.15 The Open Web Application Security Project (OWASP) is the worldwide not-for-profit
charitable organization focused on improving the security of software. What item is the primary
concern on OWASP's Top Ten Project Most Critical Web Application Security Risks?
A. Injection
B. Cross Site Scripting
C. Cross Site Request Forgery
D. Path disclosure
IT Certification Guaranteed, The Easy Way!
The top item of the OWASP 2013 OWASP's Top Ten Project Most Critical Web Application Security
Risks is injection.
Injection flaws, such as SQL, OS, and LDAP injection occur when untrusted data is sent to an
interpreter as part of a command or query. The attacker's hostile data can trick the interpreter into
executing unintended commands or accessing data without proper authorization.
References: https://www.owasp.org/index.php/Top_10_2013-Top_10

NO.16 A recent security audit revealed that there were indeed several occasions that the company's
network was breached. After investigating, you discover that your IDS is not configured properly and
therefore is unable to trigger alarms when needed. What type of alert is the IDS giving?
A. True Positive
B. False Negative
C. False Positive
D. False Positive

NO.17 A Network Administrator was recently promoted to Chief Security Officer at a local university.
One of employee's new responsibilities is to manage the implementation of an RFID card access
system to a new server room on campus. The server room will house student enrollment information
that is securely backed up to an off-site location.
During a meeting with an outside consultant, the Chief Security Officer explains that he is concerned
that the existing security controls have not been designed properly. Currently, the Network
Administrator is responsible for approving and issuing RFID card access to the server room, as well as
reviewing the electronic access logs on a weekly basis.
Which of the following is an issue with the situation?
A. Segregation of duties
B. Undue influence
C. Lack of experience
D. Inadequate disaster recovery plan

In practice, I would do this in PERL or PHP because of the required multiline matches; but I'm sure someone else can do a much better command line than me.
# 3  
The task seems easy: delete the section between "Answer:" and "NO."
Doable with sed by means of the N command and a loop. But sed has a portability issue regarding N on the last line.
So awk is the first choice here.
Code:
awk '/^Answer:/{del=1} /^NO\./{del=0; print ""} del==0' test.txt

del==0 is true if del is not initialized. A true without action defaults to {print}.
This User Gave Thanks to MadeInGermany For This Post:
# 4  
To avoid an initial newline one can test for NR>1 or del==1:
Code:
awk '/^Answer:/{del=1} (del==1 && /^NO\./){del=0; print ""} del==0' test.txt

Here comes a portable sed solution:
Code:
sed '/^Answer:/{
  :Loop
  $d; N; /\nNO\./!bLoop
  s/.*\(\n\)/\1/
}' test.txt

GNU sed needs $d, to not default-print the last "Answer:" section.
A multi-liner easiliy supports a Unix sed.
This User Gave Thanks to MadeInGermany For This Post:
# 5  
Awesome, thanks Smilie
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #977
Difficulty: Medium
A Linux Kernel Code of Conflict was introduced on 8 March 2012.
True or False?

10 More Discussions You Might Find Interesting

1. What is on Your Mind?

Answers to Recently Asked Questions about UNIX.COM

Here are some answers to some of the recent questions I have received about UNIX.COM. So, I thought I would take time to answer them here in this post: Is it expected that the original poster "thank" everyone who responds to his / her discussion thread? It is always a good practice in all... (7 Replies)
Discussion started by: Neo
7 Replies

2. UNIX for Beginners Questions & Answers

Answers for few objective questions.

Hi Unix geniuses, I need your help for the answers of few objective Q&A. i dont know if my answers are correct or not. So i really need your help to provide the answers which will help me in unix programming. (1 Reply)
Discussion started by: Vivekit82
1 Replies

3. UNIX for Dummies Questions & Answers

Help with deleting some parts from text file

Hi all, I have a fat file which contains something like this: ************************************************ blahblahblah blahblahblah Myobject1 HOME ( homecontents01 ( some junk; ) home contents02( some junk; ) ... (7 Replies)
Discussion started by: newboy
7 Replies

4. UNIX for Advanced & Expert Users

Interview questions and answers on HP Unix administration

Hi, Can some body help me to get Interview questions and answers on HP Unix administration? Thanks Krsnadasa (1 Reply)
Discussion started by: krsnadasa
1 Replies

5. Post Here to Contact Site Administrators and Moderators

Eric's Questions and Answers Blog

I hope this is ok so I will ask if I may use this thread to ask questions about programming. May I use this thread to ask questions and answer questions? If the answer is yes, this is a thread made for minimizing the amount of threads I post to ask questions about programming. Please feel... (3 Replies)
Discussion started by: Errigour
3 Replies

6. UNIX for Dummies Questions & Answers

Homework questions, book is lacking answers?

Hey you guys, There's a couple questions I cannot find the answers to anywhere in this book for my Operating Systems class, hopefully you guys could help. I believe we are in the Bourne shell. 1. Match each description to the appropriate type of variable Used in shell programs ... (7 Replies)
Discussion started by: KyleBucket
7 Replies

7. Solaris

Please give answers for this interview questions

I was not able to get answers for these interview questions. It will be appreciable and useful if any one answers this questions. (5 Replies)
Discussion started by: Sesha
5 Replies

8. Solaris

solaris 10 OS administration interview questions and answers

Hello, I am new to this forum and I am preparing for interview .Kindly can anybody send me Solaris 10 OS administration interview questions and answers (2 Replies)
Discussion started by: meet2muneer
2 Replies

9. Solaris

expecting answers for these questions?

hi all plese clarify me in the following area. 1. What is the default NFS version in solaris 5.10. If it is 3, then why it asks me to specify "-o vers=3" keyword while i am mounting a share from a RHEL 5.1 Server? 2. Can someone give the link to download packages for accessing "ntfs"... (4 Replies)
Discussion started by: kingston
4 Replies

10. Red Hat

Lots of questions about linux.

I am about 2 months new to Linux, and have only limited experience with PCLinuxOS, (kde) and Redhat. I am installing the latest version of PClinuxOS on my wife's PC right now but running into a problem I dont understand. I have never seen it before so i thought i'd ask how I can get around it.... (1 Reply)
Discussion started by: Methal
1 Replies