Remove footer record in specific condition


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Remove footer record in specific condition
# 1  
Old 12-11-2017
Remove footer record in specific condition

Hi Experts,

we have a requirement , need your help to remove the footer record in the file.

Input file :

Code:
1011070375,,21,,NG,NG,asdfsfadf,1011,,30/09/2017,ACI,USD,,0.28,,,,,,,,,,,,
1011070381,,21,,NG,NG,sgfseasdf,1011,,30/09/2017,ACI,GBP,,0.22,,,,,,,,,,,,
1011070389,,21,,NG,NG,werwfsafds,1011,,30/09/2017,ACI,USD,,0.63,,,,,,,,,,,,
1011070389,,21,,NG,NG,safsdgfe,1011,,30/09/2017,ACI,EUR,,0.63,,,,,,,,,,,,
1011070456,,94,,GB,GB,sfsdsa,1011,,31/10/2017,ACI,GBP,,0.55,,,,,,,,,,,,
JE,11-01-2017/05:38,26504


The above highlighted record needs to remove in the file
instead of "JE", some other values also will come. so start with "JE" will fails to convert dynamically.

Thanks in advance

Moderator's Comments:
Mod Comment Please post in the appropriate forum

Moderator's Comments:
Mod Comment Please use code tags

Last edited by Scrutinizer; 12-11-2017 at 02:47 AM.. Reason: code tags
# 2  
Old 12-11-2017
Please re-read the forum rules you agreed to when you joined this forum. Continued refusal to follow the rules may lead to be placed in read-only mode or in being banned from the UNIX & Linux Forums.

If values other than JE are at the start of records that need to be removed, how are those records identified?

Do you always want to remove every line that starts with a J?

Do you always want to remove every line that starts with an uppercase alphabetic character?

Do you always want to remove every line that starts with a character that is not a numeric character?

Do you always want to remove the last line in every file you process?

Do you always want to remove every line that contains exactly two commas?

What operating system are you using?

What shell are you using?

What have you tried to solve this problem on your own?
# 3  
Old 12-11-2017
Quote:
Originally Posted by Don Cragun
Please re-read the forum rules you agreed to when you joined this forum. Continued refusal to follow the rules may lead to be placed in read-only mode or in being banned from the UNIX & Linux Forums.

If values other than JE are at the start of records that need to be removed, how are those records identified?

Do you always want to remove every line that starts with a J?

Do you always want to remove every line that starts with an uppercase alphabetic character?

Do you always want to remove every line that starts with a character that is not a numeric character?

Do you always want to remove the last line in every file you process?

Do you always want to remove every line that contains exactly two commas?

What operating system are you using?

What shell are you using?

What have you tried to solve this problem on your own?
we are using the following version of os version

Code:
Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Your question :
Do you always want to remove every line that contains exactly two commas?

Yes , but need not look all line for 2 commas, usual it will be found at the footer record. these record will not be available start and middle. so our approach is to look tail of the file and matches the pattern as less than 2 or 3 commas .

thanks for your quick response

Last edited by rbatte1; 12-11-2017 at 11:27 AM.. Reason: Added CODE tags and fixed some spelling issues
# 4  
Old 12-11-2017
What about the other questions that Don Cragun asked, to help you give people in here a decent specification?

Last edited by RudiC; 12-11-2017 at 08:17 AM..
# 5  
Old 12-11-2017
If values other than JE are at the start of records that need to be removed, how are those records identified?

Do you always want to remove every line that starts with a J?
No, we need to remove footer record which contains comma less than 3 count
Do you always want to remove every line that starts with an uppercase alphabetic character?
No
Do you always want to remove every line that starts with a character that is not a numeric character?
No
Do you always want to remove the last line in every file you process?
No, because some times they are not giving footer record.in such case the actual value get removed.
Do you always want to remove every line that contains exactly two commas?
Yes. but we can apply the condition as less than 3 counts of comma.
What operating system are you using?

Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
What shell are you using?

Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
What have you tried to solve this problem on your own?
sorry i haven't any idea to solve this issue.
# 6  
Old 12-11-2017
How about
Code:
sed -r '$ {s/^([^,]*,){,2}[^,]*$//;T;d;} ' file

# 7  
Old 12-11-2017
You could also try:
Code:
awk -F, 'NF>3' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash to remove find and remove specific extension

The bash below executes and does find all the .bam files in each R_2019 folder. However set -x shows that the .bam extension only gets removed from one .bam file in each folder (appears to be the last in each). Why is it not removing the extension from each (this is $SAMPLE)? Thank you :). set... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Programming

[solved] how to remove header and footer

it still display header and footer header SQL*Plus: Release 10.2.0.1.0 - Production on Mon Sep 24 13:41:51 2012 Copyright (c) 1982, 2005, Oracle. All rights reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, Real... (0 Replies)
Discussion started by: ment0smintz
0 Replies

3. Shell Programming and Scripting

[AWK script]Counting the character in record and print them in condition

.......... (1 Reply)
Discussion started by: Antonlee
1 Replies

4. Shell Programming and Scripting

Add header and footer with record count in footer

This is my file(Target.txt) name|age|locaction abc|23|del xyz|24|mum jkl|25|kol The file should be like this 1|03252012 1|name|age|location 2|abc|23|del 2|xyz|24|mum 2|jkl|25|kol 2|kkk|26|hyd 3|4 Column 1 is row indicator for row 1 and 2, column indicator is 1,for data rows... (1 Reply)
Discussion started by: itsranjan
1 Replies

5. Shell Programming and Scripting

Grab unique record from different files on a condition

Hi, I think this is the toughest prob :wall: I have ever come across and I thankfully owe all of u for helping me cross this. cat 1.txt cat 2.txt K now. This is what I am looking for. Output.txt Here is how my output has been generated. First, the column one of each file... (6 Replies)
Discussion started by: jacobs.smith
6 Replies

6. UNIX for Dummies Questions & Answers

remove the header and footer using sed

I want to delete the header and footer in the file by using sed for that i ran the below script and my text file looks like emp.txt # This file contain employee # information abc 12300 34 'FGH' # This is confidential as per the firm rules. my intention is to remove the header... (8 Replies)
Discussion started by: vmachava
8 Replies

7. Shell Programming and Scripting

Help with File processing - Adding predefined text to particular record based on condition

I am generating a output: Name Count_1 Count_2 abc 12 12 def 15 14 ghi 16 16 jkl 18 18 mno 7 5 I am sending the output in html email, I want to add the code: <font color="red"> NAME COLUMN record </font> for the Name... (8 Replies)
Discussion started by: karumudi7
8 Replies

8. Shell Programming and Scripting

Remove the footer from a flat file by searching a pattern

Hi, I have more than 30,000 records in a flat file. I want to remove footer from the file by searching a string pattern for the footer. Example.... let the flat file test.dat contains below records. record1 record2 record3 .. .. .. record31000 Total records 31000 I want to remove the... (6 Replies)
Discussion started by: gani_85
6 Replies

9. Shell Programming and Scripting

remove header and footer rows

I would like to remove some lines from begining of file (header) and some lines from end of file (footer). The header/footer lines generated by web-browser when the user upload a file to my webserver. Example: -----------------------------7d62af20c052c Content-Disposition: form-data;... (2 Replies)
Discussion started by: seaky
2 Replies

10. Shell Programming and Scripting

Need to Chop Header and Footer record from input file

Hi, I need to chope the header and footer record from an input file and make a new output file, please let me know how i can do it in unix.thanks. (4 Replies)
Discussion started by: coolbudy
4 Replies
Login or Register to Ask a Question