Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Multiple headers in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multiple headers in a file
# 1  
Old 04-13-2011
Multiple headers in a file
Hi ,

I have a .txt file in which I have multiple headers, the header record starts with $ symbol...like the first column name is $Account.

I have to keep the header in the first line and delete all the remaining headers which are in the file.

I tried using sort adc.txt | uniq -u , but my file is really big and I don't want to so this..., is there any other way we can delete the header based on the first column i.e $ symbol from all the places except the first place...the first header record should exist for loading my data in to datastge.

Regards,
Deepti
# 2  
Old 04-13-2011
Please show us input and desired output - example files
# 3  
Old 04-13-2011
Assuming your .txt file looks like the following:
Code:
$Account AAA BBB
$HEADER1 CCC EEE
$HEADER2 DDD GGG
THIS_IS_NOT_HEADER FIELD2

Code:
awk 'NR==1 || $1 !~ /^\$/' data.txt

The script above will output:
Code:
$Account AAA BBB
THIS_IS_NOT_HEADER FIELD2

# 4  
Old 04-13-2011
Hey ,

Below is my sample source data.
Code:
$source|ABC|XYZ|DEF|.......|N
10records
$source|ABC|XYZ|DEF|.......|N
1000 records
$source|ABC|XYZ|DEF|.......|N
10000 records
$source|ABC|XYZ|DEF|.......|N
max is like 2 lakh records
$source|ABC|XYZ|DEF|.......|N


I want the out put as
Code:
$source|ABC|XYZ|DEF|.......|N
10records
1000 records
10000 records
max is like 2 lakh records


Regards,
Deepti
Last edited by Franklin52; 04-14-2011 at 10:30 AM.. Reason: Please use code tags
# 5  
Old 04-13-2011
The script I provided just did what you desire.
# 6  
Old 04-14-2011
The script is working from the command line..,

I tried to automate it for all the .txt files in the directory...I am removing the multiple headers from the file and assigning the same file name again with the below script(i.e i am removing the headers from files a,b,c and assigning the same name again to the new files)..I am getting an error like error in line 1...can you pls tell me where I am doing it wrong

Code:
while read FILENAME
do
awk 'NR==1 || $1 !~ /^\$/' < "$FILENAME" >  > "${FILENAME}.txt

"
Last edited by Franklin52; 04-14-2011 at 10:30 AM.. Reason: code tags
# 7  
Old 04-14-2011
If you got an error message, you should have post it here when asking questions, that would help people helping you.

I tested the following code, which did what you desire:
Code:
#!/bin/bash

while read FILENAME; do
    awk 'NR==1 || $1 !~ /^\$/'  $FILENAME >> out.txt
done < filenames.txt
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Row bind multiple csv files having different column headers

All, I guess by this time someone asked this kind of question, but sorry I am unable to find after a deep search. Here is my request I have many files out of which 2 sample files provided below. File-1 (with A,B as column headers) A,B 1,2 File-2 (with C, D as column headers) C,D 4,5 I... (7 Replies)
Discussion started by: ks_reddy
7 Replies

2. UNIX for Advanced & Expert Users

Cannot find logical file format for BSD file headers.

Hi. Unix rookie here. Been looking for a few days for reference documents on how BSD UNIX lays the logical file format onto a disk. Goal is to view/edit with hex editor for data repair. Lots of docs are available for how to use Unix commands (like xxd), but I want to learn the map of how Unix... (4 Replies)
Discussion started by: Chris_top_he_r
4 Replies

3. UNIX for Dummies Questions & Answers

Append file name to fasta file headers in Linux

How do we append the file name to fasta file headers in multiple fasta-files in Linux? (10 Replies)
Discussion started by: Mauve
10 Replies

4. Shell Programming and Scripting

Sar -u generates multiple column headers in csv file

Hi All, The below sar -u command generates multiple column headers in csv file Expected output should print column headers only once in the csv file shell script: $cat sar_cpu_EBS.sh #!/bin/bash while ; do sar -u 15 1 | awk '/^/ {print $1,$2,$4,$6,$7}' | tr -s ' ' ',' >>... (6 Replies)
Discussion started by: a1_win
6 Replies

5. Shell Programming and Scripting

Merging File with headers

Hi I need to merge 4 files. The issue i am facing is all the files have headers and i do not want them in the final output file. Can anybody suggest how to do it? (5 Replies)
Discussion started by: Arun Mishra
5 Replies

6. UNIX for Dummies Questions & Answers

Using sed command to remove multiple instances of repeating headers in one file?

Hi, I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies

7. Shell Programming and Scripting

Editing File Headers

Hey Guys, Absolute neewbie here. I am trying to see if it is possible to edit headers/meta-data of files in Mac OSX. I am basically trying to change an audio file header to read 16bit instead of 24bit. We have an issue with some of our software and it regularly exports 16bit audio files with... (3 Replies)
Discussion started by: andysuperaudiom
3 Replies

8. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

9. UNIX for Dummies Questions & Answers

reading file headers

Hello, I have done much googling on this, but apparently not using the right keywords. I am assuming there is some kind of header for each file on a disk which stores information such as mod time, access time, etc. I have two questions: 1) is there a way to read this header directly,... (2 Replies)
Discussion started by: Allasso
2 Replies

10. Shell Programming and Scripting

Remove text between headers while leaving headers intact

Hi, I'm trying to strip all lines between two headers in a file: ### BEGIN ### Text to remove, contains all kinds of characters ... Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c om/updates/ Antispyware-Downloadserver.com #2... (3 Replies)
Discussion started by: Trones
3 Replies
Login or Register to Ask a Question