Sponsored Content
Top Forums Shell Programming and Scripting To check the file and remove header before copying Post 302999232 by Don Cragun on Thursday 15th of June 2017 01:43:45 PM
Old 06-15-2017
Quote:
Originally Posted by rohit_shinez
Hi Rudic,

Thanks for your inputs, can this be modified in my code which runs in bash

Code:
something like this

sed '1d' $file > $file

The tail command in RudiC's suggestion copies your source files to their destinations removing "$HDCNT" lines of headers during the copy.

Your code above will do two things:
  • The shell will see the redirection and truncate the file named by the expansion of $file to size 0, and then
  • The sed command will delete the 1st line (of the empty file) named by the expansion of $file and copy the remaining (non-existent) lines to the output.
Do you really want to change your input files before you copy their contents to their destinations, or do you just need to remove the header lines from the copied files? If you don't need to modify the source files:
Code:
sed 1d "$file" > /output/aus/"$file"

will do what you want. If you need to change the source files too:
Code:
sed 1d "$file" > "$file.$$" && cp "$file.$$" "$file" && rm "$file.$$"

is a safe way to do what you want without breaking links to your source file (if any existed).
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove header(first line) and trailer(last line) in ANY given file

Hi, I need some help in removing the header (first line) and the trailer (last line) in a give file... The data file actually comes in EBCDIC format and I converted it into ASCII.. Now I need to strip off the first line and the last line.. I think we can use sed to do something like this:... (2 Replies)
Discussion started by: madhunk
2 Replies

2. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

3. Shell Programming and Scripting

Check file size and remove files

Hi, Here we have a situation where we have to check the file size and if the file size is greater than 0 bytes then remove the files from the directory. 1)EdwTrxn 2)EdwPost 3)EdwTndr 4)EdwSls 5)EdwSlsRej 6)EdwTndrRej Files will be created in the directory in the following manner. ... (5 Replies)
Discussion started by: srivsn
5 Replies

4. UNIX for Dummies Questions & Answers

Check header of file, without opening the file

if we happen to have a very big .csv file we cannot or dont care to edit, is there a way to check if the file contains a header row? (2 Replies)
Discussion started by: lydiaflamp
2 Replies

5. Shell Programming and Scripting

Copying the Header & footer Information to the Outfile.

Hi I am writing a perl script which checks for the specific column values from a file and writes to the OUT file. So the feed file has a header information and footer information. I header information isaround107 lines i.e. Starts with START-OF-FILE ....... so on .... ... (11 Replies)
Discussion started by: filter
11 Replies

6. Shell Programming and Scripting

Remove last few characters in a file but keeping Header and trailer intact

Hi All, I am trying write a simple command using AWK and SED to this but without any success. Here is what I am using: head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt Input: Header 1234567 abcdefgh... (2 Replies)
Discussion started by: nvuradi
2 Replies

7. Shell Programming and Scripting

Remove the file content based on the Header of the file

Hi All, I want to remove the content based on the header information . Please find the example below. File1.txt Name|Last|First|Location|DepId|Depname|DepLoc naga|rr|tion|hyd|1|wer|opr Nava|ra|tin|gen|2|wera|opra I have to search for the DepId and remove the data from the... (5 Replies)
Discussion started by: i150371485
5 Replies

8. UNIX for Advanced & Expert Users

How to check or remove IP address from a hashed known_hosts file?which

Hi, In my server, the hostname and IP addresses are added to the known_hosts which then be hashed by ssh-keygen -H -f. Is it possible to check if an IP address has already been added to the hashed known_hosts to avoid duplications (I think there will be problems if there are duplicated IP... (3 Replies)
Discussion started by: hce
3 Replies

9. UNIX for Dummies Questions & Answers

Check for empty line at end of a dynamic header in each file

Hi Folks, I have a requirement to develop a shell script. PFB my requirement, Requirement: I need to check an empty line after the end of each header in respective file and if a empty line is present simply echo file OK and if empty line is not present echo "Adding empty line" and add an... (6 Replies)
Discussion started by: tpk
6 Replies

10. Shell Programming and Scripting

Find header in a text file and prepend it to all lines until another header is found

I've been struggling with this one for quite a while and cannot seem to find a solution for this find/replace scenario. Perhaps I'm getting rusty. I have a file that contains a number of metrics (exactly 3 fields per line) from a few appliances that are collected in parallel. To identify the... (3 Replies)
Discussion started by: verdepollo
3 Replies
PASTE(1)						    BSD General Commands Manual 						  PASTE(1)

NAME
paste -- merge corresponding or subsequent lines of files SYNOPSIS
paste [-s] [-d list] file ... DESCRIPTION
The paste utility concatenates the corresponding lines of the given input files, replacing all but the last file's newline characters with a single tab character, and writes the resulting lines to standard output. If end-of-file is reached on an input file while other input files still contain data, the file is treated as if it were an endless source of empty lines. The options are as follows: -d list Use one or more of the provided characters to replace the newline characters instead of the default tab. The characters in list are used circularly, i.e., when list is exhausted the first character from list is reused. This continues until a line from the last input file (in default operation) or the last line in each file (using the -s option) is displayed, at which time paste begins selecting characters from the beginning of list again. The following special characters can also be used in list: newline character tab character \ backslash character Empty string (not a null character). Any other character preceded by a backslash is equivalent to the character itself. -s Concatenate all of the lines of each separate input file in command line order. The newline character of every line except the last line in each input file is replaced with the tab character, unless otherwise specified by the -d option. If '-' is specified for one or more of the input files, the standard input is used; standard input is read one line at a time, circularly, for each instance of '-'. EXIT STATUS
The paste utility exits 0 on success, and >0 if an error occurs. EXAMPLES
List the files in the current directory in three columns: ls | paste - - - Combine pairs of lines from a file into single lines: paste -s -d ' ' myfile Number the lines in a file, similar to nl(1): sed = myfile | paste -s -d ' ' - - Create a colon-separated list of directories named bin, suitable for use in the PATH environment variable: find / -name bin -type d | paste -s -d : - SEE ALSO
cut(1), lam(1) STANDARDS
The paste utility is expected to be IEEE Std 1003.2 (``POSIX.2'') compatible. HISTORY
A paste command appeared in Version 32V AT&T UNIX. BSD
June 25, 2004 BSD
All times are GMT -4. The time now is 08:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy