manipulating Fields in file using SED


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting manipulating Fields in file using SED
# 1  
Old 06-10-2009
manipulating Fields in file using SED

Hi,

I have two issues:

I have one file say file1.dat and its over 3GB. It contains pipe delimited fields. The first line in the file is the header field which tells the column names etc. and from second line it's the data fileds with pipe delimited. Something like below:

Col1|Col2|Col3|Col4|....
Aaaa|wok|dasd|wsd....
Bbbb|cube|daer|fsdfsd...



1st issue: How can I change all the data value in second column to "HARD". I do not want to change the header. The modified version needs to go to a new file FILE2.DAT and the data in new file should not get effected in the original file.

2nd issue: How can I change the data value in second field that contains "wok" to WORK.I do not want to change the header. The modified version needs to go to a new file FILE2.DAT and the data in new file should not get effected in the original file.


Would really appreciate your help.

Thanks
rkumar28
# 2  
Old 06-10-2009
All of it could be easily done with 'awk'.
You've been exposed to the awk solutions for the 4 years that you've been a member of these forums. Give it a try and do come back if/when you get stuck.
Good luck!
# 3  
Old 06-10-2009
Still need help....

Thanks Vgersh99.
I was not working on Unix for quite a few years now but I recently got a job in unix and am having some initial difficulties.
Any help would be appreciated.

Can this not be done through SED.

Thanks
rkumar28
# 4  
Old 06-10-2009
yes, it be done either with awk or sed.
Here's something to start with:
Code:
awk -F'|' 'FNR>1{$2="HARD"}1' OFS='|' myFile

# 5  
Old 06-10-2009
removing first line

Quote:
Originally Posted by vgersh99
yes, it be done either with awk or sed.
Here's something to start with:
Code:
awk -F'|' 'FNR>1{$2="HARD"}1' OFS='|' myFile

Thanks a ton this seems to work.

Is it possible to add one more logic into this to delete the first line something like SED '1d' without creating another copy of file.

Let say I have a file file1.txt....
Due to space issue, I cannot create another file....I just need to remove the header and run the awk you suggested above and save it.

Thanks again
rkumar28
# 6  
Old 06-10-2009
Code:
awk -F'|' 'FNR>1 && $2="HARD"' OFS='|' myFile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

using sed delete a line from csv file based on specific data in two separate fields

Hello, :wall: I have a 12 column csv file. I wish to delete the entire line if column 7 = hello and column 12 = goodbye. I have tried everything that I can find in all of my ref books. I know this does not work /^*,*,*,*,*,*,"hello",*,*,*,*,"goodbye"/d Any ideas? Thanks Please... (2 Replies)
Discussion started by: Chris Eagleson
2 Replies

2. Shell Programming and Scripting

Manipulating xyz file with awk-if-else or sed

Hi forum, I am really hoping somebody can please help me here. I have a dataset in xyz format, with longitude as x, latitude as y and data readings as z. eg. 0 90 -8 1 90 23 2 90 -4 etc etc etc What i am looking to do is format the data so that x and y are untouched, however in... (2 Replies)
Discussion started by: shlam16
2 Replies

3. Shell Programming and Scripting

Manipulating a variable using sed (solved)

Hi, My variable has value as this: tvar1="bool_risk_enabled" Boolean "true" Now I need to replace this true with false. Which is the best way to do this? Can we do this with sed command? Please help me. ---------- Post updated at 05:23 PM ---------- Previous update was at 05:00 PM... (2 Replies)
Discussion started by: pravintse
2 Replies

4. Shell Programming and Scripting

selecting specific fields in a file (maybe with sed?)

Hi, I have a file with following lines: chr1 10 AC=2;AF=1.00;AN=2;DP=2;Dels=0.00;HRun=0;HaplotypeScore=0.00;MQ=23.00;MQ0=0;QD=14.33;SB=-10.01 chrX 18 AB=0.52;AC=1;AF=0.50;AN=2;DP=203;DS;Dels=0.00;HRun=0;HaplotypeScore=20.01;MQ=15.63;MQ0=85;QD=12.80;SB=-1289.58 I need to extract 4... (2 Replies)
Discussion started by: menenuh
2 Replies

5. Shell Programming and Scripting

Manipulating a header file using awk or sed

Hi Guys, Is there a simple way of doing the below. Available<spaces>Assigned<spaces>Maximum<spaces>Maximum<spaces>Page<spaces>Total <spaces>Used<spaces>Pct<spaces>Max. Pct<CR> Space<spaces>Capacity<spaces>Extension<spaces>Reduction<spaces>Size<spaces>... (8 Replies)
Discussion started by: eo29
8 Replies

6. Shell Programming and Scripting

Manipulating the etc/passwd file with sed

How can i use sed to extract the user name and home directory from the /etc/passwd/ file on my server. (11 Replies)
Discussion started by: Pauline mugisha
11 Replies

7. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,... (4 Replies)
Discussion started by: axo959
4 Replies

8. Solaris

Manipulating File

Help...please. I have a log that contains Warning Authentication Failed: User GHDT88998HS doesn't exit: The User GHDT88998HS could not be found Mar 22, 2008 5:22:22AM com.hometel.ttm.auth.userlogin. about maybe a thousand entries failed user acct message How can I grab just the username... (2 Replies)
Discussion started by: rivendell500
2 Replies

9. Shell Programming and Scripting

Manipulating awk $variables using sed?

I have been searching around the forums here trying to find a solution to my problem but not getting anywhere but closer to baldness. I have a 20 column pipe "|" seperated text file. The 14th variable doesnt always exist, but will have the format of YYYYMM or YYYY if it does. I need to take... (2 Replies)
Discussion started by: r0sc0
2 Replies

10. Shell Programming and Scripting

Manipulating fields record wise

Hi all, I have an input file with no delimiter. Let us say the file is abc.txt having values for fields namely, EmpNumEnameDesigSalDept. Ofcourse the file has got several records. Every field has got a fixed start and end position. I need to assign the fields to corresponding varibles say... (1 Reply)
Discussion started by: rinku11
1 Replies
Login or Register to Ask a Question