Shell script to pull certain fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell script to pull certain fields
# 8  
Old 10-10-2016
If we only knew what you are talking about we might have some ideas.
Are you talking of rows or of columns? What tools are available and/or preferred (has been asked before)? What OS and shell version are you using?

Just some wild guessing: You want to ignore the header line, then from the second line extract field 14 (with ~ being the field separator) into variable1, and then, from the last line, extract field 14 into variable2.

Were this correct, and your shell were a recent bash or ksh, would this come close to what you want:
Code:
read var1 var2 <<< $(awk -F~ 'NR==2 {T = $14} END {print T, $14}' /tmp/datafile.txt)
echo $var1, $var2
09032016114500AM, 09032016011500PM

# 9  
Old 10-10-2016
sorry, my terminology may be bad. I think AWK or SED would be the best native tools. I think you hit on what I was after. Just one row under the FirstExternalDateTime's for the first value and the one row under FirstExternalDateTime's last value. Let me take it for a spin and get back to you!

---------- Post updated at 04:41 PM ---------- Previous update was at 03:57 PM ----------

RudiC,
First and foremost, THANK YOU...was almost perfect. The provided code returned the following, including the comma:

09032016114500AM,

It didn't seem to return the very last row. Any quick thoughts?
# 10  
Old 10-11-2016
Which is a bit hard to believe. The comma in my proposal above comes from the echo statement, and the two values are from line 2 and the LAST line of the "datafile.txt" that you attached. First thought would be the comma is in the data.

Are my assumptions in post#8 correct?
WHat results do get applying the proposal to the attached "datafile.txt"?
Does your other/new file obey the exactly same structure?
# 11  
Old 10-11-2016
Last question, do you know how I can date format the two variables of var1 and var2 so that they echo in a format such as `date '+%d/%m/%Y_%H:%M:%S'`?
# 12  
Old 10-11-2016
You still didn't answer the questions for your OS and shell. In bash, there's "parameter expansion/Substring Expansion": ${parameter:offset:length} that could fulfill your requirement. Give it a try and come back with the result, or any problem that might arise.
# 13  
Old 10-11-2016
Thanks, sorry.
uname -s shows: AIX 00F64BC74C00

Bash 3.00.16(1)

---------- Post updated at 02:05 PM ---------- Previous update was at 01:49 PM ----------

RudiC,
Please pardon my lack of experience here. I'm not sure how to use this suggestion to apply a date format 09032016011500PM into a readable format in a line of code such as
Code:
read rdfirst <<< $(awk -F~ 'NR==2 {T = $14} END{print T}' datafile.txt)
echo $rdfirst


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 10-11-2016 at 04:26 PM.. Reason: Added CODE tags.
# 14  
Old 10-11-2016
man bash:
Code:
   Parameter Expansion
.
.
.
       ${parameter:offset}
       ${parameter:offset:length}
              Substring Expansion.  Expands to up to length characters of the value of parameter starting at the character specified by offset.  If parameter is @, an  indexed
              array subscripted by @ or *, or an associative array name, the results differ as described below.  If length is omitted, expands to the substring of the value of
              parameter starting at the character specified by offset and extending to the end of the value.  length and offset  are  arithmetic  expressions  (see  ARITHMETIC
              EVALUATION below).

So, with your above $rdfirst variable containing the quoted value,
Code:
echo ${rdfirst:2:2}/${rdfirst:0:2}/
03/09/

shows how to proceed. Give it a try and come back with your results.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script to Group by Based on Multiple Fields in a file

Hi, I want to know if there is any simple approach to SUM a field based on group by of different fields for e.g. file1.txt contains below data 20160622|XXX1||50.00||50.00|MONEY|Plan1| 20160622|XXX1||100.00||100.00|MONEY|Plan1| 20160623|XXX1||25.00||25.00|MONEY|Plan1|... (3 Replies)
Discussion started by: cnu_theprince
3 Replies

2. Shell Programming and Scripting

How to get fields and get output with awk or shell script.?

I have a flat file A.txt with field seperate by a pipe 2012/11/13 20:06:11 | 284:hawk pid=014268 opened Locations 12, 13, 14, 15 for /home/hawk_t112/t112/macteam/qt/NET12/full_ddr3_2X_FV_4BD_1.qt/dbFiles/t112.proto|2012/11/14 15:19:26 | still running |norway|norway 2012/11/14 12:53:51 | ... (6 Replies)
Discussion started by: sabercats
6 Replies

3. Shell Programming and Scripting

Aligning output with null fields in shell script

Hello Gurus ! I have what probably amounts to a few simply changes to fix; however for the life of me I cannot seem to get it ti work. I need to align the output of my script (I am writing to a logfile)... here's the lines in my code: if then echo "NODE: $node" >> $logfile... (6 Replies)
Discussion started by: gvolpini
6 Replies

4. Shell Programming and Scripting

Script to pull hashes out of large text file

I am attempting to write a script that will pull out NTLM hashes from a text file that contains about 500,000 lines of data. Not all accounts contain hashes and I only need the ones that do contain hashes. Here is a sample of what the data looks like: There are thousands of other lines in... (6 Replies)
Discussion started by: chango77747
6 Replies

5. Shell Programming and Scripting

Shell script for validating fields in a file

Hi, I have not used Unix in a very long time and I am very rusty. I would appreciate any help I can get from the more experienced and experts in Shell script. I am reading one file at a time from a folder. The file is a flat file with no delimeters or carriage return. Col1 through col6 is... (5 Replies)
Discussion started by: asemota
5 Replies

6. Shell Programming and Scripting

Need script to pull multiple field from log file

I am hoping to get some help with a script to pull certain fields from a log file. User update (xx6xxx P) rpt (yy6yyy B) 2010/01/20 21:36:01.298 Remote client forward start streamid 85af 2010/01/20 21:36:01.307 rpt2 (ZZ6ZZZ G) rpt1 (YY6YYY B) urcall (CQCQCQ ) mycall (W1AW) user... (5 Replies)
Discussion started by: TedSD
5 Replies

7. Shell Programming and Scripting

Cat 114 files using grep or awk to pull muliple fields

Files xxxxxxx.txt ------------------------------------------------------------------------------------------------------------------------------------ Req.By: xxxxxxx WABUSH MINES - xxxxxx MINE (1001) Page: 1 Run on: 12/14/09... (4 Replies)
Discussion started by: sctxms
4 Replies

8. Shell Programming and Scripting

script to pull info from my email?

Hi, I need help writing a script that would pull info from an email inbox and add it to an Evolution Calendar. I'm pretty sure I can google the commands to add the info to the calendar. The part that I really need help with is getting the info from the email into the command. Basically for... (6 Replies)
Discussion started by: xinix
6 Replies

9. UNIX for Dummies Questions & Answers

Pull a file from a remote server through a shell script

Hi, I am writing a shell script to pull a file from a remote server (Let say its a windows based remote server). One of my criteria is to pull a file only if it is not empty. We have done a similar script to push a file from our end to a remote server and before pushing it we check for the... (2 Replies)
Discussion started by: sashankkrk
2 Replies

10. Shell Programming and Scripting

Can a shell script pull the first word (or nth word) off each line of a text file?

Greetings. I am struggling with a shell script to make my life simpler, with a number of practical ways in which it could be used. I want to take a standard text file, and pull the 'n'th word from each line such as the first word from a text file. I'm struggling to see how each line can be... (5 Replies)
Discussion started by: tricky
5 Replies
Login or Register to Ask a Question