How to extract specific data?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to extract specific data?
# 1  
Old 03-04-2014
How to extract specific data?

Bash scripting beginner here...

I have many folders, each folder representing one subject. Not all subjects have all the required files, so I need to somehow cycle through all the data and then extract the data only from subjects who have no files missing. I tried to output the ls command, but I don't know how to make a script that would be able to extract lines only if they are repeated x times for the same subject... Help greatly appreciated!
# 2  
Old 03-04-2014
How do you tell no subjects are missing? Just the number of files? Or do you have a list of necessary files that have to be present? And what do you mean when you say "lines"
# 3  
Old 03-04-2014
I know not to use a subject if files with a specific part in their names are missing. So yes, there is a list of necessary files.

And by "lines" I meant each row of ls output - directory of each file. My subjects are numbered so I thought I would list all the necessary files, sort by subject number, and then extract only subjects who appear at least 4 times. That's the best I could come up with so far. Haven't figured out how to script for that though.
# 4  
Old 03-04-2014
Not clear. Pls post a representative directory structure, a sample of the list of necessary files, and the desired operation of the script.
# 5  
Old 03-04-2014
Here is an example that checks for:
  • Subject/1.0 Install/start.doc
  • Subject/2.0 Handover
  • Subject/3.0 Operation
  • Subject/4.0 Retire

Change the FILES array to suit what you need

Code:
FILES=( "1.0 Install/start.doc"  "2.0 Handover"  "3.0 Operation"  "4.0 Retire" )

for subject in *
do
    IFS=$'\n' NEED=( $(printf "$subject/%s\n" "${FILES[@]}") )
    for file in "${NEED[@]}"
    do
       [ -f "$file" ] || continue 2
    done
    echo "All OK - process $subject"
    #
    # Put your processing code here
    #
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How-To Extract specific data from a file.

data.txt has several information like the below.. <SERVER>:WEB:MYDOM01:/tmp/cong/MYDOM01,/tmp/app/MYDOM01 <WEBER>:CANES:https-web01,https-web02:/web/apps/https-web01/config <SERVER>:WEB:MYDOM07:/tmp/cong/MYDOM07,/tmp/app/MYDOM07... (7 Replies)
Discussion started by: mohtashims
7 Replies

2. HP-UX

How to extract data for a specific process from ovpa?

Hi Friends, One question. Supposed I want to extract data only for process named "sqlplus" how can I do it. Any suggestions? I don't want all the data as it is not useful to me e.g.. Command I use is given below extract -xp -p -r repfile -b"03/15/13 7:00 PM" -e"03/15/13 09:30 PM" -f... (1 Reply)
Discussion started by: kunwar
1 Replies

3. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

4. Shell Programming and Scripting

Extract specific data and change its arrangment

Input: HS04636 PROGRAM source 836 7001 + ID=g1 HS04636 PROGRAM beginner 836 7001 + ID=g1.t1;Parent=g1 HS04636 PROGRAM position 836 836 + Parent=g1.t1 HS04636 PROGRAM type 836 1017 + Parent=g1.t1 HS04636 ... (2 Replies)
Discussion started by: patrick87
2 Replies

5. Shell Programming and Scripting

Extract specific read from a data

Input file: #abc_1 SAASFASFGGDSGDSGDSGSDGSDGSDGSDGSDGSDGSDGDS Output file: FASFGGDSGDS I just want to print out the read from position 5 until position 15 from the data. Below is the code that I just try but it is failed to get my desired output: grep -v '#' input_file | awk... (5 Replies)
Discussion started by: patrick87
5 Replies

6. Shell Programming and Scripting

Extract specific data content from a long list of data

My input: Data name: ABC001 Data length: 1000 Detail info Data Direction Start_time End_time Length 1 forward 10 100 90 1 forward 15 200 185 2 reverse 50 500 450 Data name: XFG110 Data length: 100 Detail info Data Direction Start_time End_time Length 1 forward 50 100 50 ... (11 Replies)
Discussion started by: patrick87
11 Replies

7. Shell Programming and Scripting

Extract all the content after a specific data

My input: >seq_1 DSASSTRRARRRRTPRTPSLRSRRSDVTCS >seq_3 RMRLRRWRKSCSERS*RRSN >seq_8 RTTGLSERPRLPTTASRSISSRWTR >seq_10 NELPLEKGSLDSISIE >seq_9 PNQGDAREPQAHLPRRQGPRDRPLQAYA+ QVQHRRHDHSRTQH*LCRRRQREDCDRLHR >seq_4 DRGKGQAGCRRPQEGEALVRRCS>seq_6 FA*GLAAQDGEA*SGRG My output: Extract all... (22 Replies)
Discussion started by: patrick87
22 Replies

8. Shell Programming and Scripting

Extract data into file with specific field specs

:confused: I have a tab delimited file that I need to extract data from and into a file with specific field specs. Each field has to be a certain amount of characters. So, the name field (from delimited file) might have only 15 characters but needs to be 25 (in new file) so I need to insert spaces... (5 Replies)
Discussion started by: criddel
5 Replies

9. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

10. Shell Programming and Scripting

extract specific data from xml format file.

Hi, I need to extract the start time value (bold, red font) under the '<LogEvent ID="Timer Start">' tag (black bold) from a file with the following pattern. There are other LogEventIDs listed in the file as well, making it harder for me to extract out the specific start time that I need. . .... (7 Replies)
Discussion started by: 60doses
7 Replies
Login or Register to Ask a Question