Visit Our UNIX and Linux User Community


How to extract specific data?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to extract specific data?
# 1  
Old 03-04-2014
How to extract specific data?

Bash scripting beginner here...

I have many folders, each folder representing one subject. Not all subjects have all the required files, so I need to somehow cycle through all the data and then extract the data only from subjects who have no files missing. I tried to output the ls command, but I don't know how to make a script that would be able to extract lines only if they are repeated x times for the same subject... Help greatly appreciated!
# 2  
Old 03-04-2014
How do you tell no subjects are missing? Just the number of files? Or do you have a list of necessary files that have to be present? And what do you mean when you say "lines"
# 3  
Old 03-04-2014
I know not to use a subject if files with a specific part in their names are missing. So yes, there is a list of necessary files.

And by "lines" I meant each row of ls output - directory of each file. My subjects are numbered so I thought I would list all the necessary files, sort by subject number, and then extract only subjects who appear at least 4 times. That's the best I could come up with so far. Haven't figured out how to script for that though.
# 4  
Old 03-04-2014
Not clear. Pls post a representative directory structure, a sample of the list of necessary files, and the desired operation of the script.
# 5  
Old 03-04-2014
Here is an example that checks for:
  • Subject/1.0 Install/start.doc
  • Subject/2.0 Handover
  • Subject/3.0 Operation
  • Subject/4.0 Retire

Change the FILES array to suit what you need

Code:
FILES=( "1.0 Install/start.doc"  "2.0 Handover"  "3.0 Operation"  "4.0 Retire" )

for subject in *
do
    IFS=$'\n' NEED=( $(printf "$subject/%s\n" "${FILES[@]}") )
    for file in "${NEED[@]}"
    do
       [ -f "$file" ] || continue 2
    done
    echo "All OK - process $subject"
    #
    # Put your processing code here
    #
done


Previous Thread | Next Thread
Test Your Knowledge in Computers #337
Difficulty: Easy
:set number in vi will add line numbers to the file and automatically save to disk.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How-To Extract specific data from a file.

data.txt has several information like the below.. <SERVER>:WEB:MYDOM01:/tmp/cong/MYDOM01,/tmp/app/MYDOM01 <WEBER>:CANES:https-web01,https-web02:/web/apps/https-web01/config <SERVER>:WEB:MYDOM07:/tmp/cong/MYDOM07,/tmp/app/MYDOM07... (7 Replies)
Discussion started by: mohtashims
7 Replies

2. HP-UX

How to extract data for a specific process from ovpa?

Hi Friends, One question. Supposed I want to extract data only for process named "sqlplus" how can I do it. Any suggestions? I don't want all the data as it is not useful to me e.g.. Command I use is given below extract -xp -p -r repfile -b"03/15/13 7:00 PM" -e"03/15/13 09:30 PM" -f... (1 Reply)
Discussion started by: kunwar
1 Replies

3. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

4. Shell Programming and Scripting

Extract specific data and change its arrangment

Input: HS04636 PROGRAM source 836 7001 + ID=g1 HS04636 PROGRAM beginner 836 7001 + ID=g1.t1;Parent=g1 HS04636 PROGRAM position 836 836 + Parent=g1.t1 HS04636 PROGRAM type 836 1017 + Parent=g1.t1 HS04636 ... (2 Replies)
Discussion started by: patrick87
2 Replies

5. Shell Programming and Scripting

Extract specific read from a data

Input file: #abc_1 SAASFASFGGDSGDSGDSGSDGSDGSDGSDGSDGSDGSDGDS Output file: FASFGGDSGDS I just want to print out the read from position 5 until position 15 from the data. Below is the code that I just try but it is failed to get my desired output: grep -v '#' input_file | awk... (5 Replies)
Discussion started by: patrick87
5 Replies

6. Shell Programming and Scripting

Extract specific data content from a long list of data

My input: Data name: ABC001 Data length: 1000 Detail info Data Direction Start_time End_time Length 1 forward 10 100 90 1 forward 15 200 185 2 reverse 50 500 450 Data name: XFG110 Data length: 100 Detail info Data Direction Start_time End_time Length 1 forward 50 100 50 ... (11 Replies)
Discussion started by: patrick87
11 Replies

7. Shell Programming and Scripting

Extract all the content after a specific data

My input: >seq_1 DSASSTRRARRRRTPRTPSLRSRRSDVTCS >seq_3 RMRLRRWRKSCSERS*RRSN >seq_8 RTTGLSERPRLPTTASRSISSRWTR >seq_10 NELPLEKGSLDSISIE >seq_9 PNQGDAREPQAHLPRRQGPRDRPLQAYA+ QVQHRRHDHSRTQH*LCRRRQREDCDRLHR >seq_4 DRGKGQAGCRRPQEGEALVRRCS>seq_6 FA*GLAAQDGEA*SGRG My output: Extract all... (22 Replies)
Discussion started by: patrick87
22 Replies

8. Shell Programming and Scripting

Extract data into file with specific field specs

:confused: I have a tab delimited file that I need to extract data from and into a file with specific field specs. Each field has to be a certain amount of characters. So, the name field (from delimited file) might have only 15 characters but needs to be 25 (in new file) so I need to insert spaces... (5 Replies)
Discussion started by: criddel
5 Replies

9. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

10. Shell Programming and Scripting

extract specific data from xml format file.

Hi, I need to extract the start time value (bold, red font) under the '<LogEvent ID="Timer Start">' tag (black bold) from a file with the following pattern. There are other LogEventIDs listed in the file as well, making it harder for me to extract out the specific start time that I need. . .... (7 Replies)
Discussion started by: 60doses
7 Replies

Featured Tech Videos