How to make a loop to read the input from a file part by part?


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to make a loop to read the input from a file part by part?
# 1  
Old 08-24-2019
How to make a loop to read the input from a file part by part?

Hi All,

We've a VDI infrastructure in AWS (AWS workspaces) and we're planning to automate the process of provisioning workspaces. Instead of going to GUI console, and launching workspaces by selecting individual users is little time consuming. Thus, I want to create them in bunches from AWS CLI (installed on centos 7). To achieve this I've created below shell script to launch the workspaces from CLI with least efforts. Please note, the ws-userlist.txt has a list of 60 users, one word per line.

Code:
for i in `cat ws-userlist.txt`;do command-to-create-workspaces DirectoryId=<directoryID>,UserName=$i,BundleId=<bundleID>;done

Now, here, there is a limitation of above workspace creation command. It can only build maximum 25 workspaces at one execution. I wonder if there any way to make the loop read the ws-userlist.txt in such a way that it read the first 25 users and take it as input for the command and execute it. once the it get executed, loop should go back to the input file, and read next set of 25 users and execute the command. Once it get executed, loop again go back to input file, and read the last 10 usernames and run the command.
Please note, username count could vary.Sometime I may have a list of 30, sometimes it could be 55.

I've basic knowledge of shell scripting. Need your expert help on this.
Thanks in advance.
# 2  
Old 08-24-2019
This
Code:
split --version
split (GNU coreutils) 8.30

seems to have the filter receive the split input lines on its stdin.
Try
Code:
split -l25 --filter="command-to-create-workspaces params" ws-userlist.txt

These 3 Users Gave Thanks to RudiC For This Post:
# 3  
Old 08-26-2019
Thanks RudiC for your reply.

I'll check it today and will let you know.
# 4  
Old 08-26-2019
If your command takes 25 parameters then you can do
Code:
xargs -L25 echo "do something with" < ws-userlist.txt

(The echo command is there for demonstration.)
Or introduce a counter in your loop:
Code:
cnt=0; for u in `cat ws-userlist.txt`; do echo "do something with $u"; if [ $((cnt+=1)) -eq 25 ]; then echo "do special action"; cnt=0; fi; done

These 2 Users Gave Thanks to MadeInGermany For This Post:
# 5  
Old 08-28-2019
Hi Rudic,

Here is the example command to create workspaces : command-to-create-workspaces DirectoryId=<directoryID>,UserName=<User>,BundleId=<bundleID>

I wonder how can I supply the input to "UserName=" option in the command using the code given by you. UserName option should pick the input from ws-userlist.txt. Please advise.

Code:
 split -l25 --filter="command-to-create-workspaces params" ws-userlist.txt

--- Post updated at 01:27 PM ---

Thanks MadeInGermany for your reply.

Could you please elaborate the code you posted? Thanks.
# 6  
Old 09-04-2019
Hi All. Thanks for your valuable suggestions.

With the help of my colleagues, I'm able to do what I was expecting to do. Here is the code which works for me.

Code:
for userid in `awk 'NR==1, NR==25' user-ids.txt`;
do
aws workspaces command to create workspaces  UserName=$userid,BundleId=<bundleID>
done
for userid in `awk 'NR==26, NR==50' user-ids.txt`;
do
aws workspaces command to create workspaces  UserName=$userid,BundleId=<bundleID>
done

# 7  
Old 09-05-2019
Thanks for showing it.
But...
in both loops exactly the same command? That does not make sense.
What is <bundleID>?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read file input in while loop does not work on AIX system

I'm working on Aix 6.1 and using ksh shell. The below works fine on Linux bash or ksh shell . while IFS= read -r dirpath ; do echo "Hi" done <<<"$var" However, any such while loop that reads the input from file or variable using <<< fails on Aix system with the below error: Below... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. Shell Programming and Scripting

Read file and add it into part of file

Hello let me explain senario. there is a file which name is config and it store main software variable: file main.conf contents: update="1" log_login="0" allow_ports="" deny_ports="21,22,23" and there is a file which name is ports.txt file ports.txt contents: 25,26,27 i... (3 Replies)
Discussion started by: nimafire
3 Replies

3. Shell Programming and Scripting

Use while loop to read file and use ${file} for both filename input into awk and as string to print

I have files named with different prefixes. From each I want to extract the first line containing a specific string, and then print that line along with the prefix. I've tried to do this with a while loop, but instead of printing the prefix I print the first line of the file twice. Files:... (3 Replies)
Discussion started by: pathunkathunk
3 Replies

4. Shell Programming and Scripting

make loop from input file

Hi Guys, I have file A.txt PP97 PP66 PP87 PP66 PP47 PP57 PP44 PP20 PP66 PP16 PP13 PP51 PP68 PP70 PP75 PP30 (2 Replies)
Discussion started by: asavaliya
2 Replies

5. Shell Programming and Scripting

[Solved] Printing a part of the last line of the specific part of a file

Hi, I have 80 large files, from which I want to get a specific value to run a Bash script. Firstly, I want to get the part of a file which contains this: Name =A xxxxxx yyyyyy zzzzzz aaaaaa bbbbbb Value = 57 This is necessary because in a file there are written more lines which... (6 Replies)
Discussion started by: wenclu
6 Replies

6. Shell Programming and Scripting

To Read a File and Insert a part of the lines into the database

Hi Guys I need to have a shell script which reads a log file and insert a part of each line into the database. Some sample lines in the file are as below. 20091112091359 MED_RQACK : user_data=60173054304,100232120,20091112091359,;ask_status=0;ask_reason=OK;msg_id=20091112091319... (5 Replies)
Discussion started by: Somanadh
5 Replies

7. Shell Programming and Scripting

How to select or make reference to, part of a field

For a field format such as AAL1001_MD82, how do I select(and use in if statement) only the last four elements( in this case MD82) or the first three elements (in this case AAL)? For instance, how do I do the following - if first three elements of $x == yyy, then ... (5 Replies)
Discussion started by: akshaykr2
5 Replies

8. UNIX for Dummies Questions & Answers

Replacing part of a text file with user input.

Ok, I am brand new to UNIX and I am trying to learn a cross between basic script and database use. I had got some ideas off the net on simple ideas for learning UNIX. I am working on creating a simple phone book program that allows myself to enter our employees from work into a phone book text... (0 Replies)
Discussion started by: georgefurbee
0 Replies

9. Shell Programming and Scripting

comparing part of header with part of detailed records.

Hi there, I am lil confused with the following issue. I have a File, which has the following header: IMSHRATE_043008_101016 a sample detailed record is :9820101 A982005000CAVG030108000000000000010169000MAR 2008 9820102 MAR 2008 D030108 ... (1 Reply)
Discussion started by: cmaroju
1 Replies

10. UNIX for Dummies Questions & Answers

read in part of the file into another file

Hi, I justs started learning Unix on my own. I have a question: What command can I use when I need to read in part of the file into another file? I remember I saw it somewhere but I don't know what it is. Thanks (3 Replies)
Discussion started by: whatisthis
3 Replies
Login or Register to Ask a Question