Sponsored Content
Top Forums Programming Open Source Splitting files using awk and reading filename value from input data Post 302976086 by RudiC on Thursday 23rd of June 2016 03:38:17 PM
Old 06-23-2016
How about posting a few lines from the input file? Some with, and some without the string of interest.

Not sure if I understood the requirement correctly, but, taking my syslog for a sample, wouldn't this fulfill your needs (redirect printout to a "user" file in /tmp/log if need be):
Code:
awk '{split ($5, U, /[[]/); T[U[1]] = $1 " " $2 " " $3} END {for (t in T) print t, T[t]}' /var/log/syslog
systemd-tmpfiles Jun 23 08:47:54
anacron Jun 23 08:46:57
dhclient: Jun 23 18:28:50
dbus Jun 23 12:03:34
CRON Jun 23 21:17:01
kernel: Jun 23 21:18:13
mtp-probe: Jun 23 11:40:12

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Moving files by splitting the path embedded in the filename

Hello All. I am having a directory /tmp/rahul which contains many files in the format @#home@#rahul@#programs@#script.pl where /home/rahul/programs is the directory where the script.pl file is to be placed. I have many files in this format. What i want is a script which read these... (7 Replies)
Discussion started by: rahulrathod
7 Replies

2. Shell Programming and Scripting

Reading in data sets into arrays from an input file.

Hye all, I would like some help with reading in a file in which the data is seperated by commas. for instance: input.dat: 1,2,34,/test for the above case, the fn. will store the values into an array -> data as follows: data = 1 data = 2 data = 34 data = /test I am trying to write... (5 Replies)
Discussion started by: sidamin810
5 Replies

3. Shell Programming and Scripting

awk reading 2 input files but not getting expected value

I'm reading 2 input files but not getting expected value. I should get an alpha value on file_1_data but not getting any. Please help. >cat test6.sh awk ' FILENAME==ARGV { file_1_data=$0; print "----- 1 Line " NR " -----" $1; next } FILENAME==ARGV { file_2_data=$0; print "----- 2... (1 Reply)
Discussion started by: pdtak
1 Replies

4. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies

5. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

6. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

7. Shell Programming and Scripting

Help with reading two input files in awk

Hello, I'm trying to write an awk program that reads two files inputs. example, file 1: 0.00017835 0.000176738 0.00018811 0.000189504 0.000188155 0.000180065 0.000178991 0.000178252 0.000182513 file 2: 1.7871769E-05 1.5139576E-16 1.5140196E-16 1.5139874E-16 1.7827407E-04 ... (5 Replies)
Discussion started by: joseamck
5 Replies

8. Shell Programming and Scripting

Splitting input CSV file into 3 files

Hi , I am receiving a CSV file that can vary in number of rows each time. I am supposed to split this file into 3 separate files like this: 1. create a file named 'File1.csv' that will contain first 3 rows of the input file 2. create file named 'File2.csv' that will contain last 3 rows of the... (7 Replies)
Discussion started by: kedrick
7 Replies

9. Shell Programming and Scripting

Splitting the Data using awk

Hello All, I have a comma delimiter file with 10 columns. I took the desired data but from $4 I need to split into two columns as 3+7 bytes. awk -F"," -v OFS=',' '{print $2,$3,$4}' foo.txt 42366,11/10/2014,5012418769 42366,11/10/2014,2046955672 42366,11/10/2014,2076802951 ... (3 Replies)
Discussion started by: karumudi7
3 Replies

10. Shell Programming and Scripting

How to embed data instead of reading user input from an array?

Hello, I am running under ubuntu1 14.04 and I have a script which is sending given process names to vanish so that I'd see less output when I run most popular tools like top etc in terminal window. In usual method it works. Whenever I restart the system, I have to enter the same data from... (2 Replies)
Discussion started by: baris35
2 Replies
CGCONFIG.CONF(5)						File Formats Manual						  CGCONFIG.CONF(5)

NAME
cgconfig.conf - libcgroup configuration file DESCRIPTION
cgconfig.conf is a configuration file used by libcgroup to define control groups, their parameters and their mount points. The file con- sists of mount , group and default sections. These sections can be in arbitrary order and all of them are optional. Any line starting with '#' is considered a comment line and is ignored. mount section has this form: mount { <controller> = <path>; ... } controller Name of the kernel subsystem. The list of subsystems supported by the kernel can be found in /proc/cgroups file. Named hierarchy can be specified as controller "name=<somename>". Do not forget to use double quotes around this controller name (see examples below). Libcgroup merges all subsystems mounted to the same directory (see Example 1) and the directory is mounted only once. path The directory path where the group hierarchy associated to a given controller shall be mounted. The directory is created automati- cally on cgconfig service startup if it does not exist and is deleted on service shutdown. If no mount section is specified, no controllers are mounted. group section has this form: group <name> { [permissions] <controller> { <param name> = <param value>; ... } ... } name Name of the control group. It can contain only characters, which are allowed for directory names. The groups form a tree, i.e. a control group can contain zero or more subgroups. Subgroups can be specified using '/' delimiter. The root control group is always created automatically in all hierarchies and it is the base of the group hierarchy. It can be explicitly specified in cgconfig.conf by using '.' as group name. This can be used e.g. to set its permissions, as shown in Example 6. When the parent control group of a subgroup is not specified it is created automatically. permissions Permissions of the given control group on mounted filesystem. root has always permission to do anything with the control group. Permissions have the following syntax: perm { task { uid = <task user>; gid = <task group>; fperm = <file permissions> } admin { uid = <admin name>; gid = <admin group>; dperm = <directory permissions> fperm = <file permissions> } } task user/group Name of the user and the group, which own the tasks file of the control group. Given fperm then specify the file permissions. Please note that the given value is not used as was specified. Instead, current file owner permis- sions are used as a "umask" for group and others permisions. For example if fperm = 777 then both group and others will get the same permissions as the file owner. admin user/group Name of the user and the group which own the rest of control group's files. Given fperm and dperm control file and directory permissions. Again, the given value is masked by the file/directory owner permissions. Permissions are only apply to the enclosing control group and are not inherited by subgroups. If there is no perm section in the control group definition, root:root is the owner of all files and default file permissions are preserved if fperm resp. dperm are not specified. controller Name of the kernel subsystem. The section can be empty, default kernel parameters will be used in this case. By specifying con- troller the control group and all its parents are controlled by the specific subsystem. One control group can be controlled by mul- tiple subsystems, even if the subsystems are mounted on different directories. Each control group must be controlled by at least one subsystem, so that libcgroup knows in which hierarchies the control group should be created. The parameters of the given controller can be modified in the following section enclosed in brackets. param name Name of the file to set. Each controller can have zero or more parameters. param value Value which should be written to the file when the control group is created. If it is enclosed in double quotes `"', it can contain spaces and other special characters. If no group section is specified, no groups are created. default section has this form: default { perm { task { uid = <task user>; gid = <task group>; fperm = <file permissions> } admin { uid = <admin name>; gid = <admin group>; dperm = <directory permissions> fperm = <file permissions> } } } Content of the perm section has the same form as in group section. The permissions defined here specify owner and permissions of groups and files of all groups, which do not have explicitly specified their permissions in their group section. EXAMPLES
Example 1 The configuration file: mount { cpu = /mnt/cgroups/cpu; cpuacct = /mnt/cgroups/cpu; } creates the hierarchy controlled by two subsystems with no groups inside. It corresponds to the following operations: mkdir /mnt/cgroups/cpu mount -t cgroup -o cpu,cpuacct cpu /mnt/cgroups/cpu Example 2 The configuration file: mount { cpu = /mnt/cgroups/cpu; "name=scheduler" = /mnt/cgroups/cpu; "name=noctrl" = /mnt/cgroups/noctrl; } group daemons { cpu { cpu.shares = "1000"; } } group test { "name=noctrl" { } } creates two hierarchies. One hierarchy named scheduler controlled by cpu subsystem, with group daemons inside. Second hierarchy is named noctrl without any controller, with group test. It corresponds to following operations: mkdir /mnt/cgroups/cpu mount -t cgroup -o cpu,name=scheduler cpu /mnt/cgroups/cpu mount -t cgroup -o none,name=noctrl none /mnt/cgroups/noctrl mkdir /mnt/cgroups/cpu/daemons echo 1000 > /mnt/cgroups/cpu/daemons/www/cpu.shares mkdir /mnt/cgroups/noctrl/tests The daemons group is created automatically when its first subgroup is created. All its parameters have the default value and only root can access group's files. Since both cpuacct and cpu subsystems are mounted to the same directory, all groups are implicitly controlled also by cpuacct subsystem, even if there is no cpuacct section in any of the groups. Example 3 The configuration file: mount { cpu = /mnt/cgroups/cpu; cpuacct = /mnt/cgroups/cpu; } group daemons/www { perm { task { uid = root; gid = webmaster; fperm = 770; } admin { uid = root; gid = root; dperm = 775; fperm = 744; } } cpu { cpu.shares = "1000"; } } group daemons/ftp { perm { task { uid = root; gid = ftpmaster; fperm = 774; } admin { uid = root; gid = root; dperm = 755; fperm = 700; } } cpu { cpu.shares = "500"; } } creates the hierarchy controlled by two subsystems with one group and two subgroups inside, setting one parameter. It corresponds to the following operations (except for file permissions which are little bit trickier to emulate via chmod): mkdir /mnt/cgroups/cpu mount -t cgroup -o cpu,cpuacct cpu /mnt/cgroups/cpu mkdir /mnt/cgroups/cpu/daemons mkdir /mnt/cgroups/cpu/daemons/www chown root:root /mnt/cgroups/cpu/daemons/www/* chown root:webmaster /mnt/cgroups/cpu/daemons/www/tasks echo 1000 > /mnt/cgroups/cpu/daemons/www/cpu.shares # + chmod the files so the result looks like: # ls -la /mnt/cgroups/cpu/daemons/www/ # admin.dperm = 755: # drwxr-xr-x. 2 root webmaster 0 Jun 16 11:51 . # # admin.fperm = 744: # --w-------. 1 root webmaster 0 Jun 16 11:51 cgroup.event_control # -r--r--r--. 1 root webmaster 0 Jun 16 11:51 cgroup.procs # -r--r--r--. 1 root webmaster 0 Jun 16 11:51 cpuacct.stat # -rw-r--r--. 1 root webmaster 0 Jun 16 11:51 cpuacct.usage # -r--r--r--. 1 root webmaster 0 Jun 16 11:51 cpuacct.usage_percpu # -rw-r--r--. 1 root webmaster 0 Jun 16 11:51 cpu.rt_period_us # -rw-r--r--. 1 root webmaster 0 Jun 16 11:51 cpu.rt_runtime_us # -rw-r--r--. 1 root webmaster 0 Jun 16 11:51 cpu.shares # -rw-r--r--. 1 root webmaster 0 Jun 16 11:51 notify_on_release # # tasks.fperm = 770 # -rw-rw----. 1 root webmaster 0 Jun 16 11:51 tasks mkdir /mnt/cgroups/cpu/daemons/ftp chown root:root /mnt/cgroups/cpu/daemons/ftp/* chown root:ftpmaster /mnt/cgroups/cpu/daemons/ftp/tasks echo 500 > /mnt/cgroups/cpu/daemons/ftp/cpu.shares # + chmod the files so the result looks like: # ls -la /mnt/cgroups/cpu/daemons/ftp/ # admin.dperm = 755: # drwxr-xr-x. 2 root ftpmaster 0 Jun 16 11:51 . # # admin.fperm = 700: # --w-------. 1 root ftpmaster 0 Jun 16 11:51 cgroup.event_control # -r--------. 1 root ftpmaster 0 Jun 16 11:51 cgroup.procs # -r--------. 1 root ftpmaster 0 Jun 16 11:51 cpuacct.stat # -rw-------. 1 root ftpmaster 0 Jun 16 11:51 cpuacct.usage # -r--------. 1 root ftpmaster 0 Jun 16 11:51 cpuacct.usage_percpu # -rw-------. 1 root ftpmaster 0 Jun 16 11:51 cpu.rt_period_us # -rw-------. 1 root ftpmaster 0 Jun 16 11:51 cpu.rt_runtime_us # -rw-------. 1 root ftpmaster 0 Jun 16 11:51 cpu.shares # -rw-------. 1 root ftpmaster 0 Jun 16 11:51 notify_on_release # # tasks.fperm = 774: # -rw-rw-r--. 1 root ftpmaster 0 Jun 16 11:51 tasks The daemons group is created automatically when its first subgroup is created. All its parameters have the default value and only root can access the group's files. Since both cpuacct and cpu subsystems are mounted to the same directory, all groups are implicitly also controlled by the cpuacct subsys- tem, even if there is no cpuacct section in any of the groups. Example 4 The configuration file: mount { cpu = /mnt/cgroups/cpu; cpuacct = /mnt/cgroups/cpuacct; } group daemons { cpuacct{ } cpu { } } creates two hierarchies and one common group in both of them. It corresponds to the following operations: mkdir /mnt/cgroups/cpu mkdir /mnt/cgroups/cpuacct mount -t cgroup -o cpu cpu /mnt/cgroups/cpu mount -t cgroup -o cpuacct cpuacct /mnt/cgroups/cpuacct mkdir /mnt/cgroups/cpu/daemons mkdir /mnt/cgroups/cpuacct/daemons In fact there are two groups created. One in the cpuacct hierarchy, the second in the cpu hierarchy. These two groups have nothing in com- mon and can contain different subgroups and different tasks. Example 5 The configuration file: mount { cpu = /mnt/cgroups/cpu; cpuacct = /mnt/cgroups/cpuacct; } group daemons { cpuacct{ } } group daemons/www { cpu { cpu.shares = "1000"; } } group daemons/ftp { cpu { cpu.shares = "500"; } } creates two hierarchies with few groups inside. One of the groups is created in both hierarchies. It corresponds to the following operations: mkdir /mnt/cgroups/cpu mkdir /mnt/cgroups/cpuacct mount -t cgroup -o cpu cpu /mnt/cgroups/cpu mount -t cgroup -o cpuacct cpuacct /mnt/cgroups/cpuacct mkdir /mnt/cgroups/cpuacct/daemons mkdir /mnt/cgroups/cpu/daemons mkdir /mnt/cgroups/cpu/daemons/www echo 1000 > /mnt/cgroups/cpu/daemons/www/cpu.shares mkdir /mnt/cgroups/cpu/daemons/ftp echo 500 > /mnt/cgroups/cpu/daemons/ftp/cpu.shares Group daemons is created in both hierarchies. In the cpuacct hierarchy the group is explicitly mentioned in the configuration file. In the cpu hierarchy the group is created implicitly when www is created there. These two groups have nothing in common, for example they do not share processes and subgroups. Groups www and ftp are created only in the cpu hierarchy and are not controlled by the cpuacct subsystem. Example 6 The configuration file: mount { cpu = /mnt/cgroups/cpu; cpuacct = /mnt/cgroups/cpu; } group . { perm { task { uid = root; gid = operator; } admin { uid = root; gid = operator; } } cpu { } } group daemons { perm { task { uid = root; gid = daemonmaster; } admin { uid = root; gid = operator; } } cpu { } } creates the hierarchy controlled by two subsystems with one group having some special permissions. It corresponds to the following opera- tions: mkdir /mnt/cgroups/cpu mount -t cgroup -o cpu,cpuacct cpu /mnt/cgroups/cpu chown root:operator /mnt/cgroups/cpu/* chown root:operator /mnt/cgroups/cpu/tasks mkdir /mnt/cgroups/cpu/daemons chown root:operator /mnt/cgroups/cpu/daemons/* chown root:daemonmaster /mnt/cgroups/cpu/daemons/tasks Users which are members of the operator group are allowed to administer the control groups, i.e. create new control groups and move pro- cesses between these groups without having root privileges. Members of the daemonmaster group can move processes to the daemons control group, but they can not move the process out of the group. Only the operator or root can do that. RECOMMENDATIONS
Keep hierarchies separated Having multiple hierarchies is perfectly valid and can be useful in various scenarios. To keeps things clean, do not create one group in multiple hierarchies. Examples 4 and 5 show how unreadable and confusing it can be, especially when reading somebody elses configuration file. Explicit is better than implicit libcgroup can implicitly create groups which are needed for the creation of configured subgroups. This may be useful and save some typing in simple scenarios. When it comes to multiple hierarchies, it's better to explicitly specify all groups and all controllers related to them. FILES
/etc/cgconfig.conf default libcgroup configuration file SEE ALSO
cgconfigparser (8) BUGS
Parameter values must be single strings without spaces. Parsing of quoted strings is not implemented. CGCONFIG.CONF(5)
All times are GMT -4. The time now is 02:09 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy