Sponsored Content
Full Discussion: Merge files in a directory
Top Forums Shell Programming and Scripting Merge files in a directory Post 302980183 by bluemind2005 on Thursday 25th of August 2016 03:14:18 AM
Old 08-25-2016
Merge files in a directory

Hey Guys,

I want to merge all files (Apache Tomcat Access Logs) for a particular date say "Aug 24" to be merged into a single file.

Is there any quick hack for that ?
Code:
[tomcat@localhost logs]$ ls -alrth access_log2016-08-*|grep "Aug 24"
-rw-rw-r--. 1 tomcat        tomcat          16M Aug 24 00:00 access_log2016-08-23.23.log
-rw-rw-r--. 1 tomcat        tomcat         6.8M Aug 24 01:00 access_log2016-08-24.00.log
-rw-rw-r--. 1 tomcat        tomcat         4.7M Aug 24 02:00 access_log2016-08-24.01.log
-rw-rw-r--. 1 tomcat        tomcat          14M Aug 24 03:00 access_log2016-08-24.02.log
-rw-rw-r--. 1 tomcat        tomcat          18M Aug 24 04:00 access_log2016-08-24.03.log
-rw-rw-r--. 1 tomcat        tomcat          15M Aug 24 05:00 access_log2016-08-24.04.log
-rw-rw-r--. 1 tomcat        tomcat         5.6M Aug 24 06:00 access_log2016-08-24.05.log
-rw-rw-r--. 1 tomcat        tomcat         8.9M Aug 24 07:00 access_log2016-08-24.06.log
-rw-rw-r--. 1 tomcat        tomcat          19M Aug 24 08:00 access_log2016-08-24.07.log
-rw-rw-r--. 1 tomcat        tomcat          32M Aug 24 09:00 access_log2016-08-24.08.log
-rw-rw-r--. 1 tomcat        tomcat          45M Aug 24 10:00 access_log2016-08-24.09.log
-rw-rw-r--. 1 tomcat        tomcat          44M Aug 24 11:00 access_log2016-08-24.10.log
-rw-rw-r--. 1 tomcat        tomcat          49M Aug 24 12:00 access_log2016-08-24.11.log
-rw-rw-r--. 1 tomcat        tomcat          51M Aug 24 13:00 access_log2016-08-24.12.log
-rw-rw-r--. 1 tomcat        tomcat          53M Aug 24 14:00 access_log2016-08-24.13.log
-rw-rw-r--. 1 tomcat        tomcat          52M Aug 24 15:00 access_log2016-08-24.14.log
-rw-rw-r--. 1 tomcat        tomcat          84M Aug 24 16:00 access_log2016-08-24.15.log
-rw-rw-r--. 1 tomcat        tomcat          57M Aug 24 17:00 access_log2016-08-24.16.log
-rw-rw-r--. 1 tomcat        tomcat          48M Aug 24 18:00 access_log2016-08-24.17.log
-rw-rw-r--. 1 tomcat        tomcat          37M Aug 24 19:00 access_log2016-08-24.18.log
-rw-rw-r--. 1 tomcat        tomcat          38M Aug 24 20:00 access_log2016-08-24.19.log
-rw-rw-r--. 1 tomcat        tomcat          40M Aug 24 21:00 access_log2016-08-24.20.log
-rw-rw-r--. 1 tomcat        tomcat          37M Aug 24 22:00 access_log2016-08-24.21.log
-rw-rw-r--. 1 tomcat        tomcat          26M Aug 24 23:00 access_log2016-08-24.22.log
[tomcat@localhost logs]$

They should be merged into single file as per date time in ascending order so access_log2016-08-23.23.log then access_log2016-08-24.00.log so on and so forth.

We can do manually like cat individual file but want to explore more smarter way. I heard from my colleague a command called plus to concatenate into single file ,but not 100% sure..

Please assist.

Last edited by RudiC; 08-25-2016 at 04:20 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge files

Hi, i have the files f1 and f2 like: files f1: c1 a1 c2 a2 c3 a3 file f2: c1 b1 c2 b2 c3 b3 i want merge the f1 and f2 file to f3 file like: c1 a1 b1 c2 a2 b3 c3 a3 b3........ .... . . please help me onthis..... (5 Replies)
Discussion started by: koti_rama
5 Replies

2. Shell Programming and Scripting

how to merge these two files?

I have two files, each of them has 12 lines, fileA has 3 columns, fileB has 1 column, like the following FileA a 1 b 2 c 3 ..blabla FileB A B C ..blabla Now I am trying to put the content of fileB as column 3 of fileA, e.g. a 1 A b 2 B c 3 C (3 Replies)
Discussion started by: fedora
3 Replies

3. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

4. Shell Programming and Scripting

Merge 2 files

Hello, i'd like a bash script to merge 2 files without duplicate lines. Example : file1 : toto titi file2 : toto tata Expected result, file3 : toto (5 Replies)
Discussion started by: Celmar
5 Replies

5. Shell Programming and Scripting

Merge files

Hello, I have a application software plink. It can merge files with some kinds of way. The command likes: plink --file 1 --merge 2 --recode --out merged That means merge file 1 and 2 then output file "merged". However I have 23 files (1,2,3,...22,23)to be merged together. How can I use... (2 Replies)
Discussion started by: zhshqzyc
2 Replies

6. UNIX for Dummies Questions & Answers

Merge files

Hi, I would like to know how can I merge files based on their coordinates, but mantaining the score of each file in the output file like: Note: 1st column is for chromosome, 2nd for start, 3rd for end of segment, 4th for score file1: 1 200 300 20 1 400 500 30 file2: 1 200 350 30 1... (1 Reply)
Discussion started by: fadista
1 Replies

7. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

8. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies

9. Shell Programming and Scripting

How to merge two files?

Dear Frens, I have two files and need to merge into one file. Like File_1: Field1 Field2 1 4 File_2: Field1 Field2 3 5 I need one single output as File_1: Field1 Field2 1 4 3 5 This means taking header from either file. (8 Replies)
Discussion started by: manisha_singh
8 Replies

10. Shell Programming and Scripting

Directory containing files,Print names of the files in the directory that are exactly same content.

Given a directory containing say a few thousand files, please output a list of all the names of the files in the directory that are exactly the same, i.e. have the same contents. func(a_directory_name) output -> {“matches”: , ... ]} e.g. func(“/home/my/files”) where the directory... (7 Replies)
Discussion started by: anuragpgtgerman
7 Replies
tomcat_selinux(8)					       SELinux Policy tomcat						 tomcat_selinux(8)

NAME
tomcat_selinux - Security Enhanced Linux Policy for the tomcat processes DESCRIPTION
Security-Enhanced Linux secures the tomcat processes via flexible mandatory access control. The tomcat processes execute with the tomcat_t SELinux type. You can check if you have these processes running by executing the ps command with the -Z qualifier. For example: ps -eZ | grep tomcat_t ENTRYPOINTS
The tomcat_t SELinux type can be entered via the unlabeled_t, proc_type, file_type, mtrr_device_t, filesystem_type, sysctl_type, tom- cat_exec_t file types. The default entrypoint paths for the tomcat_t domain are the following: all files on the system, /dev/cpu/mtrr, /usr/sbin/tomcat(6)? PROCESS TYPES
SELinux defines process types (domains) for each process running on the system You can see the context of a process using the -Z option to ps Policy governs the access confined processes have to files. SELinux tomcat policy is very flexible allowing users to setup their tomcat processes in as secure a method as possible. The following process types are defined for tomcat: tomcat_t Note: semanage permissive -a tomcat_t can be used to make the process type tomcat_t permissive. SELinux does not deny access to permissive process types, but the AVC (SELinux denials) messages are still generated. BOOLEANS
SELinux policy is customizable based on least access required. tomcat policy is extremely flexible and has several booleans that allow you to manipulate the policy and run tomcat with the tightest access possible. If you want to allow all daemons to write corefiles to /, you must turn on the daemons_dump_core boolean. Disabled by default. setsebool -P daemons_dump_core 1 If you want to enable cluster mode for daemons, you must turn on the daemons_enable_cluster_mode boolean. Enabled by default. setsebool -P daemons_enable_cluster_mode 1 If you want to allow all daemons to use tcp wrappers, you must turn on the daemons_use_tcp_wrapper boolean. Disabled by default. setsebool -P daemons_use_tcp_wrapper 1 If you want to allow all daemons the ability to read/write terminals, you must turn on the daemons_use_tty boolean. Disabled by default. setsebool -P daemons_use_tty 1 If you want to deny user domains applications to map a memory region as both executable and writable, this is dangerous and the executable should be reported in bugzilla, you must turn on the deny_execmem boolean. Enabled by default. setsebool -P deny_execmem 1 If you want to deny any process from ptracing or debugging any other processes, you must turn on the deny_ptrace boolean. Enabled by default. setsebool -P deny_ptrace 1 If you want to allow all domains to use other domains file descriptors, you must turn on the domain_fd_use boolean. Enabled by default. setsebool -P domain_fd_use 1 If you want to allow all domains to have the kernel load modules, you must turn on the domain_kernel_load_modules boolean. Disabled by default. setsebool -P domain_kernel_load_modules 1 If you want to allow all domains to execute in fips_mode, you must turn on the fips_mode boolean. Enabled by default. setsebool -P fips_mode 1 If you want to enable reading of urandom for all domains, you must turn on the global_ssp boolean. Disabled by default. setsebool -P global_ssp 1 If you want to control the ability to mmap a low area of the address space, as configured by /proc/sys/kernel/mmap_min_addr, you must turn on the mmap_low_allowed boolean. Disabled by default. setsebool -P mmap_low_allowed 1 If you want to allow confined applications to use nscd shared memory, you must turn on the nscd_use_shm boolean. Disabled by default. setsebool -P nscd_use_shm 1 If you want to disable kernel module loading, you must turn on the secure_mode_insmod boolean. Enabled by default. setsebool -P secure_mode_insmod 1 If you want to boolean to determine whether the system permits loading policy, setting enforcing mode, and changing boolean values. Set this to true and you have to reboot to set it back, you must turn on the secure_mode_policyload boolean. Enabled by default. setsebool -P secure_mode_policyload 1 If you want to allow unconfined executables to make their heap memory executable. Doing this is a really bad idea. Probably indicates a badly coded executable, but could indicate an attack. This executable should be reported in bugzilla, you must turn on the selin- uxuser_execheap boolean. Disabled by default. setsebool -P selinuxuser_execheap 1 If you want to allow all unconfined executables to use libraries requiring text relocation that are not labeled textrel_shlib_t, you must turn on the selinuxuser_execmod boolean. Enabled by default. setsebool -P selinuxuser_execmod 1 If you want to allow unconfined executables to make their stack executable. This should never, ever be necessary. Probably indicates a badly coded executable, but could indicate an attack. This executable should be reported in bugzilla, you must turn on the selin- uxuser_execstack boolean. Enabled by default. setsebool -P selinuxuser_execstack 1 If you want to support X userspace object manager, you must turn on the xserver_object_manager boolean. Enabled by default. setsebool -P xserver_object_manager 1 If you want to allow ZoneMinder to run su/sudo, you must turn on the zoneminder_run_sudo boolean. Disabled by default. setsebool -P zoneminder_run_sudo 1 MANAGED FILES
The SELinux process type tomcat_t can manage files labeled with the following file types. The paths listed are the default paths for these file types. Note the processes UID still need to have DAC permissions. file_type all files on the system FILE CONTEXTS
SELinux requires files to have an extended attribute to define the file type. You can see the context of a file using the -Z option to ls Policy governs the access confined processes have to these files. SELinux tomcat policy is very flexible allowing users to setup their tomcat processes in as secure a method as possible. STANDARD FILE CONTEXT SELinux defines the file context types for the tomcat, if you wanted to store files with these types in a diffent paths, you need to exe- cute the semanage command to sepecify alternate labeling and then use restorecon to put the labels on disk. semanage fcontext -a -t tomcat_cache_t '/srv/tomcat/content(/.*)?' restorecon -R -v /srv/mytomcat_content Note: SELinux often uses regular expressions to specify labels that match multiple files. The following file types are defined for tomcat: tomcat_cache_t - Set files with the tomcat_cache_t type, if you want to store the files under the /var/cache directory. tomcat_exec_t - Set files with the tomcat_exec_t type, if you want to transition an executable to the tomcat_t domain. tomcat_log_t - Set files with the tomcat_log_t type, if you want to treat the data as tomcat log data, usually stored under the /var/log directory. tomcat_tmp_t - Set files with the tomcat_tmp_t type, if you want to store tomcat temporary files in the /tmp directories. tomcat_unit_file_t - Set files with the tomcat_unit_file_t type, if you want to treat the files as tomcat unit content. tomcat_var_lib_t - Set files with the tomcat_var_lib_t type, if you want to store the tomcat files under the /var/lib directory. tomcat_var_run_t - Set files with the tomcat_var_run_t type, if you want to store the tomcat files under the /run or /var/run directory. Note: File context can be temporarily modified with the chcon command. If you want to permanently change the file context you need to use the semanage fcontext command. This will modify the SELinux labeling database. You will need to use restorecon to apply the labels. COMMANDS
semanage fcontext can also be used to manipulate default file context mappings. semanage permissive can also be used to manipulate whether or not a process type is permissive. semanage module can also be used to enable/disable/install/remove policy modules. semanage boolean can also be used to manipulate the booleans system-config-selinux is a GUI tool available to customize SELinux policy settings. AUTHOR
This manual page was auto-generated using sepolicy manpage . SEE ALSO
selinux(8), tomcat(8), semanage(8), restorecon(8), chcon(1), sepolicy(8) , setsebool(8) tomcat 14-06-10 tomcat_selinux(8)
All times are GMT -4. The time now is 03:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy