Linux File System Monitoring & Actions, (Tue, Dec 20th)

Tux, the Linux mascot
Tux, the Linux mascot Date 16 October 2003 Source New Tux, created using Sodipodi. Based on original image by Larry Ewing, made in GIMP. Author Larry Ewing, gg3po

There can be multiple reasons to keep an eye on a critical/suspicious file or directory. For example, you could track an attacker and wait for some access to the captured credentials in a phishing kit installed on a compromised server. You could deploy an EDR solution or an OSSEC agent that implements an FIM (‘File Integrity Monitoring”)[1]. Upon a file change, an action can be triggered. Nice, but what if you would like a quick solution but agentless? (In the scope of an incident, for example)

There is a well-known suite of API calls on Linux that track filesystem changes: inotify[2]. Around the API, a set of tools are available, like “inotifywatch” that generates an event when a file is “accessed”:

[email protected]:~$ inotifywatch /etc/hosts
Establishing watches...
Finished establishing watches, now collecting statistics.
^Ctotal  access  close_nowrite  open  filename
5      1       2              2     /etc/hosts

Another helpful command in scripts is “inotifywait”:

[email protected]:~$ inotifywait /etc/hosts
Setting up watches.
Watches established.
/etc/hosts OPEN

This one waits for some activity, and when it happens, it exists. In a shell script, it helps to wait for an event and then continue the script. That’s interesting, but what if you can’t keep a shell running? What if the shell script exists? How to handle multiple events?

I recently discovered an interesting tool to implement better file system monitoring: incron[3]. The idea is to have an “inotify cron” system. It consists of a daemon and a table manipulator. Like the regular cron, you can schedule scripts that will be executed upon a filesystem change. Here is a quick example:

[email protected]:~# incrontab -l
/var/www/wp/wp-plugins/compromized/phishing.log IN_CLOSE_WRITE /usr/local/bin/

You specify the file/directory to monitor, which access triggers the command. You can specify multiple access types. They are based on the ones defined in inotify.h:

#define IN_ACCESS 0x00000001 /* File was accessed */
#define IN_MODIFY 0x00000002 /* File was modified */
#define IN_ATTRIB 0x00000004 /* Metadata changed */
#define IN_CLOSE_WRITE 0x00000008 /* Writtable file was closed */
#define IN_CLOSE_NOWRITE 0x00000010 /* Unwrittable file closed */
#define IN_OPEN 0x00000020 /* File was opened */
#define IN_MOVED_FROM 0x00000040 /* File was moved from X */
#define IN_MOVED_TO 0x00000080 /* File was moved to Y */
#define IN_CREATE 0x00000100 /* Subfile was created */
#define IN_DELETE 0x00000200 /* Subfile was deleted */
#define IN_DELETE_SELF 0x00000400 /* Self was deleted */
#define IN_MOVE_SELF 0x00000800 /* Self was moved */

The following wildcards may be used inside the command specification:

$ Prints a dollar sign 
[email protected] Add the watched filesystem path 
$# Add the event-related file name 
$% Add the event flags (textually) 

amp; Add the event flags (numerically)
This is very efficient because you don’t have to take care of loops or keep scripts running for a long time. Just be careful that your command(s) can be triggered many times if a lot of activity is detected on the monitored files/dirs!

Happy hunting!


Xavier Mertens (@xme)
Senior ISC Handler – Freelance Cyber Security Consultant

(c) SANS Internet Storm Center. Creative Commons Attribution-Noncommercial 3.0 United States License.

Reposted from SANS. View original.

Alex Post