Monitor Partner Integration Directories for Changes
This guide shows you how to automatically detect when a partner adds, modifies, or removes files in a shared integration directory. You’ll set up a simple monitoring script that logs changes and sends alerts, so you can react quickly without manually checking the folder. All you need is a computer with Python installed and access to the directory you want to watch.
What You’ll Need
- A computer with Python 3.6 or later installed
- Read access to the partner integration directory (local or network share)
- Optional: an email account or webhook endpoint for alerts
- A text editor to create the script
Step-by-Step
- Identify the directory to monitor. Confirm the exact path, whether it’s a local folder like
C:\partners\acmeor a network share like\\server\share\partner. Ensure your user account has at least read permissions. - Install the watchdog library. Open a terminal or command prompt and run:
pip install watchdog. This library provides reliable file system monitoring without needing platform-specific code. - Create a new Python script. Name it something like
monitor.py. Open it in your text editor. - Write the monitoring script. Use the following template, replacing
YOUR_DIRECTORYwith your actual path and adjusting the alert method if needed:import time from watchdog.observers import Observer from watchdog.events import FileSystemEventHandler import logging class ChangeHandler(FileSystemEventHandler): def on_any_event(self, event): # Log every change to a file logging.info(f"{event.event_type} - {event.src_path}") # Optional: add your alert code here (e.g., send email, call webhook) if __name__ == "__main__": path = "YOUR_DIRECTORY" logging.basicConfig(filename='changes.log', level=logging.INFO, format='%(asctime)s - %(message)s') event_handler = ChangeHandler() observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start() print(f"Monitoring {path}... Press Ctrl+C to stop.") try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join() - Test the script. Run it from the terminal:
python monitor.py. While it’s running, create a test file in the monitored directory and verify that a new line appears inchanges.log. - Set up alerts (optional). To receive notifications, add code inside the event handler to send an email or POST to a webhook. For example, using the
requestslibrary to call a Slack webhook:import requests requests.post('https://hooks.slack.com/services/YOUR/WEBHOOK', json={'text': f'Change detected: {event.event_type} {event.src_path}'}) - Run the script in the background. Instead of keeping the terminal open, run it as a background process. On Linux/macOS:
nohup python monitor.py &. On Windows, use Task Scheduler to start the script at login. - Set up startup persistence. To ensure monitoring starts automatically, create a scheduled task or service. On Windows, use Task Scheduler; on Linux, add a cron job or systemd service. Test that the script starts correctly after a reboot.
Common Mistakes
- Using an incorrect path. Double-check the directory path, especially network shares that may require different syntax (e.g., forward slashes vs. backslashes).
- Forgetting to enable recursive monitoring. The
recursive=Trueparameter is essential if you need to watch subdirectories. Without it, changes in subfolders will be missed. - Ignoring permission issues. The script must have read access to the directory and write access to the log file. Run it with appropriate privileges, but avoid using admin/root unless necessary.
- Not handling temporary files. Some applications create temporary files that are quickly renamed or deleted. This can cause false alerts. Consider filtering out events for files ending in
.tmpor.swp. - Overlooking network latency. On network shares, events may be delayed or missed if the connection drops. Test the monitoring over an extended period to ensure reliability.
Tips
- Use a dedicated log rotation. To prevent the log file from growing indefinitely, set up log rotation using Python’s
logging.handlers.RotatingFileHandler. - Add error handling. Wrap the alert code in a try-except block so that a failed notification doesn’t crash the entire monitor.
- Monitor only the necessary subdirectories. If the partner directory is large, monitoring the entire tree recursively can be resource-intensive. Narrow the scope if possible.
- Test with a sample file. Before relying on the monitor, create a test file and confirm the log entry and alert (if set up) work as expected.
- Document the purpose. Add comments in your script explaining what each part does, making it easier for others (or future you) to maintain.
Troubleshooting
- No events are logged. First, verify the directory path is correct and accessible. Test by running the script and manually creating a file. If still nothing, check that watchdog is installed correctly (
pip show watchdog). On some systems, you may need to run the script with elevated privileges. - Events are logged but no alert is received. Check the alert code for errors. Test the webhook or email endpoint separately. Ensure the
requestslibrary is installed (pip install requests). - Script stops unexpectedly. Look for error messages in the terminal or in a separate error log. Common causes include permission changes, network disconnects, or memory issues. Add a try-except around the main loop to log exceptions.
- Changes are missed during high activity. Watchdog may miss events if the system is overloaded. Increase the observer’s timeout or consider using a more robust solution like inotify on Linux. Also, ensure the directory is not being accessed via a slow network connection.
- Duplicate events for the same change. Some file systems generate multiple events for a single operation (e.g., create and modify). This is normal; you can deduplicate by tracking recent file paths and ignoring events within a short time window.
Frequently Asked Questions
Can I monitor a directory on a remote server?
Yes, if the directory is mounted as a network share (e.g., SMB, NFS) and your script has access to that mount point, watchdog can monitor it. However, network latency and disconnects can cause missed events, so test thoroughly.
Does this work on Windows, macOS, and Linux?
Yes, watchdog is cross-platform. It uses native APIs (ReadDirectoryChangesW on Windows, FSEvents on macOS, inotify on Linux) to provide reliable monitoring.
How do I stop the monitoring script?
If running in the foreground, press Ctrl+C. If running in the background, find the process ID with ps aux | grep monitor.py (Linux/macOS) or use Task Manager (Windows) and kill it.
Can I monitor multiple directories at once?
Yes, create multiple observer schedules, one for each directory. Each can have its own event handler or share a common one.
What if the directory is deleted or renamed?
Watchdog will stop monitoring if the root directory is removed. You’ll need to restart the script. Consider adding a check to verify the directory exists and restart the observer if it reappears.
Is there a way to monitor only certain file types?
Yes, in the event handler, filter by file extension. For example, ignore events where event.src_path does not end with .csv.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.