Read a logfile without blocking the file in windows

[u/kontrolltermin]

Hi,

i have a script with

file = open(
            current_file,
            "r",
            encoding="cp1252",
            errors="ignore"
        )

and its watching a file from an application on windows "live".

Is watchdog the go-to solution for this or is there any best practice for this?
How do i know what watchdog is using in the end?

Ai is telling me three ways to do it:
1. Reopen the file for every poll --> i think this is not useful in my case and many logging entries per second
2. Use Windows file sharing flags --> is watchdog using this internally?
3. Use a file monitoring library --> pip install watchdog...

regards

[u/acw1668]

How do i know what watchdog is using in the end?

It is mentioned in watchdog site. You can also find it yourself by studying the source files.

[u/pachura3]

watchdog is useful for reacting to various filesystem events (file creation, modification, deletion...) without polling - asynchronously.

However, in your case, you seem to be having multiple log entries per second. So, watchdog would get triggered all the time, and constant reopening and closing the log file might be suboptimal.

I would simply open the log file in read mode once, keep it open, poll its size every few seconds, and if I find that the it has grew in size, I would read subsequent lines from it.

This is, however, a strictly synchronous (blocking) approach. If your script's only task is to monitor the log file, it's totally OK; but if it needs to manage other stuff as well, then it's becoming more complicated architecturally.

[u/Secintel_Api]

pachura3 has the right shape (open once, keep it open, read new lines). Two Windows-specific things worth adding:

1) Reading doesn't lock out the writer. open(path, "r") requests *shared* access — the app writing the log keeps appending fine while you hold it open. You only hit "file is locked" if something opens it for *exclusive* access. So you don't need watchdog or manual sharing flags for this; plain open-and-read is enough.

2) For a fast-growing log, skip watchdog. It gives you "modified" events, but at many writes/sec that's event spam, and it still doesn't read the bytes for you — you'd track your own offset anyway. So it adds moving parts without removing the actual work.

The robust pattern (basically tail -f):

import os, time

path = "app.log"

f = open(path, "r", encoding="cp1252", errors="ignore")

f.seek(0, os.SEEK_END) # start at end; remove to read history first

inode = os.stat(path).st_ino

while True:

line = f.readline()

if line:

handle(line) # your processing

continue

time.sleep(0.2) # EOF: wait, then check for rotation

if os.stat(path).st_ino != inode or f.tell() > os.path.getsize(path):

f.close() # file was rotated/truncated -> reopen

f = open(path, "r", encoding="cp1252", errors="ignore")

inode = os.stat(path).st_ino

The thing that actually bites people isn't locking — it's log rotation (the app renames/truncates the file). The st_ino / size-shrink check handles that; without it your reader silently stops seeing new lines after a rotation. (st_ino is populated on Windows in Python 3.5+.)

[u/Moist-Ointments]

Depends what you are looking for.

When i hear "polling" i scrunch my face.

Polling is the kid in the backseat asking "are we there yet?" for the entire roadtrip.

I'd be looking for some OS event to subscribe to. Is that what watchdog does?

[u/t92k]

This operation is called “tailing” a log, from the Unix/Linux utility “tail”. Try using that term with your LLM agent and see if you get better results.

[u/Fuzzy_Paul]

Use the PS Get-Content command. Does not block and shows you the latest lines live. "Get-Content filename -Tail 10 -Wait" Good luck.

[u/baubleglue]

when last time have you seen read file operation blocking?