########### Tutorials ########### This section guides you through the fundamentals of file locking. We'll learn by doing, starting with the basics and building up to more advanced patterns. ***************** Your first lock ***************** Let's create our first lock and use it to coordinate between processes. First, we'll import what we need and create a lock object: .. code-block:: python from pathlib import Path from filelock import FileLock lock = FileLock("myapp.lock") Now we have a lock object that represents a lock file on disk. We can use the lock with a context manager (the ``with`` statement): .. code-block:: python with lock: # Inside this block, we hold the lock print("I have the lock!") # Outside this block, the lock is released Run this code multiple times in different terminal windows at the same time. You'll see that only one process prints the message at a time—the others wait for their turn. The lock is working correctly! ************************ Protecting shared data ************************ File locks are most useful when protecting data that multiple processes access. Let's see how: .. code-block:: python from pathlib import Path from filelock import FileLock data_file = Path("data.txt") lock = FileLock("data.txt.lock") # Process A writes a greeting with lock: if not data_file.exists(): data_file.write_text("Hello from Process A\n") # Process B appends another greeting with lock: with data_file.open("a") as f: f.write("Hello from Process B\n") The key pattern here: **Before making changes, check what's already done.** Process A checks if the file exists before writing. Process B doesn't need to check because it's just appending. But both use the lock to ensure only one process modifies the file at a time. Run this code from two different processes. The file will contain messages from both in a consistent order. ***************** Reentrant locks ***************** Sometimes you need to acquire the same lock multiple times from the same process or thread. The lock allows this: .. code-block:: python from filelock import FileLock lock = FileLock("reentrant.lock") def helper_function(): with lock: print("Helper has the lock") with lock: print("Main code has the lock") helper_function() # Can acquire the same lock again print("Still have the lock") No deadlock occurs—the lock counts how many times it's been acquired and releases only when the count reaches zero. You can inspect this counter and the lock state at any time: .. code-block:: python lock = FileLock("reentrant.lock") print(lock.is_locked) # False print(lock.lock_counter) # 0 lock.acquire() print(lock.is_locked) # True print(lock.lock_counter) # 1 lock.acquire() print(lock.lock_counter) # 2 lock.release() print(lock.lock_counter) # 1 — still locked print(lock.is_locked) # True lock.release() print(lock.lock_counter) # 0 — fully released print(lock.is_locked) # False Key lesson: You can safely call functions that acquire a lock, even if you already hold the lock. ***************************** Multiple ways to use a lock ***************************** So far we've used the ``with`` statement. There are other ways: **Manual acquire and release:** .. code-block:: python lock.acquire() try: print("I have the lock") finally: lock.release() Always use a ``try/finally`` block to guarantee the lock is released, even if an exception occurs. **As a decorator:** .. code-block:: python @lock def protected_operation(): print("This function runs with the lock held") protected_operation() # Lock is acquired, function runs, lock is released Choose whichever feels most natural for your code. The ``with`` statement is usually clearest. Important: Always use the context manager ========================================= Avoid this pattern: .. code-block:: python FileLock("my.lock").acquire() # ⚠️ Don't do this # The lock might be released during garbage collection # before your code finishes This doesn't work reliably because if you don't assign the lock to a variable, Python's garbage collector might release it before you're done with it. Instead, always keep a reference to the lock object: .. code-block:: python lock = FileLock("my.lock") with lock: # ✓ Good # your code here pass ************************** Thread-local by default ************************** By default, each lock uses thread-local state (``thread_local=True``). This means each thread tracks its own lock counter independently. Two threads holding the same ``FileLock`` object each get their own reentrant state. Async locks default to ``thread_local=False`` because they run in a thread pool where the acquiring thread may differ from the releasing thread. See :ref:`how-to:Use locks with multiple threads` for practical examples of controlling this behavior. *************************************** Migrating from lockfile.PIDLockFile *************************************** If you're migrating from the deprecated `lockfile `_ library, :class:`SoftFileLock ` is the direct replacement for ``PIDLockFile``. It writes the process ID to the lock file and can detect stale locks. .. code-block:: python # Before (lockfile): from lockfile.pidlockfile import PIDLockFile lock = PIDLockFile("/tmp/myapp.lock") lock.acquire() print(lock.read_pid()) print(lock.is_lock_held_by_us()) lock.release() # After (filelock): from filelock import SoftFileLock lock = SoftFileLock("/tmp/myapp.lock") with lock: print(lock.pid) print(lock.is_lock_held_by_us) Key differences from ``PIDLockFile``: - ``read_pid()`` is now a property: ``lock.pid`` - ``is_lock_held_by_us()`` is now a property: ``lock.is_lock_held_by_us`` - ``break_lock()`` is now ``lock.break_lock()`` (same name) - Stale lock detection happens automatically on acquire (all platforms) - Supports context managers, reentrant locking, timeouts, and all other filelock features ************************************* Reader/writer locks on a shared NFS ************************************* If your lock file lives on a network filesystem — a slurm-mounted home directory, a Lustre cluster scratch space, or any NFS share — use :class:`SoftReadWriteLock ` rather than :class:`ReadWriteLock `. ``ReadWriteLock`` is SQLite-backed and unsafe on NFS. ``SoftReadWriteLock`` is built on :class:`SoftFileLock ` primitives and handles cross-host stale detection via a background heartbeat thread. .. code-block:: python from filelock import SoftReadWriteLock rw = SoftReadWriteLock("/shared/nfs/work.lock") with rw.read_lock(): # Any number of processes on any host can be here at the same time. data = open("/shared/nfs/data.json").read() with rw.write_lock(): # Exactly one process anywhere can be here. New readers wait behind a pending writer. open("/shared/nfs/data.json", "w").write(new_data) While the lock is held, you will see a few sidecar files on disk next to ``work.lock``: .. code-block:: text work.lock.state # short-lived state mutex, exists only during transitions work.lock.write # writer marker, exists while a writer is claiming or holding work.lock.readers/ # directory with one file per active reader A daemon heartbeat thread refreshes each marker's ``mtime`` every ``heartbeat_interval`` seconds (default 30). If a compute node crashes while holding a lock, any other node will evict the stale marker after ``stale_threshold`` seconds of no refresh (default 90, following etcd's ``LeaseKeepAlive`` convention of ``TTL / 3``). Both values are constructor arguments, so HPC deployments that hold locks for hours can raise them: .. code-block:: python rw = SoftReadWriteLock( "/shared/nfs/work.lock", heartbeat_interval=120, stale_threshold=360, ) See :doc:`concepts` for the full explanation of the heartbeat + TTL model. ************ Next steps ************ - Want to handle timeouts, cancellation, or force-release? See :doc:`how-to`. - Curious about how locks work across different platforms? Read :doc:`concepts`.