Please purchase the course to watch this video.

Full Course
Lockfiles and pid files are essential tools used in programming to manage process concurrency and ensure data integrity. A lockfile allows only one instance of an application to run at a time, preventing data corruption during operations like database access or file manipulation. However, traditional lockfile methods can lead to issues when an application terminates unexpectedly, leaving orphaned lockfiles that require manual cleanup. To address this, pid files can be implemented, which contain the process ID of the running application, allowing for automated recovery and cleanup of orphaned files. By checking if the process associated with a pid file is active, developers can decide whether to remove the pid file, thus simplifying the management of running processes. The lesson outlines the steps for creating pid files, incorporating process management techniques, and presents a functional approach to implementing file locking mechanisms.
No links available for this lesson.
In the previous module, we implemented a basic lockfile mechanism to ensure only one instance of our application runs at a time. While functional, this approach has limitations that we'll address in this lesson by implementing PID files.
Review of Lockfiles
Lockfiles are a simple but effective tool to prevent multiple instances of an application from running simultaneously, which helps ensure data integrity. Common use cases include:
- Database access
- File manipulation
- Ensuring single-instance applications
Here's the basic approach we implemented before:
func main() {
// Try to create a lockfile with O_EXCL flag to ensure exclusivity
lockFile, err := os.OpenFile("lockfile", os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
log.Fatal("Failed to create lockfile, another process might be running")
}
defer os.Remove("lockfile") // Ensure cleanup
// Set up signal handling for clean shutdown
done := make(chan os.Signal, 1)
signal.Notify(done, os.Interrupt)
fmt.Println("Application is running...")
<-done // Wait for interrupt
fmt.Println("Cleaning up and shutting down...")
}
This code:
- Tries to create a lockfile with the
O_EXCL
flag, which fails if the file already exists - If successful, sets up a deferred cleanup to remove the file when the application exits
- Listens for interrupt signals to perform a clean shutdown
The Problem with Simple Lockfiles
The issue with this approach becomes apparent when we terminate the application without proper cleanup. Let's demonstrate:
# Run the app normally
$ go run main.go
Application is running...
# In another terminal, terminate with SIGTERM
$ pkill -15 pidfile
When terminated with SIGTERM (instead of the SIGINT we're handling), the cleanup code doesn't run, leaving the lockfile in place. This "orphaned" lockfile prevents the application from running again, requiring manual cleanup.
PID Files: A Better Solution
A PID file is an enhanced lockfile that contains the Process ID of the application that created it. This additional information allows us to:
- Verify if the process that created the file is still running
- Automatically recover from orphaned files
Let's implement this approach:
func main() {
fileName := "pidfile"
// Try to create a PID file
file, err := os.OpenFile(fileName, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
// Try to recover from a possibly orphaned PID file
recoverPIDFile(fileName)
log.Fatal("Failed to create lockfile, another process might be running")
}
defer os.Remove(fileName) // Ensure cleanup
// Write our PID to the file
pid := os.Getpid()
fmt.Fprintln(file, pid)
file.Close()
// Set up signal handling for clean shutdown
done := make(chan os.Signal, 1)
signal.Notify(done, os.Interrupt)
fmt.Println("Application is running...")
<-done // Wait for interrupt
fmt.Println("Cleaning up and shutting down...")
}
Implementing PID File Recovery
Now, let's implement the recoverPIDFile
function that checks if the process in the PID file is still running:
func recoverPIDFile(fileName string) {
// Read the PID file
data, err := os.ReadFile(fileName)
if err != nil {
log.Fatal("Failed to read file:", err)
}
// Parse the PID, removing whitespace (like newlines)
pidStr := strings.TrimSpace(string(data))
pid, err := strconv.Atoi(pidStr)
if err != nil {
log.Fatal("Failed to parse PID:", err)
}
// Check if the process exists
exists, err := processExists(pid)
if err != nil {
log.Fatal("Failed to check if process exists:", err)
}
// If the process doesn't exist, remove the PID file
if !exists {
err = os.Remove(fileName)
if err != nil {
log.Fatal("Failed to remove file:", err)
}
fmt.Println("PID file cleaned up")
}
}
Checking If a Process Exists
On Unix systems, we can check if a process exists by trying to send a signal 0 to it:
func processExists(pid int) (bool, error) {
// Find the process by PID
process, err := os.FindProcess(pid)
if err != nil {
return false, fmt.Errorf("find process: %w", err)
}
// On Unix, os.FindProcess always succeeds, so we need to check
// if the process actually exists by sending signal 0
err = process.Signal(syscall.Signal(0))
exists := err == nil
return exists, nil
}
Implementing a FileLock Struct
As an exercise, we can abstract this functionality into a reusable FileLock
struct:
type FileLock struct {
fileName string
}
// TryLock attempts to acquire the file lock
// It will try to recover orphaned PID files
func (f *FileLock) TryLock() error {
// Implementation left as an exercise...
return nil
}
// Unlock releases the file lock
func (f *FileLock) Unlock() error {
// Implementation left as an exercise...
return nil
}
Exercise: Complete the FileLock Implementation
To complete the FileLock
struct:
-
Implement
TryLock
:- Try to create the PID file
- If it fails, check if it's an orphaned PID file
- If orphaned, remove it and try again
- Otherwise, return an error
- If successful, write the current PID to the file
-
Implement
Unlock
:- Check if the PID file exists
- Verify the PID in the file matches the current process
- Remove the file if it's safe to do so
Why PID Files Matter
PID files solve several problems:
- Self-Healing: Applications can recover from crashes without manual intervention
- Process Management: Other applications can find and interact with your process
- Monitoring: System tools can check if your application is running
- Cleanup: Systems can detect and clean up orphaned resources
Summary
In this lesson, we've learned:
- The limitations of simple lockfiles
- How PID files enhance lockfiles by storing process information
- How to implement recovery for orphaned PID files
- How to check if a process exists in Unix systems
- How to structure file locking into a reusable abstraction
In the next lesson, we'll explore another approach to file locking that works directly with files we're reading or writing, rather than creating separate lock or PID files.