I had been looking for a way to synchronize my python cgi scripts (several, or several instances of one) so that only one of them could access a particular table in a database at one time. I looked around on the internet but didn’t find a concrete class that could do this. I was writing my scripts for a linux based apache server, so portability didn’t matter. So here it is.
#!/usr/bin/python
#
import os
import fcntl
class lock:
def __init__(self):
self.lockf = 0
self.file = 0
def lock(self):
self.lockf = open ('/tmp/mytmpfilename.1001', 'a')
fcntl.flock (self.lockf, fcntl.LOCK_EX)
self.file = open ('/tmp/mytmpfilename.1001','w')
def unlock(self):
self.file.close ()
self.lockf.close ()
###
If you know of a better way of synchronizing processes, please let me know.
Edit.10/21/2008:
Here is more specifically what my scripts do. Perhaps, this post makes more sense in the correct context. The description below was taken from one of my comments.
All instances of my cgi script perform a read operation followed by a delete operation. Essentially, my script reads the top row from the database. Right after the information is retrieved that particular row is deleted. In other words, I am using my database as a FIFO buffer. Now if several instances of my scripts are running, more than one of them could access the first row (read it), thereby letting two different processes fetch the same element from a FIFO. One but all of them then fails with the deletion operation. The whole idea behind this locking mechanism was to avoid this. All instances of my script should always fetch a different item.
/
by Marius Gedminas
21 Oct 2008 at 10:16
Generally databases have ACID transactions for this sort of thing.
by Hongli Lai
21 Oct 2008 at 13:06
Have you considered not synchronizing them, and using optimistic concurrency control instead? That is, use transactions, and put correct constraints in your schema to ensure that it will stay consistent. Then run your transaction, and after the transaction, check whether the desired effect was achieved (which could be just checking whether no database constraint violation occurred), and if not, restart the transaction. Wikipedia has more information on optimistic concurrency control.
by admin
21 Oct 2008 at 16:22
Thanks a lot for your comments Hongli and Marius. I will look deeper into Optimisitic Concurrency Control and ACID transactions. At first glance, I don’t think I can employ any these techniques. This is basically because I need to do something very specific with the database table. Perhaps I didn’t explain it clearly in my post.
All instances of my cgi script perform a read operation followed by a delete operation. Essentially, my script reads the top row from the database. Right after the information is retrieved that particular row is deleted. In other words, I am using my database as a FIFO buffer. Now if several instances of my scripts are running, more than one of them could access the first row (read it), thereby letting two different processes fetch the same element from a FIFO. One of them then fails with the deletion operation. The whole idea behind this locking mechanism was to avoid this. All instances of my script should always fetch a different item.
I will explore these techniques further to see if they could help me better solve my problem.
by will
27 Nov 2008 at 03:05
There’s several ways to accomplish what you want using a DB and there are performance and scalability benefits to doing so (what happens if you want to have more than one box processing your fifo?)
The simplest, and most compatible method, is to use a separate table with one row in it as the lock, simply update that row as the first operation in your transaction and all other accesses will serialize behind it. Or you could just ‘lock table blah in exclusive mode’ dunno whether that’s supported in all DB servers though. You can get smarter too, but it’s only worth it if you were going to do a lot more work in the txn and wanted to maintain parallelism.