I'm running into a problem for CPSMailAccess (the next CPS webmail)
It has some enhanced features that needs to keeps things on zodb side.
So a synchro process is called to update things on zodb side when the imap server changes.
When users synchronizes their mailbox, it can take a few seconds, sometimes more, depending on the size of the box.
The first synchro takes a long of time too, because a lot of things are done, like indexings.
That makes a real problem for large scales intranets when 3000 people come to the office and synchronizes their webmail.
The 4 little available zope threads cannot handle that.
Eric Barrocca pointed me out the existence of ZAsync product from Zope Corp.
extract from the readme :
ZAsync is a Zope 2 product that allows Twisted to asynchronously
perform tasks in a separate process over ZEO on behalf of Zope
requests. Intended use cases (only the first two are realized) include
performing large numbers of long-running external database queries
without tying up Zope; allowing "fire and forget" Zope transactions
that return control to the app server while scheduling long-running
Zope transactions to be performed in the background and potentially on
another physical machine; connecting over a long-running socket with
other asynchronous processes such as instant message servers; and
performing intelligent agent types of tasks.
This product seems to fulfill the need since it can launch process on the background, thus letting the 4 "frontal" threads available for "regular" user queries.
The only care would be to protect the synchronize methods into a critical sectionto avoid conflicts.
This product brings some dependencies though (twisted, ZEO, etc...) but it worth a shot for big scale
intranets, as I can probably set the ZEO node in charge of background process to a high number of threads.
part #2 will be the results of my tries, stay tuned :-)
(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)