It's been two days that i am googling to try to find an IMAP Proxy
program that would let me speed up this kind of search:



find me all mails that contains the word "cps" in their body.



in my 30k mails mailbox, it lasts over to 30 seconds, depending on server
charge.


Why mail bodies are not indexed on Zope side

I have decided a while ago to stop indexing mail bodies on zope side, as
this is not reliable for big intranets, big account, etc... It is not a
problem of speedness, as a primary indexing of a big mailbox was taking
ages, because zasync could be use there, but it's more a problem of making
the ZODB growing very, very fast and big.


A filesystem storage linked to the ZODB ?


This would bring a lot of hard work on setting up everything to work fine
together. (one more player, you need a chaire and more room around he
table)


What could be done on IMAP side: a Python Proxy

Back to my pure IMAP search: I find this too slow, and I don't really
understand why they are no options in IMAP servers to index the messages to
speed it up things.... I can't find anything related on such tools like
Courier IMAP.



I don't want to use ZOE either, I want a server side tool that can work
with all mail clients.



I tried to find some proxies that would do this kind of work, to be placed
between client requests and the real IMAP server, but I have just found some
IMAP Proxy that would make statefull connections for webmail apps that
reconnects everytime something is done. (Connection are statefull in
CPSMailAccess)



The code of such a tool is quite simple and short but has to be very
robust:



A SocketServer.TCPServer that would listen to IMAP ports (both SSL and not
SSL) can:





  • Catch all body search queries and search in a system file that contains
    index of message bodies instead of calling the real IMAP server

  • Index incoming message before passing them to the IMAP server.




This would speed up all full text searches and bring them as fast as
classical header searches, even if it's coded in Python.



Now, this sounds easy said this way, but this is a first thaught so I guess
I do not see all problems involved.

(Post originally written by Tarek Ziadé on the old Nuxeo blogs.)