Memcached on Windows.

As you may know Memcached is a distributed caching system that is used by major website out there (facebook et.al) and there are many articles out in the wild that handle this matter on windows, I will try and put in my experience and my lessons learnt. As you are aware -  memcached is not exactly meant to run on windows and thus we have to put up with ports of the original software that my not exactly be the best to run on a production environment. I started out with the http://code.jellycan.com/memcached/ build. It was kind of old since the last time it was updated was in November 2009. It worked for a while but I could not run it with a degree of surety on production, I figured it may possibly have had some optimization done on it and some bugs lurking in there that might have been resolved. So I decided to do some bit of research in an attempt to get the latest and greatest release of memcached for windows. I stumbled upon Membase this is one of the best Memcached like services, it is basically in two parts

1.    The Membase Server is a NoSQL database. That is basically a distributed key-value data store; it is designed and optimized for the data management needs of interactive web applications, so it allows the data layer to scale out just like the web application logic tier – simply by adding more commodity servers.

2.    Memcached is part of Membase Server, transparently caching data in front of its key-value database. You can configure Membase Server to provide only Memcached services and for there for skip the use of the NoSQL database.

I decided to use the later only as what I needed was the caching framework only, and this worked for a while and worked really well coupled with the nice intuitive administration portal that comes with the install – it was exactly what I was looking for...until I attempted to throw in a large dataset for caching and it failed, We quickly diagnosed that the issue was that all memcached services have a max of 1024kb size for items that need to be stored in them – this was not acceptable as we had started implementing a caching collections instead of individual items policy for our work. Apparently there is an argument/flag passed at start-up that instructs the memcached service to start accepting items that are larger than 1mb .

memcached.exe -I

But this particular flag was not available(availed) in the memcached service on the Membase distribution. So I was regrettably back on the road, looking for an build that was new and that had capability of accepting the argument for our work.

After sometime I came across an article on how to build the memcached service on windows and to this I said a big Hallelujah!! I executed steps outlined below to do my own build.

1.    Installed mysysgit. This in addition to installing git, it also installed a compiler capable of building a 32 bit version of memcached on windows. Since apparently Microsoft does not supply a C99 compliant C compiler…go figure.

2.    Installed libevent 2.0.10 (latest at the time) on my machine by executing the following commands (from within your msysgit-shell).

$ cd /tmp

$ tar xfz libevent-2.0.10-stable.tar.gz

$ cd libevent-2.0.10-stable

$ ./configure --prefix=/usr/local

$ make all

$ make install  

3.    Built memcached by checking out and compiling the source code

$ git clone git://github.com/trondn/memcached.git

$ git checkout -t origin/engine

$ make -f win32/Makefile.mingw

I run it by executing 

        $ ./memcached.exe -E ./libs/default_engine.s

Then tested it using telnet to port 11211 and issue the "stats" command to verify that it works! On the deployment environment I just included pthreadGC2.dll from the msysgit distribution and the libs/default_engine.so in the folder I wanted memcached to run on.


Now building a x64 bit release was not as easy and I gave up!