From 362921bb08860d43565f6d290b29987a6e5cf8cf Mon Sep 17 00:00:00 2001 From: Kent Slaney Date: Thu, 6 Jun 2024 15:38:52 -0700 Subject: [PATCH] readme update from #125 --- README.rst | 101 +++++++++++++++++++++++++++++------------------------ 1 file changed, 56 insertions(+), 45 deletions(-) diff --git a/README.rst b/README.rst index 9c829098..1ae0d829 100644 --- a/README.rst +++ b/README.rst @@ -5,15 +5,16 @@ libmc |status| |pypiv| |pyversions| |wheel| |license| libmc is a memcached client library for Python without any other -dependencies in runtime. It's mainly written in C++ and Cython. libmc -can be considered as a drop in replacement for libmemcached and +dependencies at runtime. It's mainly written in C++ and Cython and +can be considered a drop in replacement for libmemcached and `python-libmemcached `__. -libmc is developing and maintaining by Douban Inc. Currently, It is -working in production environment, powering all web traffics in -douban.com. Realtime `benchmark -result `__ is -available on travis. +libmc is developed and maintained by Douban Inc. Currently, it is +working in a production environment, powering all web traffic on +`douban.com `__. [#]_ + +.. [#] The link not an endorsement of the traffic estimate site's accuracy, just + a publicly available reference point Build and Installation ---------------------- @@ -37,14 +38,14 @@ Usage: Under the hood -------------- -Under the hood, libmc consists of 2 parts: an internal fully-functional +Under the hood, libmc consists of 2 parts: an internal, fully-functional memcached client implementation in C++ and a Cython wrapper around that implementation. Dynamic memory allocation and memory-copy are slow, so -we tried our best to avoid them. The ``set_multi`` command is not -natively supported by the `memcached +we've tried our best to avoid them. libmc also supports the ``set_multi`` +command, which is not natively supported by the `memcached protocol `__. -Some techniques are applied to make ``set_multi`` command extremely fast -in libmc (compared to some other similiar libraries). +Some techniques have been applied to make ``set_multi`` command extremely fast +in libmc (compared to similiar libraries). Configuration ------------- @@ -77,24 +78,23 @@ Configuration mc.config(MC_RETRY_TIMEOUT, 5) # 5 s -- ``servers``: is a list of memcached server addresses. Each address - can be in format of ``hostname[:port] [alias]``. ``port`` and ``alias`` - are optional. If ``port`` is not given, default port ``11211`` will - be used. ``alias`` will be used to compute server hash if given, - otherwise server hash will be computed based on ``host`` and ``port`` - (i.e.: If ``port`` is not given or it is equal to ``11211``, ``host`` - will be used to compute server hash. If ``port`` is not equal to ``11211``, - ``host:port`` will be used). -- ``do_split``: Memcached server will refuse to store value if size >= - 1MB, if ``do_split`` is enabled, large value (< 10 MB) will be - splitted into several blocks. If the value is too large (>= 10 MB), - it will not be stored. default: ``True`` -- ``comp_threshold``: All kinds of values will be encoded into string - buffer. If ``buffer length > comp_threshold > 0``, it will be - compressed using zlib. If ``comp_threshold = 0``, string buffer will - never be compressed using zlib. default: ``0`` -- ``noreply``: Whether to enable memcached's ``noreply`` behaviour. - default: ``False`` +- ``servers``: a list of memcached server addresses. Each address + should be formated as ``hostname[:port] [alias]``, where ``port`` and + ``alias`` are optional. If ``port`` is not given, the default port ``11211`` + will be used. If given, ``alias`` will be used to compute the server hash, + which would otherwise be computed based on ``host`` and ``port`` + (i.e. whichever portion is given). +- ``do_split``: splits large values (up to 10MB) into chunks (<1MB). The + memcached server implementation will not store items larger than 1MB, + however in some environments it is beneficial to shard up to 10MB of data. + Attempts to store more than that are ignored. Default: ``True``. +- ``comp_threshold``: compresses large values using zlib. If + ``buffer length > comp_threshold > 0`` (in bytes), the buffer will be + compressed. If ``comp_threshold == 0``, the string buffer will never be + compressed. Default: ``0`` +- ``noreply``: controls memcached's + ``noreply`` `feature `__. + Default: ``False`` - ``prefix``: The key prefix. default: ``''`` - ``hash_fn``: hashing function for keys. possible values: @@ -110,16 +110,16 @@ Configuration implementions in libmemcached. - ``failover``: Whether to failover to next server when current server - is not available. default: ``False`` + is not available. Default: ``False`` - ``MC_POLL_TIMEOUT`` Timeout parameter used during set/get procedure. - (default: ``300`` ms) + Default: ``300`` ms - ``MC_CONNECT_TIMEOUT`` Timeout parameter used when connecting to - memcached server on initial phase. (default: ``100`` ms) -- ``MC_RETRY_TIMEOUT`` When a server is not available dur to server-end - error. libmc will try to establish the broken connection in every - ``MC_RETRY_TIMEOUT`` s until the connection is back to live.(default: - ``5`` s) + memcached server in the initial phase. Default: ``100`` ms +- ``MC_RETRY_TIMEOUT`` When a server is not available due to server-end + error, libmc will try to establish the broken connection in every + ``MC_RETRY_TIMEOUT`` s until the connection is back to live. Default: + ``5`` s **NOTE:** The hashing algorithm for host mapping on continuum is always md5. @@ -140,7 +140,7 @@ FAQ Does libmc support PHP? ^^^^^^^^^^^^^^^^^^^^^^^ -No. But if you like, you can write a wrapper for PHP based on the C++ +No, but, if you like, you can write a wrapper for PHP based on the C++ implementation. Is Memcached binary protocol supported ? @@ -151,7 +151,7 @@ No. Only Memcached ASCII protocol is supported currently. Why reinventing the wheel? ^^^^^^^^^^^^^^^^^^^^^^^^^^ -Before libmc, we're using +Before libmc, we were using `python-libmemcached `__, which is a python extention for `libmemcached `__. @@ -161,21 +161,32 @@ still some unsolved bugs. Is libmc thread-safe ? ^^^^^^^^^^^^^^^^^^^^^^ -libmc is a single-threaded memcached client. If you initialize a libmc -client in one thread but reuse that in another thread, a Python -Exception ``ThreadUnsafe`` will raise in Python. +Yes. ``libmc.ThreadedClient`` is a thread-safe client implementation. To hold +access for more than one request, ``libmc.ClientPool`` can be used with Python +``with`` statements. ``libmc.Client``, however, is a single-threaded memcached +client. If you initialize a standard client in one thread but reuse that in +another thread, a Python ``ThreadUnsafe`` Exception will be raised. Is libmc compatible with gevent? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Yes, with the help of `greenify `__, libmc is friendly to gevent. Read ``tests/shabby/gevent_issue.py`` for -details. +details. ``libmc.ThreadedClient`` and ``libmc.ClientPool`` are not compatible. +[#]_ + +.. [#] In order to use a single executable for multiple greenlet contexts, + gevent has to `copy thread memory + `__ + to and from the same stack space. This doesn't affect Python references, + which are handed off through gevent, but makes it impossible for shared + libraries to pass memory addresses across greenlets, which is required for + the worker pool. **Notice:** -`gevent.monkey.patch_all()` will override -`threading.current_thread().ident` to Greenlet's ID, +``gevent.monkey.patch_all()`` will override +``threading.current_thread().ident`` to Greenlet's ID, this will cause libmc to throw a ThreadUnSafe error or run into dead lock, you should only patch the things that you need, e.g.