Consumer.wait() blocks indefinitely

[stefankoegl]

I have a strange issue where Consumer.wait() blocks indefinitely, that is not reproducible using curl.

Basically I'm registering a Consumer for a (rather large) database with a specific since-value (53000000 in my case)

consumer = Consumer(db)
consumer.wait(self.callback, since=since, include_docs=True)

The callback receives and processes three results, and the call then blocks in

^CTraceback (most recent call last):
  File "./manage.py", line 36, in <module>
    execute_manager(settings)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 469, in execute_manager
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 222, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 255, in execute
    output = self.handle(*args, **options)
  File "/srv/mygpo/mygpo/../mygpo/maintenance/management/changescmd.py", line 59, in handle
    self.process(self.db, since)
  File "/srv/mygpo/mygpo/../mygpo/maintenance/management/changescmd.py", line 100, in process
    consumer.wait(self.callback, since=since, **params)
  File "/usr/local/lib/python2.7/dist-packages/couchdbkit-0.6.4-py2.7.egg/couchdbkit/consumer/__init__.py", line 103, in wait
    return self._consumer.wait(cb, **params)
  File "/usr/local/lib/python2.7/dist-packages/couchdbkit-0.6.4-py2.7.egg/couchdbkit/consumer/sync.py", line 56, in wait
    break
  File "/usr/local/lib/python2.7/dist-packages/restkit-4.2.1-py2.7.egg/restkit/wrappers.py", line 161, in __exit__
    self.close()
  File "/usr/local/lib/python2.7/dist-packages/restkit-4.2.1-py2.7.egg/restkit/wrappers.py", line 169, in close
    self.body.read()
  File "/usr/local/lib/python2.7/dist-packages/http_parser-0.8.1-py2.7-linux-x86_64.egg/http_parser/reader.py", line 33, in readinto
    recved = self.http_stream.stream.readinto(buf)
  File "/usr/local/lib/python2.7/dist-packages/http_parser-0.8.1-py2.7-linux-x86_64.egg/http_parser/_socketio.py", line 70, in readinto
    return _recv_into_sock(self._sock, b)
  File "/usr/local/lib/python2.7/dist-packages/http_parser-0.8.1-py2.7-linux-x86_64.egg/http_parser/_socketio.py", line 24, in <lambda>
    _recv_into_sock = lambda sock, buf: sock.recv_into(buf)
KeyboardInterrupt

In the CouchDB log the query shows up as

[Sun, 03 Mar 2013 12:08:58 GMT] [info] [<0.9352.205>] 127.0.0.1 - - GET /mygpo/_changes?feed=continuous&since=53000000&filter=episode_states%2Fhas_play_events&include_docs=true 200

If I repeat the same call with curl, it quickly returns a large number of additional results. I'd therefore rule out an issue on the CouchDB side.

The filter that is used in the query is shown below.

function(doc, req)
{
    if(doc.doc_type != "EpisodeUserState")
    {
        return false;
    }

    if(doc._deleted == true)
    {
        return false;
    }

    if(!doc.actions)
    {
        return false;
    }

    function isPlayEvent(action)
    {
        return action.action == "play";
    }

    return doc.actions.some(isPlayEvent);
}

I've tested and reproduced that with the latest version of couchdbkit, and have since upgraded couchdbkit (as well as restkit and http-parser) to their latest HEADs to rule out any recently fixed issues.

Any suggestions on how to further debug this?

[benoitc]

Proba ly an issue in te socketio wrapper. I will have a look quickly on that