UbuntuへのCeleryのインストール

Ubuntu12.10にCeleryをインストール。
Djangoアプリのバックエンドのバッチ処理用に。
Cronでも十分だけど、今後を考えて。

pipでインストール。

sudo pip install celery django-celery

インストール中にネイティブライブラリのコンパイルにPython.hが無い、という
コンパイルエラーが出る場合、python-devをaptでインストールします。

sudo apt-get install python-dev

settings.pyは

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'django.contrib.humanize',
    'south',
+    'kombu.transport.django',
+    'djcelery',
    'myapp',
)

+#celery
+BROKER_URL = "django://" 
+import djcelery
+djcelery.setup_loader() 

こんな感じで追加。

そしてmyapp/tasks.pyを作り、

from celery.task.schedules import crontab 
from celery.decorators import periodic_task

@periodic_task(run_every=crontab(hour="*", minute="*", day_of_week="*")) 
def test(*args, **kwargs):
    print "TEST!"

こんなふうに書く。

試す。

>python manage.py celery worker -B

死ぬ。こんな感じのが出る。

*** glibc detected *** /usr/bin/python: munmap_chunk(): invalid pointer: 0x00007f5b9ee73fd4 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f5b9fe10b96]
/usr/local/lib/python2.7/dist-packages/_billiard.so(+0x40db)[0x7f5b9eebd0db]
（中略）
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/billiard/forking.py", line 528, in main
    self = load(from_parent)
  File "/usr/lib/python2.7/pickle.py", line 1378, in load
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1133, in load_reduce
    value = func(*args)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/utils.py", line 130, in _unpickle_app
    return pickler()(cls, *args)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/utils.py", line 109, in __call__
    self.prepare(app, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/utils.py", line 113, in prepare
    app.conf.update(kwargs['changes'])
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 293, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 473, in conf
    return self._get_config()
  File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 338, in _get_config
    s = Settings({}, [self.prepare_config(self.loader.conf),
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 293, in __get__
    value = obj.__dict__[self.__name__] = self.__get(obj)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/base.py", line 485, in loader
    return get_loader_cls(self.loader_cls)(app=self)
  File "/usr/local/lib/python2.7/dist-packages/celery/loaders/__init__.py", line 23, in get_loader_cls
    return symbol_by_name(loader, LOADER_ALIASES)
  File "/usr/local/lib/python2.7/dist-packages/kombu/utils/__init__.py", line 80, in symbol_by_name
    module = imp(module_name, package=package, **kwargs)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/dist-packages/djcelery/loaders.py", line 15, in <module>
    from django import db
  File "/usr/local/lib/python2.7/dist-packages/django/db/__init__.py", line 11, in <module>
    if DEFAULT_DB_ALIAS not in settings.DATABASES:
  File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 184, in inner
    self._setup()
  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 42, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 95, in __init__
    raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'myproj.settings' (Is it on sys.path?): cannot import name connection

ImportErrorではあるけども、その前にglibcのエラーがドバドバ出るので
sys.path云々ではなく拡張モジュールがおかしいんだろう。

githubのissueを色々みたところ、

/usr/local/lib/python2.7/dist-packages/_billiard.so を消せ

というのがあって、たしかにそれを消すとOKなんだけども、
そうすると起動時に

[2013-01-26 22:46:14,897: WARNING/MainProcess] /usr/local/lib/python2.7/dist-packages/billiard/__init__.py:318: RuntimeWarning: force_execv is not supported as the billiard C extension is not installed

とか出てなんかちょっと気分が悪い。

個人的な趣味で、オプション付ける方を選ぶ。

>python manage.py celery worker -B

 -------------- celery@ubuntu v3.0.13 (Chiastic Slide)
---- **** -----
--- * ***  * -- [Configuration]
-- * - **** --- . broker:      django://localhost//
- ** ---------- . app:         default:0x3885090 (djcelery.loaders.DjangoLoader)
- ** ---------- . concurrency: 2 (processes)
- ** ---------- . events:      OFF (enable -E to monitor this worker)
- ** ----------
- *** --- * --- [Queues]
-- ******* ---- . celery:      exchange:celery(direct) binding:celery
--- ***** -----

[2013-01-26 22:53:13,887: WARNING/MainProcess] celery@ubuntu ready.
[2013-01-26 22:54:04,286: WARNING/PoolWorker-2] TEST!

動いた。