Setting up Filmaster locally

This page aims at guiding you through setting up the Filmaster development environment locally. You are going to need the following software:

• Python 2.6
• Django framework (1.3)
• Additional python modules
• PostgreSQL database
• A Linux box (or a lot of patience to set it up under Windows or Mac OS)

Download the source code

Here is a wiki entry explaining how to get the source of Filmaster: Obtaining the code

Installing required software

a) Django framework

We're using stable django 1.3. The installation process is described at: http://docs.djangoproject.com/en/dev/topics/install/

Install django using a standard method, i.e. 

python setup.py install

in the unpacked hacked django directory.

You'll also need gettext package to be able to generate po/mo files. Easiest way via apt-get (on a Debian-based system):

apt-get install gettext

Or install it using your favorite software manager in your distribution / OS.

b) Other dependencies

As we're using a few external modules, they need to be installed as well.

The easiers way is to set up a virtual dev environment and get all the modules from the list at once Setting up virtualenv

If it doesn't work for you for some reason, go ahead and do the following:

You need python-tz, python-openid and python-epydoc modules, python-geoip, csstidy as well as an instance of imdbpy and csstidy. They can be installed in Debian using:

apt-get install python-tz python-openid python-epydoc python-imdbpy csstidy python-geoip

You may also need: pil, python-urljr and python-yadis (Windows users claimed they had to install those modules, they are probably in Ubuntu/Debian installed by default with
Python).

Some more modules can be installed using easy_install mechanism. First install python-setuptools (the package that includes easy_install script):

sudo apt-get install python-setuptools

And then in order to install a selected package simply type:

sudo easy_install BeautifulSoup
sudo easy_install django-compress
sudo easy_install imdbpy
sudo easy_install gdata
sudo easy_install pytz
sudo easy_install vobject
sudo easy_install feedparser
sudo easy_install ybrowserauth
sudo easy_install python-twitter
sudo easy_install python-memcached
sudo easy_install mechanize

Warning: if you encounter issues with latest BeautifulSoap, fall back to the earlier version, in dependencies/BeautifulSoap* and proceed with it manually

For asynchronous background task queues install celery and rabbitmq-server:

sudo easy_install django-celery
sudo apt-get install rabbitmq-server

If something does not get installed for some reason (ybrowserauth was reported not to work via easy_install) you can install the packages manually from film20/dependencies. Simply switch to one of the following directories:

• gdata.py-1.0.13
• pytz-2008b
• ybrowserauth
• vobject-0.6.6
• feedparser
• django-compress
• BeautifulSoup
• python-twitter-0.8.2
• python-memcached-1.47

and install the module using the standard command:

python setup.py install

c) Geo location

Geo-location is turned off by default. If you want to turn it on (e.g. you need to test it locally), you need to download the data file with locations from http://dev.filmmaster.pl/static/db/GeoLiteCity.dat (guest/jakilinux) and place it at film20/film20/static. Then activate the geo module and geoMiddleware in settings.py.

d) Set up the Python app

Make sure you have main project directory on PYTHONPATH:
export PYTHONPATH=absolute_path_to_project_dir

• Create a film20/settings.py file
  Create it basing on settings.template if you haven't done it, yet. Changing the local paths should be enough to make it work.
• Run either setup_english.sh or setup_polish.sh script to create missing symlinks:

./setup_english.sh

• Alternatively, create the links yourself:

Create a symlink config/urls.py to config/urls/pl.py or config/urls/en.py depending on the configuration (choose one language):

ln -s film20/config/urls/en.py film20/config/urls.py

Similar symlink needs to be created for images and css files

ln -s film20/static/english.css film20/static/localized.css
ln -s film20/static/img/en film20/static/img/localized

We also need a folder with generic avatars

ln -s film20/static/avatars-dev film20/static/avatars

e) Database

Database settings are configured in settings.py. We're using PostgreSQL 8.3.5 database. Using other engines is not supported due to custom SQL used in a few views.

PostgreSQL, short howto:

• Install postgresql-8.3 and python-psycopg2 (its python driver)
• In case you need to compile psycopg, get it from here: http://initd.org/pub/software/psycopg/
• Create a test database:
  $ sudo su - postgres
  $ createdb test

• Next log into the test db:
  $ psql
• and create the proper database, user and password:
  create user film20;
  create database film20 with owner = film20;
  ALTER USER film20 WITH ENCRYPTED PASSWORD 'justenter';

• Add this line to /etc/postgresql/8.3/main/pg_hba.conf :
  local   all         all                               trust

• Restart postgres:
  # /etc/init.d/postgresql-8.3 restart

Creating tables

• enter the relevant data (db, user, pass) in settings.py
• run in film20 directory:
  $ manage.py syncdb
• One more thing you need to do is to create a django Site record:

Setting the site ID

Filmaster as most django sites require a single record in django_site table to function normally. The SITE_ID in settings.py has to point to this one.

• Log in to the psql console and simply insert one row like:

film20=> insert into django_site (id, domain, name) values (1, 'localhost', 'Filmaster.com');

• And you should have something like that in the database now:

film20=> select * from django_site;
 id |    domain     |     name   
1 | localhost  | Filmaster.com

Done! Good job!

Loading some test data

Now you can start hacking with Filmaster. It would be probably a good idea to import a few movies first and enter some test data. You can do it manually using the run_imdb_fetcher.py script.

Easiest way to do it is to:

• go to http://localhost:8000/add-films/ and add a new films providing the IMDB links
• then accept this film at http://localhost:8000/add-films/admin
• run this script to trigger fetching the film metadata from IMDB:

python run_imdb_fetcher.py -f

Now you should have some movies in database. Rate them here: http://localhost:8000/rate-movies/ to get some test ratings and start hacking around!

Loading a database dump

You can also try load a sample database dump we provide.

If you'd like to work on some more real life data, we have a simple database dump that you can use. It weights a lot so we don't want to expose it publicly but drop us an e-mail at filmaster@filmaster.com and we'll be glad to provide you with the FTP connection details.

Note that we provide database dumps for regular contributors so if you are only starting, it's better to use the raw database.

Running the webapp

Eventually, run the app local server (by default we're using the django dev server to develop locally but if you prefer apache, you can set it up as well) using the manage.py script:

python manage.py runserver

Open a web browser (or IE) with url like: http://127.0.0.1:8000/
The admin interface is here: http://127.0.0.1:8000/admin
If you want to access the db as administrator you'll need to create a user account and then grant it the proper permissions manually in database.
TODO: create a simple script that does that!
And for avatars, as we don't want to keep them in svn

To execute queued background tasks run celery worker:

python manage.py celeryd

The DEV server

DEV server is localized at: http://reloaded.filmmaster.pl

It should always host a reasonably up-to-date version of the app (filmaster-reloaded branch).

If dev does not work, please e-mail dev@filmaster.pl.

If you checked something in that you believe makes the dev server not functional, try to correct it yourself by analyzing the recent commits in filmaster-reloaded and if anything needs to be done on the dev server itself (like a settings.py change or a database change, also contect dev@filmaster.com to make us perform the tasks if you don't have the rights to ssh to the server to do it yourself).

Unit tests

The unit tests can be run by issuing a command like:

./manage.py test # whole app

./manage.py test import_ratings # only a specific module (like import_ratings)

Not that we have tons of tests. They are on constant TODO list, but we're planning to actually prioritize this task as regression testing becomes more and more of an issue.

In order to perform the tests, the postgres user needs permissions to drop/create tables (needs to be a super user). To achieve that, execute:

psql> ALTER USER film20 SET superuser = true;

or if you have PostgreSQL>8.3, do:

psql> ALTER USER film20 SUPERUSER

If you're using sqlite for unit tests, you don't need to worry about it.