Hi.
This is my first post to this mailing list, and in fact to any Debian mailing list in general so I'd like to introduce myself first. To keep Your interest focused on the problem I moved this to the footer of this message. With that of the way here's the core topic:
I'd like to define and document the recommended best practice of packaging Django 1.0+ (and especially 1.3) web applications and projects for Debian. I started this on the Debian wiki at http://wiki.debian.org/DjangoPackagingDraft
The topics I want to cover are:
* Packaging Django applications: - where to install code - where to install templates - where to install static files (as understood by http://docs.djangoproject.com/en/dev/howto/static-files/) - where to install documentation (if any)
* Packaging Django projects: - where to install code (like urls, settings, etc) - where to install templates - how to do settings (debconf integration / configuration files / python module vs real config file?) - what to with web server integration (aka http://localhost/packagename vs do-it-yourself-after-installation) - how do to database configuration and integration (dbconfig-common?)
There are some things I did not want to enforce because either there is no clear best method or I don't feel that the topic is relevant before specifying more fundamental pieces. In particular this includes:
- Shipping web server configuration files so that application can work out of the box after installation. - Handling of project configuration (it's both executable code and administrator-oriented settings file)
I started a demo native package (still work in progress, comments welcome) called django-hello that contains a simple django project with one application. I deliberately used django-staticfiles as that's how upcoming web applications will distribute their static files. The code is at lp:~zkrynicki/+junk/django_hello. There is an Ubuntu PPA with python-django-staticfiles package that is required at https://launchpad.net/~zkrynicki/+archive/lava).
There are some compromises I needed to make though. This package is using python-support as I still need to target the current Ubuntu LTS release (lucid) where dh_python2 is not available. I'm more than happy to convert this package to dh_python2 assuming that the same rules can be applied in a backwards compatible manner to Lucid packages.
That's it. I'd like to know what you think.
About me:
My name is Zygmunt Krynicki. I'm work in the Linaro Validation team and I'd like to start a process that will make the output of the Linaro Validation team available to Debian users. I'd like to maintain the packages we're making and learn to do those packages properly.
Most of my work revolves around server-side technology, specifically around Django web applications. I did not find any strong guidelines on how to package Django web applications and projects for Debian. Therefore I decided that I'd like to contribute to Debian and help define and apply those guidelines to various Django applications as they become available.
Best regards Zygmunt Krynicki
PS: I cross-posted to linaro-dev to make sure people there that might be interested in this discussion can notice it and participate.
On Tue, Mar 8, 2011 at 9:50 PM, Zygmunt Krynickiu zkrynicki@ubuntu.com wrote:
I'd like to define and document the recommended best practice of packaging Django 1.0+ (and especially 1.3) web applications and projects for Debian. I started this on the Debian wiki at http://wiki.debian.org/DjangoPackagingDraft
Some things I (as a sysadmin user of Debian) would like to suggest for all web apps, django based or not:
Use debconf to ask if the sysadmin wants to setup a site and then ask for enough detail to do so. The package should ship a script that can be called either manually by the sysadmin or automatically by the maintainer scripts (using debconf answers) to setup a site or a sub-path of a site.
Database (with South etc) and configuration upgrades (with Config::Model or similar) are great, please do them as automatically as possible after confirmation using debconf and or provide scripts for the sysadmin to do them.
Data: I like to differentiate between data supporting the package and data that belongs to the sysadmin/site. IMO the former belongs in /var somewhere and the latter at a path chosen by the sysadmin. I don't think this is gotten right by much software, including by all database packages.
W dniu 08.03.2011 18:10, Paul Wise pisze:
On Tue, Mar 8, 2011 at 9:50 PM, Zygmunt Krynickiuzkrynicki@ubuntu.com wrote:
I'd like to define and document the recommended best practice of packaging Django 1.0+ (and especially 1.3) web applications and projects for Debian. I started this on the Debian wiki at http://wiki.debian.org/DjangoPackagingDraft
Some things I (as a sysadmin user of Debian) would like to suggest for all web apps, django based or not:
Use debconf to ask if the sysadmin wants to setup a site and then ask for enough detail to do so. The package should ship a script that can be called either manually by the sysadmin or automatically by the maintainer scripts (using debconf answers) to setup a site or a sub-path of a site.
The current web apps policy recommends [1] that by default the application should be accessible at http://SERVER/PACKAGE and I understand that decision (it should be possible to install all the web apps without conflicts).
The problem of how to do that generally is still valid (there are a lot of web servers to support potentially). I would start with an initial set that must support apache2 with mod_wsgi. In the future we can offer support for additional web servers.
Ideally this might work with some abstraction layer such as wwwconfig-common. Web applications could drop a file in /usr/share/wwwconfig-common/webapps.d/PACKAGE. Then if the administrator wants to reconfigure the "webapps" running on his server all he needs to do is edit something like /etc/webapps-common/webapps.conf and dpkg-reconfigure webapps-common.
Each application would only need to declare a few pieces: - stuff to serve statically (either as DOMAIN or URL-PREFIX) - wsgi file - name
For example, as JSON:
{ "name": "django-hello", "static-content": [ { "pathname": "/var/lib/django-hello/static", "url_prefix": "static/", "domain_prefix": "static.", "description": "Static resources used by django-hello" }, { "pathname": "/var/lib/django-hello/media", "url_prefix": "media/", "domain_prefix": "media." "description": "User submitted resources used by django-hello" } ], "wsgi": "/var/lib/django-hello/django.wsgi" }
The www-common middleware would then store a record for each application. This record would contain: - enabled or disabled status - custom sub-URL if desired - custom DOMAIN if desired
Again, as JSON:
{ "name": "django-hello", "custom_url": false, "custom_domain": false, "enabled": true }
This would serve it as http://SERVER/django-hello/
Another example:
{ "name": "django-hello", "custom_url": "/", "custom_domain": "hello.example.org", "enabled": true }
This would serve it as http://hello.example.org/
All combinations of those two options would be supported. Sub URL could be set to / to mark a single application as main. A custom DOMAIN would limit such entries to the specified virtual host.
The glue code would then read the database of web applications, read the configuration of each web application and regenerate the webserver-specific configuration files.
Database (with South etc) and configuration upgrades (with Config::Model or similar) are great, please do them as automatically as possible after confirmation using debconf and or provide scripts for the sysadmin to do them.
South support is rather easy. I still need to check what should happen in the downgrade use case and where exactly to store a backup of the database but in general it should be possible to support this correctly.
Configuration upgrade is more difficult as it really depends on what the maintainers of a web application desire to expose as configuration and what is merely an implementation detail that should not be configurable by the system administrator. I agree that supporting this is important (and especially supporting upgrades that don't break things). I tend to lean to a recommendation that application settings are handled explicitly by particular web application. That is: a web application should define a configuration file with all the things it publicly supports. IMHO this would be usually limited to email settings (do we have webapps-email-common?), database settings (dbconfig-common), administrator name, pathname of the "storage area" for user content. The rest should be implementation detail. We can have a django module for supporting such configuration system. It would have the advantage of not requiring code changes (the way django currently requires) and could be a healthy thing for the wider django community in general.
I'm a little worried about upgrade verbosity. In particular, I would like to allow silent (unattended) upgrades if possible. I think with proper debconf settings it's possible to ask the question "do you want to upgrade database schema to version $FOO" with "yes, I do" being the silent default. The issue here is that unless you upgrade you cannot really start the application properly which should (AFAIR) be treated equally as upgrade failure. That's rather extreme IMHO. I'd rather require that applications can be safely upgraded because the system will create appropriate database/media files backups before it triggers the process.
Data: I like to differentiate between data supporting the package and data that belongs to the sysadmin/site. IMO the former belongs in /var somewhere and the latter at a path chosen by the sysadmin. I don't think this is gotten right by much software, including by all database packages.
How do you differentiate which data belongs to the sysadmin/site and which to the package? Do I understand correctly that user-submitted content is "application data" and static resources are "package data"?
Thanks for the feedback! Zygmunt Krynicki
[1] http://webapps-common.alioth.debian.org/draft/html/ch-httpd.html#s-httpd-loc...