Tom Moroz, Chris Rossi and Calvin Hendryx-Parker have been talking about KARL.
Tom Moroz of the Open Society Institute (OSI) starts on a bit of background on what KARL is.
OSI has 7 ‚core‘ offices in US and Europe. It has national foundations in 32 countries. They have a program and grant giving activity in over 60 countries and >20 Program Areas / Initiatives.
KARL was begun in August 2006, in May 2007 KARL v1.0 beta release, January 2008 an official launch, November 2008 KARL v2.0, June 2009 KARL v3.0. OSI first developed it for their own needs but later saw that many NGOs have the same problems so they tried to open it up for their use cases as well.
He then showed some of the features of KARL.
Chris Rossi then took over and explained a bit about the technical background. KARL moved away from Plone to use BFG and he tries to explain the reasoning:
- Plone is great at what it does. Content Management in a nice, general way. This is the sweet spot of Plone.
- What do you do when your project is outside of the sweet spot?
- Further you need to exted Plone, the more you fight the framework. Result is a high developer frustration.
- Using BFG, they were able to maintain programming paradigms familiar to Zope/Plone developers without using/fighting Plone.
The move from KARL 1 to KARL 2:
- Entirely new UI built on top of Plone — very different from Plone’s UI.
- Much framework fighting
- Much integration with non-Plone technologies, aka Frankenplone.
- Because of the Frankenplone infrastructure they ended up with bad performance and poor reliability
- It also resulted in difficult maintenance
- Difficult to provide something reasonable for non-OSI deployments.
What are the alternatives to Plone?
- Zope2, primarily used by Plone, time tested.
- Zope3, big, many dependencies, app developers encouraged to use CA.
- Grok, an attempt to make Zope3 easier to use (but it’s still Zope3 and still big and complicated).
- Django, nice, but full stack like Zope2 (you need to use their answers for all you do).
- Pylons/TG2, also nice but geared towards RDB/URL dispatch
- Several others, little traction, not Zope like.
With BFG:
- It is in the repoze namespace, but is not really repoze.
- Like Zope, but much smaller, leaner
- Speed and simplicity: „Pay only for what you eat“. If you don’t need it, it’s not there. If you do need it, it is easy to put it in.
- Documentation: An undocumented feature or API is a bug.
- Testing: 100% test coverage. Untested code is broken code.
- Collaborative culture: Using and promoting non-Zope technologies when they are useful.
- Easy to adopt for Zope developers. (Maybe non-Zope developers as well.)
KARL technologies
Python: WSGI, WebOB, Paste Deploy, virtualenv, eggs, repoze.who, Supervisor, TinyMCE, jQuery, nose, twill, mod_wsgi.
Zope: ZPT (Chameleon), traversal, views, security, ZODB/ZEO, catalog, component architecture, buildout
Facts:
- 50-150 rps per core, fully dynamic and personalized, no caching
- 50,000+ lines of code (including tests)
- 1,500+ unit tests, 8-10 seconds
- startup time: <2 seconds
- 946 MB ZODB +11.8 GB blobs, 3,700 users, 736 communities, 90k documents
- <600MB RAM
KARL project
- 13 committers, 8 organizations
- LaunchPad: 708 total issues, 35 milestones, 59 UAT, 5 evaluations, 3 migrations
- Zero open bugs
- Hosting by Six Feet Up
- Web Site: www.karlproject.org
Then Calvin talks about hosting which he started with KARL 2. He explains what the difference is and how deployment now is much easier because the framework is smaller.
They are running it on Xen and CentOS. It’s fully managed and monitored. Of course you can also run KARL yourself on your server.
KARL 3.0 migration
- Plone-based KARL 2 to the BFG-based KARL 3
- 4.7 GB of data and 62,000 pieces of content
- Created dummy content to baseline and test the migration process
- Using a GenericSetup to export XML content and built a one time content importer for KARL 3
- Migrated users group information stored in Postgres to KARL 3
- Both systems were ZODB backed, but the newer KARL system included BLOB support
- Other pilots were migrated a few weeks later
Recent & Future KARL Hosting
- Moved to new datacenter in Q3 2009 (more robust)
- Disaster Recovery Support Q1 2010
- Online Realtime Backup Q1 2010
He also mentioned EC2 but there you are dependant on Amazon’s infrastructure.
The path forward:
- On-going software development
- Development of governance model to make available to both large and small organizations
- The importance of „soft“ technology and methodologies