KARL: knowledge management & collaboration for NGOs (Plone Conference 2009) - mrtopf.demrtopf.de

KARL: knowledge management & collaboration for NGOs (Plone Conference 2009)

Tom Moroz, Chris Rossi and Calvin Hendryx-Parker have been talking about KARL.

Tom Moroz of the Open Society Institute (OSI) starts on a bit of background on what KARL is.

OSI has 7 ‚core‘ offices in US and Europe. It has national foundations in 32 countries. They have a program and grant giving activity in over 60 countries and >20 Program Areas / Initiatives.

KARL was begun in August 2006, in May 2007 KARL v1.0 beta release, January 2008 an official launch, November 2008 KARL v2.0, June 2009 KARL v3.0. OSI first developed it for their own needs but later saw that many NGOs have the same problems so they tried to open it up for their use cases as well.

He then showed some of the features of KARL.

Chris Rossi then took over and explained a bit about the technical background. KARL moved away from Plone to use BFG and he tries to explain the reasoning:

  • Plone is great at what it does. Content Management in a nice, general way. This is the sweet spot of Plone.
  • What do you do when your project is outside of the sweet spot?
  • Further you need to exted Plone, the more you fight the framework. Result is a high developer frustration.
  • Using BFG, they were able to maintain programming paradigms familiar to Zope/Plone developers without using/fighting Plone.

The move from KARL 1 to KARL 2:

  • Entirely new UI built on top of Plone — very different from Plone’s UI.
  • Much framework fighting
  • Much integration with non-Plone technologies, aka Frankenplone.
  • Because of the Frankenplone infrastructure they ended up with bad performance and poor reliability
  • It also resulted in difficult maintenance
  • Difficult to provide something reasonable for non-OSI deployments.

What are the alternatives to Plone?

  • Zope2, primarily used by Plone, time tested.
  • Zope3, big, many dependencies, app developers encouraged to use CA.
  • Grok, an attempt to make Zope3 easier to use (but it’s still Zope3 and still big and complicated).
  • Django, nice, but full stack like Zope2 (you need to use their answers for all you do).
  • Pylons/TG2, also nice but geared towards RDB/URL dispatch
  • Several others, little traction, not Zope like.

With BFG:

  • It is in the repoze namespace, but is not really repoze.
  • Like Zope, but much smaller, leaner
  • Speed and simplicity: „Pay only for what you eat“. If you don’t need it, it’s not there. If you do need it, it is easy to put it in.
  • Documentation: An undocumented feature or API is a bug.
  • Testing: 100% test coverage. Untested code is broken code.
  • Collaborative culture: Using and promoting non-Zope technologies when they are useful.
  • Easy to adopt for Zope developers. (Maybe non-Zope developers as well.)

KARL technologies

Python: WSGI, WebOB, Paste Deploy, virtualenv, eggs, repoze.who, Supervisor, TinyMCE, jQuery, nose, twill, mod_wsgi.

Zope: ZPT (Chameleon), traversal, views, security, ZODB/ZEO, catalog, component architecture, buildout

Facts:

  • 50-150 rps per core, fully dynamic and personalized, no caching
  • 50,000+ lines of code (including tests)
  • 1,500+ unit tests, 8-10 seconds
  • startup time: <2 seconds
  • 946 MB ZODB +11.8 GB blobs, 3,700 users, 736 communities, 90k documents
  • <600MB RAM

KARL project

  • 13 committers, 8 organizations
  • LaunchPad: 708 total issues, 35 milestones, 59 UAT, 5 evaluations, 3 migrations
  • Zero open bugs
  • Hosting by Six Feet Up
  • Web Site: www.karlproject.org

Then Calvin talks about hosting which he started with KARL 2. He explains what the difference is and how deployment now is much easier because the framework is smaller.

They are running it on Xen and CentOS. It’s fully managed and monitored. Of course you can also run KARL yourself on your server.

KARL 3.0 migration

  • Plone-based KARL 2 to the BFG-based KARL 3
  • 4.7 GB of data and 62,000 pieces of content
  • Created dummy content to baseline and test the migration process
  • Using a GenericSetup to export XML content and built a one time content importer for KARL 3
  • Migrated users group information stored in Postgres to KARL 3
  • Both systems were ZODB backed, but the newer KARL system included BLOB support
  • Other pilots were migrated a few weeks later

Recent & Future KARL Hosting

  • Moved to new datacenter in Q3 2009 (more robust)
  • Disaster Recovery Support Q1 2010
  • Online Realtime Backup Q1 2010

He also mentioned EC2 but there you are dependant on Amazon’s infrastructure.

The path forward:

  • On-going software development
  • Development of governance model to make available to both large and small organizations
  • The importance of „soft“ technology and methodologies

2 Kommentare » Schreibe einen Kommentar

  1. Hi Nicolas, nice hearing about CubicWeb. One clarification though. The point of the talk wasn't to pitch KARL as an alternative to Plone. But rather, to pitch the idea that, when you fall outside Plone's sweet spot you shouldn't fight the framework. Instead, you should write your own application, using the stuff you know from the world of Zope/Plone.

    I realize that some might choose to jump completely out and do Django, Pylons, TG2, or CubicWeb. BFG is aimed at people that like some of the technologies in Plone and want to use them a la carte.