Architecture

Database and model classes

Overview

We have 4 top level content classes user work with:

page.Page

When the user creates a Norm (respectively Paper, or german “Papier”) through the web interface, he is internally creating a Page with the page.Page.function set to page.Page.NORM. The text of the page is saved in Text objects in page.Page.texts, that may have different variants.

Page is a subclass of delegateable.Delegateable.

proposal.Proposal

The actual text of the proposal is saved as a Page object with the Page.function page.Page.DESCRIPTIION in an attribute page.Page.description.

In a Page a user can propose changes to a Page (Norm). These are saved as selection.Selection objects (see below).

Proposal is a subclass of delegateable.Delegateable.

milestone.Milestone
A Milestone is used to represent a date or a deadline. Each Delegtable have a relation to one Milestone.
comment.Comment
Comments can be created for delegateable.Delegateable objects. The reference to the Delegateable is available as comment.Comment.topic. Each comment is associated with a poll.Poll to rate content. If it is an answer to another Comment this is referenced with comment.Comment.reply_id. The text of an comment is stored as a revision.Revision. If a comment is edited, a new Revision is created. Revisions have similar function for Comments like Texts have for Pages.

Some of the more interesting models and support classes are:

selection.Selection
A Selection is a “reference” object for the relation between Pages (Norms) and Proposals and stores additional information, especially when the relation was created or removed.
text.Text
Text objects are only used to save text for page.Page objects. See Page.
revision.Revision
Store the text of a Comment. Similar to Text. See Comment.
poll.Poll
A Poll is an object representing a particular poll process a :class:Comment, :class:Proposal or :class:Page/Norm Single votes relate to the :class:Poll object.
vote.Vote
A Vote is a single vote from a user. It knows about the user whose vote it is, and if it was a delegated vote the useris of the delegatee that voted for the user.
delegation.Delegation
Created when a user delegates the voting to an delegatee. It knows about the user who (principal_id) delegated, who is the delegatee (agent_id) for which poll (scope_id - the id of the delegateable object).
delegateable.Delegateable
Base class for Pages and Proposals for which a user can delegate the voting. Sqlalchemy’s joint table inheritance is used.
tally.Tally
A Tally saves the linear history of the :class:Poll noting which vote occured and what is the sum number of for/against/abstains.
watch.Watch
Users can subscribe to content changes and will be notified depending settings in their preferences. A Watch object is created in these cases.
meta.Indexable
Mixin class for content that should be indexed in Solr. It defines only one method meta.Indexable.to_index() that collects data from the models and will be uses automatically when a model object participates in a transaction.

Almost all model classes have a classmethod .create() to create a new instance of the model and setup the necessary data or relationships. Furthermore methods like .find() and .all() as convenient query method that support limiting the query to the current model.instance.Instance or in-/exclude deleted model instances.

Diagrams

Tables

../_images/adhocracy-tables.png

Diagram of the mapped tables

../_images/adhocracy-classes.png

Diagramm of all model classes

Updating the diagrams

To update the diagramms install graphviz and easy_install sqlalchemy_schemadisplay into the environment adhocracy is installed in. Then run python /adhocracy/scripts/generate-db-diagrams.py. It will create the diagrams as GIF files. Finally replace the GIF files in adhocracy/docs/development with the new versions.

Delegateables

A user can delegate his vote to another user for comment.Comment, proposal.Proposal and page.Page. This functionality is enabled by inheriting from Delegateable

Inheritance is done with sqlalchemy’s joint table inheritance where the delegateable table is polymorphic on delegateable.Delegateable.type

Page Models

The model class page.Page has 2 uses in adhocracy that are differentiated by the value of page.Page.function.

  • A Page represents a Norm if page.Page.function is page.Page.NORM. This is the primary usage of Page.
  • For every proposal.Proposal a page is created and available as proposal.Proposal.description to manage the text and text versions of the proposal. page.Page.function is page.Page.DESCRIPTION in this case.

Pages are delegateable and inherit from delegateable.Delegateable.

Variants and Versions

The text of the Page is not saved in the page table but created as a text.Text object. A page can contain different variants of the text, and for each variant an arbitrary number of versions, e.g.:

  • initial text
    • version 1
    • version 2
    • ...
  • other text variant
    • version 1
    • version 2
    • ...
  • ...

Text variants are used for Norms. For the initial text, variant is set to text.Text.HEAD. This text variant is handled special in the UI and labeled Status Quo in Norms. Other variants can be freely named. All text variants are listed in Page.variants.

Each variant can have a arbitrary number of versions. The newest version of the text is called the head (not to confuse with the default text variant Text.HEAD). You can get the newest version of a specific variant with Page.variant_head(). The newest versions of all variants is available as Page.heads. A shortcut to obtain the newest version of the HEAD variant is Page.head.

Text variants are not used for pages that are used as the description of proposal.Proposal objects (proposal.Proposal.description).

The poll tally of a variant or all variant can be optained with Page.variant_tally() or Page.variant_tallies()

Polls are set up per variant, not for the Page object.

Page Hierarchies

Page objects (that have the funciton Norm) can be organized in a tree stucture by setting another Page (Norm) object as one of the Page.parents of the current page. Parents can be an arbitrary number of delegateable.Delegateable objects, but only one, not already deleted Page with the Page.function Page.NORM is allowed. Parents are taken into account when we compute a delegation graph.

The subpages of a page are available as Page.subpages.

Other functionality

Beside that Pages have functions and attributes to handle purging, deleting, renaming, Selections (Page (Norm) <-> Proposal relationships) and other things. See the api documentation for Page

Indexing/Searching

Indexing and searching is done with sql(alchemy) and solr. Indexing with solr is done asyncronously most of the time while updates of the rdbm is done syncronously most of the time. The asyncronous indexing is done throug a rabbitmq job queue.

Types of search indexes

Beside rdbm intern indexes adhocracy maintains application specific indexes (that partly act as audit trails too):

  • solr’ is used for full text and tag searches. It is an document oriented index. The index schema can be found in `adhocracy/solr/schema.xml
  • new tally.Tally objects are created with every new or changed vote and provide the current total of votes.

Update application layer indexes

adhocracy implements an sqlalchemy Mapperextension with hooks.HookExtension that provides hook method to sqlalchemy that will be called before and after insert, update and delete operations for every model instance that is part of a commit. To determinate what to do it will inspect the model instance for fitting hook methods.

The asyncronous system roughly works like this:

  • hooks defines a list of event hook methods names that are also used as event identifiers (:const:.PREINSERT, :const:.PREDELETE, :const:.PREUPDATE, :const:.POSTINSERT, :const:.POSTDELETE, :const:.POSTUPDATE)
  • All model classes defined in adhocracy.models.refs.TYPES are patched by init_queue_hooks(). A function that posts a message message to the job queue (_handle_event) is patched in as all the method names listed above. The post to the job queue contains the the entity (model class) and the event identifier.
  • A number of callback functions is registered by adhocracy.lib.democracy.init_democracy() with the help of register_queue_callback() in the hooks REGISTRY
  • Everytime one of the patched models is inserted, updated, or deleted, a generic job is inserted into the jobqueue that contains the changed model instance and the event identifier.
  • The background thread (paster background <ini-file>) picks up the jobs and calls handle_queue_message() which calls all registered callbacks.

To have indexing and searching working propperly you need:

  • a working rabbitmq
  • a working solr
  • a running background process to process the jobs pushed into the rabbitmq job queue (paster background <ini-file>)

Read the install documentation for setup information.

Authentication and Permissions