Previous topic

libearth.session — Isolate data from other installations

Next topic

libearth.subscribe — Subscription list

This Page

libearth.stage — Staging updates and transactions

Stage is a similar concept to Git’s one. It’s a unit of updates, so every change to the repository should be done through a stage.

It also does more than Git’s stage: Route. Routing system hide how document should be stored in the repository, and provides the natural object-mapping interface instead.

Stage also provides transactions. All operations on staged documents should be done within a transaction. You can open and close a transaction using with statement e.g.:

with stage:
    subs = stage.subscriptions
    stage.subscriptions = some_operation(subs)

Transaction will merge all simultaneous updates if there are multiple updates when it’s committed. You can easily achieve thread safety using transactions.

Note that it however doesn’t guarantee data integrity between multiple processes, so you have to use different session ids when there are multiple processes.

class libearth.stage.BaseStage(session, repository)

Base stage class that routes nothing yet. It should be inherited to route document types. See also Route class.

It’s a context manager, which is possible to be passed to with statement. The context maintains a transaction, that is required for all operations related to the stage:

with stage:
    v = stage.some_value
    stage.some_value =  operate(v)

If any ongoing transaction is not present while the operation requires it, it will raise TransactionError.

  • session (Session) – the current session to stage
  • repository (Repository) – the repository to stage

(collections.Sequence) The repository key of the directory where session list are stored.


Get the current ongoing transaction. If any transaction is not begun yet, it raises TransactionError.

Returns:the dirty buffer that should be written when the transaction is committed
Return type:DirtyBuffer
Raises TransactionError:
 if not any transaction is not begun yet
read(document_type, key)

Read a document of document_type by the given key in the staged repository.

  • document_type (type) – the type of document to read. it has to be a subclass of MergeableDocumentElement
  • key (collections.Seqeuence) – the key to find the document in the repository

found document instance

Return type:


Raises libearth.repository.RepositoryKeyError:

when the key cannot be found


This method is intended to be internal. Use routed properties rather than this. See also Route.

repository = None

(Repository) The staged repository.

session = None

(Session) The current session of the stage.


(collections.Set) List all sessions associated to the repository. It includes the session of the current stage.


Touch the latest staged time of the current session into the repository.


This method is intended to be internal.

transactions = None

(collections.MutableMapping) Ongoing transactions. Keys are the context identifier (that get_current_context_id() returns), and values are pairs of the DirtyBuffer that should be written when the transaction is committed, and stack information.

write(key, document, merge=True)

Save the document to the key in the staged repository.

  • key (collections.Sequence) – the key to be stored
  • document (MergeableDocumentElement) – the document to save
  • merge (bool) – merge with the previous revision of the same session (if exists). True by default

actually written document

Return type:



This method is intended to be internal. Use routed properties rather than this. See also Route.

class libearth.stage.Directory(stage, document_type, key_spec, indices, key)

Mapping object which represents hierarchy of routed key path.

  • stage (BaseStage) – the current stage
  • document_type (type) – the same to Route.document_type
  • key_spec (collections.Sequence) – the same to Route.key_spec value
  • indices (collections.Sequence) – the upper indices that are already completed
  • key (collections.Sequence) – the upper key that are already completed


The constructor is intended to be internal, so don’t instantiate it directory. Use Route instead.

class libearth.stage.DirtyBuffer(repository, lock)

Memory-buffered proxy for the repository. It’s used for transaction buffer which maintains updates to be written until the ongoing transaction is committed.

  • repository (Repository) – the bare repository where the buffer will flush() to
  • lock (threading.RLock) – the common lock shared between dirty buffers of the same stage


This class is intended to be internal.

flush(_dictionary=None, _key=None)

Flush all buffered updates to the repository.

repository = None

(Repository) The bare repository where the buffer will flush() to.

class libearth.stage.Route(document_type, key_spec)

Descriptor that routes a document_type to a particular key path pattern in the repository.

key_spec could contain some format strings. Format strings can take a keyword (session) and zero or more positional arguments.

For example, if you route a document type without any positional arguments in key_spec format:

class Stage(BaseStage):
    '''Stage example.'''

    metadata = Route(
        ['metadata', '{session.identifier}.xml']

Stage instance will has a metadata attribute that simply holds Metadata document instance (in the example):

>>> stage.metadata  # ['metadata', 'session-id.xml']
<Metadata ...>

If you route something with one or more positional arguments in key_spec format, then it works in some different way:

class Stage(BaseStage):
    '''Stage example.'''

    seating_chart = Route(
        ['students', 'col-{0}', 'row-{1}', '{session.identifier}.xml']

In the above routing, two positional arguments were used. It means that the seating_chart property will return two-dimensional mapping object (Directory):

>>> stage.seating_chart  # ['students', ...]
< ['students']>
>>> list(stage.seating_chart)
['A', 'B', 'C', 'D']
>>> b = stage.seating_chart['B']  # ['students', 'col-B', ...]
< ['students', 'col-B']>
>>> list(stage.seating_chart['B'])
['1', '2', '3', '4', '5', '6']
>>> stage.seating_chart['B']['6']  \
... # ['students', 'col-B', 'row-6', 'session-id.xml']
<Student B6>
  • document_type (type) – the type of document to route. it has to be a subclass of MergeableDocumentElement
  • key_spec (collections.Sequence) – the repository key pattern that might contain some format strings e.g. ['docs', '{0}', '{session.identifier}.xml']`. positional values are used for directory indices (if present), and ``session keyword value is used for identifying sessions
document_type = None

(type) The type of the routed document. It is a subtype of MergeableDocumentElement.

key_spec = None

(collections.Sequence) The repository key pattern that might contain some format strings.

class libearth.stage.Stage(session, repository)

Staged documents of Earth Reader.


(collections.MutableMapping) The map of feed ids to Feed objects.


(SubscriptionList) The set of subscriptions.

exception libearth.stage.TransactionError

The error that rises if there’s no ongoing transaction while it’s needed to update the stage, or if there’s already begun ongoing transaction when the new transaction get tried to begin.


Compile a format_string to regular expression pattern. For example, 'string{0}like{1}this{{2}}' will be compiled to /^string(.*?)like(.*?)this\{2\}$/.

Parameters:format_string (str) – format string to compile
Returns:compiled pattern object
Return type:re.RegexObject

Identifies which context it is (greenlet, stackless, or thread).

Returns:the identifier of the current context
Fork me on GitHub