libearth.stage — Staging updates and transactions

Stage is a similar concept to Git’s one. It’s a unit of updates, so every change to the repository should be done through a stage.

It also does more than Git’s stage: Route. Routing system hides how document should be stored in the repository, and provides the natural object-mapping interface instead.

Stage also provides transactions. All operations on staged documents should be done within a transaction. You can open and close a transaction using with statement e.g.:

with stage:
    subs = stage.subscriptions
    stage.subscriptions = some_operation(subs)

Transaction will merge all simultaneous updates if there are multiple updates when it’s committed. You can easily achieve thread safety using transactions.

Note that it however doesn’t guarantee data integrity between multiple processes, so you have to use different session ids when there are multiple processes.

class libearth.stage.BaseStage(session, repository)

Base stage class that routes nothing yet. It should be inherited to route document types. See also Route class.

It’s a context manager, which is possible to be passed to with statement. The context maintains a transaction, that is required for all operations related to the stage:

with stage:
    v = stage.some_value
    stage.some_value =  operate(v)

If any ongoing transaction is not present while the operation requires it, it will raise TransactionError.

Parameters:
  • session (Session) – the current session to stage
  • repository (Repository) – the repository to stage
SESSION_DIRECTORY_KEY = ['.sessions']

(collections.Sequence) The repository key of the directory where session list are stored.

get_current_transaction(pop=False)

Get the current ongoing transaction. If any transaction is not begun yet, it raises TransactionError.

Returns:the dirty buffer that should be written when the transaction is committed
Return type:DirtyBuffer
Raises:TransactionError – if not any transaction is not begun yet
read(document_type, key)

Read a document of document_type by the given key in the staged repository.

Parameters:
Returns:

found document instance

Return type:

MergeableDocumentElement

Raises:

libearth.repository.RepositoryKeyError – when the key cannot be found

Note

This method is intended to be internal. Use routed properties rather than this. See also Route.

repository = None

(Repository) The staged repository.

session = None

(Session) The current session of the stage.

sessions

(collections.Set) List all sessions associated to the repository. It includes the session of the current stage.

touch()

Touch the latest staged time of the current session into the repository.

Note

This method is intended to be internal.

transactions = None

(collections.MutableMapping) Ongoing transactions. Keys are the context identifier (that get_current_context_id() returns), and values are pairs of the DirtyBuffer that should be written when the transaction is committed, and stack information.

write(key, document, merge=True)

Save the document to the key in the staged repository.

Parameters:
  • key (collections.Sequence) – the key to be stored
  • document (MergeableDocumentElement) – the document to save
  • merge (bool) – merge with the previous revision of the same session (if exists). True by default
Returns:

actually written document

Return type:

MergeableDocumentElement

Note

This method is intended to be internal. Use routed properties rather than this. See also Route.

class libearth.stage.Directory(stage, document_type, key_spec, indices, key)

Mapping object which represents hierarchy of routed key path.

Parameters:
  • stage (BaseStage) – the current stage
  • document_type (type) – the same to Route.document_type
  • key_spec (collections.Sequence) – the same to Route.key_spec value
  • indices (collections.Sequence) – the upper indices that are already completed
  • key (collections.Sequence) – the upper key that are already completed

Note

The constructor is intended to be internal, so don’t instantiate it directory. Use Route instead.

class libearth.stage.DirtyBuffer(repository, lock)

Memory-buffered proxy for the repository. It’s used for transaction buffer which maintains updates to be written until the ongoing transaction is committed.

Parameters:
  • repository (Repository) – the bare repository where the buffer will flush() to
  • lock (threading.RLock) – the common lock shared between dirty buffers of the same stage

Note

This class is intended to be internal.

flush(_dictionary=None, _key=None)

Flush all buffered updates to the repository.

repository = None

(Repository) The bare repository where the buffer will flush() to.

class libearth.stage.Route(document_type, key_spec)

Descriptor that routes a document_type to a particular key path pattern in the repository.

key_spec could contain some format strings. Format strings can take a keyword (session) and zero or more positional arguments.

For example, if you route a document type without any positional arguments in key_spec format:

class Stage(BaseStage):
    '''Stage example.'''

    metadata = Route(
        Metadata,
        ['metadata', '{session.identifier}.xml']
    )

Stage instance will has a metadata attribute that simply holds Metadata document instance (in the example):

>>> stage.metadata  # ['metadata', 'session-id.xml']
<Metadata ...>

If you route something with one or more positional arguments in key_spec format, then it works in some different way:

class Stage(BaseStage):
    '''Stage example.'''

    seating_chart = Route(
        Student,
        ['students', 'col-{0}', 'row-{1}', '{session.identifier}.xml']
    )

In the above routing, two positional arguments were used. It means that the seating_chart property will return two-dimensional mapping object (Directory):

>>> stage.seating_chart  # ['students', ...]
<libearth.directory.Directory ['students']>
>>> list(stage.seating_chart)
['A', 'B', 'C', 'D']
>>> b = stage.seating_chart['B']  # ['students', 'col-B', ...]
<libearth.directory.Directory ['students', 'col-B']>
>>> list(stage.seating_chart['B'])
['1', '2', '3', '4', '5', '6']
>>> stage.seating_chart['B']['6']  \
... # ['students', 'col-B', 'row-6', 'session-id.xml']
<Student B6>
Parameters:
  • document_type (type) – the type of document to route. it has to be a subclass of MergeableDocumentElement
  • key_spec (collections.Sequence) – the repository key pattern that might contain some format strings e.g. ['docs', '{0}', '{session.identifier}.xml']`. positional values are used for directory indices (if present), and ``session keyword value is used for identifying sessions
document_type = None

(type) The type of the routed document. It is a subtype of MergeableDocumentElement.

key_spec = None

(collections.Sequence) The repository key pattern that might contain some format strings.

class libearth.stage.Stage(session, repository)

Staged documents of Earth Reader.

feeds

(collections.MutableMapping) The map of feed ids to Feed objects.

subscriptions

(SubscriptionList) The set of subscriptions.

exception libearth.stage.TransactionError

The error that rises if there’s no ongoing transaction while it’s needed to update the stage, or if there’s already begun ongoing transaction when the new transaction get tried to begin.

libearth.stage.compile_format_to_pattern(format_string)

Compile a format_string to regular expression pattern. For example, 'string{0}like{1}this{{2}}' will be compiled to /^string(.*?)like(.*?)this\{2\}$/.

Parameters:format_string (str) – format string to compile
Returns:compiled pattern object
Return type:re.RegexObject
libearth.stage.get_current_context_id()

Identifies which context it is (greenlet, stackless, or thread).

Returns:the identifier of the current context