libearth.repository
— Repositories¶
Repository
abstracts storage backend e.g. filesystem.
There might be platforms that have no chance to directly access
file system e.g. iOS, and in that case the concept of repository
makes you to store data directly to Dropbox or Google Drive
instead of filesystem. However in the most cases we will simply use
FileSystemRepository
even if data are synchronized using
Dropbox or rsync.
In order to make the repository highly configurable it provides the way
to lookup and instantiate the repository from url. For example,
the following url will load FileSystemRepository
which sets
path
to /home/dahlia/.earthreader/
:
file:///home/dahlia/.earthreader/
For extensibility every repository class has to implement from_url()
and to_url()
methods, and register it as an entry point of
libearth.repositories
group e.g.:
[libearth.repositories]
file = libearth.repository:FileSystemRepository
Note that the entry point name (file
in the above example) becomes
the url scheme to lookup the corresponding repository class
(libearth.repository.FileSystemRepository
in the above example).
-
class
libearth.repository.
FileIterator
(path, buffer_size)¶ Read a file through
Iterator
protocol, with automatic closing of the file when it ends.Parameters: - path (
str
) – the path of file - buffer_size (
numbers.Integral
) – the size of bytes that would be produced each step
- path (
-
exception
libearth.repository.
FileNotFoundError
¶ Raised when a given path does not exist.
-
class
libearth.repository.
FileSystemRepository
(path, mkdir=True, atomic=False)¶ Builtin implementation of
Repository
interface which uses the ordinary file system.Parameters: Raises: - FileNotFoundError – when the
path
doesn’t exist - NotADirectoryError – when the
path
is not a directory
- FileNotFoundError – when the
-
exception
libearth.repository.
NotADirectoryError
¶ Raised when a given path is not a directory.
-
class
libearth.repository.
Repository
¶ Repository interface agnostic to its underlying storage implementation.
Stage
objects can deal with documents to be stored using the interface.Every content in repositories is accessible using keys. It actually abstracts out “filenames” in “file systems”, hence keys share the common concepts with filenames. Keys are hierarchical, like file paths, so consists of multiple sequential strings e.g.
['dir', 'subdir', 'key']
. You canlist()
all subkeys in the upper key as well e.g.:repository.list(['dir', 'subdir'])
-
exists
(key)¶ Return whether the
key
exists or not. It returnsFalse
if it doesn’t exist instead of raisingRepositoryKeyError
.Parameters: key ( collections.Sequence
) – the key to find whether it existsReturns: True
only if the givenkey
exists, orFalse
if not existsReturn type: bool
Note
Every subclass of
Repository
has to overrideexists()
method to implement details.
-
classmethod
from_url
(url)¶ Create a new instance of the repository from the given
url
. It’s used for configuring the repository in plain text e.g.*.ini
.Note
Every subclass of
Repository
has to overridefrom_url()
static/class method to implement details.Parameters: url ( urllib.parse.ParseResult
) – the parsed url tupleReturns: a new repository instance Return type: Repository
Raises: ValueError – when the given url is not invalid
-
list
(key)¶ List all subkeys in the
key
.Parameters: key ( collections.Sequence
) – the incomplete key that might have subkeysReturns: the set of subkeys (set of strings, not set of string lists) Return type: collections.Set
Raises: RepositoryKeyError – the key
cannot be found in the repository, or it’s not a directoryNote
Every subclass of
Repository
has to overridelist()
method to implement details.
-
read
(key)¶ Read the content from the
key
.Parameters: key ( collections.Sequence
) – the key which stores the content to readReturns: byte string chunks Return type: collections.Iterable
Raises: RepositoryKeyError – the key
cannot be found in the repository, or it’s not a fileNote
Every subclass of
Repository
has to overrideread()
method to implement details.
-
to_url
(scheme)¶ Generate a url that
from_url()
can accept. It’s used for configuring the repository in plain text e.g.*.ini
. URLscheme
is determined by caller, and given through argument.Note
Every subclass of
Repository
has to overrideto_url()
method to implement details.Parameters: scheme – a determined url scheme Returns: a url that from_url()
can acceptReturn type: str
-
write
(key, iterable)¶ Write the
iterable
into thekey
.Parameters: - key (
collections.Sequence
) – the key to stores theiterable
- iterable (
collections.Iterable
) – the iterable object yiels chunks of the whole content. every chunk has to be a byte string
Note
Every subclass of
Repository
has to overridewrite()
method to implement details.- key (
-
-
exception
libearth.repository.
RepositoryKeyError
(key, *args, **kwargs)¶ Exception which rises when the requested key cannot be found in the repository.
-
key
= None¶ (
collections.Sequence
) The requested key.
-
-
libearth.repository.
from_url
(url)¶ Load the repository instance from the given configuration
url
.Note
If
setuptools
is not installed it will only supportfile://
scheme andFileSystemRepository
.Parameters: url (
str
,urllib.parse.ParseResult
) – a repository configuration urlReturns: the loaded repository instance
Return type: Raises: - LookupError – when the corresponding repository type to
the given
url
scheme cannot be found - ValueError – when the given
url
is invalid
- LookupError – when the corresponding repository type to
the given