Using TOZ Store

Data Structure

Next, you need to determine what structure would be best for your data records. Data records within TozStore are composed of three main fields: the record type, the un-encrypted metadata, and the encrypted data object. These three fields are what you provide when writing records, with unencrypted meta being optional.

The record type is used to group records together for the purpose of sharing. TozStore's sharing process allows you to share record types between clients. Access control to specific records is all or nothing based off of record types, so be sure to keep that in mind when structuring your data.

The un-encrypted metadata is an optional field that can be used to search for your records. Some intrinsic meta is generated when you create your record such as the record id (UUID), the time it was created, the last time it was modified, etc. You may want more friendly ways to retrieve your data in the future, and this is where metadata comes in (remember the data is encrypted and only you can decrypt it). The metadata is stored in plain-text giving you the ability to search upon them with the TozStore SDK's. Keep in mind these are plain-text and should not be used to store sensitive data.

Finally, the encrypted data object is where your sensitive data lives. The structure of this object is a string to string key-value store, where the value is encrypted and the key is un-encrypted. Here is a more general overview of how TozStore works.

my_data = { 'some_plain_key':'encrypted_sensitive_value', ... }

Below is a simple example showing you how to write a record in TozStore.

  //data will be encrypted as the write operation moves through the SDK
  let data = {
    'first_name': 'Jon',
    'last_name': 'Snow',
    'phone': '555-555-1212',
  }
  
  //metadata is left in plain text to enable queries on your data
  let metadata = {
    'house' : 'Stark'
  }
  
  //record types help to group your data together
  let record_type = 'contact'
  let record = await client.write(record_type, data, metadata)

Note that your data and metadata objects should be flat. Treat TozStore as a string to string key value store.

Sharing

Sharing in TozStore is how you grant access to your data to other clients. These can be clients within your account or cross account. Sharing is great since it retains control of the data within your account. You can revoke access at any time and this does not duplicate the data itself. All access to records is handled through cryptographic controls. Here's an example of how sharing data works with TozStore.

Python

import e3db
from e3db.types import Search

# Instantiate your client
client = e3db.Client(config())

# For this example, instantiate a third party client
third_party = e3db.Client(third_party_config())

record_type = 'share_me'
client.share(record_type, third_party.client_id)
client.write(record_type, {'test':'data'})

# Obtain records written by client from the third_party client
results = third_party.search(Search(include_all_writers=True) \
                                .match(record_types=[record_type]))

for r in results:
   print(r.to_json())

Searching (Beta)

Within TozStore your data is encrypted and can't be searched by us, but the meta you provide is searchable. We support a number of search options on this meta that allow you to parse through your data without exposing the secure information. To recap record creation, when you write a record you have the option to provide un-encrypted meta that is stored along with your data. This meta is accepted in format of a string to string map, like the format of encrypted data. This plain meta map can be used for searching, but the key and values of the mapping are also expanded into additional searchable fields.

Python

import e3db
from e3db.types import Search

# Instantiate your client
client = e3db.Client(config())

apricot_data = {"recipe": "one ripe apricot"}
apricot_plain = {"flavor":"apricot"}
client.write("jam", apricot_data, apricot_plain)

strawberry_data = {"recipe": "many strawberries"}
strawberry_plain = {"flavor":"strawberry"}
client.write("jam", strawberry_data, strawberry_plain )

# Get our strawberry record
strawberry_results = client.search(Search().match(plain=strawberry_plain))
for jam in strawberry_results:
   print(jam.to_json())
   
# match records where record_types == "jam" AND values == "apricot"
apricot_query = Search().match(condition="AND", 
                              record_types=["jam"], 
                              values=["apricot"])
apricot_results = client.search(apricot_query)
for jam in apricot_results:
   print(jam.to_json())

The examples above highlights some simple searches on TozStore. Currently, Search is only supported in the Python SDK and you can find more complex examples that include time filters, fuzzy, regex, and other meta fields on the Github page.