Got a question that the wiki doesn't answer? Ask on the forum (preferred), or join us on IRC.

About the ads

CommandHelper/Persistance Network Integration

From SK's Wiki
Jump to: navigation, search

If you are writing an external system that integrates with MethodScript's Persistance Network, there are several implementation details that you must be aware of. Depending on your level of integration, it may be easiest to use a particular option to gather data from the system; if you're using a fully external system that isn't written in Java, you can use the cmdline tools to gather information about the system, edit/add keys, with a JSON based response system. If you're writing something in Java, it may be easiest to link to the jar, and use the built in methods for managing the system. If you're writing a fully standalone system that doesn't use the cmdline tools to edit the data, you'll need to be aware of the specific implementation details of each data source, as well as using conventions and standards to ensure consistent usage of the system.



Both the Java API and cmdline API support all the same core operations, but with the Java API, you are able to recieve POJOs instead of having to deal with parsing the return values for particular functions, and convenience methods exist for gather more complicated data in a pre-parsed format. All functions work the same in either API, but the differences between the two interfaces is noted below, the main difference being that the cmdline API always returns the most reasonable "toString'd" version of an output. All the operations below return an implied String, and take Strings as arguments. The available operations are:

getDataSourceURI(keyName) Returns the URI for the specified key, based on the network's filter settings.
notifyChange(dataSourceURI) Notifies the network architecture that changes to this data source have occured, and for non-transient data sources, this will cause a refresh of the data in compliant systems.
lastChange(dataSourceURI) Returns the last time that this data source was modified, which should be compared to the timestamp of when your last model refresh occured. If this is stale, it should refresh the model from the source.
getKeyValue(keyName) Returns the value for the specified key
getKeyValues(namespace) Returns a list of all keys and values within this namespace
setKeyValue(keyName, value) Sets the value of a key
getKeyValueURI(dataSourceURI, keyName) Gets the key value from the specified data source, skipping the filters altogether. This is usually not recommended.
setKeyValueURI(dataSourceURI, keyName, value) Sets the key value in the specified data source, skipping the filters altogether. This is usually not recommended.
getAllKeys() Returns a list of all keys in the system (note, slow)
getAllKeysURI(dataSourceURI) Returns a list of all keys in this particular data source

Java API

In addition to the operations listed above, the following operations are also supported only in the Java API:

DataSource getDataSource(String keyName) Returns a fully constructed data source for the specified key, based on the network's filter settings.
DataSource getDataSource(URI uri) Returns a fully constructed data source specified by the URI given.

Additionally, some methods have different return types and different parameter types, but those are noted in the Javadoc.

Cmdline API

To access the operations via cmdline, use the following switches:

--persistance-network Puts the cmdline in persistance network API mode.
--human-readable Optional. If set, the data is returned in a human readable mode, not in a json. This is for reference only, and contains the same information the json would have had.
--operation The name of the operation you wish to perform (the "function name")
--parameters The parameters to send to the operation

Example: java -jar CommandHelper.jar --persistance-network --operation setKeyValue --parameters "" "This is the value to send"

The response will be a JSON with the following fields:

  • responseType - Will be either one of "exception" or "void" or "value". Depending on what this is, the following will be set

If responseType is exception:

  • type - The name of the exception that occured
  • message - The message in this exception
  • data - (Optional) If the exception also has other data, it would be put here

If responseType is value:

  • type - The data type of the value that is being returned, which will be one of: "string", "boolean", "array", "integer", or "float".
  • data - The actual data being returned

If responseType is void, no other parameters will be set.

Data Source Implementation Details

It is very important to note how each data source stores its data. In the Java code, the minimum supported features are having the ability to store a string key->string value pair, where the key is unique. However, for certain data source implementations, this means massaging the data some, to prevent simply having to use flat file storage. For this reason, there are a few reserved characteristics in the persistance framework, to allow for maximum compatibility across all storage types.

Keys and values must be strings, not binary data. Keys are limited to the following signature, with a few caveats: [a-zA-Z0-9_]+(?[\.a-zA-Z0-9_])* A single underscore is reserved due to the inability for some data source implementations to easily actualize a tree structure where a node can contain both children and data. In cases where a node has both data and children, the child "_" represents its data, and the rest of the children represent the actual children. Take this example, where we have two values, and are using the json storage:

  • a.b.c1 = value1
  • a.b.c2 = value2

Stored as a json, this is easy enough:

{"a":{"b":{"c1":"value1", "c2":"value2"}}}

However, consider this case (which is valid in the persistance framework):

  • a.b = value1
  • a.b.c = value2

It is impossible to store this as a json:

{"a":{"b":<what goes here? "value1" or {c:"value2"}? We can't put both>}}

In this case, we store it as such:


An underscore is used, because it will work in all systems currently supported, and if a new system appears that won't support it, it probably wouldn't support the normal keys anyways, and would likely require an inefficient implementation anyways.

In the case of values that are accessible from MethodScript, they are always stored as partial JSONs, that is, individual values are stored as if they are fully serialized into a json value, but are not contained in an object or an array. Normal arrays are stored as [arrays], and associative arrays are stored as {objects}. If values are stored that are not accessible from MethodScript, the only requirement is that they are strings. For the examples below that show actual file contents, assume the following keys are set:

  • a.b: value1
  • a.b.c1: value2
  • a.b.c2: value3

Serialized Persistance

The serialized persistance file is a simple Java serialization of a HashMap<String, String>. The key is stored in a "namespace.key" format.

It is binary data, but as you can see, the keys and values are present if you look at the generated file with a text editor:

loadFactorI^@   thresholdxp?@^@^@^@^@^@^Lw^H^@^@^@^P^@^@^@^Ct^@^Fa.b.c1t^@^Fvalue2t^@^^@^Fvalue1t^@^Fa.b.c2t^@^Fvalue3x


Keys are stored in namespaces according to the YML structure.

This is the above key set, assuming prettyprint is on:

  b: {
    c1: value2,
    c2: value3,
    _: value1

With prettyprint off, you can see that the file size is less, but harder to read:

  b: {c1: value2, c2: value3, _: value1}


Keys are stored in a flat file ini structure, with "=" characters separating keys and values.