NOTE: This document is a work in progress. You can submit an issue if you find a problem or have a suggestion. The source for this documentation is in our repository at locutus/docs/src. We welcome pull requests.
Introduction
What is Locutus?
Locutus is a global, observable, decentralized key-value store. Values are arbitrary blocks of data, called the contract's "state." Keys are cryptographic contracts that specify:
- Whether a given state is permitted under this contract
- How the state can be modified over time
- How two valid states can be merged
- How to efficiently synchronize a contract's state between peers
Locutus is a true decentralized peer-to-peer network, and is robust and scalable, through its use of a small-world network.
Applications on Locutus can be built in any language that is supported by web browsers, including JavaScript and WebAssembly. These applications are distributed over Locutus and can create, retrieve, and update contracts through a WebSocket connection to the local Locutus peer.
Writing a Contract
Locutus contracts can be written in any language that compiles to WebAssembly. This includes Rust, and AssemblyScript, among many others.
A contract consists of the WebAssembly code itself and its "parameters," which are additional data like cryptographic keys. This makes it easy to configure contracts without having to recompile them.
A contract can be retrieved using a key, which is a cryptographic hash derived from the contract's WebAssembly code together with its parameters.
Small world routing
Locutus peers self-organize into a small-world network to allow contracts to be found in a fast, scalable, and decentralized way.
Every peer in Locutus is assigned a number between 0 and 1 when it first joins the network, this is the peer's "location". The small world network topology ensures that peers with similar locations are more likely to be connected.
Contracts also have a location, which is derived from the contract's key. Peers cache contracts close to their locations.
Writing an Application
Creating a decentralized application on Locutus is very similar to creating a normal web application. You can use familiar frameworks like React, Bootstrap, Angular, Vue.js, and so on.
The main difference is that instead of connecting to a REST API running on a server, the web application connects to the Locutus peer running on the local computer through a WebSocket connection.
Through this the application can:
- Create new contracts and their associated state
- Retrieve contracts and their state
- Modify contract state when permitted by the contract
How to use Contracts
Contracts are extremely flexible. they can be used to create decentralized data structures like hashmaps, inverted indices for keyword search, or efficient buffers for streaming audio and video.
Component Ecosystem
Applications in Locutus don't need to be built from scratch, they can be built on top of components provided by us or others.
Reputation system
Allows users to build up reputation over time based on feedback from those they interact with. Think of the feedback system in services like Uber, but with Locutus it will be entirely decentralized and cryptographically secure. It can be used for things like spam prevention (with IM and email), or fraud prevention (with an online store).
This is conceptually similar to Freenet's Web of Trust plugin.
Arbiters
Arbiters are trusted services that can perform tasks and authenticate the results, such as verifying that a contract had a particular state at a given time, or that external blockchains (Bitcoin, Ethereum, Solana etc) contain specific transactions. Trust is achieved through the reputation system.
NOTE: This document is a work in progress. You can submit an issue if you find a problem or have a suggestion. The source for this documentation is in our repository at locutus/docs/src. We welcome pull requests.
Development Guide
This guide will walk through how to develop a simple distributed web application using Locutus. To do that, we'll be using Rust for the contracts themselves and Typescript for developing the web application.
At the time of writing (September 2022) the Locutus network is not yet active. We've published this guide so that people can experiment with building and running Locutus applications locally, and provide feedback.
You can see some examples of working applications and contracts in the apps
directory of the locutus repository, e.g.:
- freenet-microblogging (WIP)
Installation
Development for Locutus requires installing some dependencies:
1. Rust & Cargo
Locutus is developed in Rust, on Linux/Mac this will install Rust and its build tool Cargo which Locutus also requires:
$ curl https://sh.rustup.rs -sSf | sh
2. Locutus Dev Tool (LTD)
Once you have a working installation of Cargo you can install the Locutus dev tools:
$ cargo install locutus
This command will install ldt
(Locutus Dev Tool) and a working node that can be used for local development.
2.1 Usage
You can find more information about the available commands by executing ldt
with the --help
argument:
$ ldt --help
Locutus Development Tool 0.0.2
The Freenet Project Inc.
USAGE:
ldt [DATA_DIR] <SUBCOMMAND>
ARGS:
<DATA_DIR> Overrides the default data directory where Locutus files are stored
OPTIONS:
-h, --help Print help information
-V, --version Print version information
SUBCOMMANDS:
build Builds and packages a contract
execute Node CLI
help Print this message or the help of the given subcommand(s)
new Create a new Locutus contract and/or app
publish Publishes a new contract to the network
run-local A CLI utility for testing out contracts against a Locutus local node
Creating a new contract
You can create a new contract skeleton by executing the new
command with ldt
. Two contract types are supported currently by the tool, regular contracts, and web application container contracts. Currently, the following technological stacks are supported (more to be added in the future):
- Regular contracts:
- Rust (default)
- Web applications:
- Container development:
- Rust (default)
- Web/state development:
- Typescript. (default: using npm and webpack)
- JavaScript.
- Rust (WIP).
- Container development:
We will need to create a directory that will hold our web app and initialize it:
$ mkdir -p my-app/web
$ mkdir -p my-app/backend
$ cd my-app/web
$ ldt new web-app
will create the skeleton for a web application and its container contract for Locutus ready for development at the my-app/web
directory.
Making a container contract
The first thing that we need is to write the code for our container contract. This contract's role is to contain the web application code itself, allowing it to be distributed over Locutus.
The new
command has created the source ready to be modified for us, in your favorite editor open the following file:
$ ./container/src/lib.rs
In this case, and for simplicity's sake, the contract won't be performing any functions, but in a realistic scenario, this contract would include some basic security functionality like verifying that whoever is trying to update the contract has the required credentials.
To make our contract unique so it doesn't collide with an existing contract, we can generate a random signature that will be embedded with the contract.
For example in the lib.rs
file we will write the following:
use locutus_stdlib::prelude::*;
pub const RANDOM_SIGNATURE: &[u8] = &[6, 8, 2, 5, 6, 9, 9, 10];
struct Contract;
#[contract]
impl ContractInterface for Contract {
fn validate_state(
_parameters: Parameters<'static>,
_state: State<'static>,
_related: RelatedContracts<'static>,
) -> Result<ValidateResult, ContractError> {
unimplemented!()
}
fn validate_delta(
_parameters: Parameters<'static>,
_delta: StateDelta<'static>,
) -> Result<bool, ContractError> {
unimplemented!()
}
fn update_state(
_parameters: Parameters<'static>,
_state: State<'static>,
_data: Vec<UpdateData<'static>>,
) -> Result<UpdateModification<'static>, ContractError> {
unimplemented!()
}
fn summarize_state(
_parameters: Parameters<'static>,
_state: State<'static>,
) -> Result<StateSummary<'static>, ContractError> {
unimplemented!()
}
fn get_state_delta(
_parameters: Parameters<'static>,
_state: State<'static>,
_summary: StateSummary<'static>,
) -> Result<StateDelta<'static>, ContractError> {
unimplemented!()
}
}
That's a lot of information, let's unpack it:
use locutus_stdlib::prelude::*;
Here we are importing the necessary types and traits to write a Locutus contract successfully using Rust.
pub const RANDOM_SIGNATURE: &[u8] = &[6, 8, 2, 5, 6, 9, 9, 10];
This will make our contract unique, notice the pub
qualifier so the compiler doesn't remove this constant because is unused and is included in the output of the compiler.
struct Contract;
#[contract]
impl ContractInterface for Contract {
...
}
Here we create a new type, Contract
for which we will be implementing the ContractInterface
trait. To know more details about the functionality of a contract, delve into the details of the contract interface.
Notice the #[contract]
macro call, this will generate the necessary code for the WASM runtime to interact with your contract ergonomically and safely. Trying to use this macro more than once in the same module will result in a compiler error, and only the code generated at the top-level module will be used by the runtime.
As a rule of thumb, one contract will require implementing the `ContractInterface`` exactly once.
Creating a web application
Now we have a working example of a contract, but our contract is an empty shell, which does not do anything yet. To change this, we will start developing our web application.
To do that, we can go and modify the code of the contract state, which in this case is the web application. Locutus offers a standard library (stdlib) that can be used with Typescript/JavaScript to facilitate the development of web applications and interfacing with your local node, so we will make our package.json
contains the dependency:
{
"dependencies": {
"@locutus/locutus-stdlib": "0.0.2"
}
}
Open the file src/index.ts
in a code editor and you can start developing the web application.
An important thing to notice is that our application will need to interface with our local node, the entry point for our machine to communicate with other nodes in the network. The stdlib offers a series of facilities in which you will be able to communicate with the network ergonomically.
Here is an example of how you could write your application to interact with the node:
import { LocutusWsApi } from "@locutus/locutus-stdlib/webSocketInterface";
const handler = {
onPut: (_response: PutResponse) => {},
onGet: (_response: GetResponse) => {},
onUpdate: (_up: UpdateResponse) => {},
onUpdateNotification: (_notif: UpdateNotification) => {},
onErr: (err: HostError) => {},
onOpen: () => {},
};
const API_URL = new URL(`ws://${location.host}/contract/command/`);
const locutusApi = new LocutusWsApi(API_URL, handler);
const CONTRACT = "DCBi7HNZC3QUZRiZLFZDiEduv5KHgZfgBk8WwTiheGq1";
async function loadState() {
let getRequest = {
key: Key.fromSpec(CONTRACT),
fetch_contract: false,
};
await locutusApi.get(getRequest);
}
Let's unpack this code:
const handler = {
onPut: (_response: PutResponse) => {},
onGet: (_response: GetResponse) => {},
onUpdate: (_up: UpdateResponse) => {},
onUpdateNotification: (_notif: UpdateNotification) => {},
onErr: (err: HostError) => {},
onOpen: () => {},
};
const API_URL = new URL(`ws://${location.host}/contract/command/`);
const locutusApi = new LocutusWsApi(API_URL, handler);
This type provides a convenient interface to the WebSocket API. It receives an object which handles the different responses from the node via callbacks. Here you would be able to interact with DOM objects or other parts of your code.
const CONTRACT = "DCBi7HNZC3QUZRiZLFZDiEduv5KHgZfgBk8WwTiheGq1";
async function loadState() {
let getRequest = {
key: Key.fromSpec(CONTRACT),
fetch_contract: false,
};
await locutusApi.get(getRequest);
}
Here we use the API wrapper to make a get request (which requires a key and specifies if we require fetching the contract code or not) to get the state for a contract with the given address. The response from the node will be directed to the onGet
callback. You can use any other methods available in the API to interact with the node.
Writing the backend for our web application
In the creating a new contract section we described the contract interface, but we were using it to write a simple container contract that won't be doing anything in practice, just carrying around the front end of your application. The core logic of the application, and a back end where we will be storing all the information, requires another contract. So we will create a new contract in a different directory for it:
$ cd ../backend
$ ldt new contract
This will create a regular contract, and we will need to implement the interface on a type that will handle our contract code. For example:
use locutus_stdlib::prelude::*;
pub const RANDOM_SIGNATURE: &[u8] = &[6, 8, 2, 5, 6, 9, 9, 10];
struct Contract;
struct Posts(...)
impl Posts {
fn add_post(&mut self, post: Post) { ... }
}
struct Post(...)
#[contract]
impl ContractInterface for Contract {
fn update_state(
_parameters: Parameters<'static>,
state: State<'static>,
data: Vec<UpdateData<'static>>,
) -> Result<UpdateModification<'static>, ContractError> {
let mut posts: Posts = serde_json::from_slice(&state).map_err(|_| ContractError::InvalidState)?;
if let Some(UpdateData::Delta(delta)) = data.pop() {
let new_post: Posts = serde_json::from_slice(&delta).map_err(|_| ContractError::InvalidState);
posts.add_post(new_post)?;
} else {
Err(ContractError::InvalidUpdate)
}
Ok(UpdateModification::valid(posts.into()))
}
...
}
In this simple example, we convert a new incoming delta to a post and the state to a list of posts we maintain, and we append the post to the list of posts. After that, we convert back the posts list to an state and return that.
If we subscribe to the contract changes or our web app, we will receive a notification with the updates after they are successful, and we will be able to render them in our browser. We can do that, for example, using the API:
function getUpdateNotification(notification: UpdateNotification) {
let decoder = new TextDecoder("utf8");
let updatesBox = DOCUMENT.getElementById("updates") as HTMLPreElement;
let newUpdate = decoder.decode(Uint8Array.from(notification.update));
let newUpdateJson = JSON.parse(newUpdate);
updatesBox.textContent = updatesBox.textContent + newUpdateJson;
}
Building and packaging a contract
Now that we have the front end and the back end of our web app, we can package the contracts and run them in the node to test them out.
In order to do that, we can again use the development tool to help us out with the process. But before doing that, let's take a look at the manifesto format and understand the different parameters that allow us to specify how this contract should be compiled (check the manifest details for more information). In the web app directory, we have a locutus.toml
file which contains something similar to:
[contract]
type = "webapp"
lang = "rust"
...
[webapp.state-sources]
source_dirs = ["dist"]
This means that the dist
directory will be packaged as the initial state for the webapp (that is the code the browser will be interpreting and in the end, rendering).
If we add the following keys to the manifesto:
[webapp.dependencies]
posts = { path = "../backend" }
The WASM code from the backend
contract will be embedded in our web application state, so it will be accessible as a resource just via the local HTTP gateway access and then we can re-use it for publishing additional contracts.
Currently, wep applications follow a standarized build procedure in case you use ldt
and assumptions about your system. For example, in the case of a type = "webapp"
contract, if nothing is specified, it will assume you have npm
and the tsc
compiler available at the directory level, as well as webpack
installed.
This means that you have installed either globally or at the directory level, e.g. globally:
$ npm install -g typescript
$ npm install -g webpack
$ npm install -g webpack-cli
or locally (make sure your package.json
file has the required dependencies):
$ npm install typescript --save-dev
$ npm install webpack --save-dev
$ npm install webpack-cli --save-dev
If, however, you prefer to follow a different workflow, you can write your own by enabling/disabling certain parameters or using a blank template. For example:
[contract]
lang = "rust"
[state]
files = ["my_packaged_web.tar.xz"]
Would just delegate the work of building the packaged tar
to the developer. Or:
[contract]
type = "webapp"
lang = "rust"
[webapp]
lang = "typescript"
[webapp.typescript]
webpack = false
would disable usign webpack
at all.
Now that we understand the details, and after making any necessary changes, in each contract directory we run the following commands:
$ ldt build
This command will read your contract manifest file (locutus.toml
) and take care of building the contract and packaging it, ready for the node and the network to consume it.
Under the ./build/locutus
directory, you will see both a *.wasm
file, which is the contract file, and contract-state
, in case it applies, which is the initial state that will be uploaded when initially putting the contract.
Web applications can access the code of backend contracts directly in their applications and put new contracts (that is, assigning a new location for the code, plus any parameters that may be generated dynamically by the web app, and the initial state for that combination of contract code + parameters) dynamically.
Let's take a look at the manifest for our web app container contract:
Testing out contracts in the local node
Once we have all our contracts sorted and ready for testing, we can do this in local mode in our node. For this the node must be running, we can make sure that is running by running the following command as a background process or in another terminal; since we have installed it:
$ locutus-node
You should see some logs printed via the stdout of the process indicating that the node HTTP gateway is running.
Once the HTTP gateway is running, we are ready to publish the contracts to our local Locutus node:
$ cd ../backend && ldt publish --code="./build/locutus/backend.wasm" --state="./build/locutus/contract-state"
$ cd ../web && ldt publish --code="./build/locutus/web.wasm" --state="./build/locutus/contract-state"
In this case, we're not passing any parameters (so our parameters will be an empty byte array), and we are passing an initial state without the current backend contract. In typical use, both the parameters would have meaningful data, and the backend contract may be dynamically generated from the app and published from there.
Once this is done, you can start your app just by pointing to it in the browser: http://127.0.0.1:50509/contract/web/<CONTRACT KEY>
For example http://127.0.0.1:50509/contract/web/CYXGxQGSmcd5xHRJNQygPwmUJsWS2njh3pdVjfVz9EV/
Iteratively you can repeat this process of modifying, and publishing locally until you are confident with the results and ready to publish your application.
Since the web is part of your state, you are always able to update it, pointing to new contracts, and evolving it over time.
Limitations
-
Publishing to the Locutus network is not yet supported.
-
Only Rust is currently supported for contract development, but we'll support more languages like AssemblyScript in the future.
-
Binaries for all the required tools are not yet available, they must be compiled from source
The Manifest Format
The locutus.toml
file for each application/contract is called its manifest. It is written in the TOML format. Manifest files consist of the following sections:
- [contract] — Defines a contract.
- type — Contract type.
- lang — Contract source language.
- output_dir — Output path for build artifacts.
- [webapp] — Configuration for web application containers.
- [state] — Optionally seed a state.
The [contract]
section
The type
field
[contract]
...
type = "webapp"
The type of the contract being packaged. Currently the following types are supported:
standard
, the default type, it can be ellided. This is just a standard contract.webapp
, a web app container contract. Additionally to the container contract the web application source will be compiled and packaged as the state of the contract.
The lang
field
[contract]
...
lang = "rust"
The programming language in which the contract is written. If specified the build tool will compile the contract. Currently only Rust is supported.
The output_dir
field
[contract]
...
output_dir = "./other/output/dir/"
An optional path to the output directory for the build artifacts. If not set the output will be written to the relative directory ./build/locutus
from the manifest file directory.
The [webapp]
section
An optional section, only specified in case of webapp
contracts.
The lang
field
[webapp]
...
lang = "typescript"
The programming language in which the web application is written. Currently the following languages are supported:
The metadata
field
[webapp]
...
metadata = "/path/to/metadata/file"
An optional path to the metadata for the webapp, if not set the metadata will be empty.
The [webapp.typescript]
options section
Optional section specified in case of the the typescript
lang.
The following fields are supported:
[webapp.typescript]
webpack = true
webpack
— if set webpack will be used when packaging the contract state.
The [webapp.javascript]
options section
Optional section specified in case of the the javascript
lang.
The following fields are supported:
[webapp.javascript]
webpack = true
webpack
— if set webpack will be used when packaging the contract state.
The [webapp.state-sources]
options section
[webapp.state-sources]
source_dirs = ["path/to/sources"]
files = ["*/src/**.js"]
Specifies the sources for the state of the contract, this will be later on unpacked and accessible at the HTTP gateway from the Locutus node. Includes any web sources (like .html or .js files). The source_dirs
field is a comma separated array of directories that should be appended to the root of the state, the files
field is a comma separated array of glob compatible patterns to files that will be appendeded to the state.
At least one of source_dirs
or files
fields are required.
The [webapp.dependencies]
section
[webapp.dependencies]
...
posts = { path = "../contracts/posts" }
An optional list of contract dependencies that will be embedded and available in the state of the contract.
Each entry under this entry represents an alias to the contract code, it must include a path
field that specifies the relative location of the dependency from this manifesto directory.
If dependencies are specified they will be compiled and appended to the contract state, under the contracts
directory, and as such, become available from the HTTP gateway. A dependencies.json
file will be automatically generated and placed under such directory that maps the aliases to the file and hash of the code generated for the dependencies.
In this way the "parent" container contract can use those contracts code to put/update new values through the websocket API in an ergonomic manner.
The [state]
section
[state]
files = ["*/src/**.js"]
An optional section for standard contracts in case they want to seed an state initially, it will take a single file and make it available at the build directory.
Building with Docker Images
Prerequisites
Make sure docker is installed and working, and has the docker compose
command.
Contract DB Storage
The docker image stores its data at /root/.local/share/locutus
inside the
container. This is mapped to /tmp/locutus-docker
outside the container.
Build the base docker image of Locutus
All the docker related files are in the docker
subdirectory.
Requires that Docker be installed and working. Then, in the root directory of the repo:
To build the docker locutus container:
cd docker
docker compose build
Running Locutus Node from the docker image
Note: Currently the node will not pick up new contracts when they are published. Make sure the node is stopped and re-started after new contracts are added.
docker compose up
Running the ldt
tool from the docker image
There is a shell script in the docker
sub directory which makes running ldt
from inside the container against source held outside the container easier. It
behaves just like the ldt
tool, except as stated below.
Getting help from ldt
/location/of/locutus/docker/ldt.sh --help
Building Contracts
To BUILD a contract, we need to define 1 or 2 env vars:
PROJECT_SRC_DIR
= Root of the Project being build and defaults topwd
so if you are in your project root, no need to set it.CONTRACT_SRC_DIR
= Relative DIR under PROJECT_SRC_DIR to the Contract to build. eg,./web
would build a contract in theweb
subdirectory of thePROJECT_SRC_DIR
. Note: This MUST be a subdirectory.
eg (in the root of the project):
CONTRACT_SRC_DIR=./web /location/of/locutus/docker/ldt.sh build
Publishing Contracts
From the base directory of the contract project.
/location/of/locutus/docker/ldt.sh publish --code target/wasm32-unknown-unknown/release/freenet_microblogging_web.wasm --state web/build/locutus/contract-state
Contract Interface
Terms
- Contract State - data associated with a contract that can be retrieved by Applications and Components.
- Delta - Represents a modification to some state - similar to a diff in source code
- Parameters - Data that forms part of a contract along with the WebAssembly code
- State Summary - A compact summary of a contract's state that can be used to create a delta
Interface
Locutus contracts must implement the contract interface from crates/locutus-stdlib/src/interface.rs:
{{#include ../../crates/locutus-stdlib/src/interface.rs:contractifce}}
Parameters
, State
, and StateDelta
are all wrappers around simple [u8]
byte arrays for maximum efficiency and flexibility.
Contract Interaction
In the (hopefully) near future we'll be adding the ability for contracts to read each other's state while validating and updating their own, see issue #167 for the latest on this.
Glossary
Application
Software that uses Locutus as a back-end. This includes native software distributed independenly of Locutus but which uses Locutus as a back-end (perhaps bundling Locutus), and web applications that are distributed over Locutus and run in a web browser.
Contract
A contract is WebAssembly code with associated data like the contract state. The role of the contract is to determine:
- Is the state valid for this contract?
- Under what circumstances can the state be modified or updated? (see Delta)
- How can two valid states be merged to produce a third valid state?
Container Contract
A contract that contains an application or component as state, accessed through the web proxy.
For example, if the contract id is 6C2KyVMtqw8D5wWa8Y7e14VmDNXXXv9CQ3m44PC9YbD2
then visiting http://localhost:PORT/contract/web/6C2KyVMtqw8D5wWa8Y7e14VmDNXXXv9CQ3m44PC9YbD2
will cause the application/component to be retrieved from Locutus, decompressed, and sent to the browser where it can execute.
Contract State
Data associated with a contract that can be retrieved by Applications and Components. For efficiency and flexibility, contract state is represented as a simple [u8]
byte array.
Delta
Represents a modification to some state - similar to a diff in source code. The exact format of a delta is determined by the contract. A contract will determine whether a delta is valid - perhaps by verifying it is signed by someone authorized to modify the contract state. A delta may be created in response to a State Summary as part of the State Synchronization mechanism.
Parameters
Data that forms part of a contract along with the WebAssembly code. This is supplied to the contract as a parameter to the contract's functions. Parameters are typically be used to configure a contract, much like the parameters of a constructor function.
For example, the parameters could contain a hash of the state itself. The contract would then use it to verify that the state hashes to that value. This would create a contract that is guaranteed to contain the same state. In the original Freenet, this was known as a content hash key.
State Summary
Given a contract state, this is a small piece of data that can be used to determine a delta between two contracts as part of the state synchronization mechanism. The format of a state summary is determined by the state's contract.
State Synchronization
Given two valid states for a contract, the state synchronization mechanism allows the states to be efficiently merged over the network to ensure eventual consistency.
Web Application
Software built on Locutus and distributed through Locutus.
Applications run in the browser and can be built with tools like React, TypeScript, and Vue.js. An application may use multiple components and contracts.
Applications are compressed and distributed via a container contract.