Official DSNP over Frequency Schemas
Matching DSNP Version: v1.2.0
Use Schemas as Library
Install
npm install @dsnp/frequency-schemas
Use Schema
import { dsnp } from "@dsnp/frequency-schemas";
console.log(dsnp.getSchema("broadcast"));
Get Schema Id from Chain
import { dsnp } from "@dsnp/frequency-schemas";
import { ApiPromise } from "@polkadot/api";
const api = ApiPromise.create(/* ... */);
console.log(await dsnp.getSchemaId(api, "broadcast"));
Frequency mainnet and testnet have well-known Ids defined in dsnp/index.ts
.
Other configurations default to assuming npm run deploy
has been run on a fresh chain (which is usually the case for a localhost instance), but can be overridden:
dsnp.setSchemaMapping(api.genesisHash.toString(), {
// format is dsnpName: { version: schemaId, ... }
"tombstone": { "1.2": 64 },
"broadcast": { "1.2": 67 },
// ...
});
console.log(await dsnp.getSchemaId(api, "broadcast")); // yields 67
With Parquet Writer
npm install @dsnp/parquetjs
import { parquet } from "@dsnp/frequency-schemas";
import { ParquetWriter } from "@dsnp/parquetjs";
const [parquetSchema, writerOptions] = parquet.fromFrequencySchema("broadcast");
const writer = await ParquetWriter.openFile(parquetSchema, "./file.parquet", writerOptions);
writer.appendRow({
announcementType: 2,
contentHash: "0x1234567890abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
fromId: 78187493520,
url: "https://spec.dsnp.org/DSNP/Types/Broadcast.html",
});
await writer.close();
Use to Deploy Schemas
Setup
- Pull the repository
- Install dependencies
npm install
Usage
To deploy/register all schemas
npm run deploy
by default it will deploy to the localhost
node on port 9944 using the Alice sudo test account.
Two environment variables allow you to change these defaults:
DEPLOY_SCHEMA_ACCOUNT_URI="//Alice"
DEPLOY_SCHEMA_ENDPOINT_URL="ws://localhost:9944"
e.g.
DEPLOY_SCHEMA_ACCOUNT_URI="//Bob" DEPLOY_SCHEMA_ENDPOINT_URL="ws://127.0.0.1:9944" npm run deploy profile
To register a single schema
e.g. To register the "profile" schema
npm run deploy profile
Note: Requires a sudo key if deploying to a testnet.
Mainnet will use the proposal system (proposeToCreateSchema
).
Additional Tools
Help
npm run deploy help
Read all Schemas from a Chain
DEPLOY_SCHEMA_ENDPOINT_URL="ws://127.0.0.1:9944" npm run read
Will output various information about the schemas on the chain as well as attempt to match known DSNP schemas.
Example Output
## Connection Information
┌─────────┬─────────────────────┬────────────────────────────────────────────┐
│ (index) │ key │ value │
├─────────┼─────────────────────┼────────────────────────────────────────────┤
│ 0 │ 'endpointUrl' │ 'wss://frequency-seal.liberti.social:9944' │
│ 1 │ 'clientVersion' │ '0.1.0-377bbe37fbe' │
│ 2 │ 'specName' │ 'frequency-rococo' │
│ 3 │ 'specVersion' │ '1' │
│ 4 │ 'latestBlockNumber' │ '16' │
└─────────┴─────────────────────┴────────────────────────────────────────────┘
## Schema Information
There are 8 schemas on the connected chain.
## Schema Id 1
┌─────────┬──────────────────────┬───────────────────────────────┐
│ (index) │ key │ value │
├─────────┼──────────────────────┼───────────────────────────────┤
│ 0 │ 'schema_id' │ '1' │
│ 1 │ 'model_type' │ 'Parquet' │
│ 2 │ 'payload_location' │ 'IPFS' │
│ 3 │ 'matchesDSNPSchemas' │ 'dsnp.broadcast,dsnp.profile' │
└─────────┴──────────────────────┴───────────────────────────────┘
## Schema Model
[
{
"name": "announcementType",
"column_type": {"INTEGER": {"bit_width": 32, "sign": true}},
"compression": "GZIP",
"bloom_filter": false
},
{
"name": "contentHash",
"column_type": "BYTE_ARRAY",
"compression": "GZIP",
"bloom_filter": true
},
{
"name": "fromId",
"column_type": {"INTEGER": {"bit_width": 64, "sign": false}},
"compression": "GZIP",
"bloom_filter": true
},
{
"name": "url",
"column_type": "STRING",
"compression": "GZIP",
"bloom_filter": false
}
]
...
Find Frequency Schema Ids that Match DSNP Schema Versions
This script will look up and verify schemas in the schema registry that match the DSNP names and versions defined in dsnp/index.ts
.
DEPLOY_SCHEMA_ENDPOINT_URL="ws://127.0.0.1:9944" npm run find
Use with Docker
This repo deploys dsnp/instant-seal-node-with-deployed-schemas
to Docker Hub.
It is based on a Frequency Standalone Docker with the schemas automatically deployed on top of it with the image defaults including using "instant sealing" mode.
Note: --platform=linux/amd64
is because as frequencychain
images are only published for the linux/amd64
platform.
Run Locally
For any local testing do the following:
docker pull --platform=linux/amd64 dsnp/instant-seal-node-with-deployed-schemas:latest
docker run --platform=linux/amd64 --rm -p 9944:9944 dsnp/instant-seal-node-with-deployed-schemas:latest
Build Locally
docker build --platform=linux/amd64 -t dsnp/instant-seal-node-with-deployed-schemas:latest -t dsnp/instant-seal-node-with-deployed-schemas:<versionNumberHere> .
Pushing Docker Image
To match with the Frequency version, a new tag should be pushed to update the docker version of this image each time frequency releases a new version. The following steps explain how to properly do a release for this.
- Go to the Frequency repo to see what the latest release version is.
- In this repo, check that main is properly passing its tests and building here
- Go to main:
git checkout main && git pull --rebase
- Make sure to pull all latest tags as well:
git pull --tags
- Tag the build to match the frequency version but appended with "docker/":
git tag docker/{insert version number}
. For example, if the version number is v1.0.0, then the tag should bedocker/v1.0.0
Push the tag up:git push --tags
- Monitor the build
- When that finishes successfully, check Docker Hub to verify that the image was pushed up