Notorious Public Menace

    @ddn/server

    0.0.2 • Public • Published

    DDN Server

    ddn-server is a Data Distribution Node (DDN) server based on DDN with the Dat protocol.

    ddn-server is for you if:

    • You're comfortable with some server administration (or want to learn!)
    • You want to keep your dat:// website/s online
    • You want to publish your dat:// website/s to a domain name

    Table of contents

    Features

    • Keeps dat:// websites online
    • Publishes dat:// websites to DNS shortnames
    • Mirrors dat:// websites to https://
    • Provisions and manages HTTPS with Let's Encrypt
    • Provides optional Pinning Service API endpoints

    Install

    If you already have Node.js (8.0+) and npm installed on your server, get started by installing DDN server with npm or npx.

    npm install -g @ddn/server

    Otherwise, install Node.js and npm first:

    Having trouble installing? See Troubleshooting.

    Running server

    To run ddn-server manually, simply invoke the ddn-server command:

    ddn-server

    To keep ddn-server running, you'll need to daemonize it. We like using pm2.

    # install pm2
    npm i -g pm2
    
    # start server with pm2
    pm2 start ddn-server

    To stop the daemon, run:

    # stop server
    pm2 stop ddn-server
    

    Command line flags

    • --config <path>
      • Use the config file at the given path instead of the default ~/.ddn-server.yml. Overrides the value of the DDN_SERVER_CONFIG env var.

    Environment variables

    • DDN_SERVER_CONFIG=cfg_file_path
      • Specify an alternative path to the config than ~/.ddn-server.yml
    • NODE_ENV=debug|staging|production
      • Set to debug or staging to use the Let's Encrypt testing servers.

    Examples

    ddn-server uses a configuration file (~/.ddn-server.yml by default) for managing its behavior. These examples show various configurations.

    See all configuration options

    Example: set up a website with a domain name and HTTP mirroring

    This configuration file will "seed" or "pin" the files at dat://123...456, publish those files to dat://alice.com, and mirror them to https://alice.com.

    This example uses a domain name, so in order for the domain name to resolve correctly, you'll need to update your DNS configuration first. In this case, you could set an A record that points to the ddn-server server's IP address.

    dats:
      - url: dat://123...456
        domains:
          - alice.com
    httpMirror: true
    letsencrypt:
      email: alice@mail.com
      agreeTos: true
    

    Example: seed multiple websites, with no domain names

    This configuration simply "seeds" or "pins" the files at dat://123...456 and dat:///456...789. No domain name is required for this configuration.

    dats:
      - url: dat://123...456
      - url: dat://456...789

    Example: seed multiple websites, enable Web API

    In addition to being used to manage a website, ddn-server can be used with more specific configurations, like providing a mini-pinning service for a group of friends.

    This example enables the Pinning Service API, which makes it possible for you (and maybe a few friends!) to publish to your ddn-server instance with ddn, the Dat CLI, or any client that supports the Pinning Service API.

    If you choose to use a domain name for your API endpoint, be sure to configure your DNS to point to your ddn-server server.

    See full webapi reference

    # enable publishing to `ddn-server` from Eok and the Dat CLI
    webapi:                # set to false to disable
      domain:              # the domain name for the web API (optional)
      username:            # the username for publishing from ddn or the Dat CLI
      password:            # the password for publishing from ddn or the Dat CLI

    Configuration

    Configuration file

    ddn-server uses ~/.ddn-server.yml as its default configuration file. You can specify an alternative config file using a command line flag or an environment variable.

    directory: ~/.ddn-server # where your data will be stored
    httpMirror: true       # enables HTTP mirroring
    ports:
      http: 80             # HTTP port for redirects or non-TLS serving
      https: 443           # HTTPS port for serving mirrored content and DNS data
    letsencrypt:           # set to false to disable Let's Encrypt
      email:               # you must provide your email to LE for admin
      agreeTos: true       # you must agree to the LE terms (set to true)
    dashboard:             # set to false to disable
      port: 8089           # port for accessing the metrics dashboard
    
    # enable publishing to server from ddn and the Dat CLI
    webapi:                # set to false to disable
      domain:              # enter your web API's domain here (optional, unless Let's Encrypt TLS is wanted on the web API)
      username:            # the username for publishing from ddn or the Dat CLI
      password:            # the password for publishing from ddn or the Dat CLI
    
    # enter your pinned dats here
    dats:
      - url:               # URL of the dat to be pinned
        domains:           # (optional) the domains of the dat
    
    # enter any proxied routes here
    proxies:
      - from:              # the domain to accept requests from
        to:                # the domain (& port) to target
    
    # enter any redirect routes here
    redirects:
      - from:              # the domain to accept requests from
        to:                # the domain to redirect to

    dashboard

    Default false

    Set to true to enable the Prometheus metrics dashboard.

    dashboard:             # set to false to disable
      port: 8089           # port for accessing the metrics dashboard

    dashboard.port

    Default: 8089

    The port to serve the Prometheus metrics dashboard.

    dats

    A listing of the Dat archives to seed.

    dats:
      - url: dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03/
        domains:
          - mysite.com
          - my-site.com

    dats.*.url

    The Dat URL of the site to host. Should be a 'raw' dat url (no DNS hostname).

    Example values:

    # raw key
    1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03
    
    # URL with trailing slash
    dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03/
    
    # URL with no trailing slash
    dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03
    

    dats.*.domains

    Additional domains of the Dat archive. Can be a string or a list of strings. Each string should be a domain name.

    To use dats.*.domains, you'll first need to configure the DNS entry for your domain name to point to your server. For instance, to point alice.com with ddn-server, you'll need to update your DNS configuration to point alice.com to your server server's IP address.

    Example values:

    mysite.com
    foo.bar.edu
    best-site-ever.link
    

    dats.*.name

    DEPRECATED. See the v2.0.0 migration guide.

    The name of the Dat archive. Sets a subdomain relative to the Web API domain, similar to the way that Hashbase does. Must be unique on the ddn-server instance.

    dats.*.otherDomains

    DEPRECATED. Use the domains field instead.

    Additional domains of the Dat archive. Can be a string or a list of strings. Each string should be a domain name. Example values:

    mysite.com
    foo.bar.edu
    best-site-ever.link
    

    directory

    Default: ~/.ddn-server

    The directory where ddn-server will store your Dat archive's files.

    domain

    DEPRECATED. See the v2.0.0 migration guide.

    The DNS domain of your server instance.

    httpMirror

    Default: false

    Set to true to provide https mirroring of your Dat archives.

    letsencrypt

    Default: false

    Set to true to enable Lets Encrypt's automatic TLS certificate provisioning.

    letsencrypt:           # set to false to disable Let's Encrypt
      email:               # you must provide your email to LE for admin
      agreeTos: true       # you must agree to the LE terms (set to true)

    letsencrypt.email

    The email to send Let's Encrypt notices to.

    letsencrypt.agreeTos

    Do you agree to the terms of service of Lets Encrypt? Required, must be true.

    ports

    The ports for HTTP and HTTPS.

    ports:
      http: 80             # HTTP port for redirects or non-TLS serving
      https: 443           # HTTPS port for serving mirrored content and DNS data

    ports.http

    Default: 80

    The port for serving HTTP sites.

    HTTP automatically redirects to HTTPS.

    ports.https

    Default: 443

    The port for serving HTTPS sites.

    proxies

    A listing of domains to proxy. Useful when your server has other services running that you need available.

    proxies:
      - from: my-proxy.com
        to: http://localhost:8080

    proxies.*.from

    The domain to proxy from. Should be a domain name.

    Example values:

    mysite.com
    foo.bar.edu
    best-site-ever.link
    

    proxies.*.to

    The protocol, domain, and port to proxy to. Should be an origin.

    Example values:

    https://mysite.com/
    http://localhost:8080/
    http://127.0.0.1:123/
    

    redirects

    A listing of domains to redirect.

    redirects:
      - from: my-old-site.com
        to: https://my-site.com

    redirects.*.from

    The domain to redirect from. Should be a domain name.

    Example values:

    mysite.com
    foo.bar.edu
    best-site-ever.link
    

    redirects.*.to

    The base URL to redirect to. Should be an origin.

    Example values:

    https://mysite.com/
    http://localhost:8080/
    http://127.0.0.1:123/
    

    webapi

    Default: false

    Set to true to enable the Pinning Service API, which enables publishing to DDN server with ddn, the Dat CLI, or any other client that supports the Pinning Service API.

    # enable publishing to `ddn-server` from ddn and the Dat CLI
    webapi:                # set to false to disable
      domain:              # the domain of the web api (optional)
      username:            # the username for publishing from ddn or the Dat CLI
      password:            # the password for publishing from ddn or the Dat CLI

    webapi.username

    The username for your pinning service API.

    webapi.password

    The password for your pinning service API.

    webapi.domain

    The DNS domain of your server Web API. Optional, but required if you want Let's Encrypt to provide your Web API with a TLS certificate.

    Advanced examples

    Example: proxies

    If your ddn-server instance is running on ports 80/443, and you have other Web servers running on your server, you might need ddn-server to proxy to those other servers. You can do that with the proxies config. Here's an example proxy rule:

    See full proxies reference

    proxies:
      - from: my-proxy.com
        to: http://localhost:8080

    Example: redirecting requests

    Sometimes you need to redirect old domains to new ones. You can do that with the redirects rule. Here's an example redirect rule:

    See full redirects reference

    redirects:
      - from: my-old-site.com
        to: https://my-site.com

    Example: using a metrics dashboard

    ddn-server has built-in support for Prometheus, which can be visualized with Grafana.

    ./grafana-screenshot.png

    DDN server exposes its metrics at port 8089. Prometheus periodically scrapes the metrics and stores them in a database. Grafana uses those metrics and provides a provides a nice dashboard visualization. It's a little daunting at first, but setup should be relatively painless.

    Steps:

    1. Install Prometheus on your server
    2. Install Grafana on your server
    3. Update the prometheus.yml config
    4. Start Prometheus and Grafana
    5. Login to Grafana
    6. Add Prometheus as a data source to Grafana (it should be running at localhost:9090
    7. Import this Grafana dashboard

    Your prometheus.yml config should include have the scrape_configs option set like this:

    scrape_configs:
      - job_name: 'prometheus'
        static_configs:
          - targets: ['localhost:9090']
      - job_name: 'server'
        static_configs:
          - targets: ['localhost:8089']

    Example: running server behind Apache or Nginx

    If you're running ddn-server on a server that uses Apache or Nginx, you may need to change your config to disable HTTPS. For instance, if you're using nginx and proxying to port 8080, update your config to disable Let's Encrypt and to set the HTTP port:

    letsencrypt: false
    ports:
      http: 8080

    You will need to add all domains to your Nginx/Apache config.

    Example: running server in a docker container

    1. Install Docker. If you're on Linux, remember to configure Docker to start on boot. Don't know of the equivalent for other systems.

    2. Clone the project. Edit .ddn-server.yml according to your needs. Most importantly: Change username and password.
      If you don't want to think of a username and a password, just use this but increase the length.

    3. In the project root, run this command:

    docker build -t ddn-server:latest . && docker run -d --name=ddn-server --restart=always -p 80:80 -p 443:443 -p 3282:3282 ddn-server:latest
    

    Notes:

    1. Not an expert in Docker security or configuration.
    2. if you have ddn on the same machine, you may want to change the dat port -p 3282:3282 to something like -p 9999:3282.
    3. To debug the running container:
      • Run docker ps -a to see the container running status.
      • Run docker logs ddn-server to see the logs.
      • Run docker exec -it ddn-server sh to get into a terminal.
    4. Didn't think about how you'd install a newer version of ddn-server while keeping the old configuration and data.

    Troubleshooting

    Installing build dependencies

    When installing ddn-server, you may need to install additional build dependencies:

    sudo apt-get install libtool m4 automake libcap2-bin build-essential
    

    Port setup (EACCES error)

    For ddn-server to work correctly, you need to be able to access port 80 (http), 443 (https), and 3282 (dat). Your firewall should be configured to allow traffic on those ports.

    If you get an EACCES error on startup, you either have a process using the port already, or you lack permission to use the port. Try lsof -i tcp:80 or lsof -i tcp:443 to see if there are any processes bound to the ports you need.

    If the ports are not in use, then it's probably a permissions problem. We recommend using the following command to solve that:

    # give node perms to use ports 80 and 443
    sudo setcap cap_net_bind_service=+ep `readlink -f \`which node\``
    

    This will give nodejs the rights to use ports 80 and 443. This is preferable to running ddn-server as root, because that carries some risk of a bug in ddn-server allowing somebody to control your server.

    Support

    ddn-server is built by the DDN team. You can help us continue our work on ddn, ddn-server, and more. Thank you!

    Licence

    Copyright (c) 2019 DDN.link Copyright (c) 2018 Blue Link Labs

    Keywords

    Install

    npm i @ddn/server

    DownloadsWeekly Downloads

    0

    Version

    0.0.2

    License

    MIT

    Unpacked Size

    87.1 kB

    Total Files

    27

    Last publish

    Collaborators

    • bjtailor
    • haoei
    • piupiua
    • imfly
    • evanlai