penny-collector

    0.0.1 • Public • Published

    penny-collector

    Codeship Status for MRN-Code/penny-collector

    alpha warning

    penny-collector is not ready for public consumption. it is published in order to facilitate a basic demo

    install

    • npm i -g penny-collector. Don't forget to update often via npm update -g penny-collector.
    • penny-collector --save-config your-config-file.json
      • a custom config file can be tailored for your needs by the friendly folks at COINS
        • feel free to take a peek at the source in config/default.json if you'd like to make your own!
    • @note: penny-collector require nodejs 4.x+

    about

    penny-collector is a data logging, submitting, and archiving utility.

    penny-collector is designed to be run a system daemon. Contact your IT department for help on how to set this up. The COINS team can assist in this process. We can provide Ubuntu upstart script or help using a nodejs process monitor in setting up the daemon.

    data submission

    penny-collector can submit data in two modes, metadata mode and full-file mode.

    metadata submissions

    penny-collector can connect to COINS and submit information about research data. For example, study staff wish to submit records of "scan" images to COINS. They may also wish to submit "behavioral" data about their research participants. In this regard, staff may be able to associate and aggregate comprehensive data together. COINS can help study managers assess which participants have received which types of scans (where, when, how, etc), and query metadata about those events. Although in this mode the "raw" data is not available, the metadata is very often all that is required for research management.

    full file submissions

    penny-collector can transfer and archive data on the COINS system. This strategy is for users who want to use COINS as a storage service.

    archiver

    If you are not using penny-collector to submit files for storage at COINS, you may use penny-collector's archive functionality to move your files to archive storage based on rules. Provide a target.basePath and fields to compose your file paths from in target.fields. The fields will derive from your file meta-data, which we call a penny-model.

    API Docs

    Head over to the official docs to check out the API documentation.

    ToDo

    • add in real halfpenny validations!
    • how to handle network failure (400 HTTP)
      • retry logic // e.g.: halfpenny.validateFileMeta([ ... array, of, file, meta ], { retry: true, retry: 3, retry: { attempts: 3, backoff: function() {} } })
      • lost and found
      • report (report API error response)
    • how to handle api down (500s), same as 400s
    • 403 Forbidden
      • what happens if validation loop is successful, same as others, but invalid
    • 401 unauthorized,
      • process crash (via rethrow)

    development

    We recommend that, you, Ms. or Mr. Developer:

    • add a local.json to config/
    {
        "watchPath": ".test-watch",
        "target": {
            "basePath": ".test-target",
            "fields": ["ursi"]
        },
        "logFile": ".penny-collector.log"
    }

    appendix

    #!upstart 
    description "COINS penny-collector"
    author      "MRN-Code"
     
    start on startup
    stop on shutdown
     
    script
        exec sudo -E -u your_user penny-collector
    end script

    credits

    logo: margdking

    Install

    npm i penny-collector

    DownloadsWeekly Downloads

    2

    Version

    0.0.1

    License

    MIT

    Last publish

    Collaborators

    • cdaringe
    • jwtlake
    • mstone121
    • rsskmr
    • ruwang
    • swashcap