Nauseating Pumpkin Mush

    vdev
    TypeScript icon, indicating that this package has built-in type declarations

    0.13.4 • Public • Published

    About

    vdev is a set of utilities for high velocity devops focused on enabling aKubernetes centric development and operation model.

    IMPORTANT vdev 0.10.0+ Command command line ./node_modules/.bin/vdev changed syntax. See below for new commands.

    NOTE: At this point this tool is very focused on a specific set of technologies (docker/kubernetes), nodeJs/TypeScript, postgres as db, and google cloud platform). For anybody external to BriteSnow dev (or its clients), it might be considered experimental as API and cli might changes. Feel free to cherry-pick what you might find useful (all MIT license).

    vdev has two main constructs:

    • blocks are module or service (when dockerfile) that get built. vDev recognize if there is a tsconfig.json, package.json, Dockerfile and perform the appropriate built. As of now, vdev does not have a plugin feature, but this will come in the future. Block can also have webBundles which can be .js, .ts, .pcss (postcss), or .tmpl (handlebars assume), and vdev will do the appropriate compile.

    • realms are kubernetes/cluster contexts where container blocks and yaml resources get deployed to. Realm commands allow to change easily between those k8s contexts and gcp project in one command, and provide simple yet powerfull yaml handlebars templating scheme. Today supports GCP, but the goal is to add AWS soon.

    Requirements:

    Minimum requirement:

    • node.js > 10
    • Docker for mac with Kubernetes support (current edge)
    • Typescript installed globally (for now) if some typescript blocks
    • docker for 'npm test'

    GCP requirement (for deployment):

    • gcloud command lines instaleld with the appropriate authentication.

    AWS support will be added when it will suport Kubernetes.

    Usage

    Setup

    • In the project, install vdev npm install vdev
    • In the root project, have a vdev.yaml file as below
    • Options to run the commands (both option are independent)
      • Option 1 (npm run vdev ...): In package.json in the "scripts" object add
        • "vdev": "node ./node_modules/.bin/vdev"
        • Then can use as npm run vdev dbuild to docker build the blocks.
      • Option 2 (shell alias): In user ~/.profile add a line like
        • alias v="node ./node_modules/.bin/vdev"
        • Then, from project root, can douse as v dbuild to docker build the blocks (does not need to have option 1 to work)

    Commands example

    Assuming Option 1 above, and running this command project root.

    # --- Docker
    ## Build all block with Dockerfile (or the one marked defaultImages in vdev)
    node run vdev dbuild
    ## Build a specific list
    node run vdev dbuild web-server,agent
    
    ## Push docker image(s) to target registry (all or the defaultImages)
    node run vdev dpush
    ## Push specific list
    node run vdev dpush web-server,agent
    
    # --- Kubernetes
    ## Kubectl apply resource files in k8s directory for the current realm (or all specified in defaultConfigurations)
    node run vdev kapply 
    ## Only some configuration files (with .yaml extension)
    node run vdev kapply _config,web-server,agent
    
    ## Kubectl apply (same rule as above)
    node run vdev kcreate web-server,agent
    
    ## Kubectl delete (same rule as above)
    node run vdev kdelete 
    
    ## Kubectl exec (one service only)
    node run vdev kexec web-server
    
    ## Kubectl logs (all pods log)
    node run vdev klogs 
    ## Only some service log
    node run vdev klogs web-server,agent
    
    ## Kubernetes server/service restart. Will call the `/service/restart.sh` of one or more services (not recreating/restart the pod, just call the scripts `/service/restart.sh`)
    node run vdev krestart 
    
    ## (internal) just run the templating on a k8s configuration files and output it in the .out/_realm_/ folder for inspection or manual manipulation
    node run vdev ktemplate
    
    # --- Misc
    ## Update the app version from multiple file
    node run vdev version
    ## Remove the package-lock.json and node_modules of all block (not root)
    node run vdev nclean
    ## Build block (local) with web bundle
    node run vdev build web 
    ## Build a specific web bundle
    node run vdev build web/css 
    
    # --- Storage
    ## List 
    node run vdev sls dev:some/prefix
    ## download
    node run vdev sdown dev:some/file.png
    ## upload (or can work with file)
    node run vdev sup /some/local/file.png dev:some/folder/
    ## copy
    node run vdev scp dev:some/file.png prod:other/file.png
    

    vdev.yaml example

    First let's define the vdev.yaml at the root of the monorepo.

    system: myapp
    baseBlockDir: services/ # used as the base dir .dir (as '${baseDir}/${blockName}/'')
    k8sDir: k8s/ # root 
    imageTag: '{{__version__}}'
    
    realms:
      _common:  # common/default values that will be used for each realm
        yamlDir: gke/ # so `k8s/gke` will be the yamlDir if not overriden below.
    
      dev:
        context: docker-for-desktop
        yamlDir: dev/ # Override the common yaml dir to `k8s/dev/` 
                      #(dev has hooks for debugging and such, and commong yaml files will just add more complexity)
    
      stage: # for aws
        context: arn:aws:eks:us-west-2:33333344444:cluster/eks-cluster-01
        registry: 33333344444.dkr.ecr.us-west-2.amazonaws.com/
    
      stage-azure: # for asure 
        context: stage-asure-ref
        registry: thirdpartyservices.azurecr.io/  
    
      prod: # for gcp (must have project, registry inferred)
        confirmOnDelete: true
        context: gke_my-gcp-prod-project-_us-west1-prod-cluser
        project: my-gcp-prod-project
        loadBalancerIP: '35.33.23.18'
    
    blocks:
      - db
      - queue
      - agent
      - importer
      - name: web
        dir: web/
        baseDistDir: web-server/src/main/webapp
        webBundles:
          - name: lib
            entries: ./src/lib-bundle.js
            dist: ./js/lib-bundle.js
          - name: app
            entries: ./src/**/*.ts
            dist: ./js/app-bundle.js
            rollupOptions:
              globals:
                d3: window.d3
                mvdom: window.mvdom
                handlebars: window.Handlebars        
          - name: css
            entries: ./src/**/*.pcss
            dist: ./css/all-bundle.css
          - name: tmpl
            entries: ./src/**/*.tmpl
            dist: ./js/templates.js
      - name: web-server
        dir: web-server/
        dbuildDependencies: web # build dependency for when running dbuild (no effect on build).

    API

    PSQL

    import * as fs from 'fs-extra-plus';
    import { assertRealm, getCurrentRealm, psqlImport } from '../main';
    
    const sqlDir = './sql';
    
    /**
     * NOTE assuming recreateDb is called with minimit argv
     * Convention and order of execution is: 
     * 	- sql files starting with '0' will be ran a postgres users
     *  - sql files starting with '1' will be ran a dbPrefix_user on dbPrefix_db
     *  - sql files starting with 'drop-' will be ran a dbPrefix_user on dbPrefix_db
     * @param argv 
     */
    async function recreateDb(argv: ParsedArgs) {
    	const host = argv._[0] || "localhost";
    
    	const realm = assertRealm(await getCurrentRealm());
    	const dbPrefix = realm.system + '_';
    
    	await psqlImport({ user: "postgres", db: "postgres", host: host }, await fs.glob('0*.sql', sqlDir));
    	await psqlImport({ user: dbPrefix + "user", db: dbPrefix + "db", host: host }, await fs.glob('1*.sql', sqlDir));
    	await psqlImport({ user: dbPrefix + "user", db: dbPrefix + "db", host: host }, await fs.glob('drop-*.sql', sqlDir));
    }

    Keywords

    none

    Install

    npm i vdev

    DownloadsWeekly Downloads

    10

    Version

    0.13.4

    License

    MIT

    Unpacked Size

    211 kB

    Total Files

    90

    Last publish

    Collaborators

    • jeremychone