Necrotizing Pineapple Music

    @mieweb/wikigdrive

    1.1.0 • Public • Published

    wikiGDrive

    Google Drive to MarkDown synchronization

    CircleCI Total alerts Language grade: JavaScript Project Dependencies devDependencies Status

    Install from NPM

    View this project on NPM NPM downloads Publish wikigdrive to NPM

    npm i -g @mieweb/wikigdrive
    

    App setup

    1. Go to console https://console.developers.google.com/
    2. Create New Project
    3. Enable Apis -> add Google Drive API
    4. Enable Apis -> Add Google Docs API
    5. Credentials -> Create Credentials (OAuth Client ID) -> Other ( see authorization section )

    Usage and options

    Init workdir with (creates internal .wgd directory):

    wikigdrive init --drive "https://drive.google.com/drive/folders/FOLDER_ID"
    
    --service_account=wikigdrive.json
    --config /location/of/.wgd - Location of config file
    --dest /location/of/downloaded/content - Destination for downloaded and converted markdown files
    
    --drive_id - An ID of the drive
    
    --client_id - ID of google app, alternatively can be passed in .env or through environment variable CLIENT_ID;
    --client_secret - Secret of google app, alternatively can be passed in .env or through environment variable CLIENT_SECRET;
    
    --link_mode - Style of internal markdown links
    --link_mode mdURLs - `/filename.md`
    --link_mode dirURLs - `/filename/`
    --link_mode uglyURLs - `/filename.html` - see https://gohugo.io/getting-started/configuration/
    
    --without-folder-structure    Download documents into single, flat folder
    

    Run one time documents pull

    wikigdrive pull
    

    Run continuous documents watch

    wikigdrive watch --git_update_delay=10
    
    --watch [mtime|changes] - Run program in loop, watch for gdrive changes
    --git_update_delay=x - trigger git update hook after x minutes
    

    Run server mode for webhooks support (TODO: not implemented yet)

    wikigdrive server
    

    Example usage with Hugo Generator

    1. Install hugo https://gohugo.io/getting-started/quick-start/

    2. Create a New Site

    hugo new site quickstart
    
    1. Add a Theme
    cd quickstart
    git init
    git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke
    echo 'theme = "ananke"' >> config.toml
    
    1. Install wikigdrive
    npm i -g @mieweb/wikigdrive
    
    1. Sync GDrive
    wikigdrive init --drive "https://drive.google.com/drive/folders/FOLDER_ID" --dest ./content --link_mode uglyURLs
    wikigdrive pull
    

    Note that by default you need to use uglyURLs with Hugo. https://gohugo.io/content-management/urls/#ugly-urls

    1. Generate HTML
    hugo
    

    or start server for development:

    hugo server
    

    Example usage with Hexo Generator

    1. Install hexo https://hexo.io/docs/index.html
    npm i -g hexo-cli
    
    1. Create a New Site
    hexo init quickstart
    
    1. Add a Theme

    By default, hexo installs landscape theme. If you need another one check: https://hexo.io/docs/themes

    1. Install wikigdrive
    npm i -g @mieweb/wikigdrive
    
    1. Sync GDrive
    wikigdrive init --drive "https://drive.google.com/drive/folders/FOLDER_ID" --dest ./source --link_mode uglyURLs
    wikigdrive pull
    
    1. Generate HTML
    hexo generate
    

    or start server for development:

    hexo serve
    

    Authorization

    There are two methods: individual credentials or a service account.

    ***Note: If the authentication is successful, but the account does not have access to documents in gdrive, there is currently no way to know if the directory is empty or just not possible to see.

    FAQ

    What is the purpose of this tool?

    To enable collaborative editing of documentation and the ability to publish that documentation as well as linking it to revision control system branches (like in git)

    Why use Google at all. Why not use markdown and GitHub?

    No collaboration in real-time. Also, markdown requires skill when managing screenshots and diagrams that are not easily accomplished in markdown.

    Why not just use Google Docs?

    Would love it if it were possible, but drive does not offer the ability to publish pages cleanly. The URLs are not SEO friendly. Would love it if there was a driveId map where every document could be given a friendly name (aka its title on the drive). Then (like Wikipedia has disambiguation pages), a reader could be redirected to the proper content. Google doesn’t, so this project is an attempt to fill that gap.

    Also, Google does not have a good blame system for contributions to a document. Hopefully this is fixed someday but in the meantime, GitHub on markdown can help fill the void.

    Why markdown?

    All ears for a different preferred format. It’s easy to read when editing directly and when doing a diff for changes it’s clean

    What about mismatches in Docs vs Markdown

    There are features of Google Docs that are not going to be supported. Like coloring text, page breaks, headers, comments, etc. These features are not core to our goals for clean WYSIYYM.

    Keeping a WYSIWYM style ensures a good mobile experience to view and edit.

    Why not make a website front end to a Google shared drive?

    Our goals are to be able to take versions of the content and commit them along with a version of the code at a point in time. By just making a website, it would allow for real-time viewing of the content but no way to go to a specific version of the documentation at a given time.

    A website front end is a goal for real-time testing of the viewing experience, but initially, we want to make markdown that can be committed.

    Internals

    .wgd dir structure

    drive.json:

    {
      "drive": "https://drive.google.com/drive/folders/FOLDER_ID",
      "drive_id": "",
      "dest": "/home/user/mieweb/wikigdrive-test",
      "flat_folder_structure": false,
      "link_mode": "mdURLs",
      "service_account": "wikigdrive.json"
    }
    

    files.json is indexed with Google's fileId

    • id - Google's fileId
    • name - Title set inside google docs. It is not unique
    • mimeType - Google's mime type or 'conflict' or 'redirect'
    • modifiedTime - Server-size mtime
    • desiredLocalPath - slugified name. It is not unique, wikigdrive handles redirects so it is NOT real path in local system
    • localPath - real local path, unique with handled conflicts and redirects (in case of title rename)
    • lastAuthor - Google's last author if available
    • dirty - file needs to be downloaded
    • conflictId - unique numeric id for file within files of same desiredLocalPath (used to append localPath)
    • conflicting - array of fileIds when mimeType = 'conflict'
    • counter - current number of existing conflicts when mimeType = 'conflict'
    {
        "123123123": {
            "id": "123123123",
            "name": "A title of document",
            "mimeType": "application/vnd.google-apps.document",
            "modifiedTime": "2020-02-27T20:20:20.123Z",
            "desiredLocalPath": "a-title-of-document",
            "lastAuthor": "John Smith",
            "localPath": "a-title-of-document"
        }
    }
    

    binaryFiles.json is indexed with md5 sums:

    • localPath - path to file, generated using md5 sum
    • md5Checksum - md5 sum

    transform.json is indexed with file id

    • localPath - path to transformed markdown file
    • modifiedTime - fetched from google server
    {
        "123123123": {
            "localPath": "external_path/123123123.png",
            "md5Checksum": "123123123"
        }
    }
    

    Install

    npm i @mieweb/wikigdrive

    DownloadsWeekly Downloads

    3

    Version

    1.1.0

    License

    ISC

    Unpacked Size

    5.9 MB

    Total Files

    83

    Last publish

    Collaborators

    • horner
    • wreiske