@contentstack/datasync-manager
    TypeScript icon, indicating that this package has built-in type declarations

    1.2.1 • Public • Published

    Contentstack

    Contentstack is a headless CMS with an API-first approach. It is a CMS that developers can use to build powerful cross-platform applications in their favorite languages. Build your application frontend, and Contentstack will take care of the rest. Read More.

    Contentstack DataSync Manager

    Contentstack DataSync lets you sync your Contentstack data with your database, enabling you to save data locally and serve content directly from your database. It is a combination of four powerful modules that is DataSync Webhook Listener, DataSync Manager, DataSync Asset Store Filesystem, DataSync Content Store — Filesystem and MongoDB.

    The DataSync Manager is one of the four important components of Contentstack DataSync. When any publish, unpublish, or delete operations are performed on assets or content, the DataSync Manager fetches the data and sends it to Content Store. It uses Contentstack's Sync APIs to sync data from Contentstack with your preferred database — Filesystem and MongoDB in our case.

    Prerequisite

    • nodejs v8+

    Usage

    This is how DataSync Manager works with DataSync boilerplate:

    const assetStore = require('@contentstack/datasync-asset-store-filesystem')
    const contentStore = require('@contentstack/datasync-content-store-filesystem')
    const listener = require('@contentstack/webhook-listener')
    const syncManager = require('@contentstack/datasync-manager') // <--
    const config = require('./config')
    
    // Set asset, content store, listener and config to Datasync Manager
    syncManager.setAssetStore(assetStore)
    syncManager.setContentStore(contentStore)
    syncManager.setListener(listener)
    syncManager.setConfig(config)
    
    // start DataSync manager
    syncManager.start()
      .then(() => {
        console.log('Contentstack sync started successfully!')
      })
      .catch(console.error)

    You can replace @contentstack/datasync-content-store-filesystem used above, with @contentstack/datasync-content-store-mongodb and switch content store databases.

    Configuration

    • Here's a list of contentstack's configuration keys
    Key Name Default Description
    apiKey Required. Your stack's API key
    deliveryToken Required. Your environment's delivery token
    sync_token Token from where you'd like to start the process
    pagination_token Token from where you'd like to start the process
    MAX_RETRY_LIMIT 6 Number of times the API call would retry, if the server fails
    • Here's a list of configuration keys for contentstack datasync-manager:
    Property Default Description
    cooloff 3000 Number of ms the app would wait before making a subsequent Sync API request
    enableAssetReferences true This would map out all the asset fields and modify/map them to the content type they belong to. Ex:
    • Before
    • "asset_field": "bltassetuid123"
    • After
    • "asset_field": { "reference_to": "_assets", "values": "bltassetuid123" }
    enableContentReferences true This would map out all the reference fields and modify/map them to the content type they belong to. Ex:
    • Before
    • "categories": "bltcategoryuid123"
    • After
    • "categories": { "reference_to": "category", "values": "bltcategoryuid123" }
    limit 100 Number of items fetched in each Sync API request made.
    maxsize 2097152 Maximum file size of files (ex: .ledger file)
    queue.pause_threshold 10000 The min-max internal queue size of contentstack-sync-manager. The DataSync manager would pause making Sync API requests once the internal queue size exceeds pause_threshold count.
    queue.resume_threshold 5000 The min-max internal queue size of contentstack-sync-manager. The DataSync manager will resume Sync API requests once the queue size goes below resume_threshold count.
    saveFilteredItems true When enabled, the app will log items that have been filtered out. Items can be filtered out due to the following:
    • Missing required keys
    • Language filters applied via config
    filters
    • content_type_uid
    • locale
    • action
    Filters allow you to narrow down the items that'd be returned by the SYNC API response. Ex: {filters: {content_type_uid: ['blogs']}, locale: ['en-us'], action: ['publish']}

    And here's an example to get you started:

    {
      // exmaple to override default values
      contentstack: {
          apiKey: '',
          deliveryToken: '',
          sync_token: '',
          MAX_RETRY_LIMIT: 5
        }
      },
      syncManager: {
          cooloff: 2000,
          limit: 80,
          filters: {
            content_type_uid: ['blogs'],
            locale: ['es-es', 'en-us'],
            action: ['publish', 'unpublish']
          }
        }
      }
    }

    Further Reading

    Support and Feature requests

    If you have any issues working with the library, please file an issue here at Github. You can send us an e-mail at support@contentstack.com if you have any support or feature requests. Our support team is available 24/7 on the intercom. You can always get in touch and give us an opportunity to serve you better!

    Licence

    This repository is published under the MIT license.

    Install

    npm i @contentstack/datasync-manager

    DownloadsWeekly Downloads

    14

    Version

    1.2.1

    License

    MIT

    Unpacked Size

    730 kB

    Total Files

    103

    Last publish

    Collaborators

    • shafeeqd959
    • mynk
    • kego1992
    • roshangade
    • iyerrama29
    • uukkoji
    • cloudaccounts
    • gautam.lodhiya
    • hiteshshetty