Manganese

Download and organize manga on your computers using popular websites crawling.

Contains both a programmatically usable API module, and a command-line interface which can be used on its own.

Based on the starkana-manga-crawler project, but this time, with support for multiple websites through a plugin system.

Command-line interface

install and setup

Using npm:

npm install manganese -g   # For the cli

You then need to install a plugin. The plugin is responsible of the actual downloading work.

# This is just just one plugin, more will come soon!
manganese install mangatown-manganese

You can find all available plugins by using

manganese find-plugin # don't freak out if it's slow, it's normal
# and then do
manganese install <that-plugin-that-you-like>

Should you want to remove one that you don't like (you can also simply not use it):

manganese remove <that-plugin-that-you-hate>

You can use manganese to control your library from the command line! For now, you can only download things, but more functionalities will be coming soon, such as keeping a list of manga that you like and download all the new releases.

downloading

Once you have a plugin installed, you can start downloading chapters using

manganese download|dl <plugin> <series> <chapter(s)> [options] 
// Examples
// Naruto chapter 1
manganese download manganese-plugin-example Naruto 1
// One Piece chapters 1 to 10 (1 and 10 included)
manganese dl manganese-plugin-example "One Piece" 1:10
// All Bleach chapters starting from chapter 600 onward (600 included)
manganese dl manganese-plugin-example Bleach 600:$
// Change the name of the series if you wish using -n
manganese dl manganese-plugin-example "The Breaker: New Waves" -n TBNW 1 

See number-ranger for how to format your chapter range queries.

Subscription

You can subscribe to series, and then request to download all the new chapters for the series you're subscribed to.

Subscribing

manganese subscribe|sub -p <plugin> [options] <series>
// Examples
// Subscribe to One Piece and Bleach
manganese subscribe -p manganese-plugin-example "One Piece" Bleach
// You can change the name of the series, but then only for one series
manganese sub -p manganese-plugin-example "The Breaker: New Waves" -n TBNW

// Unsubscribe using -u
manganese sub -u Bleach

Updating

Now you can get the latest chapters for all your subscriptions using only one command.

// Update all series
manganese update|up
// Update only some series
manganese update|up "One Piece" Bleach

This will only download those that you don't have yet, meaning starting from (but not included) the latest chapter you currently possess. How does manganese know how far along in the series you are? It will use both of the following techniques:

• It will simply look at your manga folder(s), and figure out what the latest chapter is by looking at each folder's name and chapter number
• The subscription file stored on your computer that registers all your subscriptions also stores your progress. Everytime you download chapters for one of your subscriptions (through download or update), this file will be updated. Tip: This file could be shared on Dropbox for example so that you can syncronize multiple computers. If wanted, you can disable updating your subscriptions by using --nocache.

module - programmatic use

install and setup

Using npm:

npm install manganese -g   # For the cli

downloader

This is the most useful sub-module when you only want to do simple operations like specific chapter downloads.

downloader.download(jobs, config, cb, progressCb)

This method starts fetch jobs in parallel, which will create (if successful and needed) download jobs that it will put in an internal queue. Those jobs will then be downloaded in parallel, after all the fetching has been completed.

Arguments

• jobs: Array of jobs to run, of the form

// To download "Naruto" chapters 1 to 5 (included) and "One Piece" 10, using the manganese-plugin-example plugin.
var jobs = [{
    plugin: 'manganese-plugin-example',
    series: 'Naruto',
    chapters: [{
        start: 1,
        end: 5
}, {
    plugin: 'manganese-plugin-example',
    series: 'One Piece',
    chapters: [{
        start: 10
        // Plus some additional fields that each plugin might find necessary or useful, such as the chapter's url.
    }]
}];

• config: Configuration object

/*
Will download to /my/folder. Complete path with the previous jobs example would download to
- /my/folder/Naruto/Naruto 1, ..., /my/folder/Naruto/Naruto 5
- /my/folder/One Piece/One Piece 10
*/
var config = [{
    // Where the chapters will be downloaded to
    // Naruto chapter 1 will be downloaded to '/my/folder/Naruto/Naruto 1'
    destFolder: '/my/folder',
    // Number of chapters to be downloaded at the same time
    chapterConcurrency: Number, // default: 1
    // Number of pages to be downloaded at the same time.
    // The umber of concurrent network operations wil therefore be close to chapterConcurrency * pageConcurrency.
    pageConcurrency: Number // default: 5
}];

• cb:

• cb(error) if an error occurs somewhere in the process
• cb(null, jobList) after all fetch and download jobs have ended. jobList will be an array of jobs that will be started in an internal queue and given back as is to the corresponding plugin, of the form:

[
    {
        plugin: 'manganese-plugin-example',
        series: 'Naruto',
        chapter: 1,
        dest: '/my/folder/Naruto/Naruto 1',
        // Plus some additional fields that each plugin might find necessary or useful, such as the chapter's url.
    }
]

installer

installer.install(plugins, cb)

You can install manganese-plugins by name, which will be called using npm install <plugin>. It will make sure that the plugins are really manganese-plugins (i.e. that they have that keyword in their package.json).

subscriber

subscriber.subscribe(config, subscriptions, cb)

Subscribe to one or more series.

Arguments

• config: Configuration object. Only needs to contain subscriptionFile, which needs to be a path to a file where subscriptions will be registered.

• subscriptions: Object or array of objects of the form

{
    plugin: 'manganese-plugin-example',
    series: 'One Piece',
    name: 'OP', // Name to use for your folders, defaults to series
    latest: 12 // Latest downloaded chapter, defaults to -Infinity
}

• cb:

• cb(error) if an error occurs somewhere in the process
• cb(null, subscriptions) updated list of subscriptions

subscriber.unsubscribe(config, subscriptions, cb)

Unsubscribe to one or more series.

Arguments

• config: Configuration object. Only needs to contain subscriptionFile, which needs to be a path to a file where subscriptions will be registered.

• subscriptions: String or array of strings, corresponding to series names.

* `cb`:
- `cb(error)` if an error occurs somewhere in the process
- `cb(null, subscriptions)` updated list of subscriptions

### subscriber.updateLatest(config, subscriptions, cb)

Update the latest field for a subscription. Will only modify it it the value is higher than the current value. 

__Arguments__
* `config`: Configuration object. Only needs to contain `subscriptionFile`, which needs to be a path to a file where subscriptions will be registered.

* `subscriptions`: Object or array of objects of the form
```js
{
    name: 'Naruto', // Series name
    latest: 12 // Latest downloaded chapter
}

• cb:

• cb(error) if an error occurs somewhere in the process
• cb(null, subscriptions) updated list of subscriptions

updater

updater.update(config, jobs, cb, progressCb)

Subscribe to one or more series.

Arguments

• config: Configuration object of the form

{
    // Folder to look into to find current series progress, and to save chapters into
    destFolder: '/some/path',
    // Additional folder to look into
    readFolders: [
        '/some/other/path',
        '/yet/another/path'
    ]
}

• jobs: Object or array of objects of the form

{
    plugin: 'manganese-plugin-example',
    series: 'One Piece',
    name: 'OP' // Name to use for your folders, defaults to series
}

• cb: Result of downloader.download() function.

• cb(error) if an error occurs somewhere in the process
• cb(null, subscriptions) updated list of subscriptions

• progressCb: progressCb that will be passed as is to downloader.download() function.

How to contribute

The manganese project

Do you want to improve manganese? Feel free to submit issues and/or pull requests. Any advice and constructives comments is welcome too.

Writing your own plugin

Do you want to write your own plugin? I don't have a documentation for it yet, but instead of waiting for it, you should check out mangatown-manganese and do pretty much like it. It's simply a node module that has been published on npm.

Some quick notes on writing plugins and future objectives:

• The plugin should correspond to only one website.
• The plugin must at least expose two methods:
  • listJobs: fetch an array of jobs (1 job = 1 chapter) that will then be queued, based on the request made.
  • downloadChapter: download a chapter. It will be given an item of the job array received from listJobs, with additional fields such as the destination folder where you have to save the chapter's content. If it can help you, put as much useful information in listJobs' results (such as the url for the chapter), but keep in mind that it should be a very lightweight method and that the grunt work should be in downloadChapter.
• When you are done writing the plugin, publish it to npm, and make sure that it has 'manganese-plugin' in the keywords field of the package.json, or else users won't be able to find or install it.
• I expect to at some point add extra functionalities, which will need to be implemented by the plugin. I will if possible try to make the transition smooth and allow plugins without the new functionalities to still be usable.
• I currently write a 100% mocked test suite with no i/o so that test run fast and succeed without any additional configuration or Internet access, but I'm still not sure this is the way to go, as it won't detect breaking changes coming from the website.
• Check both manganese and mangatown-manganese's test suites to understand what is expected.
• If you wish for a feature in manganese, advice or simply have questions, feel free to contact me.